Sniffer news

news in version 8

since 8 RC9

Old cleaning spool replaced by new different cleaning mechanism which was developed to minimize I/O operations and it also finally brings more features. Cleaning procedure iterates through new index files and unlink it without needs to scan directories and count the space on each cleaning iteration.

cleaning procedure runs every hour and checks maximum size or maximum days. If you set maxpoolsize it will wipe out the oldest data every hour until the size is reached. maxpooldays keeps maximum number of data to set days. The same is for sip rtp and graph so you can keep sip pcaps longer than rtp pcaps for example. All options can be activated at once it is good to always have maxpoolsize = N where the N is maximum disk space you are willing to use by sniffer. All sizes are in MB maxpoolsize = 102400

#maxpooldays            =
#maxpoolsipsize         =
#maxpoolsipdays         =

#maxpoolrtpsize         =
#maxpoolrtpdays         =

#maxpoolgraphsize       =
#maxpoolgraphdays       =

update (1.8.2013) new in 8 RC7 -

• implement new option cdrproxy = yes (enabled by default) : in case SIP session travels accross several proxies (and Call-ID header does NOT change) and you would like to track all sip proxies and make them searchable in GUI / database. If disabled in cdr.sipcalledip will be destination IP from the first INVITE. If enabled in cdr.sipcalledip will be destination IP from the last INVITE and all IP from middle INVITE will be inserted in cdr_proxy table. In the GUI is new proxy column.
• add partitions for register_state and register_failed
• fix capture rules - if there is NULL in column ignore the flag completely. This fixes issue when the rule is created and user wants to override only one flag and leave others untouched (NULL)

complete changelog [1]

Version 8 implements new way of buffering packets from kernel ringbuffer into dynamically allocated buffer which has now no memory limit. The old vmbuffer was allocated statically up to maximum 4GB memory but the new buffer grows dynamically up to desired memory. What is the buffer good for? In case storage disk is temporarily overloaded and the sniffer is storing all packets to disk the buffer is growing faster than the I/O rate. For some high network traffic 4GB buffer was not enough and can handle only few minutes under high pressure. The new buffer can also be compressed (~50% ratio) thus doubling the overall memory. If the memory buffer is completely filled disk buffer is used which is also new feature for those who wants to not loose any single packet in any situation. Storage for disk buffer is recommended dedicated which is not shared with pcap or mysql (so it can guarantee desired I/O throughput). The new buffer also reduces number of packet copies to only 1 instead of 3 reducing CPU by 10%.

With the new buffer mechanism we also implemented new mirroring option which sends compressed buffer over TCP socket to remote voipmonitor sniffer. This approach is now recommended way how to do software packet mirroring between two linux servers. Sometimes it is not desired to overload Linux based PBX/SBC by voipmonitor sniffer so the sniffer acts only as mirror. The mirroring can do auto-reconnect in case of connection failure and can use the new buffering mechanism - in memory and on disk with compression which means that the mirroring can be interrupted for as long as allocated memory or disk space.

With version 8 it is now possible to add capture rules with "skip" flag which completely skips processing certain calls based on IP or Tel. number rules.

New debug messages - if voipmonitor sniffer is running with at least "-v 1" you can watch several metrics:

tail -f /var/log/syslog (on debian/ubuntu)
tail -f /var/log/messages (on redhat/centos)

calls[365][401] cdrqueue[0] heap[0.1%] heapoverruns[0] comp[41.3%] traffic[25.9Mb/s] t0CPU[6.3%] t1CPU[1.6%] t2CPU[1.4%]
calls[365][405] cdrqueue[0] heap[0.0%] heapoverruns[0] comp[41.3%] traffic[25.2Mb/s] t0CPU[6.4%] t1CPU[1.6%] t2CPU[1.5%]
calls[346][386] cdrqueue[0] heap[0.1%] heapoverruns[0] comp[41.3%] traffic[25.6Mb/s] t0CPU[6.3%] t1CPU[1.6%] t2CPU[1.5%]

• calls - [X][Y] - X is actual calls in voipmonitor memory. Y is total calls in voipmonitor memory (actual + queue buffer)
• cdrqueue - is number of calls waiting to be written to MySQL. If this number is growing the MySQL is not able to handle it. See Scaling#innodb_flush_log_at_trx_commit
• heapoverruns - if this number grows the heap buffer was completely filled. In this case the primary thread will stop reading packets from ringbuffer and if the ringbuffer is full packets will be lost - this occurrence will be logged to syslog.
• comp - compression buffer ratio (if enabled)
• t0CPU - This is %CPU utilization for thread 0. Thread 0 is process reading from kernel ring buffer. Once it is over 90% it means that the current setup is hitting limit processing packets from network card. Please write to support@voipmonitor.org if you hit this limit.
• t1CPU - This is %CPU utilization for thread 1. Thread 1 is process reading packets from thread 0, adding it to the buffer and compress it (if enabled).
• t2CPU - This is %CPU utilization for thread 2. Thread 2 is process which parses all SIP packets. If >90% there the sensor is hitting limit - please contact support@voipmonitor.org if you see >90%.