Sniffing modes: Difference between revisions

From VoIPmonitor.org
(Review: oprava pre na syntaxhighlight, oprava úrovně nadpisu, konverze markdown tabulky na wiki formát)
(Review: odstranění duplicitního obsahu z jiné stránky, oprava formátování (Warning/Note šablony), jednotná AI Summary sekce)
Line 29: Line 29:
The VoIPmonitor sensor interface will be put into promiscuous mode automatically. To capture from multiple interfaces, set <code>interface = any</code> in <code>voipmonitor.conf</code> and enable promiscuous mode manually on each NIC (e.g., <code>ip link set dev eth1 promisc on</code>).
The VoIPmonitor sensor interface will be put into promiscuous mode automatically. To capture from multiple interfaces, set <code>interface = any</code> in <code>voipmonitor.conf</code> and enable promiscuous mode manually on each NIC (e.g., <code>ip link set dev eth1 promisc on</code>).


{| class="wikitable" style="background:#d4edda; border:1px solid #28a745;"
{{Warning|1='''Critical for Multiple Mirrored Interfaces:''' When sniffing from multiple mirrored interfaces, VLANs, or switch ports, packets may arrive as duplicates (same traffic from multiple SPAN sources). This can cause incomplete calls, missing audio, or incorrect SIP/RTP session reassembly. Add <code>auto_enable_use_blocks = yes</code> to <code>voipmonitor.conf</code>. This enables automatic packet deduplication and defragmentation. See [[Sniffer_configuration#auto_enable_use_blocks|Sniffer_configuration]] for details.}}
|-
! colspan="2" style="background:#28a745; color: white;" | Critical for Multiple Mirrored Interfaces
|-
| style="vertical-align: top;" | '''Issue:'''
| When sniffing from multiple mirrored interfaces, VLANs, or switch ports, packets may arrive as duplicates (same traffic from multiple SPAN sources). This can cause incomplete calls, missing audio, or incorrect SIP/RTP session reassembly.
|-
| style="vertical-align: top;" | '''Solution:'''
| Add <code>auto_enable_use_blocks = yes</code> to <code>voipmonitor.conf</code>. This enables automatic packet deduplication and defragmentation, ensuring packets from multiple sources are correctly identified and merged. See [[Sniffer_configuration#auto_enable_use_blocks|Sniffer_configuration]] for details.
|}


=== B. Software-based Tunnelling ===
=== B. Software-based Tunnelling ===
Line 48: Line 39:
** <code>udp_port_vxlan = 4789</code> (common in cloud environments)
** <code>udp_port_vxlan = 4789</code> (common in cloud environments)
* '''Proprietary & Other Protocols:'''
* '''Proprietary & Other Protocols:'''
** [[audiocodes tunneling|AudioCodes Tunneling]] (uses <code>udp_port_audiocodes</code> or <code>tcp_port_audiocodes</code>)
** [[Audiocodes_tunneling|AudioCodes Tunneling]] (uses <code>udp_port_audiocodes</code> or <code>tcp_port_audiocodes</code>)
** HEP (Homer Encapsulation Protocol)
** HEP (Homer Encapsulation Protocol)
** IPFIX (for Oracle SBCs) (enable <code>ipfix*</code> options)
** IPFIX (for Oracle SBCs) (enable <code>ipfix*</code> options)
Line 54: Line 45:
==== HEP (Homer Encapsulation Protocol) ====
==== HEP (Homer Encapsulation Protocol) ====


HEP is a lightweight protocol for capturing and mirroring VoIP packets. Many SBCs and sip proxies (such as Kamailio, OpenSIPS, FreeSWITCH) support HEP to send a copy of traffic to a monitoring server.
HEP is a lightweight protocol for capturing and mirroring VoIP packets. Many SBCs and SIP proxies (such as Kamailio, OpenSIPS, FreeSWITCH) support HEP to send a copy of traffic to a monitoring server.


'''Configuration in voipmonitor.conf:'''
'''Configuration in voipmonitor.conf:'''
Line 79: Line 70:
* Environments where standard tunnels (GRE/ERSPAN) are not available
* Environments where standard tunnels (GRE/ERSPAN) are not available


'''Note:''' There is also <code>hep_kamailio_protocol_id_fix = yes</code> for Kamailio-specific protocol ID issues.
{{Note|1=There is also <code>hep_kamailio_protocol_id_fix = yes</code> for Kamailio-specific protocol ID issues.}}


'''Known Limitations:'''
'''Known Limitations:'''
Line 94: Line 85:
===== No HEP Correlation ID Support =====
===== No HEP Correlation ID Support =====


**VoIPmonitor does not use HEP correlation ID (captureNodeID) to correlate SIP and RTP packets.**
'''VoIPmonitor does not use HEP correlation ID (captureNodeID) to correlate SIP and RTP packets.'''


When SIP signaling and RTP media are encapsulated in HEP and arrive from different HEP sources (different capture nodes or sensors), VoIPmonitor cannot correlate them into a single CDR using the HEP protocol metadata.
When SIP signaling and RTP media are encapsulated in HEP and arrive from different HEP sources (different capture nodes or sensors), VoIPmonitor cannot correlate them into a single CDR using the HEP protocol metadata.


{| class="wikitable" style="background:#fff3cd; border:1px solid #ffc107;"
{{Warning|1='''HEP Correlation Limitation:'''
|-
* HEP Source A sends SIP packets
! colspan="2" style="background:#ffc107;" | HEP Correlation Limitation
|-
| style="vertical-align: top;" | '''What does not work:'''
| * HEP Source A sends SIP packets
* HEP Source B sends RTP packets for the same call
* HEP Source B sends RTP packets for the same call
* VoIPmonitor tries to use HEP captureNodeID/correlation ID to merge them
* VoIPmonitor tries to use HEP captureNodeID/correlation ID to merge them
* Result: SIP and RTP are NOT correlated; the call appears incomplete or missing
* '''Result:''' SIP and RTP are NOT correlated; the call appears incomplete or missing
|-
 
| style="vertical-align: top;" | '''Why:'''
VoIPmonitor extracts the payload from HEP encapsulation and correlates using standard SIP Call-ID, To/From tags, and RTP SSRC fields. It does not utilize the HEP envelope metadata fields (correlation ID, capture node ID, etc.) for cross-sensor correlation.
| VoIPmonitor extracts the payload from HEP encapsulation and correlates using standard SIP Call-ID, To/From tags, and RTP SSRC fields. It does not utilize the HEP envelope metadata fields (correlation ID, capture node ID, etc.) for cross-sensor correlation.
|-
| style="vertical-align: top;" | '''Workaround (Feature Request VS-1703):'''
| Currently, there is no available workaround. The only options are:
* Wait for a future release that adds HEP correlation ID support (feature request VS-1703 has been logged)
* Pursue a custom paid implementation to add this functionality


This is fundamentally different from VoIPmonitor's standard "single sniffer requirement" for raw packet capture. The HEP protocol limitation is specific to how HEP encapsulation is processed.
'''Workaround (Feature Request VS-1703):''' Currently, there is no available workaround. The only options are to wait for a future release that adds HEP correlation ID support (feature request VS-1703 has been logged) or pursue a custom paid implementation.}}
|}


=== Pre-Deployment Compatibility Verification ===
=== Pre-Deployment Compatibility Verification ===
Line 133: Line 113:
'''Verification Workflow:'''
'''Verification Workflow:'''


1. **Configure Mirroring in Test Mode:** Set up the SPAN, RSPAN, ERSPAN, or tunnel configuration to forward a small subset of VoIP traffic to a test sensor or VM.
# '''Configure Mirroring in Test Mode:''' Set up the SPAN, RSPAN, ERSPAN, or tunnel configuration to forward a small subset of VoIP traffic to a test sensor or VM.
 
# '''Capture Test Calls:'''
2. **Capture Test Calls:**
#* Make a few test calls through your VoIP system.
  * Make a few test calls through your VoIP system.
#* Using <code>tcpdump</code> or <code>tshark</code>, capture the mirrored traffic into a pcap file:
  * Using <code>tcpdump</code> or <code>tshark</code>, capture the mirrored traffic into a pcap file:
#:<syntaxhighlight lang="bash">
    <syntaxhighlight lang="bash">
# Example: Capture SIP and RTP from the mirrored interface
# Example: Capture SIP and RTP from the mirrored interface
sudo tcpdump -i eth0 -s0 port 5060 -w /tmp/compatibility_test.pcap
sudo tcpdump -i eth0 -s0 port 5060 -w /tmp/compatibility_test.pcap
    </syntaxhighlight>
</syntaxhighlight>
 
# '''Verify Packet Capture:'''
3. **Verify Packet Capture:**
#* Confirm the pcap contains both SIP signaling and RTP audio:
  * Confirm the pcap contains both SIP signaling and RTP audio:
#:<syntaxhighlight lang="bash">
    <syntaxhighlight lang="bash">
tshark -r /tmp/compatibility_test.pcap -Y "sip || rtp"
tshark -r /tmp/compatibility_test.pcap -Y "sip || rtp"
    </syntaxhighlight>
</syntaxhighlight>
  * Check for expected packet sizes, codecs, and call flow.
#* Check for expected packet sizes, codecs, and call flow.
 
# '''Submit for Analysis:''' Send the pcap file to VoIPmonitor support along with details about:
4. **Submit for Analysis:** Send the pcap file to VoIPmonitor support along with details about:
#* Your network hardware (Cisco router model, switch model, SBC model)
  * Your network hardware (Cisco router model, switch model, SBC model)
#* Mirroring method (SPAN, RSPAN, ERSPAN, GRE, VXLAN, etc.)
  * Mirroring method (SPAN, RSPAN, ERSPAN, GRE, VXLAN, etc.)
#* Any special configurations (VLAN tags, MPLS labels, encapsulation)
  * Any special configurations (VLAN tags, MPLS labels, encapsulation)
#* Your planned deployment (on-host vs. dedicated sensor, client/server vs. standalone)
  * Your planned deployment (on-host vs. dedicated sensor, client/server vs. standalone)
# '''Feedback and Adjustment:''' Support will analyze the pcap and confirm if VoIPmonitor can process your specific traffic structure. They may recommend configuration changes (e.g., adjusting <code>sipport</code>, enabling tunnel decapsulation, modifying TCP/UDP port settings) or identify incompatible traffic patterns.
 
5. **Feedback and Adjustment:** Support will analyze the pcap and confirm if VoIPmonitor can process your specific traffic structure. They may recommend configuration changes (e.g., adjusting <code>sipport</code>, enabling tunnel decapsulation, modifying TCP/UDP port settings) or identify incompatible traffic patterns.


'''Benefits of Pre-Deployment Testing:'''
'''Benefits of Pre-Deployment Testing:'''
Line 179: Line 155:
'''Configuration Steps:'''
'''Configuration Steps:'''


1. **Create a Dedicated Sensor VM:** Deploy a VoIPmonitor sensor instance in your cloud environment. This VM should be sized appropriately for your expected traffic volume.
# '''Create a Dedicated Sensor VM:''' Deploy a VoIPmonitor sensor instance in your cloud environment. This VM should be sized appropriately for your expected traffic volume.
# '''Configure Cloud Mirroring Policy:''' In your cloud provider's console, create a mirroring policy:
#* Select source VMs or subnets where your VoIP traffic (PBX/SBC) originates.
#* Set the destination to the internal IP of your VoIPmonitor sensor VM.
#* Ensure the encapsulation protocol is compatible with VoIPmonitor (VXLAN is recommended and most common).
# '''Critical: Bidirectional Capture:''' Configure the mirroring policy to capture traffic '''in BOTH directions''':
#* <code>INGRESS</code> (incoming traffic to sources)
#* <code>EGRESS</code> (outgoing traffic from sources)
#* <code>BOTH</code> or <code>EITHER</code> is recommended


2. **Configure Cloud Mirroring Policy:** In your cloud provider's console, create a mirroring policy:
{{Warning|1=Capturing only ingress or only egress will result in incomplete call data and broken CDRs.}}
  * Select source VMs or subnets where your VoIP traffic (PBX/SBC) originates.
  * Set the destination to the internal IP of your VoIPmonitor sensor VM.
  * Ensure the encapsulation protocol is compatible with VoIPmonitor (VXLAN is recommended and most common).


3. **Critical: Bidirectional Capture:** Configure the mirroring policy to capture traffic '''in BOTH directions''':
<ol start="4">
  * <code>INGRESS</code> (incoming traffic to sources)
<li>'''Configure VoIPmonitor Sensor:'''</li>
  * <code>EGRESS</code> (outgoing traffic from sources)
</ol>
  * <code>BOTH</code> or <code>EITHER</code> is recommended


  <code>WARNING: Capturing only ingress or only egress will result in incomplete call data and broken CDRs.</code>
<syntaxhighlight lang="ini">
# Enable VXLAN support for cloud packet mirroring
udp_port_vxlan = 4789


4. **Configure VoIPmonitor Sensor:**
# Interface configuration
interface = eth0


  <syntaxhighlight lang="ini">
# SIP ports
  # Enable VXLAN support for cloud packet mirroring
sipport = 5060
  udp_port_vxlan = 4789


  # Interface configuration
# Optional: Filter at source to save bandwidth
  interface = eth0
# Configure cloud mirroring filters to forward only SIP/RTP traffic
</syntaxhighlight>


  # SIP ports
<ol start="5">
  sipport = 5060
<li>'''VM Sizing for Cloud Sensor:''' Properly size the sensor VM instance:</li>
</ol>
* '''vCPU:''' Allow 1-2 cores per 100 concurrent calls (adjusted for codec complexity and packet rate).
* '''RAM:''' 4GB minimum for production; more if using on-disk compression or high PCAP retention.
* '''Storage:''' Use SSD or high-throughput block storage for the <code>spooldir</code>. VoIPmonitor is I/O intensive — persistent disk performance is critical to avoid packet loss.
* '''Network:''' Ensure sufficient NIC bandwidth; mirroring multiple high-traffic sources can saturate the sensor's interface.


  # Optional: Filter at source to save bandwidth
<ol start="6">
  # Configure cloud mirroring filters to forward only SIP/RTP traffic
<li>'''NTP Synchronization:''' Accurate timekeeping is critical. Ensure all VMs (sources, sensor, and related infrastructure) use the cloud provider's internal NTP servers or a reliable external NTP source.</li>
  </syntaxhighlight>
</ol>
 
5. **VM Sizing for Cloud Sensor:** Properly size the sensor VM instance:
  * <code>vCPU:</code> Allow 1-2 cores per 100 concurrent calls (adjusted for codec complexity and packet rate).
  * <code>RAM:</code> 4GB minimum for production; more if using on-disk compression or high PCAP retention.
  * <code>Storage:</code> Use SSD or high-throughput block storage for the <code>spooldir</code>. VoIPmonitor is I/O intensive — persistent disk performance is critical to avoid packet loss.
  * <code>Network:</code> Ensure sufficient NIC bandwidth; mirroring multiple high-traffic sources can saturate the sensor's interface.
 
6. **NTP Synchronization:** Accurate timekeeping is critical. Ensure all VMs (sources, sensor, and related infrastructure) use the cloud provider's internal NTP servers or a reliable external NTP source.


'''Best Practices for Cloud Mirroring:'''
'''Best Practices for Cloud Mirroring:'''


* <code>Filter at the Source:</code> Use cloud mirroring filters to forward only SIP signaling and RTP audio ports. Sending all network traffic (HTTP, SSH, etc.) wastes CPU and bandwidth.
* '''Filter at the Source:''' Use cloud mirroring filters to forward only SIP signaling and RTP audio ports. Sending all network traffic (HTTP, SSH, etc.) wastes CPU and bandwidth.
* <code>Monitor Network Limits:</code> Cloud NICs have bandwidth limits (e.g., 10 Gbps). Mirroring multiple high-traffic sources may saturate the sensor VM's interface.
* '''Monitor Network Limits:''' Cloud NICs have bandwidth limits (e.g., 10 Gbps). Mirroring multiple high-traffic sources may saturate the sensor VM's interface.
* <code>MTU Considerations:</code> VXLAN adds ~50 bytes of overhead. If original packets are near 1500 bytes MTU, encapsulated packets may exceed it, causing fragmentation or drops. Ensure network path supports jumbo frames or proper fragmentation handling.
* '''MTU Considerations:''' VXLAN adds ~50 bytes of overhead. If original packets are near 1500 bytes MTU, encapsulated packets may exceed it, causing fragmentation or drops. Ensure network path supports jumbo frames or proper fragmentation handling.
* <code>Test Load:</code> Start with filtered ports and a subset of traffic, monitor performance, then expand to full production volume.
* '''Test Load:''' Start with filtered ports and a subset of traffic, monitor performance, then expand to full production volume.


'''Alternative: Client/Server Architecture with On-Host Sensors'''
'''Alternative: Client/Server Architecture with On-Host Sensors'''
Line 252: Line 232:
'''Configuration Steps:'''
'''Configuration Steps:'''


1. **Mount remote spools on GUI server:**
# '''Mount remote spools on GUI server:'''


  Using NFS:
Using NFS:
  <syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
  # On GUI server, mount remote spool directory
# On GUI server, mount remote spool directory
  sudo mount -t nfs 10.224.0.101:/var/spool/voipmonitor /mnt/voipmonitor/sensor1
sudo mount -t nfs 10.224.0.101:/var/spool/voipmonitor /mnt/voipmonitor/sensor1
  sudo mount -t nfs 10.224.0.102:/var/spool/voipmonitor /mnt/voipmonitor/sensor2
sudo mount -t nfs 10.224.0.102:/var/spool/voipmonitor /mnt/voipmonitor/sensor2


  # Add to /etc/fstab for persistent mounts
# Add to /etc/fstab for persistent mounts
  10.224.0.101:/var/spool/voipmonitor  /mnt/voipmonitor/sensor1  nfs  defaults  0  0
10.224.0.101:/var/spool/voipmonitor  /mnt/voipmonitor/sensor1  nfs  defaults  0  0
  10.224.0.102:/var/spool/voipmonitor  /mnt/voipmonitor/sensor2  nfs  defaults  0  0
10.224.0.102:/var/spool/voipmonitor  /mnt/voipmonitor/sensor2  nfs  defaults  0  0
  </syntaxhighlight>
</syntaxhighlight>


  Using SSHFS:
Using SSHFS:
  <syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
  # On GUI server, mount remote spool via SSHFS
# On GUI server, mount remote spool via SSHFS
  sshfs voipmonitor@10.224.0.101:/var/spool/voipmonitor /mnt/voipmonitor/sensor1
sshfs voipmonitor@10.224.0.101:/var/spool/voipmonitor /mnt/voipmonitor/sensor1
  sshfs voipmonitor@10.224.0.102:/var/spool/voipmonitor /mnt/voipmonitor/sensor2
sshfs voipmonitor@10.224.0.102:/var/spool/voipmonitor /mnt/voipmonitor/sensor2


  # Add to /etc/fstab for persistent mounts (with key-based auth)
# Add to /etc/fstab for persistent mounts (with key-based auth)
  voipmonitor@10.224.0.101:/var/spool/voipmonitor  /mnt/voipmonitor/sensor1  fuse.sshfs  defaults,IdentityFile=/home/voipmonitor/.ssh/id_rsa  0  0
voipmonitor@10.224.0.101:/var/spool/voipmonitor  /mnt/voipmonitor/sensor1  fuse.sshfs  defaults,IdentityFile=/home/voipmonitor/.ssh/id_rsa  0  0
  </syntaxhighlight>
</syntaxhighlight>


2. **Configure PCAP spooldir path in GUI:**
<ol start="2">
<li>'''Configure PCAP spooldir path in GUI:'''</li>
</ol>


  In the GUI, go to '''Settings > System Configuration > Sniffer data path''' and set it to search multiple spool directories. Each directory is separated by a colon (<code>:</code>).
In the GUI, go to '''Settings > System Configuration > Sniffer data path''' and set it to search multiple spool directories. Each directory is separated by a colon (<code>:</code>).


  <syntaxhighlight lang="text">
<syntaxhighlight lang="text">
  Sniffer data path: /var/spool/voipmonitor:/mnt/voipmonitor/sensor1:/mnt/voipmonitor/sensor2
Sniffer data path: /var/spool/voipmonitor:/mnt/voipmonitor/sensor1:/mnt/voipmonitor/sensor2
  </syntaxhighlight>
</syntaxhighlight>


  The GUI will search these paths in order when looking for PCAP files.
The GUI will search these paths in order when looking for PCAP files.


3. **Register remote sensors in GUI:**
<ol start="3">
<li>'''Register remote sensors in GUI:'''</li>
</ol>


  Go to '''Settings > Sensors''' and register each remote sensor:
Go to '''Settings > Sensors''' and register each remote sensor:
  * '''Sensor ID:''' Must match <code>id_sensor</code> in each remote's <code>voipmonitor.conf</code>
* '''Sensor ID:''' Must match <code>id_sensor</code> in each remote's <code>voipmonitor.conf</code>
  * '''Name:''' Descriptive name (e.g., "Site 1 - London")
* '''Name:''' Descriptive name (e.g., "Site 1 - London")
  * '''Manager IP, Port:''' Optional with NFS/SSHFS mount (leave empty if mounting spools directly)
* '''Manager IP, Port:''' Optional with NFS/SSHFS mount (leave empty if mounting spools directly)


'''Important Notes:'''
'''Important Notes:'''
Line 340: Line 324:
Missing data (both CDRs and PCAPs) for a specific time period is typically caused by network unavailability between the VoIPmonitor probe and the NFS/SSHFS storage server.
Missing data (both CDRs and PCAPs) for a specific time period is typically caused by network unavailability between the VoIPmonitor probe and the NFS/SSHFS storage server.


* **1. Check system logs for NFS or SSHFS errors:**
'''1. Check system logs for NFS or SSHFS errors:'''


  <syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
  # Check for NFS-specific errors
# Check for NFS-specific errors
  journalctl -u voipmonitor --since "2024-01-01" --until "2024-01-02"
journalctl -u voipmonitor --since "2024-01-01" --until "2024-01-02"


  # Look for specific patterns in syslog
# Look for specific patterns in syslog
  grep "nfs: server.*not responding" /var/log/syslog
grep "nfs: server.*not responding" /var/log/syslog
  grep "nfs.*timed out" /var/log/syslog
grep "nfs.*timed out" /var/log/syslog
  grep "I/O error" /var/log/syslog
grep "I/O error" /var/log/syslog


  # For SSHFS issues
# For SSHFS issues
  grep "sshfs.*Connection reset" /var/log/syslog
grep "sshfs.*Connection reset" /var/log/syslog
  grep "sshfs.*Transport endpoint is not connected" /var/log/syslog
grep "sshfs.*Transport endpoint is not connected" /var/log/syslog
  </syntaxhighlight>
</syntaxhighlight>


  Key error messages to look for:
Key error messages to look for:
  * <code>nfs: server 192.168.1.100 not responding, timed out</code> - NFS server unreachable
* <code>nfs: server 192.168.1.100 not responding, timed out</code> - NFS server unreachable
  * <code>nfs: server 192.168.1.100 OK</code> - Connection restored after interruption
* <code>nfs: server 192.168.1.100 OK</code> - Connection restored after interruption
  * <code>Stale file handle</code> - NFS mount needs remounting
* <code>Stale file handle</code> - NFS mount needs remounting
  * <code>Transport endpoint is not connected</code> - SSHFS mount disconnected
* <code>Transport endpoint is not connected</code> - SSHFS mount disconnected


* **2. Verify network connectivity to the storage server:**
'''2. Verify network connectivity to the storage server:'''


  <syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
  # Ping test to the NFS/SSHFS server
# Ping test to the NFS/SSHFS server
  ping 192.168.1.100
ping 192.168.1.100


  # Trace the network path to identify bottlenecks
# Trace the network path to identify bottlenecks
  traceroute 192.168.1.100
traceroute 192.168.1.100


  # Test DNS resolution if using hostnames
# Test DNS resolution if using hostnames
  nslookup storage-server.domain.com
nslookup storage-server.domain.com
  </syntaxhighlight>
</syntaxhighlight>


* **3. Ensure the NFS/SSHFS server is running and accessible:**
'''3. Ensure the NFS/SSHFS server is running and accessible:'''


  <syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
  # On the probe/sensor side - check if mount is active
# On the probe/sensor side - check if mount is active
  mount | grep nfs
mount | grep nfs
  mount | grep fuse.sshfs
mount | grep fuse.sshfs


  # Check mount status for all mounted spool directories
# Check mount status for all mounted spool directories
  stat /mnt/voipmonitor/sensor1
stat /mnt/voipmonitor/sensor1


  # On the NFS server side - verify services are running
# On the NFS server side - verify services are running
  systemctl status nfs-server
systemctl status nfs-server
  systemctl status sshd
systemctl status sshd
  </syntaxhighlight>
</syntaxhighlight>


* **4. Check for mount-specific issues:**
'''4. Check for mount-specific issues:'''


  <syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
  # Test NFS mount manually (unmount and remount)
# Test NFS mount manually (unmount and remount)
  sudo umount /mnt/voipmonitor/sensor1
sudo umount /mnt/voipmonitor/sensor1
  sudo mount -t nfs 10.224.0.101:/var/spool/voipmonitor /mnt/voipmonitor/sensor1
sudo mount -t nfs 10.224.0.101:/var/spool/voipmonitor /mnt/voipmonitor/sensor1


  # Check /etc/fstab for mount errors
# Check /etc/fstab for mount errors
  sudo mount -a  # Test all mounts in /etc/fstab
sudo mount -a  # Test all mounts in /etc/fstab


  # Verify mount permissions
# Verify mount permissions
  ls -la /mnt/voipmonitor/sensor1
ls -la /mnt/voipmonitor/sensor1
  </syntaxhighlight>
</syntaxhighlight>


==== Common Causes of Missing Data ====
==== Common Causes of Missing Data ====
Line 436: Line 420:
To minimize data loss from NFS/SSHFS connectivity issues:
To minimize data loss from NFS/SSHFS connectivity issues:


* **Use TCP for NFS** (more reliable than UDP):
'''Use TCP for NFS''' (more reliable than UDP):
  <syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
  # Mount NFS with TCP explicitly
# Mount NFS with TCP explicitly
  sudo mount -t nfs -o tcp 10.224.0.101:/var/spool/voipmonitor /mnt/voipmonitor/sensor1
sudo mount -t nfs -o tcp 10.224.0.101:/var/spool/voipmonitor /mnt/voipmonitor/sensor1
  </syntaxhighlight>
</syntaxhighlight>


* **Use the <code>hard,nofail</code> mount options:**
'''Use the <code>hard,nofail</code> mount options:'''
  <syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
  # In /etc/fstab
# In /etc/fstab
  10.224.0.101:/var/spool/voipmonitor  /mnt/voipmonitor/sensor1  nfs  hard,nofail,tcp  0  0
10.224.0.101:/var/spool/voipmonitor  /mnt/voipmonitor/sensor1  nfs  hard,nofail,tcp  0  0
  </syntaxhighlight>
</syntaxhighlight>
  * <code>hard</code>: Make NFS operations wait indefinitely instead of timing out
* <code>hard</code>: Make NFS operations wait indefinitely instead of timing out
  * <code>nofail</code>: Do not fail if the mount is unavailable at boot time
* <code>nofail</code>: Do not fail if the mount is unavailable at boot time


* **Monitor mount status:**
'''Monitor mount status:''' Set up automated monitoring to alert when NFS/SSHFS mounts become unresponsive or disconnected.
  Set up automated monitoring to alert when NFS/SSHFS mounts become unresponsive or disconnected.


* **Consider Client/Server mode as alternative:**
'''Consider Client/Server mode as alternative:''' If NFS/SSHFS connectivity is unreliable, consider using the modern [[Sniffer_distributed_architecture|Client/Server architecture]] instead, which uses encrypted TCP channels and is more resilient to network interruptions.
  If NFS/SSHFS connectivity is unreliable, consider using the modern [[Sniffer_distributed_architecture|Client/Server architecture]] instead, which uses encrypted TCP channels and is more resilient to network interruptions.


=== Modern Mode: Client/Server Architecture (v20+) — Recommended ===
=== Modern Mode: Client/Server Architecture (v20+) — Recommended ===
Line 463: Line 445:
# '''Packet Mirroring:''' Remote sensors forward the entire raw packet stream to the central server, which performs all processing and storage. Ideal for low-resource remote sites.
# '''Packet Mirroring:''' Remote sensors forward the entire raw packet stream to the central server, which performs all processing and storage. Ideal for low-resource remote sites.


==== Architecture Diagrams (PlantUML) ====
==== Architecture Diagrams ====


<kroki lang="plantuml">
<kroki lang="plantuml">
  @startuml
@startuml
  skinparam shadowing false
skinparam shadowing false
  skinparam defaultFontName Arial
skinparam defaultFontName Arial
  skinparam rectangle {
skinparam rectangle {
    BorderColor #4A90E2
  BorderColor #4A90E2
    BackgroundColor #FFFFFF
  BackgroundColor #FFFFFF
    stereotypeFontColor #333333
  stereotypeFontColor #333333
  }
}
  skinparam packageBorderColor #B0BEC5
skinparam packageBorderColor #B0BEC5
  skinparam packageBackgroundColor #F7F9FC
skinparam packageBackgroundColor #F7F9FC


  title Client/Server Architecture — Local Processing Mode
title Client/Server Architecture — Local Processing Mode


  package "Remote Site" {
package "Remote Site" {
    [Remote Probe/Sensor] as Remote
  [Remote Probe/Sensor] as Remote
    database "Local Storage (PCAP)" as RemotePCAP
  database "Local Storage (PCAP)" as RemotePCAP
  }
}


  package "Central Site" {
package "Central Site" {
    [Central VoIPmonitor Server] as Central
  [Central VoIPmonitor Server] as Central
    database "Central MySQL/MariaDB" as CentralDB
  database "Central MySQL/MariaDB" as CentralDB
    [Web GUI] as GUI
  [Web GUI] as GUI
  }
}


  Remote -[#2F6CB0]-> Central : Encrypted TCP/60024\nCDRs only
Remote -[#2F6CB0]-> Central : Encrypted TCP/60024\nCDRs only
  Remote --> RemotePCAP : Stores PCAP locally
Remote --> RemotePCAP : Stores PCAP locally
  Central --> CentralDB : Writes CDRs
Central --> CentralDB : Writes CDRs
  GUI -[#2F6CB0]-> Central : Queries data & requests PCAPs
GUI -[#2F6CB0]-> Central : Queries data & requests PCAPs
  Central -[#2F6CB0]-> RemotePCAP : Fetches PCAPs on demand (TCP/5029)
Central -[#2F6CB0]-> RemotePCAP : Fetches PCAPs on demand (TCP/5029)
  @enduml
@enduml
  </kroki>
</kroki>


<kroki lang="plantuml">
<kroki lang="plantuml">
  @startuml
@startuml
  skinparam shadowing false
skinparam shadowing false
  skinparam defaultFontName Arial
skinparam defaultFontName Arial
  skinparam rectangle {
skinparam rectangle {
    BorderColor #4A90E2
  BorderColor #4A90E2
    BackgroundColor #FFFFFF
  BackgroundColor #FFFFFF
    stereotypeFontColor #333333
  stereotypeFontColor #333333
  }
}
  skinparam packageBorderColor #B0BEC5
skinparam packageBorderColor #B0BEC5
  skinparam packageBackgroundColor #F7F9FC
skinparam packageBackgroundColor #F7F9FC


  title Client/Server Architecture — Packet Mirroring Mode
title Client/Server Architecture — Packet Mirroring Mode


  package "Remote Site" {
package "Remote Site" {
    [Remote Probe/Sensor\n(Low Resource)] as Remote
  [Remote Probe/Sensor\n(Low Resource)] as Remote
  }
}


  package "Central Site" {
package "Central Site" {
    [Central VoIPmonitor Server] as Central
  [Central VoIPmonitor Server] as Central
    database "Central MySQL/MariaDB" as CentralDB
  database "Central MySQL/MariaDB" as CentralDB
    database "Central Storage (PCAP)" as CentralPCAP
  database "Central Storage (PCAP)" as CentralPCAP
    [Web GUI] as GUI
  [Web GUI] as GUI
  }
}


  Remote -[#2F6CB0]-> Central : Encrypted TCP/60024\nRaw packet stream
Remote -[#2F6CB0]-> Central : Encrypted TCP/60024\nRaw packet stream
  Central --> CentralDB : Writes CDRs
Central --> CentralDB : Writes CDRs
  Central --> CentralPCAP : Processes & stores PCAPs
Central --> CentralPCAP : Processes & stores PCAPs
  GUI -[#2F6CB0]-> Central : Queries data & downloads PCAPs
GUI -[#2F6CB0]-> Central : Queries data & downloads PCAPs
  @enduml
@enduml
  </kroki>
</kroki>


==== Step-by-Step Configuration Guide ====
==== Step-by-Step Configuration Guide ====


; Prerequisites
'''Prerequisites'''
* VoIPmonitor v20+ on all sensors.
* VoIPmonitor v20+ on all sensors.
* Central database reachable from the central server instance.
* Central database reachable from the central server instance.
Line 538: Line 520:
* NTP running everywhere (see '''Time Synchronization''' below).
* NTP running everywhere (see '''Time Synchronization''' below).


; Scenario A — Local Processing (default, low WAN usage)
'''Scenario A — Local Processing (default, low WAN usage)'''
 
<syntaxhighlight lang="ini">
<syntaxhighlight lang="ini">
# /etc/voipmonitor.conf on the REMOTE sensor (LOCAL PROCESSING)
# /etc/voipmonitor.conf on the REMOTE sensor (LOCAL PROCESSING)
Line 573: Line 556:
</syntaxhighlight>
</syntaxhighlight>


; Scenario B — Packet Mirroring (centralized processing/storage)
'''Scenario B — Packet Mirroring (centralized processing/storage)'''
 
<syntaxhighlight lang="ini">
<syntaxhighlight lang="ini">
# /etc/voipmonitor.conf on the REMOTE sensor (PACKET MIRRORING)
# /etc/voipmonitor.conf on the REMOTE sensor (PACKET MIRRORING)
Line 664: Line 648:
* '''Why is <code>id_sensor</code> required everywhere?''' The GUI uses it to tag and filter calls by capture source.
* '''Why is <code>id_sensor</code> required everywhere?''' The GUI uses it to tag and filter calls by capture source.
* '''Local Processing still fetches PCAPs from remote — who connects to whom?''' The GUI requests via the central server; the central server then connects to the remote sensor's <code>TCP/5029</code> to retrieve the PCAP.
* '''Local Processing still fetches PCAPs from remote — who connects to whom?''' The GUI requests via the central server; the central server then connects to the remote sensor's <code>TCP/5029</code> to retrieve the PCAP.
== Related Documentation ==
* [[Sniffer_distributed_architecture|Distributed Architecture: Client-Server Mode]] - Detailed client/server configuration
* [[Scaling|Scaling and Performance Tuning Guide]] - For performance optimization
* [[Sniffer_troubleshooting|Sniffer Troubleshooting]] - For systematic diagnostic procedures
* [[Cloud|Cloud Service Configuration]] - For cloud deployment specifics
* [[Systemd_for_voipmonitor_service_management|Systemd Service Management]] - For service management best practices
== AI Summary for RAG ==
== AI Summary for RAG ==
'''Summary:''' This guide covers the deployment topologies for VoIPmonitor. It contrasts running the sensor on the same host as a PBX versus on a dedicated server. For dedicated sensors, it details methods for forwarding traffic, including hardware-based port mirroring (SPAN) and various software-based tunneling protocols (IP-in-IP, GRE, TZSP, VXLAN, HEP, AudioCodes, IPFIX). HEP (Homer Encapsulation Protocol) is a lightweight protocol for capturing and mirroring VoIP packets. When `hep = yes`, VoIPmonitor listens for HEPv3 (and compatible HEPv2) packets and extracts the original VoIP traffic from the encapsulation. CRITICAL HEP LIMITATION: VoIPmonitor does NOT use HEP correlation ID (captureNodeID) to correlate SIP and RTP packets. When SIP signaling and RTP media are encapsulated in HEP and arrive from different HEP sources (different capture nodes or sensors), VoIPmonitor cannot correlate them into a single CDR using HEP protocol metadata. This is feature request VS-1703 and there is currently no available workaround. The article covers cloud service packet mirroring options (GCP Packet Mirroring, AWS Traffic Mirroring, Azure Virtual Network TAP) with critical requirements: bidirectional capture (ingress and egress) and proper VM sizing (vCPU, RAM, storage I/O). The core of the article explains distributed architectures for multi-site monitoring, comparing the "classic" standalone remote sensor model with the modern, recommended "client/server" model. It details the two operational modes of the client/server architecture: local processing (sending only CDRs, PCAPs remain remote with central-proxied fetch) and packet mirroring (sending full, raw packets for central processing), which is ideal for low-resource endpoints. The article also explains an alternative approach for classic remote sensors: mounting PCAP spools via NFS or SSHFS when TCP/5029 access to sensors is blocked by firewalls, including troubleshooting steps for missing data due to NFS/SSHFS connectivity issues (checking logs for "not responding, timed out" errors, verifying network connectivity with ping/traceroute, and ensuring NFS/SSHFS server is running and accessible). The guide concludes with step-by-step configuration, firewall rules, critical parameter notes, and the importance of NTP plus first-start DB initialization.
 
'''Summary:''' This guide covers the deployment topologies for VoIPmonitor. It contrasts running the sensor on the same host as a PBX versus on a dedicated server. For dedicated sensors, it details methods for forwarding traffic, including hardware-based port mirroring (SPAN) and various software-based tunneling protocols (IP-in-IP, GRE, TZSP, VXLAN, HEP, AudioCodes, IPFIX). HEP (Homer Encapsulation Protocol) is a lightweight protocol for capturing and mirroring VoIP packets. When <code>hep = yes</code>, VoIPmonitor listens for HEPv3 (and compatible HEPv2) packets and extracts the original VoIP traffic from the encapsulation. CRITICAL HEP LIMITATION: VoIPmonitor does NOT use HEP correlation ID (captureNodeID) to correlate SIP and RTP packets. When SIP signaling and RTP media are encapsulated in HEP and arrive from different HEP sources (different capture nodes or sensors), VoIPmonitor cannot correlate them into a single CDR using HEP protocol metadata. This is feature request VS-1703 and there is currently no available workaround. The article covers cloud service packet mirroring options (GCP Packet Mirroring, AWS Traffic Mirroring, Azure Virtual Network TAP) with critical requirements: bidirectional capture (ingress and egress) and proper VM sizing (vCPU, RAM, storage I/O). The core of the article explains distributed architectures for multi-site monitoring, comparing the "classic" standalone remote sensor model with the modern, recommended "client/server" model. It details the two operational modes of the client/server architecture: local processing (sending only CDRs, PCAPs remain remote with central-proxied fetch) and packet mirroring (sending full, raw packets for central processing), which is ideal for low-resource endpoints. The article also explains an alternative approach for classic remote sensors: mounting PCAP spools via NFS or SSHFS when TCP/5029 access to sensors is blocked by firewalls, including troubleshooting steps for missing data due to NFS/SSHFS connectivity issues (checking logs for "not responding, timed out" errors, verifying network connectivity with ping/traceroute, and ensuring NFS/SSHFS server is running and accessible). The guide concludes with step-by-step configuration, firewall rules, critical parameter notes, and the importance of NTP plus first-start DB initialization.


'''Keywords:''' deployment, architecture, topology, on-host, dedicated sensor, port mirroring, SPAN, RSPAN, traffic mirroring, tunneling, GRE, TZSP, VXLAN, HEP, HEP correlation ID, captureNodeID, HEP limitation, HEP SIP RTP correlation, AudioCodes, IPFIX, cloud mirroring, GCP, AWS, Azure, Packet Mirroring, Traffic Mirroring, Virtual Network TAP, ingress, egress, bidirectional, VM sizing, remote sensor, multi-site, client server mode, packet mirroring, local processing, firewall rules, NTP, time synchronization, cloud mode, NFS, SSHFS, spooldir mounting, NFS troubleshooting, SSHFS troubleshooting, missing data, network connectivity
'''Keywords:''' deployment, architecture, topology, on-host, dedicated sensor, port mirroring, SPAN, RSPAN, traffic mirroring, tunneling, GRE, TZSP, VXLAN, HEP, HEP correlation ID, captureNodeID, HEP limitation, HEP SIP RTP correlation, AudioCodes, IPFIX, cloud mirroring, GCP, AWS, Azure, Packet Mirroring, Traffic Mirroring, Virtual Network TAP, ingress, egress, bidirectional, VM sizing, remote sensor, multi-site, client server mode, packet mirroring, local processing, firewall rules, NTP, time synchronization, cloud mode, NFS, SSHFS, spooldir mounting, NFS troubleshooting, SSHFS troubleshooting, missing data, network connectivity
Line 687: Line 681:
* Can VoIPmonitor correlate SIP and RTP packets that arrive from different HEP sources?
* Can VoIPmonitor correlate SIP and RTP packets that arrive from different HEP sources?
* Is there a workaround for HEP SIP/RTP correlation across multiple HEP capture nodes?
* Is there a workaround for HEP SIP/RTP correlation across multiple HEP capture nodes?
* How do I configure GRE, ERSPAN, and VXLAN tunneling for VoIPmonitor?mory issues:
* How do I configure GRE, ERSPAN, and VXLAN tunneling for VoIPmonitor?
 
{| class="wikitable" style="background:#fff3cd; border:1px solid #ffc107;"
|-
! colspan="2" style="background:#ffc107;" | max_buffer_mem Guidance by Bottleneck Type
|-
| style="vertical-align: top;" | '''RAM-Based Memory Issue (Step 3)'''
| Local server cannot process traffic fast enough.<br>'''Solution:''' INCREASE <code>max_buffer_mem</code> (e.g., 4000-8000 MB) to give more headroom
|-
| style="vertical-align: top;" | '''Network Bottleneck (distributed mode)'''
| Probe cannot send packets to central server fast enough.<br>'''Solution:''' DECREASE <code>max_buffer_mem</code> (e.g., from 8000 to 2000 MB) so the buffer fails faster without exhausting RAM, enabling quicker recovery from network congestion.
|}
 
Edit <code>/etc/voipmonitor.conf</code> on the probe:
<syntaxhighlight lang="ini">
[general]
# REDUCE max_buffer_mem for network throughput bottlenecks
# This causes the buffer to fail sooner (releasing memory) instead of
# continuing to occupy RAM waiting for a network connection that cannot keep up
max_buffer_mem = 2000
 
# Enable compression to reduce network traffic
packetbuffer_compress = yes
</syntaxhighlight>
 
Restart the probe service:
<syntaxhighlight lang="bash">
systemctl restart voipmonitor
</syntaxhighlight>
 
For detailed documentation on distributed architecture configuration, see [[Sniffer_distributed_architecture|Distributed Architecture: Client-Server Mode]].
 
=== Related Issues ===
 
For performance tuning and scaling guidance, see:
* [[Scaling|Scaling and Performance Tuning Guide]]
* [[IO_Measurement|I/O Performance Measurement]]
* [[High-Performance_VoIPmonitor_and_MySQL_Setup_Manual|High-Performance Setup]]
 
== Related Documentation ==
 
* [[Scaling|Scaling and Performance Tuning Guide]] - For performance optimization
* [[Anti-fraud|Anti-Fraud Rules]] - For attack detection and mitigation
* [[Sniffer_troubleshooting|Sniffer Troubleshooting]] - For systematic diagnostic procedures
* [[High-Performance_VoIPmonitor_and_MySQL_Setup_Manual|High-Performance Setup]] - For optimizing high-traffic deployments
* [[Systemd_for_voipmonitor_service_management|Systemd Service Management]] - For service management best practices
 
== AI Summary for RAG ==
 
'''Summary:''' Emergency procedures for recovering VoIPmonitor from critical failures. Topics covered: (1) "Too high load" error - intentional protection feature, not a crash; assess CPU with htop per-core, compare Load Average to core count (LA < cores = acceptable). (2) Excessive CPU / runaway process - use kill -9 PID to terminate, root causes include SIP REGISTER floods, pcapcommand directive, RTP thread overload, audio features. (3) OOM memory exhaustion - check dmesg -T for killed processes, reduce innodb_buffer_pool_size and max_buffer_mem. (4) Service restart loop with "MEMORY IS FULL" and "Cannot bind to port 5029" - has 5 root causes: (a) Kernel storage errors (Step 0: dmesg -T), (b) Disk I/O bottleneck (Step 1: iostat -x 1, dd write test), (c) RAM exhaustion (Step 3: increase max_buffer_mem, packetbuffer_compress), (d) Adaptive jitterbuffer overload (Step 4: disable jitterbuffer_adapt=no), (e) Network throughput bottleneck in distributed mode (Step 6: switch packetbuffer_sender=no for Local Processing). Port 5029 binding error (Step 5): clear zombie processes with killall -9 voipmonitor. (5) Packet loss on sensors due to CPU overload - check journalctl for Load Average/heap, use sniffer_stat API for packets_dropped, add CPU cores or use temporary mitigation (callslimit, ringbuffer, silencedetect=no). (6) System hangs - collect core dump with gdb/gcore before restart for developer analysis.
 
'''Keywords:''' emergency recovery, high CPU, system unresponsive, runaway process, kill -9, too high load, Live Sniffer, htop, load average, CPU cores, SIP REGISTER flood, pcapcommand, OOM, out of memory, dmesg, MEMORY IS FULL, HEAP FULL, packetbuffer, disk I/O, iostat, %util, dd oflag=direct, max_buffer_mem, innodb_buffer_pool_size, Cannot bind to port 5029, zombie process, restart loop, jitterbuffer_adapt, MOS_adaptive, NFS, nfsiostat, packetbuffer_sender, Local Processing, Packet Mirroring, distributed mode, network bottleneck, packets_dropped, sniffer_stat, ringbuffer, callslimit, core dump, gdb, gcore, iDRAC, iLO, IPMI
 
'''Key Questions:'''
* What does "too high load" error mean in Live Sniffer?
* How to interpret load average against CPU cores?
* What to do when VoIPmonitor consumes excessive CPU or becomes unresponsive?
* How to forcefully terminate a runaway VoIPmonitor process?
* What causes SIP REGISTER flood CPU spikes and how to mitigate?
* How to diagnose OOM events with dmesg?
* What causes "packetbuffer: MEMORY IS FULL" error?
* How to distinguish between RAM exhaustion, disk I/O bottleneck, and network bottleneck?
* What is the first diagnostic step for MEMORY IS FULL errors (dmesg -T)?
* How to use iostat to diagnose disk I/O bottleneck?
* How to perform write speed test to spool directory?
* What does "Cannot bind to port 5029" error mean and how to fix?
* How is jitterbuffer_adapt related to MEMORY IS FULL + HEAP FULL errors?
* What are the trade-offs when disabling jitterbuffer_adapt?
* How to fix MEMORY IS FULL in distributed mode (packetbuffer_sender)?
* How to switch from Packet Mirroring to Local Processing mode?
* How to check packet loss on sensors using sniffer_stat API?
* What temporary configurations reduce CPU load (callslimit, ringbuffer, silencedetect)?
* How to collect core dump for developer analysis during system hang?
* How to recover when binary is not found after crash?

Revision as of 18:03, 6 January 2026


This guide provides a comprehensive overview of VoIPmonitor's deployment models. It covers the fundamental choice between on-host and dedicated sensors, methods for capturing traffic, and detailed configurations for scalable, multi-site architectures.

Core Concept: Where to Capture Traffic

The first decision in any deployment is where the VoIPmonitor sensor (sniffer) will run.

1. On-Host Capture (on the PBX/SBC)

The sensor can be installed directly on the same Linux server that runs your PBX or SBC.

  • Pros: Requires no extra hardware, network changes, or port mirroring. It is the simplest setup.
  • Cons: Adds CPU, memory, and disk I/O load to your production voice server. If these resources are critical, a dedicated sensor is the recommended approach.

Platform Note: The VoIPmonitor sensor runs exclusively on Linux. While some PBXs are available on Windows (e.g., 3CX Windows edition, certain legacy systems), the sensor cannot be installed on Windows. For Windows-based PBXs, you must use a dedicated Linux sensor with traffic mirroring (see below).

2. Dedicated Sensor

A dedicated Linux server runs only the VoIPmonitor sensor. This is the recommended approach for production environments as it isolates monitoring resources from your voice platform. To use a dedicated sensor, you must forward a copy of the network traffic to it using one of the methods below.

When a Dedicated Sensor is Required:

  • Windows-based PBXs (e.g., 3CX Windows edition) - VoIPmonitor sensor is Linux-only
  • When your PBX/SBC server has limited CPU, RAM, or disk I/O resources
  • When you want zero monitoring impact on your production voice platform
  • When capturing from multiple sites with a centralized collector

Methods for Forwarding Traffic to a Dedicated Sensor

A. Hardware Port Mirroring (SPAN/RSPAN)

This is the most common and reliable method. You configure your physical network switch to copy all traffic from the switch ports connected to your PBX/SBC to the switch port connected to the VoIPmonitor sensor. This feature is commonly called Port Mirroring, SPAN, or RSPAN. Consult your switch's documentation for configuration details.

The VoIPmonitor sensor interface will be put into promiscuous mode automatically. To capture from multiple interfaces, set interface = any in voipmonitor.conf and enable promiscuous mode manually on each NIC (e.g., ip link set dev eth1 promisc on).

⚠️ Warning: Critical for Multiple Mirrored Interfaces: When sniffing from multiple mirrored interfaces, VLANs, or switch ports, packets may arrive as duplicates (same traffic from multiple SPAN sources). This can cause incomplete calls, missing audio, or incorrect SIP/RTP session reassembly. Add auto_enable_use_blocks = yes to voipmonitor.conf. This enables automatic packet deduplication and defragmentation. See Sniffer_configuration for details.

B. Software-based Tunnelling

When hardware mirroring is not an option, many network devices and PBXs can encapsulate VoIP packets and send them to the sensor's IP address using a tunnel. VoIPmonitor natively supports a wide range of protocols.

  • Built-in Support: IP-in-IP, GRE, ERSPAN
  • UDP-based Tunnels: Configure the corresponding port in voipmonitor.conf:
    • udp_port_tzsp = 37008 (for MikroTik's TZSP)
    • udp_port_l2tp = 1701
    • udp_port_vxlan = 4789 (common in cloud environments)
  • Proprietary & Other Protocols:
    • AudioCodes Tunneling (uses udp_port_audiocodes or tcp_port_audiocodes)
    • HEP (Homer Encapsulation Protocol)
    • IPFIX (for Oracle SBCs) (enable ipfix* options)

HEP (Homer Encapsulation Protocol)

HEP is a lightweight protocol for capturing and mirroring VoIP packets. Many SBCs and SIP proxies (such as Kamailio, OpenSIPS, FreeSWITCH) support HEP to send a copy of traffic to a monitoring server.

Configuration in voipmonitor.conf:

# Enable HEP support
hep = yes

# Port to listen for HEP packets (default: 9060)
hep_bind_port = 9060

# Optional: Bind to specific IP address
# hep_bind_ip = 0.0.0.0

# Optional: Enable UDP binding (default: yes)
hep_bind_udp = yes

When hep = yes, VoIPmonitor listens for HEPv3 (and compatible HEPv2) packets and extracts the original VoIP traffic from the encapsulation.

Use Cases:

  • Remote SBCs or PBXs export traffic to a centralized VoIPmonitor server
  • Kamailio/FreeSWITCH siptrace module integration
  • Environments where standard tunnels (GRE/ERSPAN) are not available

ℹ️ Note: There is also hep_kamailio_protocol_id_fix = yes for Kamailio-specific protocol ID issues.

Known Limitations:

HEP Timestamp Precision

HEP3 packets include a timestamp field that represents when the packet was captured at the source. VoIPmonitor uses this HEP timestamp for the call record. If the source HEP server has an unreliable or unsynchronized time source, this can cause incorrect timestamps in the captured calls.

Currently, there is no built-in configuration option to ignore the HEP timestamp and instead use the time when VoIPmonitor receives the packet. If you need this functionality, please:

  • Request the feature on the product roadmap (no guaranteed ETA)
  • Consider a custom development project for a fee
No HEP Correlation ID Support

VoIPmonitor does not use HEP correlation ID (captureNodeID) to correlate SIP and RTP packets.

When SIP signaling and RTP media are encapsulated in HEP and arrive from different HEP sources (different capture nodes or sensors), VoIPmonitor cannot correlate them into a single CDR using the HEP protocol metadata.

⚠️ Warning: HEP Correlation Limitation:

  • HEP Source A sends SIP packets
  • HEP Source B sends RTP packets for the same call
  • VoIPmonitor tries to use HEP captureNodeID/correlation ID to merge them
  • Result: SIP and RTP are NOT correlated; the call appears incomplete or missing

VoIPmonitor extracts the payload from HEP encapsulation and correlates using standard SIP Call-ID, To/From tags, and RTP SSRC fields. It does not utilize the HEP envelope metadata fields (correlation ID, capture node ID, etc.) for cross-sensor correlation.

Workaround (Feature Request VS-1703): Currently, there is no available workaround. The only options are to wait for a future release that adds HEP correlation ID support (feature request VS-1703 has been logged) or pursue a custom paid implementation.

Pre-Deployment Compatibility Verification

Before full production deployment, especially when integrating VoIPmonitor with network hardware (Cisco/Juniper routers, SBCs), or complex mirroring setups (RSPAN, ERSPAN, tunnels), it is highly recommended to verify that VoIPmonitor can correctly capture and process packets from your specific environment.

This approach allows you to identify compatibility issues early, without committing to a full deployment that may need adjustments.

Typical Use Cases:

  • Deploying a dedicated sensor with SPAN/RSPAN from a Cisco router or switch
  • Using ERSPAN to forward VoIP traffic across routers
  • Capturing from proprietary SBCs or VoIP gateways (Cisco C2951, AudioCodes, etc.)
  • Implementing newer or complex tunneling protocols (VXLAN, GRE with specific configurations)

Verification Workflow:

  1. Configure Mirroring in Test Mode: Set up the SPAN, RSPAN, ERSPAN, or tunnel configuration to forward a small subset of VoIP traffic to a test sensor or VM.
  2. Capture Test Calls:
    • Make a few test calls through your VoIP system.
    • Using tcpdump or tshark, capture the mirrored traffic into a pcap file:
    # Example: Capture SIP and RTP from the mirrored interface
    sudo tcpdump -i eth0 -s0 port 5060 -w /tmp/compatibility_test.pcap
    
  3. Verify Packet Capture:
    • Confirm the pcap contains both SIP signaling and RTP audio:
    tshark -r /tmp/compatibility_test.pcap -Y "sip || rtp"
    
    • Check for expected packet sizes, codecs, and call flow.
  4. Submit for Analysis: Send the pcap file to VoIPmonitor support along with details about:
    • Your network hardware (Cisco router model, switch model, SBC model)
    • Mirroring method (SPAN, RSPAN, ERSPAN, GRE, VXLAN, etc.)
    • Any special configurations (VLAN tags, MPLS labels, encapsulation)
    • Your planned deployment (on-host vs. dedicated sensor, client/server vs. standalone)
  5. Feedback and Adjustment: Support will analyze the pcap and confirm if VoIPmonitor can process your specific traffic structure. They may recommend configuration changes (e.g., adjusting sipport, enabling tunnel decapsulation, modifying TCP/UDP port settings) or identify incompatible traffic patterns.

Benefits of Pre-Deployment Testing:

  • Confirms VoIPmonitor compatibility with your specific hardware and network setup
  • Identifies configuration needs before full production deployment
  • Saves time by avoiding trial-and-error during go-live
  • Provides documented proof of concept for stakeholders
  • Allows tuning of sensor resources (CPU/RAM/disk) based on actual traffic characteristics

If verification fails or reveals incompatibilities, support can often suggest alternative approaches or configuration adjustments before you proceed.

Cloud Packet Mirroring (GCP, AWS, Azure)

Cloud providers offer native packet mirroring services that can forward traffic to a dedicated VoIPmonitor sensor. These services typically use VXLAN or GRE encapsulation.

Supported Cloud Services:

  • Google Cloud Platform (GCP): Packet Mirroring
  • Amazon Web Services (AWS): Traffic Mirroring
  • Microsoft Azure: Virtual Network TAP

Configuration Steps:

  1. Create a Dedicated Sensor VM: Deploy a VoIPmonitor sensor instance in your cloud environment. This VM should be sized appropriately for your expected traffic volume.
  2. Configure Cloud Mirroring Policy: In your cloud provider's console, create a mirroring policy:
    • Select source VMs or subnets where your VoIP traffic (PBX/SBC) originates.
    • Set the destination to the internal IP of your VoIPmonitor sensor VM.
    • Ensure the encapsulation protocol is compatible with VoIPmonitor (VXLAN is recommended and most common).
  3. Critical: Bidirectional Capture: Configure the mirroring policy to capture traffic in BOTH directions:
    • INGRESS (incoming traffic to sources)
    • EGRESS (outgoing traffic from sources)
    • BOTH or EITHER is recommended

⚠️ Warning: Capturing only ingress or only egress will result in incomplete call data and broken CDRs.

  1. Configure VoIPmonitor Sensor:
# Enable VXLAN support for cloud packet mirroring
udp_port_vxlan = 4789

# Interface configuration
interface = eth0

# SIP ports
sipport = 5060

# Optional: Filter at source to save bandwidth
# Configure cloud mirroring filters to forward only SIP/RTP traffic
  1. VM Sizing for Cloud Sensor: Properly size the sensor VM instance:
  • vCPU: Allow 1-2 cores per 100 concurrent calls (adjusted for codec complexity and packet rate).
  • RAM: 4GB minimum for production; more if using on-disk compression or high PCAP retention.
  • Storage: Use SSD or high-throughput block storage for the spooldir. VoIPmonitor is I/O intensive — persistent disk performance is critical to avoid packet loss.
  • Network: Ensure sufficient NIC bandwidth; mirroring multiple high-traffic sources can saturate the sensor's interface.
  1. NTP Synchronization: Accurate timekeeping is critical. Ensure all VMs (sources, sensor, and related infrastructure) use the cloud provider's internal NTP servers or a reliable external NTP source.

Best Practices for Cloud Mirroring:

  • Filter at the Source: Use cloud mirroring filters to forward only SIP signaling and RTP audio ports. Sending all network traffic (HTTP, SSH, etc.) wastes CPU and bandwidth.
  • Monitor Network Limits: Cloud NICs have bandwidth limits (e.g., 10 Gbps). Mirroring multiple high-traffic sources may saturate the sensor VM's interface.
  • MTU Considerations: VXLAN adds ~50 bytes of overhead. If original packets are near 1500 bytes MTU, encapsulated packets may exceed it, causing fragmentation or drops. Ensure network path supports jumbo frames or proper fragmentation handling.
  • Test Load: Start with filtered ports and a subset of traffic, monitor performance, then expand to full production volume.

Alternative: Client/Server Architecture with On-Host Sensors

Instead of cloud packet mirroring, consider installing VoIPmonitor sensors directly on each PBX/SBC VM using the Client/Server architecture:

  • Install sensor on each Asterisk/SBC VM (on-host capture)
  • Sensors process calls locally or forward packets via packetbuffer_sender to a central collector
  • Eliminates mirroring overhead and potential incomplete capture issues
  • May have better performance for high-traffic environments

Distributed Deployment Models

For monitoring multiple remote offices or a large infrastructure, a distributed model is essential. This involves a central GUI/Database server collecting data from multiple remote sensors.

Classic Mode: Standalone Remote Sensors

In this traditional model, each remote sensor is a fully independent entity.

  • How it works: The remote sensor processes packets and stores PCAPs locally. It connects directly to the central MySQL/MariaDB database to write CDRs. For PCAP retrieval, the GUI typically needs network access to each sensor's management port (default TCP/5029).
  • Pros: Simple conceptual model.
  • Cons: Requires opening firewall ports to each sensor and managing database credentials on every remote machine.

Alternative PCAP Access: NFS/SSHFS Mounting

For environments where direct TCP/5029 access to remote sensors is impractical (e.g., firewalls, VPN limitations), you can mount remote spool directories on the central GUI server using NFS or SSHFS.

Use Cases:

  • Firewall policies block TCP/5029 but allow SSH or NFS traffic
  • Remote sensors have local databases that need to be queried separately
  • You want the GUI to access PCAPs directly from mounted filesystems instead of proxying through TCP/5029

Configuration Steps:

  1. Mount remote spools on GUI server:

Using NFS:

# On GUI server, mount remote spool directory
sudo mount -t nfs 10.224.0.101:/var/spool/voipmonitor /mnt/voipmonitor/sensor1
sudo mount -t nfs 10.224.0.102:/var/spool/voipmonitor /mnt/voipmonitor/sensor2

# Add to /etc/fstab for persistent mounts
10.224.0.101:/var/spool/voipmonitor  /mnt/voipmonitor/sensor1  nfs  defaults  0  0
10.224.0.102:/var/spool/voipmonitor  /mnt/voipmonitor/sensor2  nfs  defaults  0  0

Using SSHFS:

# On GUI server, mount remote spool via SSHFS
sshfs voipmonitor@10.224.0.101:/var/spool/voipmonitor /mnt/voipmonitor/sensor1
sshfs voipmonitor@10.224.0.102:/var/spool/voipmonitor /mnt/voipmonitor/sensor2

# Add to /etc/fstab for persistent mounts (with key-based auth)
voipmonitor@10.224.0.101:/var/spool/voipmonitor  /mnt/voipmonitor/sensor1  fuse.sshfs  defaults,IdentityFile=/home/voipmonitor/.ssh/id_rsa  0  0
  1. Configure PCAP spooldir path in GUI:

In the GUI, go to Settings > System Configuration > Sniffer data path and set it to search multiple spool directories. Each directory is separated by a colon (:).

Sniffer data path: /var/spool/voipmonitor:/mnt/voipmonitor/sensor1:/mnt/voipmonitor/sensor2

The GUI will search these paths in order when looking for PCAP files.

  1. Register remote sensors in GUI:

Go to Settings > Sensors and register each remote sensor:

  • Sensor ID: Must match id_sensor in each remote's voipmonitor.conf
  • Name: Descriptive name (e.g., "Site 1 - London")
  • Manager IP, Port: Optional with NFS/SSHFS mount (leave empty if mounting spools directly)

Important Notes:

  • Each remote sensor must have a unique id_sensor configured in voipmonitor.conf
  • Remote sensors write directly to their local MySQL database (or possibly to a central database)
  • Filter calls by site using the id_sensor column in the CDR view
  • Ensure mounted directories are writable by the GUI user for PCAP uploads
  • For better performance, use NFS with async or SSHFS with caching options

Filtering and Site Identification:

  • In the CDR view, use the Sensor dropdown filter to select specific sites
  • Alternatively, filter by IP address ranges using CDR columns
  • The id_sensor column in the database uniquely identifies which sensor captured each call
  • Sensor names can be customized in Settings > Sensors for easier identification

Comparison: TCP/5029 vs NFS/SSHFS

Approach Network Traffic Firewall Requirements Performance Use Case
TCP/5029 Proxy (Standard) On-demand fetch per request TCP/5029 outbound from GUI to sensors Better (no continuous mount overhead) Most deployments
NFS Mount Continuous (filesystem access) NFS ports (usually 2049) bidirectional Excellent (local filesystem speed) Local networks, high-throughput
SSHFS Mount Continuous (encrypted filesystem) SSH (TCP/22) outbound from GUI Good (some encryption overhead) Remote sites, cloud/VPN

Troubleshooting NFS/SSHFS Mounts

If you experience missing CDRs or PCAP files for a specific time period, or if the GUI reports files not found despite sensors receiving traffic, the issue is often NFS/SSHFS connectivity between the probe and storage server.

Check for NFS/SSHFS Connectivity Issues

Missing data (both CDRs and PCAPs) for a specific time period is typically caused by network unavailability between the VoIPmonitor probe and the NFS/SSHFS storage server.

1. Check system logs for NFS or SSHFS errors:

# Check for NFS-specific errors
journalctl -u voipmonitor --since "2024-01-01" --until "2024-01-02"

# Look for specific patterns in syslog
grep "nfs: server.*not responding" /var/log/syslog
grep "nfs.*timed out" /var/log/syslog
grep "I/O error" /var/log/syslog

# For SSHFS issues
grep "sshfs.*Connection reset" /var/log/syslog
grep "sshfs.*Transport endpoint is not connected" /var/log/syslog

Key error messages to look for:

  • nfs: server 192.168.1.100 not responding, timed out - NFS server unreachable
  • nfs: server 192.168.1.100 OK - Connection restored after interruption
  • Stale file handle - NFS mount needs remounting
  • Transport endpoint is not connected - SSHFS mount disconnected

2. Verify network connectivity to the storage server:

# Ping test to the NFS/SSHFS server
ping 192.168.1.100

# Trace the network path to identify bottlenecks
traceroute 192.168.1.100

# Test DNS resolution if using hostnames
nslookup storage-server.domain.com

3. Ensure the NFS/SSHFS server is running and accessible:

# On the probe/sensor side - check if mount is active
mount | grep nfs
mount | grep fuse.sshfs

# Check mount status for all mounted spool directories
stat /mnt/voipmonitor/sensor1

# On the NFS server side - verify services are running
systemctl status nfs-server
systemctl status sshd

4. Check for mount-specific issues:

# Test NFS mount manually (unmount and remount)
sudo umount /mnt/voipmonitor/sensor1
sudo mount -t nfs 10.224.0.101:/var/spool/voipmonitor /mnt/voipmonitor/sensor1

# Check /etc/fstab for mount errors
sudo mount -a  # Test all mounts in /etc/fstab

# Verify mount permissions
ls -la /mnt/voipmonitor/sensor1

Common Causes of Missing Data

Symptom Most Likely Cause Troubleshooting Step
Gap in data during a specific time period NFS/SSHFS server unreachable Check logs for "not responding, timed out"
Stale file handle errors NFS server rebooted or export changed Remount NFS share
Connection resets Network interruption or unstable connection Check network stability and ping times
Very slow file access Network latency or bandwidth saturation Monitor network throughput
GUI shows "File not found" Mount point dismounted Check mount status and remount if needed

Preventative Measures

To minimize data loss from NFS/SSHFS connectivity issues:

Use TCP for NFS (more reliable than UDP):

# Mount NFS with TCP explicitly
sudo mount -t nfs -o tcp 10.224.0.101:/var/spool/voipmonitor /mnt/voipmonitor/sensor1

Use the hard,nofail mount options:

# In /etc/fstab
10.224.0.101:/var/spool/voipmonitor  /mnt/voipmonitor/sensor1  nfs  hard,nofail,tcp  0  0
  • hard: Make NFS operations wait indefinitely instead of timing out
  • nofail: Do not fail if the mount is unavailable at boot time

Monitor mount status: Set up automated monitoring to alert when NFS/SSHFS mounts become unresponsive or disconnected.

Consider Client/Server mode as alternative: If NFS/SSHFS connectivity is unreliable, consider using the modern Client/Server architecture instead, which uses encrypted TCP channels and is more resilient to network interruptions.

Modern Mode: Client/Server Architecture (v20+) — Recommended

This model uses a secure, encrypted TCP channel between remote sensors (clients) and a central sensor instance (server). The GUI communicates with the central server only, which significantly simplifies networking and security.

This architecture supports two primary modes:

  1. Local Processing: Remote sensors process packets locally and send only lightweight CDR data over the encrypted channel. PCAPs remain on the remote sensor. On-demand PCAP fetch is proxied via the central server (to the sensor's TCP/5029).
  2. Packet Mirroring: Remote sensors forward the entire raw packet stream to the central server, which performs all processing and storage. Ideal for low-resource remote sites.

Architecture Diagrams

Step-by-Step Configuration Guide

Prerequisites

  • VoIPmonitor v20+ on all sensors.
  • Central database reachable from the central server instance.
  • Unique id_sensor per sensor (< 65536).
  • NTP running everywhere (see Time Synchronization below).

Scenario A — Local Processing (default, low WAN usage)

# /etc/voipmonitor.conf on the REMOTE sensor (LOCAL PROCESSING)

id_sensor               = 2          # unique per sensor (< 65536)
server_destination      = 10.224.0.250
server_destination_port = 60024
server_password         = your_strong_password

packetbuffer_sender     = no         # local analysis; sends only CDRs
interface               = eth0       # or: interface = any
sipport                 = 5060       # example; add your usual sniffer options

# No MySQL credentials here — remote sensor does NOT write to DB directly.
# /etc/voipmonitor.conf on the CENTRAL server (LOCAL PROCESSING network)

server_bind             = 0.0.0.0
server_bind_port        = 60024
server_password         = your_strong_password

mysqlhost               = 10.224.0.201
mysqldb                 = voipmonitor
mysqluser               = voipmonitor
mysqlpassword           = db_password

cdr_partition           = yes        # partitions for CDR tables
mysqlloadconfig         = yes        # allows DB-driven config if used

interface               =            # leave empty to avoid local sniffing
# The central server will proxy on-demand PCAP fetches to sensors (TCP/5029).

Scenario B — Packet Mirroring (centralized processing/storage)

# /etc/voipmonitor.conf on the REMOTE sensor (PACKET MIRRORING)

id_sensor               = 3
server_destination      = 10.224.0.250
server_destination_port = 60024
server_password         = your_strong_password

packetbuffer_sender     = yes        # send RAW packet stream to central
interface               = eth0       # capture source; no DB settings needed
# /etc/voipmonitor.conf on the CENTRAL server (PACKET MIRRORING)

server_bind             = 0.0.0.0
server_bind_port        = 60024
server_password         = your_strong_password

mysqlhost               = 10.224.0.201
mysqldb                 = voipmonitor
mysqluser               = voipmonitor
mysqlpassword           = db_password

cdr_partition           = yes
mysqlloadconfig         = yes

# As this server does all analysis, configure as if sniffing locally:
sipport                 = 5060
# ... add your usual sniffer/storage options (pcap directories, limits, etc.)

Firewall Checklist (Quick Reference)

  • Modern Client/Server (v20+):
    • Central Server: Allow inbound TCP/60024 from remote sensors. Allow inbound TCP/5029 from GUI (management/API to central sensor).
    • Remote Sensors (Local Processing only): Allow inbound TCP/5029 from the central server (for on-demand PCAP fetch via proxy). Outbound TCP/60024 to the central server.
  • Cloud Mode:
    • Remote Sensors: Allow outbound TCP/60023 to cloud.voipmonitor.org.

Configuration & Checklists

Parameter Notes (clarifications)

  • id_sensor — Mandatory in any distributed deployment (Classic or Client/Server). Must be unique per sensor (< 65536). The value is written to the database and used by the GUI to identify where a call was captured.
  • cdr_partition — In Client/Server, enable on the central server instance that writes to the database. It can be disabled on remote "client" sensors that only mirror packets.
  • mysqlloadconfig — When enabled, the sensor can load additional parameters dynamically from the sensor_config table in the database. Typically enabled on the central server sensor that writes to DB; keep disabled on remote clients which do not access DB directly.
  • interface — Use a specific NIC (e.g., eth0) or any to capture from multiple NICs. For any ensure promiscuous mode on each NIC.

Initial Service Start & Database Initialization

After installation, the first startup against a new/empty database is critical.

  1. Start the service: systemctl start voipmonitor
  2. Follow logs to ensure schema/partition creation completes:
    • journalctl -u voipmonitor -f
    • or tail -f /var/log/syslog | grep voipmonitor

You should see creation of functions and partitions shortly after start. If you see errors like Table 'cdr_next_1' doesn't exist, the sensor is failing to initialize the schema — usually due to insufficient DB privileges or connectivity. Fix DB access and restart the sensor so it can finish initialization.

Time Synchronization

Accurate and synchronized time is critical for correlating call legs from different sensors. All servers (GUI, DB, and all Sensors) must run an NTP client (e.g., chrony or ntpdate) to keep clocks in sync.

Comparison of Remote Deployment Modes

Deployment Model Packet Processing Location PCAP Storage Location Network Traffic to Central Server GUI Connectivity
Classic Standalone Remote Remote Minimal (MySQL CDRs) GUI ↔ each Sensor (management port)
Modern Client/Server (Local Processing) Remote Remote Minimal (Encrypted CDRs) GUI ↔ Central Server only (central proxies PCAP fetch)
Modern Client/Server (Packet Mirroring) Central Central High (Encrypted full packets) GUI ↔ Central Server only

FAQ & Common Pitfalls

  • Do remote sensors need DB credentials in Client/Server? No. Only the central server instance writes to DB.
  • Why is id_sensor required everywhere? The GUI uses it to tag and filter calls by capture source.
  • Local Processing still fetches PCAPs from remote — who connects to whom? The GUI requests via the central server; the central server then connects to the remote sensor's TCP/5029 to retrieve the PCAP.

Related Documentation

AI Summary for RAG

Summary: This guide covers the deployment topologies for VoIPmonitor. It contrasts running the sensor on the same host as a PBX versus on a dedicated server. For dedicated sensors, it details methods for forwarding traffic, including hardware-based port mirroring (SPAN) and various software-based tunneling protocols (IP-in-IP, GRE, TZSP, VXLAN, HEP, AudioCodes, IPFIX). HEP (Homer Encapsulation Protocol) is a lightweight protocol for capturing and mirroring VoIP packets. When hep = yes, VoIPmonitor listens for HEPv3 (and compatible HEPv2) packets and extracts the original VoIP traffic from the encapsulation. CRITICAL HEP LIMITATION: VoIPmonitor does NOT use HEP correlation ID (captureNodeID) to correlate SIP and RTP packets. When SIP signaling and RTP media are encapsulated in HEP and arrive from different HEP sources (different capture nodes or sensors), VoIPmonitor cannot correlate them into a single CDR using HEP protocol metadata. This is feature request VS-1703 and there is currently no available workaround. The article covers cloud service packet mirroring options (GCP Packet Mirroring, AWS Traffic Mirroring, Azure Virtual Network TAP) with critical requirements: bidirectional capture (ingress and egress) and proper VM sizing (vCPU, RAM, storage I/O). The core of the article explains distributed architectures for multi-site monitoring, comparing the "classic" standalone remote sensor model with the modern, recommended "client/server" model. It details the two operational modes of the client/server architecture: local processing (sending only CDRs, PCAPs remain remote with central-proxied fetch) and packet mirroring (sending full, raw packets for central processing), which is ideal for low-resource endpoints. The article also explains an alternative approach for classic remote sensors: mounting PCAP spools via NFS or SSHFS when TCP/5029 access to sensors is blocked by firewalls, including troubleshooting steps for missing data due to NFS/SSHFS connectivity issues (checking logs for "not responding, timed out" errors, verifying network connectivity with ping/traceroute, and ensuring NFS/SSHFS server is running and accessible). The guide concludes with step-by-step configuration, firewall rules, critical parameter notes, and the importance of NTP plus first-start DB initialization.

Keywords: deployment, architecture, topology, on-host, dedicated sensor, port mirroring, SPAN, RSPAN, traffic mirroring, tunneling, GRE, TZSP, VXLAN, HEP, HEP correlation ID, captureNodeID, HEP limitation, HEP SIP RTP correlation, AudioCodes, IPFIX, cloud mirroring, GCP, AWS, Azure, Packet Mirroring, Traffic Mirroring, Virtual Network TAP, ingress, egress, bidirectional, VM sizing, remote sensor, multi-site, client server mode, packet mirroring, local processing, firewall rules, NTP, time synchronization, cloud mode, NFS, SSHFS, spooldir mounting, NFS troubleshooting, SSHFS troubleshooting, missing data, network connectivity

Key Questions:

  • Can I use cloud packet mirroring (GCP/AWS/Azure) with VoIPmonitor?
  • How should I configure cloud packet mirroring for ingress and egress traffic?
  • What is the difference between the classic remote sensor and the modern client/server mode?
  • When should I use packet mirroring (packetbuffer_sender) instead of local processing?
  • What are the firewall requirements for the client/server deployment model?
  • How can I access PCAP files from remote sensors if TCP/5029 is blocked?
  • How do I configure NFS or SSHFS to mount remote PCAP spools?
  • How do I configure the GUI sniffer data path for multiple mounted spools?
  • How do I troubleshoot missing CDRs or PCAPs when using NFS or SSHFS mounts?
  • What should I look for in logs to diagnose NFS connectivity issues?
  • Can I run the sensor on the same machine as my Asterisk/FreeSWITCH server?
  • What is a SPAN port and how is it used with VoIPmonitor?
  • Why is NTP important for a distributed VoIPmonitor setup?
  • What is HEP and how do I configure VoIPmonitor to receive HEP packets?
  • Does VoIPmonitor use HEP correlation ID (captureNodeID) to correlate SIP and RTP packets?
  • Can VoIPmonitor correlate SIP and RTP packets that arrive from different HEP sources?
  • Is there a workaround for HEP SIP/RTP correlation across multiple HEP capture nodes?
  • How do I configure GRE, ERSPAN, and VXLAN tunneling for VoIPmonitor?