Reprocess Historical Calls: Difference between revisions

From VoIPmonitor.org
(Add documentation for reprocessing historical calls with process_spooldir.pl to enable features retroactively (e.g., save_sip_responses))
 
(Create new page: Reprocess Historical Calls guide)
 
Line 1: Line 1:
{{DISPLAYTITLE:Reprocessing Historical Calls with New Configuration}}
{{DISPLAYTITLE:Reprocess Historical Calls}}
[[Category:Administration]]
[[Category:Tools]]


'''This guide explains how to re-process captured packet archives (PCAP files) using a different configuration than was originally used at capture time. This is useful for enabling features retroactively, such as storing intermediate SIP responses, analyzing calls with new detection rules, or testing configuration changes without affecting live monitoring.'''
= Reprocess Historical Calls =


== Use Cases
This guide explains how to re-analyze previously captured PCAP files using VoIPmonitor. This is useful for applying new configurations, testing detection methods, or recovering data after schema changes.


Common scenarios where historical call reprocessing is needed:
== Use Cases ==


* '''Enable SIP response storage retroactively''' - Capture intermediate SIP responses (e.g., 491 Request Pending) from previously captured calls when <code>save_sip_responses</code> was not enabled
{| class="wikitable"
* '''Test new detection algorithms''' - Apply updated fax/DTMF detection or jitter analysis to historical data
|-
* '''Replay with different capture rules''' - Test how a new <code>capture_rules</code> configuration would have behaved on past traffic
! Use Case !! Description
* '''Upgrade schema compatibility''' - Re-process old PCAPs to take advantage of new database schema features
|-
* '''Debug analysis issues''' - Re-run a problematic CDR through the packet processor to debug parsing logic
| '''Pre-deployment verification''' || Upload sample traffic (VoLTE, WebRTC, TLS) to confirm VoIPmonitor can parse the protocol before deploying live mirroring
|-
| '''Retroactive SIP response capture''' || Enable <code>save_sip_responses = yes</code> to find intermediate SIP codes (like 491) from calls captured before this feature was enabled
|-
| '''Testing new detection methods''' || Apply updated algorithms for fax/DTMF/silence detection to historical data
|-
| '''Configuration experimentation''' || Test how new capture rules would have behaved on past traffic
|-
| '''Schema upgrades''' || Reprocess old PCAPs to leverage new database features
|-
| '''License reset recovery''' || Reprocess PCAPs after truncating CDR tables (see [[License#License_Lock|License Lock]])
|}


== Safety Considerations
== Methods ==


{{Warning|1=Reprocessing historical data can be resource-intensive and should be performed on non-production equipment or during maintenance windows. The process can consume significant CPU, I/O, and database resources depending on the volume of data being reprocessed.}}
=== Method 1: Single PCAP File (-r option) ===


Important safety notes:
Process a single PCAP file and generate CDRs:


* Always re-process into a '''separate database''', not your primary production database
<syntaxhighlight lang="bash">
* The original PCAP files remain unchanged; reprocessing only creates new CDR entries
voipmonitor --config-file /etc/voipmonitor.conf -r /path/to/capture.pcap
* Re-processing does not delete or modify data in the source PCAP archives
</syntaxhighlight>
* Disk space is required for both the original PCAPs and the new reprocessed CDR data
* Network packet capture is '''not required''' during reprocessing; this is an offline batch process


== Step-by-Step Guide
'''GUI alternative:''' Use '''Tools → Load PCAP''' to upload files via web interface.


=== Step 1: Create a New Empty Database
{{Note|1=Configure the upload sniffer config path in '''Settings → System Configuration → Upload sniffer conf path'''}}


Create a separate MySQL/MariaDB database for the reprocessed data. Do not use your main production database for this process.
=== Method 2: Directory Processing (--readpcapdir) ===


<syntaxhighlight lang="sql">
Process all PCAPs in a directory (e.g., the spooldir):
-- Connect to MySQL as an administrative user
mysql -u root -p


-- Create a new empty database
<syntaxhighlight lang="bash">
CREATE DATABASE voipmonitor_reprocess CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
voipmonitor --config-file /etc/voipmonitor.conf --readpcapdir /var/spool/voipmonitor
 
-- Create a dedicated user for this database (optional but recommended)
CREATE USER 'voipmonitor_reprocess'@'localhost' IDENTIFIED BY 'your_secure_password';
GRANT ALL PRIVILEGES ON voipmonitor_reprocess.* TO 'voipmonitor_reprocess'@'localhost';
FLUSH PRIVILEGES;
</syntaxhighlight>
</syntaxhighlight>


The VoIPmonitor sensor will automatically create the required tables when it starts processing with this database.
This recursively processes the spooldir structure (<code>YYYY-MM-DD/HH/MM/</code>).


=== Step 2: Create a Reprocess Configuration File
=== Method 3: Continuous Directory Scanning (scanpcapdir) ===


Create a new configuration file (e.g., <code>/etc/voipmonitor-reprocess.conf</code>) with your desired settings:
For near-real-time processing of PCAP files as they are created (e.g., by tcpdump):


<syntaxhighlight lang="ini">
<syntaxhighlight lang="ini">
# /etc/voipmonitor-reprocess.conf
# /etc/voipmonitor.conf
[general]
scanpcapdir = /dev/shm/voipmonitor
 
scanpcapmethod = newfile
# Point to the NEW reprocessing database (NOT your main production database)
mysqlhost = localhost
mysqlsocket = /var/run/mysqld/mysqld.sock
mysqlport = 3306
mysqlusername = voipmonitor_reprocess
mysqlpassword = your_secure_password
mysqldb = voipmonitor_reprocess
 
# Enable the features you want to apply to historical calls
# Example: Store all SIP responses including intermediate ones
save_sip_responses = yes
 
# Other configuration based on what you want to change
# Example: Re-run with new SIP history settings
save_sip_history = responses
 
# Keep other settings appropriate for your environment
spooldir = /var/spool/voipmonitor
 
# CRITICAL: No interface or live capture settings required
# This is an offline reprocessing job
</syntaxhighlight>
</syntaxhighlight>


{{Tip|You can copy your existing <code>/etc/voipmonitor.conf</code> as a starting point and then modify it for reprocessing. Remove or comment out options related to live capture (like <code>interface</code>, <code>sipport</code>, etc.) since reprocessing reads from disk only.}}
{| class="wikitable"
 
|-
=== Step 3: Run the Reprocessing Script
! Parameter !! Description
 
|-
Use the <code>process_spooldir.pl</code> Perl script to read PCAP files from the spooldir and re-process them with your new configuration:
| <code>scanpcapdir</code> || Directory to monitor for new PCAP files
|-
| <code>scanpcapmethod</code> || Detection method: <code>newfile</code> (default) or <code>rename</code>
|}


Example tcpdump command creating 5-second PCAP files:
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
# Basic syntax
tcpdump -i eth0 -G 5 -w /dev/shm/voipmonitor/%s.pcap
perl /var/www/html/voipmonitor/scripts/process_spooldir.pl \
  --voipcfg=/etc/voipmonitor-reprocess.conf \
  --spooldir=/var/spool/voipmonitor \
  --fromdate='2024-12-01 00:00' \
  --todate='2024-12-31 23:59'
</syntaxhighlight>
</syntaxhighlight>


'''Parameters:'''
== Processing to a Separate Database ==


* <code>--voipcfg</code>: Path to your reprocessing configuration file (REQUIRED)
{{Warning|1='''Never reprocess directly to your production database.''' Always use a separate database for reprocessed data.}}
* <code>--spooldir</code>: Path to the spooldir directory containing the archived PCAP files
* <code>--fromdate</code>: Start date for reprocessing (YYYY-MM-DD HH:MM format)
* <code>--todate</code>: End date for reprocessing (YYYY-MM-DD HH:MM format)
* <code>--pcapfilter</code>: Optional BPF filter to only reprocess matching packets (e.g., <code>--pcapfilter='udp port 5060'</code>)
* <code>--verbose</code>: Enable detailed output for troubleshooting
* <code>--threads</code>: Number of parallel threads (default: 2)


{{Note|1=The date range selects files based on the date/time hierarchy in the spooldir structure (<code>/var/spool/voipmonitor/YYYY-MM-DD/HH/</code>), NOT based on call start times within PCAP files.}}
=== Step 1: Create Isolated Database ===
 
=== Step 4: Monitor the Reprocess Progress
 
Reprocessing can take time depending on your data volume and system performance. Monitor progress with:


<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
# In another terminal, monitor the script output
mysql -u root -p -e "CREATE DATABASE voipmonitor_reprocess;"
# (If you didn't run with --verbose, check logs below)
 
# Check database size growth
mysql -u voipmonitor_reprocess -p -e "SELECT COUNT(*) AS cdr_count FROM voipmonitor_reprocess.cdr;"
 
# Monitor CPU and I/O usage
htop
 
# Check MySQL performance
mysqladmin -u root -p processlist
</syntaxhighlight>
</syntaxhighlight>


The script will output progress information including files processed, calls found, and errors encountered.
=== Step 2: Create Dedicated Configuration ===


=== Step 5: View Reprocessed Data in the GUI
<syntaxhighlight lang="bash">
 
cp /etc/voipmonitor.conf /etc/voipmonitor-reprocess.conf
Once reprocessing is complete, you need to configure the GUI to connect to the new database:
 
{{Note|1=You can either switch the primary GUI database connection (affecting all users) or create a dedicated read-only access for viewing reprocessed data without disrupting normal operations.}}
 
'''Option A: Temporarily Switch GUI Database (for quick review):'''
 
<syntaxhighlight lang="php">
// Edit /var/www/html/voipmonitor/configuration.php
// Change the database connection parameters:
 
$dbhost = 'localhost';
$dbname = 'voipmonitor_reprocess';  // Changed from 'voipmonitor'
$dbuser = 'voipmonitor_reprocess';
$dbpass = 'your_secure_password';
</syntaxhighlight>
</syntaxhighlight>


Refresh the GUI in your browser. You will now see the reprocessed calls with your new configuration options applied.
Edit the reprocess config:


'''Option B: Use Separate Virtual Host (recommended):'''
<syntaxhighlight lang="ini">
# /etc/voipmonitor-reprocess.conf
mysqldb = voipmonitor_reprocess


Create a separate Apache/Nginx virtual host pointing to a copy of the GUI configured to connect to the reprocessing database. This allows you to view both production and reprocessed data simultaneously.
# Disable live capture
 
interface =
=== Step 6: Analyze the Results
 
Now you can search for the data that your new configuration enabled:
 
* To find calls with specific intermediate SIP responses, use the '''SIP Messages''' interface in the GUI and filter by response code (e.g., 491)
* Compare the reprocessed CDR structure with original CDRs to verify the changes are correct
* Export reports from the reprocessed data as needed
 
== Common Reprocessing Scenarios
 
=== Enable SIP Response Storage for Historical Calls
 
This is the most common use case - you want to find calls containing intermediate SIP responses that were not captured originally.
 
'''Configuration:'''
<syntaxhighlight lang="ini">
[general]
# ... database settings as above ...


# Store all SIP responses (not just final ones)
# Apply desired new settings
save_sip_responses = yes
save_sip_responses = yes
silencedetect = yes
inbanddtmf = yes


# Also store SIP response text for searching
# Different manager port to avoid conflicts
save_sip_history = responses
managerport = 5030
</syntaxhighlight>
</syntaxhighlight>


'''After reprocessing:'''
=== Step 3: Run Reprocessing ===
1. Navigate to '''CDR → SIP Messages''' in the GUI
2. Use the '''Response''' filter to search for specific codes (e.g., 491, 180, 183)
3. Click on entries to see the full SIP message details and associated CDR


=== Re-process with Enhanced Audio Detection
<syntaxhighlight lang="bash">
 
voipmonitor --config-file /etc/voipmonitor-reprocess.conf --readpcapdir /var/spool/voipmonitor
Update audio analysis settings on historical calls:
</syntaxhighlight>
 
<syntaxhighlight lang="ini">
[general]
# ... database settings as above ...
 
# Enable in-band DTMF detection (was disabled before)
inbanddtmf = yes


# Enable silence detection (was disabled before)
=== Step 4: Connect GUI to Reprocessed Data ===
silencedetect = yes
silencethreshold = 512


# Save energy levels for analysis
Either:
save-energylevels = yes
* Change <code>mysqldb</code> in GUI's <code>configuration.php</code> temporarily, or
</syntaxhighlight>
* Create a second GUI instance pointing to the reprocess database


=== Test New Capture Rules
== PCAP to Audio Conversion ==


See how a new <code>capture_rules</code> configuration would have affected PCAP retention:
Extract audio from a PCAP file without database:


<syntaxhighlight lang="ini">
<syntaxhighlight lang="ini">
[general]
# /tmp/voipmonitor-audio.conf
# ... database settings as above ...
spooldir = /tmp/audio_output
 
saveaudio = ogg
# Define new capture rules
nocdr = yes
capture_rules = /etc/capture_rules_new.conf
savesip = no
 
savertp = no
# Save all PCAPs for testing (do not use this in production)
maxpooldays = 0
maxpoolsize = 0
</syntaxhighlight>
</syntaxhighlight>


{{Warning|1=Setting <code>maxpooldays = 0</code> and <code>maxpoolsize = 0</code> disables automatic PCAP cleanup. Only use this for testing specific date ranges and clean up manually afterward.}}
<syntaxhighlight lang="bash">
 
voipmonitor --config-file=/tmp/voipmonitor-audio.conf -k -v1 -r /path/to/call.pcap
== Troubleshooting
 
=== Script Errors or Hangs
 
If <code>process_spooldir.pl</code> encounters issues:
 
{syntaxhighlight lang="bash}
# Run with verbose logging to see detailed progress
perl /var/www/html/voipmonitor/scripts/process_spooldir.pl \
  --voipcfg=/etc/voipmonitor-reprocess.conf \
  --spooldir=/var/spool/voipmonitor \
  --fromdate='2024-12-01 00:00' \
  --todate='2024-12-31 23:59' \
  --verbose 2>&1 | tee /tmp/reprocess.log
 
# Check for common issues:
# - Disk space: df -h
# - MySQL connection: mysql -u voipmonitor_reprocess -p voipmonitor_reprocess
# - PCAP file permissions: ls -l /var/spool/voipmonitor
}</syntaxhighlight>
 
=== Database Connection Issues
 
Ensure the MySQL user has proper permissions:
 
<syntaxhighlight lang="sql">
-- Verify user permissions
SHOW GRANTS FOR 'voipmonitor_reprocess'@'localhost';
 
-- The user should have ALL PRIVILEGES on the voipmonitor_reprocess database
</syntaxhighlight>
</syntaxhighlight>


=== No Data in Database After Reprocessing
Audio formats: <code>ogg</code>, <code>wav</code>, <code>mp3</code>
 
If the reprocess script completes without errors but the database is empty:
 
* Verify the <code>--fromdate</code> and <code>--todate</code> ranges match the spooldir structure
* Check that PCAP files exist in the specified date ranges: <code>ls -la /var/spool/voipmonitor/2024-12-01/</code>
* Verify the <code>spooldir</code> parameter points to the correct location
* Review the script output for any skipped files or warnings
 
=== CDR Count Doesn't Match Expectations
 
If you reprocessed a known number of calls but the CDR count differs:
 
* The reprocessing script processes PCAP files, not individual calls
* Multiple CDRs may be generated from a single PCAP file
* Some calls may have been filtered out by the new configuration settings (e.g., <code>cdronlyanswered = yes</code>)
* Check the <code>cdr_ignore_response</code> setting which may filter certain response codes


== Performance Optimization Tips
== Performance Considerations ==


* '''Threading''': Use <code>--threads=N</code> to parallelize processing on multi-core systems
* '''CPU:''' Reprocessing is CPU-intensive. Monitor <code>t0CPU</code> usage.
* '''Date Ranges''': Process data in smaller date chunks (e.g., one day at a time) for better progress tracking
* '''I/O:''' SSD recommended for large datasets. Spooldir uses nested TAR archives.
* **Off-Peak Hours**: Run reprocessing jobs during low traffic periods to minimize impact on production systems
* '''Memory:''' Same requirements as live capture.
* '''Disk I/O''': If possible, place the reprocessing SQL files and PCAP archives on separate disk drives
* '''Parallelization:''' Use multiple instances with different date ranges for faster processing.
* '''Index Creation''': Consider creating database indexes after reprocessing is complete rather than during the process


=syntaxhighlight lang="bash"
{{Tip|1=Schedule reprocessing during off-peak hours to avoid impacting live monitoring.}}
# Example: Process one month in weekly chunks to track progress better
for week in 1 2 3 4; do
  start_date="2024-12-01"
  end_date=$(date -d "$start_date + 7 days - 1 second" +%Y-%m-%d\ %H:%M)


  perl /var/www/html/voipmonitor/scripts/process_spooldir.pl \
== Troubleshooting ==
    --voipcfg=/etc/voipmonitor-reprocess.conf \
    --spooldir=/var/spool/voipmonitor \
    --fromdate="$start_date 00:00" \
    --todate="$end_date 23:59" \
    --verbose 2>&1 | tee "/tmp/reprocess_week$week.log"


  start_date=$end_date
{| class="wikitable"
done
|-
</syntaxhighlight>
! Problem !! Solution
 
|-
== Cleanup After Reprocessing
| No CDRs generated || Verify PCAP contains complete SIP dialogs (INVITE through BYE). Check <code>sipport</code> matches ports in PCAP.
 
|-
Once you have analyzed the reprocessed data and no longer need the reprocessing database:
| CDR count mismatch || Some calls may span multiple PCAP files. Ensure all related files are processed.
 
|-
<syntaxhighlight lang="bash">
| Permission errors || Run as root or ensure voipmonitor user has read access to PCAP files.
# Stop any GUI connections to the reprocessing database
|-
# Either revert GUI configuration or remove the virtual host
| Database connection failed || Verify MySQL credentials in config and that target database exists.
 
|-
# Remove the database and user
| Missing audio || Ensure PCAP contains both SIP and RTP packets. Check <code>savertp = yes</code>.
mysql -u root -p -e "DROP DATABASE voipmonitor_reprocess;"
|}
mysql -u root -p -e "DROP USER 'voipmonitor_reprocess'@'localhost';"
 
# Remove the temporary configuration file
rm /etc/voipmonitor-reprocess.conf
</syntaxhighlight>


== Related Documentation
== See Also ==


* [[Sniffer_configuration|Sniffer Configuration Reference]] - Detailed explanation of all configuration options
* [[Tools#Load_PCAP|Tools: Load PCAP]]
* [[Manual_PCAP_Extraction_from_spooldir|Manual PCAP Extraction]] - Extracting individual call PCAPs
* [[Data_Cleaning|Data Cleaning and Retention]]
* [[Data_Cleaning|Data Cleaning]] - Managing PCAP and database retention
* [[License#License_Lock|License Lock Recovery]]
* [[Manual_export_of_pcap_files_from_spooldir|Manual PCAP Export from Spooldir]]


== AI Summary for RAG
== AI Summary for RAG ==


'''Summary:''' This document explains how to re-process historical PCAP archives with a different VoIPmonitor configuration using the <code>process_spooldir.pl</code> Perl script. Common use cases include enabling <code>save_sip_responses</code> retroactively to find intermediate SIP response codes (like 491) in old calls, testing new detection algorithms, or applying new configuration options. The process requires creating a separate database, writing a reprocessing config file (<code>--voipcfg</code>), specifying date ranges (<code>--fromdate</code>/<code>--todate</code>), and optionally using threading (<code>--threads</code>). Reproduction is an offline batch operation that reads from spooldir TAR archives and writes new CDR entries without modifying the original PCAP files. After reprocessing, the GUI can be connected to the new database (either by temporarily switching configuration or creating a separate virtual host) to view the results.
'''Summary:''' Guide for reprocessing historical PCAP files in VoIPmonitor. Three methods available: (1) Single file with <code>-r</code> option or GUI Load PCAP tool, (2) Directory processing with <code>--readpcapdir</code> for bulk reprocessing, (3) Continuous scanning with <code>scanpcapdir</code> config for near-real-time tcpdump integration. Critical safety rule: always use separate database for reprocessed data, never production. Common use cases include pre-deployment protocol verification (VoLTE, WebRTC), retroactive SIP response capture, testing new detection methods (silence, DTMF), and license lock recovery after CDR truncation. For audio extraction without database, use <code>nocdr=yes</code> with <code>saveaudio</code> option.


'''Keywords:''' reprocess, historical data, process_spooldir.pl, save_sip_responses, intermediate SIP responses, 491, SIP response code, offline processing, reprocessing, retroactive analysis, PCAP re-analysis, new database, voipcfg, fromdate, todate
'''Keywords:''' reprocess, pcap, readpcapdir, scanpcapdir, historical calls, offline processing, batch processing, audio extraction, pre-deployment verification, retroactive capture, separate database, voipmonitor -r, Load PCAP, scanpcapmethod


'''Key Questions:'''
'''Key Questions:'''
* How do I enable SIP response storage for calls that were already captured?
* How do I reprocess historical PCAP files in VoIPmonitor?
* How can I find intermediate SIP response codes like 491 in historical data?
* How do I upload and process a PCAP file to verify protocol support?
* What is process_spooldir.pl and how do I use it?
* How do I extract audio from a PCAP file without a database?
* How do I re-process captured PCAP files with a different configuration?
* How do I process multiple PCAP files from a directory?
* Can I apply new VoIPmonitor settings to old call data?
* How do I set up continuous PCAP scanning from tcpdump output?
* How do I create a separate database for reprocessing analysis?
* Can I reprocess calls to apply new configuration settings retroactively?
* Is it safe to re-process historical data without affecting live monitoring?
* How do I safely reprocess data without affecting production?
* How do I view reprocessed CDRs in the GUI?
* How do I recover calls after truncating the CDR table?
* Can I test new capture rules on past traffic?

Latest revision as of 16:51, 8 January 2026


Reprocess Historical Calls

This guide explains how to re-analyze previously captured PCAP files using VoIPmonitor. This is useful for applying new configurations, testing detection methods, or recovering data after schema changes.

Use Cases

Use Case Description
Pre-deployment verification Upload sample traffic (VoLTE, WebRTC, TLS) to confirm VoIPmonitor can parse the protocol before deploying live mirroring
Retroactive SIP response capture Enable save_sip_responses = yes to find intermediate SIP codes (like 491) from calls captured before this feature was enabled
Testing new detection methods Apply updated algorithms for fax/DTMF/silence detection to historical data
Configuration experimentation Test how new capture rules would have behaved on past traffic
Schema upgrades Reprocess old PCAPs to leverage new database features
License reset recovery Reprocess PCAPs after truncating CDR tables (see License Lock)

Methods

Method 1: Single PCAP File (-r option)

Process a single PCAP file and generate CDRs: 

voipmonitor --config-file /etc/voipmonitor.conf -r /path/to/capture.pcap

GUI alternative: Use Tools → Load PCAP to upload files via web interface.

ℹ️ Note: Configure the upload sniffer config path in Settings → System Configuration → Upload sniffer conf path

Method 2: Directory Processing (--readpcapdir)

Process all PCAPs in a directory (e.g., the spooldir): 

voipmonitor --config-file /etc/voipmonitor.conf --readpcapdir /var/spool/voipmonitor

This recursively processes the spooldir structure (YYYY-MM-DD/HH/MM/).

Method 3: Continuous Directory Scanning (scanpcapdir)

For near-real-time processing of PCAP files as they are created (e.g., by tcpdump): 

# /etc/voipmonitor.conf
scanpcapdir = /dev/shm/voipmonitor
scanpcapmethod = newfile
Parameter Description
scanpcapdir Directory to monitor for new PCAP files
scanpcapmethod Detection method: newfile (default) or rename

Example tcpdump command creating 5-second PCAP files: 

tcpdump -i eth0 -G 5 -w /dev/shm/voipmonitor/%s.pcap

Processing to a Separate Database

⚠️ Warning: Never reprocess directly to your production database. Always use a separate database for reprocessed data.

Step 1: Create Isolated Database

mysql -u root -p -e "CREATE DATABASE voipmonitor_reprocess;"

Step 2: Create Dedicated Configuration

cp /etc/voipmonitor.conf /etc/voipmonitor-reprocess.conf

Edit the reprocess config: 

# /etc/voipmonitor-reprocess.conf
mysqldb = voipmonitor_reprocess

# Disable live capture
interface =

# Apply desired new settings
save_sip_responses = yes
silencedetect = yes
inbanddtmf = yes

# Different manager port to avoid conflicts
managerport = 5030

Step 3: Run Reprocessing

voipmonitor --config-file /etc/voipmonitor-reprocess.conf --readpcapdir /var/spool/voipmonitor

Step 4: Connect GUI to Reprocessed Data

Either:

  • Change mysqldb in GUI's configuration.php temporarily, or
  • Create a second GUI instance pointing to the reprocess database

PCAP to Audio Conversion

Extract audio from a PCAP file without database: 

# /tmp/voipmonitor-audio.conf
spooldir = /tmp/audio_output
saveaudio = ogg
nocdr = yes
savesip = no
savertp = no

voipmonitor --config-file=/tmp/voipmonitor-audio.conf -k -v1 -r /path/to/call.pcap

Audio formats: ogg, wav, mp3

Performance Considerations

  • CPU: Reprocessing is CPU-intensive. Monitor t0CPU usage.
  • I/O: SSD recommended for large datasets. Spooldir uses nested TAR archives.
  • Memory: Same requirements as live capture.
  • Parallelization: Use multiple instances with different date ranges for faster processing.

💡 Tip: Schedule reprocessing during off-peak hours to avoid impacting live monitoring.

Troubleshooting

Problem Solution
No CDRs generated Verify PCAP contains complete SIP dialogs (INVITE through BYE). Check sipport matches ports in PCAP.
CDR count mismatch Some calls may span multiple PCAP files. Ensure all related files are processed.
Permission errors Run as root or ensure voipmonitor user has read access to PCAP files.
Database connection failed Verify MySQL credentials in config and that target database exists.
Missing audio Ensure PCAP contains both SIP and RTP packets. Check savertp = yes.

See Also

AI Summary for RAG

Summary: Guide for reprocessing historical PCAP files in VoIPmonitor. Three methods available: (1) Single file with -r option or GUI Load PCAP tool, (2) Directory processing with --readpcapdir for bulk reprocessing, (3) Continuous scanning with scanpcapdir config for near-real-time tcpdump integration. Critical safety rule: always use separate database for reprocessed data, never production. Common use cases include pre-deployment protocol verification (VoLTE, WebRTC), retroactive SIP response capture, testing new detection methods (silence, DTMF), and license lock recovery after CDR truncation. For audio extraction without database, use nocdr=yes with saveaudio option.

Keywords: reprocess, pcap, readpcapdir, scanpcapdir, historical calls, offline processing, batch processing, audio extraction, pre-deployment verification, retroactive capture, separate database, voipmonitor -r, Load PCAP, scanpcapmethod

Key Questions:

  • How do I reprocess historical PCAP files in VoIPmonitor?
  • How do I upload and process a PCAP file to verify protocol support?
  • How do I extract audio from a PCAP file without a database?
  • How do I process multiple PCAP files from a directory?
  • How do I set up continuous PCAP scanning from tcpdump output?
  • Can I reprocess calls to apply new configuration settings retroactively?
  • How do I safely reprocess data without affecting production?
  • How do I recover calls after truncating the CDR table?
Retrieved from "https://www.voipmonitor.org/doc/index.php?title=Reprocess_Historical_Calls&oldid=10204"