Data Cleaning: Difference between revisions

From VoIPmonitor.org
(Add custom autocleaning section with IP filter cleanup workflow)
(Fix incorrect association of _2 parameters with tar_move - they are for spooldir_2 used with capture rules (TP-38))
 
(46 intermediate revisions by the same user not shown)
Line 1: Line 1:
This guide explains how VoIPmonitor manages data retention for both captured packets (PCAP files) and Call Detail Records (CDRs) in the database. Proper configuration is essential for managing disk space and maintaining long-term database performance.
{{DISPLAYTITLE:Data Cleaning and Retention}}
[[Category:Configuration]]
[[Category:Administration]]
 
This guide explains how VoIPmonitor manages data retention for PCAP files and database records.


== Overview ==
== Overview ==


VoIPmonitor generates two primary types of data that require periodic cleaning:
VoIPmonitor generates two types of data requiring periodic cleanup:
* '''PCAP Files:''' Raw packet captures of SIP/RTP/GRAPH data stored on the filesystem in the spool directory. These can consume significant disk space.
* '''CDR Data:''' Call metadata stored in the MySQL database. Large tables can slow down GUI performance if not managed properly.
 
The system uses two separate, independent mechanisms to manage the retention of this data:
 
<kroki lang="plantuml">
@startuml
skinparam shadowing false
skinparam defaultFontName Arial
skinparam rectangle {
  BorderColor #4A90E2
  BackgroundColor #FFFFFF
}
 
rectangle "VoIPmonitor Sensor" as sensor
 
package "Filesystem Storage" {
  folder "/var/spool/voipmonitor" as spool {
    file "SIP PCAPs" as sip
    file "RTP PCAPs" as rtp
    file "GRAPH files" as graph
    file "AUDIO files" as audio
  }
}
 
database "MySQL Database" {
  collections "cdr" as cdr
  collections "cdr_next" as cdrnext
  collections "register_state" as reg
  collections "sip_msg" as sipmsg
}
 
sensor --> spool : writes
sensor --> cdr : writes
 
rectangle "Filesystem Cleaner\n(maxpoolsize/maxpooldays)" as fscleaner #E8F5E9
rectangle "Database Cleaner\n(cleandatabase)" as dbcleaner #E3F2FD
 
fscleaner --> spool : deletes old files
dbcleaner --> cdr : drops partitions
 
note bottom of fscleaner : Runs every 5 minutes\nDeletes oldest data first
note bottom of dbcleaner : Daily partition drop\nInstant operation
@enduml
</kroki>
 
== Filesystem Cleaning (PCAP Spool Directory) ==
 
The sensor stores captured call data in a structured directory tree on the local filesystem.
 
=== Reducing Data Collection at Source ===
 
Before configuring cleanup policies, consider reducing the amount of data captured. This is often the most effective long-term solution for storage management.
 
==== Save Only RTP Headers (Major Space Saver) ====
 
RTP packets typically contain the full audio payload, which consumes the majority of disk space. If you only need call quality statistics (MOS, jitter, packet loss) and not actual audio playback, switch to saving RTP headers only.
 
Edit <code>/etc/voipmonitor.conf</code>:
 
<syntaxhighlight lang="ini">
# Change from full RTP to headers only
savertp = header
</syntaxhighlight>


{| class="wikitable"
{| class="wikitable"
|-
|-
! Setting !! Storage Impact !! Use Case
! Data Type !! Storage !! Cleanup Mechanism !! Key Parameters
|-
|-
| <code>savertp = yes</code> || High (~10x more) || Requires ability to play back audio from PCAPs
| '''PCAP Files''' || Filesystem (spool directory) || Cleanspool process (every 5 min) || <code>maxpoolsize</code>, <code>maxpooldays</code>
|-
|-
| <code>savertp = header</code> || Low || Only CDR statistics needed, no audio playback required
| '''CDR Records''' || MySQL database || Partition dropping (instant) || <code>cleandatabase</code>
|}
|}


With <code>savertp = header</code>, VoIPmonitor still captures all necessary metadata for MOS scoring, jitter analysis, packet loss statistics, and quality graphs, but does not store the actual audio payload. This can reduce storage consumption by up to 90%.
{{Note|These are '''independent systems''' - filesystem cleanup does not affect database records and vice versa.}}


'''Important:''' After changing from <code>savertp = yes</code> to <code>savertp = header</code>, existing PCAP files will remain playable. New calls will only contain RTP headers.
=== Quick Reference ===


{| class="wikitable" style="background-color: #FFF3CD;"
{| class="wikitable"
|-
|-
! colspan="2" style="background: #856404; color: white;" | Critical: Distributed Architecture Consideration
! Purpose !! Parameter !! Example
|-
|-
| style="vertical-align: top;" | '''<code>packetbuffer_sender</code> Architecture:'''
| Limit total PCAP size || <code>maxpoolsize</code> || <code>maxpoolsize = 512000</code> (500 GB)
|
If you are using the Packet Mirroring/Client-Server mode (<code>packetbuffer_sender=yes</code> on remote sensors), the <code>savertp</code> setting must be applied on the '''central server''' where packet analysis is performed, not on the individual sensors.
 
{|
|-
|-
! Architecture !! Where to Apply <code>savertp = header</code>
| Limit RTP specifically || <code>maxpoolrtpsize</code> || <code>maxpoolrtpsize = 102400</code> (100 GB)
|-
|-
| <code>packetbuffer_sender = no</code> (Local Processing) || On each sensor/probe's configuration file
| Limit PCAP age || <code>maxpooldays</code> || <code>maxpooldays = 30</code>
|-
|-
| <code>packetbuffer_sender = yes</code> (Packet Mirroring) || On the central server's configuration file (not on sensors)
| CDR retention (days) || <code>cleandatabase</code> || <code>cleandatabase = 30</code>
|}
 
For distributed deployment details, see [[Sniffer_distributed_architecture#Controlling_Packet_Storage_in_Packet_Mirroring_Mode|Client-Server Architecture]].
|}
 
For more configuration options, see [[Sniffer_configuration#Saving_Options|Sniffer Configuration - Saving Options]].
 
=== Spool Directory Location ===
 
By default, all data is stored in <code>/var/spool/voipmonitor</code>. This location can be changed by setting the <code>spooldir</code> option in <code>voipmonitor.conf</code>.
 
==== Relocating the Spool Directory to a Different Partition ====
 
If your default partition is running out of space, you can move the spool directory to a larger partition or dedicated disk. This is particularly useful when MySQL Error 28 ("No space left on device") occurs despite retention settings being in place.
 
{| class="wikitable"
|-
|-
! Situation !! Recommended Action
| CDR retention (size) || <code>cleandatabase_size</code> || <code>cleandatabase_size = 512000</code> (+ <code>cleandatabase_size_force = true</code>)
|-
|-
| <code>/var/lib/mysql</code> full but other partitions available || Move spool to larger partition and update <code>spooldir</code> in <code>voipmonitor.conf</code>
| Save only RTP stats || <code>savertp</code> || <code>savertp = header</code> (saves ~90% space)
|-
| Multiple disks available || Use dedicated partition for PCAP storage
|-
| GUI on same server || Sync GUI <code>SNIFFER_DATA_PATH</code> to match new <code>spooldir</code>
|}
|}


===== Procedure to Relocate Spooldir =====
== Filesystem Cleaning (PCAP Files) ==


;Step 1: Identify Available Space
=== How Cleanspool Works ===
Check all partitions to find a suitable location:
<syntaxhighlight lang="bash">
df -h
</syntaxhighlight>


;Step 2: Create New Spool Directory
The sniffer maintains a file index '''in memory'''. Every 5 minutes, the cleanspool thread checks retention limits and deletes the oldest files when limits are exceeded.
Create the directory on the destination partition:
<syntaxhighlight lang="bash">
# Example: Use /mnt/pcaps partition
mkdir -p /mnt/pcaps/voipmonitor


# Set correct ownership for both sniffer and GUI access
<kroki lang="plantuml">
chown voipmonitor:voipmonitor /mnt/pcaps/voipmonitor
@startuml
chmod 755 /mnt/pcaps/voipmonitor
skinparam shadowing false
</syntaxhighlight>
skinparam defaultFontSize 11
participant "Sniffer" as S
participant "Cleanspool" as C
database "In-Memory\nIndex" as MEM
collections "Filesystem" as FS


;Step 3: Update VoIPmonitor Configuration
S -> FS: Write PCAP
Edit <code>/etc/voipmonitor.conf</code> to point to the new location:
S -> MEM: Update index
<syntaxhighlight lang="ini">
... Every 5 minutes ...
[general]
C -> MEM: Find oldest files\nexceeding limits
spooldir = /mnt/pcaps/voipmonitor
loop Delete old files
</syntaxhighlight>
    C -> FS: DELETE
    C -> MEM: Remove entry
end
note over MEM : Persisted to\n.cleanspool_cache
@enduml
</kroki>


;Step 4: Update GUI Configuration (Critical)
The <code>.cleanspool_cache</code> files in hourly directories serve as '''persistent storage for fast restart''' - they allow quick index reload without scanning the entire filesystem.


If the GUI runs on the same server, you MUST update the path the GUI uses to access the spool. This is defined in the GUI configuration file:
{{Note|1='''Legacy Indexing:''' Old VoIPmonitor versions stored metadata in MySQL <code>files</code> table. This can be enabled with <code>cleanspool_use_files = yes</code> (deprecated, do not use).}}


* Debian/Ubuntu: <code>/var/www/html/voipmonitor/config/configuration.php</code>
=== Retention Configuration ===
* RHEL/CentOS: <code>/var/www/voipmonitor/config/configuration.php</code>
 
Edit the GUI configuration file:
<syntaxhighlight lang="php">
// Find and update this line to match your new spooldir
define('SNIFFER_DATA_PATH', '/mnt/pcaps/voipmonitor');
</syntaxhighlight>
 
'''Important:''' The <code>SNIFFER_DATA_PATH</code> in the GUI config MUST match the <code>spooldir</code> setting in <code>voipmonitor.conf</code>. If these are mismatched, the GUI will be unable to locate and display call recordings.
 
;Step 5: Restart Services
Apply all changes by restarting the services:
<syntaxhighlight lang="bash">
# Restart the VoIPmonitor sensor
systemctl restart voipmonitor
 
# If using Apache/Nginx, reload web server
systemctl reload apache2    # Debian/Ubuntu
# or
systemctl reload nginx      # RHEL/CentOS
</syntaxhighlight>
 
;Step 6: Verify Configuration
<syntaxhighlight lang="bash">
# Check if new spool directory is being used
ls -ld /mnt/pcaps/voipmonitor
 
# Verify GUI can access the path
grep SNIFFER_DATA_PATH /path/to/gui/config/configuration.php
 
# Check sensor logs for any spool-related errors
tail -f /var/log/voipmonitor.log
</syntaxhighlight>
 
===== Optional: Move Existing Data =====
 
If you want to migrate existing PCAP files to the new location:
<syntaxhighlight lang="bash">
# Stop the sensor to prevent writes during migration
systemctl stop voipmonitor
 
# Move existing data (this may take a long time)
mv /var/spool/voipmonitor/* /mnt/pcaps/voipmonitor/
 
# Restart the sensor
systemctl start voipmonitor
</syntaxhighlight>


=== Retention Configuration ===
Limits can be set by '''size''' (MB) or '''age''' (days). When both are configured, the first limit reached triggers cleanup.


The cleaning process runs automatically every 5 minutes and removes the oldest data based on the rules you define in <code>voipmonitor.conf</code>. You can set limits based on total size (in Megabytes) or age (in days). If both a size and day limit are set for the same data type, the first limit that is reached will trigger the cleaning.
==== Global Limits ====


{| class="wikitable"
{| class="wikitable"
|-
|-
! Parameter !! Default Value !! Description
! Parameter !! Default !! Description
|-
| <code>maxpoolsize</code> || 102400 (100 GB) || The total maximum disk space for '''all''' captured data (SIP, RTP, GRAPH, AUDIO).
|-
| <code>maxpooldays</code> || (unset) || The maximum number of days to keep '''all''' captured data.
|-
|-
| <code>maxpoolsipsize</code> || (unset) || A specific size limit for SIP PCAP files only.
| <code>maxpoolsize</code> || 102400 (100 GB) || Maximum total size for all PCAP data
|-
|-
| <code>maxpoolsipdays</code> || (unset) || A specific age limit for SIP PCAP files only.
| <code>maxpooldays</code> || (unset) || Maximum age in days for all PCAP data
|-
| <code>maxpoolrtpsize</code> || (unset) || A specific size limit for RTP PCAP files only.
|-
| <code>maxpoolrtpdays</code> || (unset) || A specific age limit for RTP PCAP files only.
|-
| <code>maxpoolgraphsize</code> || (unset) || A specific size limit for GRAPH files only.
|-
| <code>maxpoolgraphdays</code> || (unset) || A specific age limit for GRAPH files only.
|-
| <code>maxpoolaudiosize</code> || (unset) || A specific size limit for converted audio files (WAV/OGG) only.
|-
| <code>maxpoolaudiodays</code> || (unset) || An age limit for converted audio files (WAV/OGG) only.
|}
|}


=== Understanding Directory Size Differences (SIP vs RTP Retention) ===
==== Per-Type Limits ====
 
If you configure different retention periods for SIP and RTP data (for example, <code>maxpoolsipdays = 90</code> and <code>maxpoolrtpdays = 30</code>), you will notice a significant size difference between directories within different time windows.


{| class="wikitable"
{| class="wikitable"
|-
|-
! Age of Directory !! Contents !! Size
! Data Type !! Size Parameter !! Days Parameter
|-
| SIP signaling || <code>maxpoolsipsize</code> || <code>maxpoolsipdays</code>
|-
|-
| 0-30 days || Both SIP and RTP PCAP files || Large
| RTP audio || <code>maxpoolrtpsize</code> || <code>maxpoolrtpdays</code>
|-
|-
| 30-90 days || SIP PCAP files only (RTP deleted) || Smaller
| Quality graphs || <code>maxpoolgraphsize</code> || <code>maxpoolgraphdays</code>
|-
|-
| 90+ days || None (both SIP and RTP deleted) || Empty or absent
| Converted audio (WAV/OGG) || <code>maxpoolaudiosize</code> || <code>maxpoolaudiodays</code>
|}
|}


This behavior is '''expected and by design''':
==== Recommended Configuration ====


* Directories within the <code>maxpoolrtpdays</code> retention window contain both SIP and RTP data, making them significantly larger (often 5-10x larger).
Limit RTP (largest files) while keeping SIP longer for troubleshooting:
* Directories older than <code>maxpoolrtpdays</code> but within <code>maxpoolsipdays</code> contain only SIP data, so they are much smaller.
* The automatic cleanup process removes RTP files after <code>maxpoolrtpdays</code> and SIP files after <code>maxpoolsipdays</code>.


This is not an error or configuration issue. It is the expected result of having different retention periods for different data types.
<syntaxhighlight lang="ini">
 
# /etc/voipmonitor.conf
==== Diagnosis: Compare Directory Sizes ====
maxpoolrtpsize = 102400  # 100 GB limit for RTP
 
maxpoolsize = 512000      # 500 GB overall limit
To verify this behavior, check your spool directory:
 
<syntaxhighlight lang="bash">
cd /var/spool/voipmonitor
 
# Show directories sorted by size
du -h --max-depth=1 ./ | sort -rh | head -20
 
# Example output:
# 80G    ./2025-01          # Current month (has SIP+RTP)
# 15G    ./2024-12          # Previous month (SIP only, RTP deleted)
# 120G  .
</syntaxhighlight>
</syntaxhighlight>


Compare with your configuration:
==== Secondary Storage Limits (spooldir_2) ====


<syntaxhighlight lang="bash">
When using <code>spooldir_2</code> for secondary storage (configured via [[Capture_rules|capture rules]] with "Store pcaps to second spooldir" enabled), use <code>_2</code> suffix parameters for cleaning:
grep -E "maxpoolsip|maxpoolrtp" /etc/voipmonitor.conf


# Example configuration:
<code>maxpoolsize_2</code>, <code>maxpooldays_2</code>, <code>maxpoolrtpsize_2</code>, etc.
# maxpoolsipdays = 90    # SIP kept for 90 days
# maxpoolrtpdays = 30    # RTP kept for 30 days
</syntaxhighlight>


If the size difference matches your retention configuration, this is expected behavior and no action is needed.
{{Note|1=<code>spooldir_2</code> is '''independent''' from <code>tar_move</code>. The <code>_2</code> suffix parameters apply only to the secondary spooldir configured via capture rules, not to <code>tar_move</code> destination storage.}}


=== Troubleshooting: Disk Full / Files Disappearing ===
=== Emergency Cleanup ===


If you see errors when attempting to extract older calls from the GUI, or if call files are disappearing too quickly, your spool directory may have reached its size limit.
{{Warning|1=Emergency cleanup activates when disk is nearly full and '''ignores all <code>maxpool*</code> settings'''.}}


==== Diagnosis: Check Disk Usage ====
{| class="wikitable"
 
# Identify the sensor/probe responsible for the missing data.
# SSH into the sensor/probe and navigate to the spooldir.
# Check the disk usage:
<syntaxhighlight lang="bash">
cd /var/spool/voipmonitor
du -h --max-depth=1 ./
 
# Example output:
# 150G    ./2025-01
# 120G    ./2024-12
# 90G    ./2024-11
# 360G    .
</syntaxhighlight>
 
# Compare with the configured limit:
<syntaxhighlight lang="bash">
grep maxpoolsize /etc/voipmonitor.conf
# Example output: maxpoolsize = 102400  (100 GB in MB)
</syntaxhighlight>
 
==== Resolution: Increase Spooldir Size ====
 
If the actual usage exceeds the configured limit, increase <code>maxpoolsize</code>:
 
<syntaxhighlight lang="ini">
# Edit /etc/voipmonitor.conf
[general]
maxpoolsize = 716800  # 700 GB in MB
maxpooldays = 90      # Optional: Keep data for last 90 days
</syntaxhighlight>
 
Apply changes:
<syntaxhighlight lang="bash">
systemctl restart voipmonitor
</syntaxhighlight>
 
==== Emergency Cleanup Triggers ====
 
Even when your <code>maxpooldays</code> setting appears correct, PCAP files may still be deleted prematurely due to '''emergency cleanup triggers'''. These are safety mechanisms that override normal retention settings to prevent system crashes from disk exhaustion.
 
{| class="wikitable" style="background-color: #FFF3CD;"
|-
|-
! colspan="2" style="background: #856404; color: white;" | Common Cause: Retention Not Working as Expected
! Parameter !! Default !! Triggers When
|-
|-
| style="vertical-align: top;" | '''Symptom:'''
| <code>autocleanspoolminpercent</code> || 1 || Disk usage reaches 99%
| You set <code>maxpooldays = 60</code>, but files are deleted after 2 weeks (14 days).
|-
|-
| style="vertical-align: top;" | '''Root Cause:'''
| <code>autocleanmingb</code> || 5 || Free space below 5 GB
| Emergency cleanup triggered by <code>autocleanspoolminpercent</code> or <code>autocleanmingb</code>.
|}
|}


Emergency cleanup is activated when either of these conditions is met:
When triggered, oldest data is deleted aggressively until thresholds are cleared. The <code>cleanspool_enable_fromto</code> time window is ignored.
 
=== Control Parameters ===


{| class="wikitable"
{| class="wikitable"
|-
|-
! Parameter !! Default !! Description !! Behavior
! Parameter !! Default !! Description
|-
|-
| <code>autocleanspoolminpercent</code> || 1 || Minimum free disk percentage that triggers emergency cleanup | When the partition reaches this threshold (e.g., 99% full), aggressive cleaning starts ''immediately'' regardless of <code>maxpooldays</code> setting.
| <code>cleanspool</code> || yes || Enable/disable spool cleaning
|-
|-
| <code>autocleanmingb</code> || 5 || Minimum free disk space in GB that triggers emergency cleanup | When free space falls below this value (e.g., less than 5 GB left), emergency cleaning is triggered.
| <code>cleanspool_enable_fromto</code> || 0-24 || Restrict cleaning to hours (e.g., <code>1-5</code> for 1-5 AM)
|-
| <code>maxpool_clean_obsolete</code> || no || Delete files not in index (use with caution)
|-
| <code>all_unlink_log</code> || no || Log all file deletions
|}
|}


===== How Emergency Cleanup Works =====
=== Reducing Data at Source ===


The cleaning process checks disk space before each operation:
==== Save RTP Headers Only ====


<syntaxhighlight lang="bash">
If you only need quality metrics (MOS, jitter, packet loss) without audio playback:
# Check current disk usage
df -h /var/spool/voipmonitor


# Example output:
<syntaxhighlight lang="ini">
# Filesystem      Size  Used Avail Use% Mounted on
savertp = header
# /dev/sdb1      2.0T  1.96T  40G  98% /var/spool/voipmonitor
</syntaxhighlight>
</syntaxhighlight>


If <code>Use%</code exceeds <code>100 - autocleanspoolminpercent</code> (e.g., 99%), emergency cleanup is triggered.
This reduces storage by up to '''90%''' while preserving all quality statistics.
 
'''Emergency cleanup behavior:'''
* Files are deleted starting from the oldest until the emergency threshold is cleared
* This happens regardless of your <code>maxpooldays</code> or <code>maxpoolsize</code> settings
* The normal cleaning cycle (every 5 minutes) is triggered immediately
* Once the threshold is cleared, normal retention rules resume
 
===== Diagnosis: Check Emergency Cleanup Triggers =====
 
<syntaxhighlight lang="bash">
# Check if emergency triggers are set
grep -E "autocleanspoolminpercent|autocleanmingb" /etc/voipmonitor.conf
 
# Check current disk usage
df -h /var/spool/voipmonitor


# Check available space on the partition
# If Avail is close to autocleanmingb (5GB), emergency cleanup may be triggering
</syntaxhighlight>


===== Resolution: Adjust Emergency Triggers or Increase Disk Space =====


If emergency cleanup is causing premature deletion, you have two options:
==== Disable RTP Entirely ====


;Option 1: Adjust Emergency Triggers
If SIP signaling retention is the priority and you do not need RTP data at all:
Increase the thresholds to allow more data before emergency cleanup triggers:


<syntaxhighlight lang="ini">
<syntaxhighlight lang="ini">
# Edit /etc/voipmonitor.conf
savertp = no
[general]
</syntaxhighlight>
maxpooldays = 60


# Reduce emergency sensitivity (allow partition to fill more before trigger)
This disables RTP packet storage entirely, reducing PCAP size by approximately '''20x'''. Use this when:
autocleanspoolminpercent = 5  # Default is 1, increase to 5 to allow 95% usage
* Storage limits are insufficient for even header-only RTP
autocleanmingb = 20          # Default is 5, increase to 20 GB
* SIP troubleshooting is the primary need (no audio analysis required)
</syntaxhighlight>
* You need to retain SIP signaling for much longer periods


Apply changes:
{{Warning|Setting <code>savertp = no</code> removes all RTP data including quality statistics. Use <code>savertp = header</code> instead if you need MOS, jitter, or packet loss metrics.}}
<syntaxhighlight lang="bash">
==== Selective Recording ====
systemctl restart voipmonitor
</syntaxhighlight>


;Option 2: Increase Disk Space
To record full audio only for specific calls:
If the partition is genuinely full, add more storage or move the spooldir to a larger partition (see [[#Relocating_the_Spool_Directory_to_a_Different_Partition|Relocating Spooldir]]).
# Set <code>savertp = header</code> globally
# Create capture rules with <code>recordRTP=ON</code> for exceptions


===== Example Troubleshooting Scenario =====
See [[Capture_rules]] for details.


'''User report:''' "I set <code>maxpooldays = 60</code> for 60-day retention, but PCAPs are deleted after only 14 days."
== Database Cleaning (CDR Records) ==


'''Investigation:'''
=== Partitioning Method ===
<syntaxhighlight lang="bash">
# 1. Check retention settings
grep maxpooldays /etc/voipmonitor.conf
# maxpooldays = 60  ✓ Correct


# 2. Check disk usage
VoIPmonitor uses daily partitioning. Dropping old partitions is instant (milliseconds) regardless of row count.
df -h /var/spool/voipmonitor
# Filesystem      Size  Used Avail Use% Mounted on
# /dev/sdb1      500G  498G  2G  99.6% /var/spool/voipmonitor


# 3. Check emergency triggers
<syntaxhighlight lang="ini">
grep autoclean /etc/voipmonitor.conf
# Keep CDR records for 30 days
# autocleanspoolminpercent = 1 (default)
cleandatabase = 30
# autocleanmingb = 5 (default)
</syntaxhighlight>
</syntaxhighlight>


'''Root Cause:'''
=== Database Cleaning Parameters ===
* Partition is 99.6% full, which exceeds the 99% threshold (100 - 1)
* Only 2 GB free space remains, below the 5 GB <code>autocleanmingb</code> limit
* Emergency cleanup triggers, deleting files until at least 5 GB is free
 
'''Solution:'''
* Increase <code>autocleanspoolminpercent = 5</code> to allow 95% usage before emergency trigger
* Or increase <code>autocleanmingb = 20</code> to allow more data before cleanup
* Or add more disk space to the partition
 
For the complete parameter reference, see [[Sniffer_configuration#Spool_Cleaning|Sniffer Configuration - Spool Cleaning]].
 
=== Custom Autocleaning: One-Time Cleanup with Filters ===
 
The GUI provides a '''custom autocleaning''' feature that allows you to perform one-time cleanup of existing recordings based on specific criteria, such as IP address or telephone number. This is useful when you need to clean up data for a specific subset of calls without affecting global retention settings.
 
==== Use Case: Cleaning Old Recordings for a Specific IP ===
 
After configuring [[Capture_rules|capture rules]] to stop recording future calls from a specific IP address, you may still have existing RTP recordings from that IP. Custom autocleaning allows you to remove these old recordings.
 
Example scenario:
* You configured a capture rule to discard RTP for IP <code>192.168.1.50</code>
* Only new calls will be affected by this rule
* Existing recordings for this IP must be cleaned up manually
 
GUI cleanup steps:
# Navigate to '''Settings > Custom Autocleaning'''
# Create a new autocleaning rule
# Set the criteria (e.g., "Delete RTP older than 1 day")
# In the '''Common Filter''' section, specify the target IP address (<code>192.168.1.50</code>)
# Save and apply the rule
# Once the cleanup is complete, remove the autocleaning rule
 
This rule will run once and clean up all matching old recordings, then the capture rule will prevent future recordings.
 
==== Comparison with Global Retention ===


{| class="wikitable"
{| class="wikitable"
|-
|-
! Feature !! Custom Autocleaning !! Global Retention
! Parameter !! Default !! Description
|-
|-
| '''Scope''' || Targeted (specific IP, number, filter) || All calls
| <code>cleandatabase</code> || 0 (disabled) || Global retention for CDR, register_state, register_failed, sip_msg
|-
|-
| '''Purpose''' || One-time cleanup of existing data || Ongoing automated cleanup
| <code>cleandatabase_cdr</code> || 0 || CDR table (includes <code>message</code> table)
|-
|-
| '''Configuration''' || GUI with CDR filters | <code>maxpoolsize</code>, <code>maxpooldays</code>
| <code>cleandatabase_rtp_stat</code> || 2 || RTP statistics table
|-
|-
| '''Flexibility''' || High - can use any CDR filter criteria | Low - time/size only
| <code>cleandatabase_register_state</code> || 0 || Registration state
|}
 
==== Filters Available in Custom Autocleaning ===
 
The custom autocleaning interface supports the same filter options as the CDR view, including:
 
* '''IP filters''' - Source or destination IP addresses
* '''Telephone number filters''' - Caller or called numbers
* '''Time-based filters''' - Calls older than X days
* '''Other CDR criteria''' - Duration, MOS, packet loss, etc.
 
For details on using these filters, see [[Call_Detail_Record_-_CDR#Filtering_Calls|CDR Filtering]].
 
==== Important Notes ===
 
* Custom autocleaning rules should typically be deleted after they have run once
* Leaving a rule active will continuously clean up matching data, which may not be desired
* The cleanup affects PCAP files stored on disk, not CDR records in the database
* For database cleanup, use the global <code>cleandatabase</code> parameters
 
=== How Cleanspool Works: Filesystem Index vs Database ===
 
It is important to understand that the <code>cleanspool</code> feature operates on the '''filesystem, not the database'''. This is a critical distinction for configuring data retention correctly.
 
==== Filesystem-Based Cleaning ====
 
The cleanspool process maintains its own index in the <code>.cleanspool_cache</code> file located in the spool directory. This filesystem index is completely independent of the VoIPmonitor database.
 
{| class="wikitable"
|-
|-
! Aspect !! Cleanspool (Filesystem) !! Database Cleaning
| <code>cleandatabase_register_failed</code> || 0 || Failed registrations
|-
|-
| '''Where it operates''' || Deletes PCAP files from disk || Deletes CDR records from MySQL
| <code>cleandatabase_register_time_info</code> || 0 || Registration timing ('''NOT''' covered by global)
|-
|-
| '''Index used''' || <code>.cleanspool_cache</code> (filesystem scan) || Database tables/partitions
| <code>cleandatabase_sip_msg</code> || 0 || SIP messages (OPTIONS/SUBSCRIBE/NOTIFY)
|-
|-
| '''Retention limits''' || <code>maxpoolsize</code>, <code>maxpooldays</code> || <code>cleandatabase</code>, <code>cleandatabase_size</code>
| <code>cleandatabase_ss7</code> || 0 || SS7 records
|-
|-
| '''File selection criteria''' || Based on filesystem index, NOT database records || Based on <code>calldate</code> in cdr table
| <code>partition_operations_enable_fromto</code> || 1-5 || Time window for partition operations (24h format)
|}
|}


==== Key Behavior: Files Not in Database ====
{{Warning|1=<code>register_time_info</code> is NOT covered by global <code>cleandatabase</code>. Set <code>cleandatabase_register_time_info</code> explicitly.}}


The cleanspool process will delete files based solely on the filesystem cache index and the <code>maxpoolsize</code>/<code>maxpooldays</code> limits. This means:
=== Size-Based Cleaning ===


* Files '''will be deleted even if they are not in the database''' (orphaned PCAPs)
To limit database by size instead of time:
* Files '''will be deleted even if database records still exist''' (e.g., after database backup/restore)
* To protect specific data, you must move it '''outside the spool directory entirely''', not just to a different subfolder


==== The .cleanspool_cache File ====
<syntaxhighlight lang="ini">
cleandatabase_size = 512000        # 500 GB limit in MB
cleandatabase_size_force = true    # Required to enable
</syntaxhighlight>


When cleanspool runs, it scans the spool directory structure (<code>YYYY-MM-DD/HH</code>) and indexes all PCAP files into <code>.cleanspool_cache</code>. During the next cleaning cycle, it uses this cache to:
=== Multi-Sensor Environments ===


1. Sort files by age (oldest first)
When multiple sensors share a database, only '''ONE''' sensor should manage partitions:
2. Delete files until <code>maxpoolsize</code> or <code>maxpooldays</code> limits are met


Important behaviors:
<syntaxhighlight lang="ini">
# On all sensors EXCEPT one:
disable_partition_operations = yes


* By default (<code>maxpool_clean_obsolete = no</code>), cleanspool only deletes files that are indexed in <code>.cleanspool_cache</code>
# On the designated sensor:
* Files that appear but are not in the cache are '''ignored''' until the next full scan updates the cache
partition_operations_enable_fromto = 4-6
* With <code>maxpool_clean_obsolete = yes</code>, cleanspool will also delete any files in the spool directory that are not in the cache
</syntaxhighlight>
 
{| class="wikitable" style="background-color: #FFF3CD;"
|-
! colspan="2" style="background: #856404; color: white;" | Important: Protect Data Moves
|-
| style="vertical-align: top;" | '''<code>maxpool_clean_obsolete = no</code> (default):'''
|
If you manually move files into the spool directory, they will not be indexed immediately. Cleanspool will ignore them until a full scan updates the cache. To prevent deletion, move data outside the spool directory entirely, not to a different subfolder within the spool.
|}


==== Configuration Parameter Reference ====
=== Limits ===


; <code>maxpool_clean_obsolete = no</code> (default)
* '''MySQL/MariaDB partition limit:''' ~8000 partitions per table (~22 years with daily partitioning)
: Cleanspool only deletes files that are indexed in the <code>.cleanspool_cache</code> filesystem index. Files that are not in the cache are ignored.
* '''Aurora DB partition limit:''' ~800 partitions per table - much lower than standard MySQL
* '''CDR record limit:''' No practical limit - uses BIGINT


; <code>maxpool_clean_obsolete = yes</code>
=== Aurora DB / Limited Partition Environments ===
: Cleanspool will delete ALL files in the spool directory, including those not found in the cache. This is useful if you have manually moved files or want to clean orphaned files.


'''Warning:''' This parameter controls filesystem cache behavior, not database record matching. Cleanspool always operates independently of the database.
For databases with low partition limits (like Amazon Aurora), you can pre-create partitions before starting the sniffer and prevent runtime partition creation:


=== Maintenance: Re-indexing the Spool Directory ===
<syntaxhighlight lang="ini">
# Pre-create partitions up to a specific date before starting
create_new_partitions_until = 2026-01-31


VoIPmonitor maintains an index of all created PCAP files to perform cleaning efficiently without scanning the entire directory tree. If this index becomes corrupt, or if you manually move files into the spool, old data may not be deleted correctly.
# Disable only partition creation (dropping still works for cleanup)
disable_partition_operations_create = yes


To trigger a manual re-index via the manager API:
# Or disable only partition dropping (creation still works)
# disable_partition_operations_drop = yes


<syntaxhighlight lang="bash">
# Number of partitions to create ahead (default 2)
# Open a manager API session
create_new_partitions = 2
echo 'manager_file start /tmp/vmsck' | nc 127.0.0.1 5029
</syntaxhighlight>


# Send the re-index command
{{Note|1=For Aurora DB with ~800 partition limit: with daily partitions, this gives ~2 years of data. Plan retention accordingly using <code>cleandatabase</code>.}}
echo reindexfiles | nc -U /tmp/vmsck
</syntaxhighlight>


Note: This command requires <code>netcat</code> with support for UNIX sockets (<code>-U</code>). For alternative methods, see the [[Encryption_in_manager_api_customer|Manager API documentation]].
== Advanced Topics ==


== Database Cleaning (CDR Retention) ==
=== Spool Directory Location ===


Managing the size of the <code>cdr</code> table and other large tables is critical for GUI performance.
Default: <code>/var/spool/voipmonitor</code>


=== Partitioning Method (Recommended) ===
Structure: <code>YYYY-MM-DD/HH/MM/{SIP|RTP|GRAPH|AUDIO}/files...</code>


Since version 7, VoIPmonitor utilizes '''database partitioning''', which splits large tables into smaller, daily segments. This is the recommended method for managing database retention.
==== Relocating the Spool ====


{| class="wikitable" style="width:100%;"
<syntaxhighlight lang="bash">
|-
# 1. Create new directory
! Aspect !! Description
mkdir -p /mnt/storage/voipmonitor
|-
chown voipmonitor:voipmonitor /mnt/storage/voipmonitor
| '''How it works''' || Set <code>cleandatabase = 30</code> in <code>voipmonitor.conf</code> to keep the last 30 days of data.
|-
| '''Why it's better''' || Dropping old partitions is instantaneous (milliseconds), regardless of row count. Zero database load.
|-
| '''Requirement''' || Partitioning is enabled by default on new installations.
|}


==== Quick Start: Global Retention ====
# 2. Update sniffer config
# /etc/voipmonitor.conf:
# spooldir = /mnt/storage/voipmonitor


For most deployments, configure one parameter in <code>voipmonitor.conf</code>:
# 3. Update GUI config (config/configuration.php):
# define('SNIFFER_DATA_PATH', '/mnt/storage/voipmonitor');


<syntaxhighlight lang="ini">
# 4. Restart
# Keep all records for 30 days
systemctl restart voipmonitor
cleandatabase = 30
</syntaxhighlight>
</syntaxhighlight>


The <code>cleandatabase</code> parameter acts as a global default for all <code>cleandatabase_*</code> options and applies to:
=== Tiered Storage (tar_move) ===
* <code>cdr</code> - Call Detail Records
* <code>message</code> - SIP MESSAGE texts
* <code>sip_msg</code> - SIP OPTIONS/SUBSCRIBE/NOTIFY messages
* <code>register_state</code> - SIP registration states
* <code>register_failed</code> - Failed registration attempts


==== Retention Parameters ====
Extend retention using secondary storage:


{| class="wikitable"
<syntaxhighlight lang="ini">
|-
spooldir = /var/spool/voipmonitor
! Parameter !! Default !! Description
tar_move = yes
|-
tar_move_destination_path = /mnt/archive/voipmonitor
| <code>cleandatabase</code> || 0 (disabled) || Master retention setting in days.
</syntaxhighlight>
|-
| <code>cleandatabase_cdr</code> || 0 || Specific retention for <code>cdr</code> and <code>message</code> tables.
|-
| <code>cleandatabase_rtp_stat</code> || 2 || Retention for detailed RTP statistics.
|-
| <code>cleandatabase_sip_msg</code> || 0 || Retention for OPTIONS/SUBSCRIBE/NOTIFY.
|-
| <code>cleandatabase_size</code> || (unset) || Alternative: size-based limit in MB (requires version 2024.05.1+).
|-
| <code>partition_operations_enable_fromto</code> || 1-5 || Time window for partition operations (e.g., 1-5 AM).
|}


===== Size-Based Database Cleaning =====
Files in secondary storage remain accessible via GUI.


In addition to time-based retention, VoIPmonitor supports size-based database cleanup using <code>cleandatabase_size</code>. This is useful when you want to limit the database size regardless of data age.
==== S3 Cloud Storage ====


<syntaxhighlight lang="ini">
Use <code>rclone</code> instead of <code>s3fs</code> to avoid GUI unresponsiveness:
# Limit database to 50 GB
cleandatabase_size = 51200


# Force immediate cleanup if size exceeded (required for size-based)
<syntaxhighlight lang="bash">
cleandatabase_size_force = true
rclone mount bucket-name /mnt/s3-archive \
  --allow-other --dir-cache-time 30s --vfs-cache-mode off
</syntaxhighlight>
</syntaxhighlight>


{| class="wikitable"
=== Custom Autocleaning (GUI) ===
|-
! Parameter !! Description
|-
| <code>cleandatabase_size</code> || Maximum database size in MB. When exceeded, the oldest partitions are dropped until the size is below this limit.
|-
| <code>cleandatabase_size_force</code> || Must be set to <code>true</code> to enable size-based cleanup. Without this, <code>cleandatabase_size</code> will not trigger cleanup.
|}


'''Behavior:'''
For one-time cleanup of specific recordings (by IP, phone number, etc.):
* The size check is performed daily during partition operations
# Navigate to Settings > Custom Autocleaning
* When the size exceeds <code>cleandatabase_size</code> + 5% buffer, the oldest partitions are dropped
# Create rule with filters
* This ensures the database stays within the specified size limit
# Apply and remove rule after completion
* Works independently of <code>cleandatabase</code> (time-based retention)


'''Note:''' Size-based cleanup only affects partitioned tables. Non-partitioned tables are not affected.
== Troubleshooting ==


More details: [[Sniffer_configuration#Database_Cleaning|Sniffer Configuration - Database Cleaning]].
=== Files Disappearing Faster Than Expected ===


=== Legacy Method: Manual Deletion (Not Recommended) ===
;1. Check if emergency cleanup is active
<syntaxhighlight lang="bash">
df -h /var/spool/voipmonitor
# If >95% full, emergency cleanup is running
</syntaxhighlight>


For very old, non-partitioned databases, you would need custom scripts with <code>DELETE FROM cdr WHERE calldate < ...</code> queries.
;2. Check GUI configuration override
When <code>mysqlloadconfig = yes</code> (default), GUI settings override config file. Check: Settings > Sensors > wrench icon.


'''Warning:''' Manual DELETE on large tables is extremely slow and resource-intensive. A single operation on millions of rows can take hours and impact GUI performance.
;3. Set appropriate limits
Set <code>maxpoolsize</code> to 90-95% of disk capacity to leave buffer for growth.


== Troubleshooting Disk Space Issues ==
=== Disk Space Not Reclaimed After Database Cleanup ===


=== Disk Space Not Reclaimed After Cleanup ===
Check <code>innodb_file_per_table</code>:
 
If automatic cleanup runs but disk space is not freed from the MySQL data directory, check the <code>innodb_file_per_table</code> setting:


<syntaxhighlight lang="sql">
<syntaxhighlight lang="sql">
Line 676: Line 347:
</syntaxhighlight>
</syntaxhighlight>


{| class="wikitable"
If OFF, enable in <code>my.cnf</code>:
|-
! Value !! Behavior
|-
| '''ON''' || Each table/partition has its own <code>.ibd</code> file. Dropping partitions reclaims space immediately.
|-
| '''OFF''' || All data in shared <code>ibdata1</code> file. Dropping partitions does '''not''' reduce file size.
|}


==== Solutions ====
;Option 1: Enable for Future Tables
Add to <code>/etc/my.cnf</code> or <code>/etc/mysql/my.cnf</code>:
<syntaxhighlight lang="ini">
<syntaxhighlight lang="ini">
[mysqld]
[mysqld]
innodb_file_per_table = 1
innodb_file_per_table = 1
</syntaxhighlight>
</syntaxhighlight>
<syntaxhighlight lang="bash">
systemctl restart mysql
</syntaxhighlight>
Note: This only affects NEW tables/partitions. Existing data in <code>ibdata1</code> remains.


;Option 2: Reclaim Space from Existing Tables
=== MySQL Error 28: No Space Left ===
<syntaxhighlight lang="sql">
 
OPTIMIZE TABLE cdr;
Enable size-based cleaning:
</syntaxhighlight>
'''Warning:''' Requires significant free disk space to duplicate table data. May crash if disk is nearly full.


;Option 3: Export and Re-import
<syntaxhighlight lang="ini">
<syntaxhighlight lang="bash">
cleandatabase_size = 512000
mysqldump -u root -p voipmonitor > voipmonitor_backup.sql
cleandatabase_size_force = true
mysql -u root -p -e "DROP DATABASE voipmonitor; CREATE DATABASE voipmonitor;"
mysql -u root -p voipmonitor < voipmonitor_backup.sql
</syntaxhighlight>
</syntaxhighlight>


=== Monitoring Database Health ===
Other causes: Inode exhaustion (<code>df -i</code>), MySQL tmpdir full.


==== SQL Queue Metrics ====
=== Verify Database Cleaning ===


The sensor tracks queue metrics visible in GUI → Settings → Sensors → Status:
<syntaxhighlight lang="sql">
 
SELECT PARTITION_NAME, TABLE_ROWS
{| class="wikitable"
FROM information_schema.PARTITIONS
|-
WHERE TABLE_NAME = 'cdr'
! Metric !! Description !! Healthy Range
ORDER BY PARTITION_ORDINAL_POSITION DESC
|-
LIMIT 10;
| '''SQLq''' || CDRs waiting to be written to database || Near 0, sporadic spikes OK
|-
| '''SQLf''' || Failed database write attempts || Zero (not growing)
|}
 
* Consistently high/growing SQLq → database cannot keep up
* Non-zero/growing SQLf → database errors or connectivity issues
 
See [[SQL_queue_is_growing_in_a_peaktime|SQL Queue Troubleshooting]] for details.
 
==== System Monitoring ====
 
<syntaxhighlight lang="bash">
# Check system load
cat /proc/loadavg
 
# Monitor disk I/O (shows only active processes)
iotop -o
</syntaxhighlight>
</syntaxhighlight>


High I/O from <code>mysqld</code> processes may indicate slow storage or poorly tuned MySQL settings.
If partition count matches your <code>cleandatabase</code> setting, cleaning IS working.


== MySQL Performance Settings ==
For SQL queue issues and database performance, see [[Database_troubleshooting]].


For high-performance operation with partitioning:
== See Also ==


<syntaxhighlight lang="ini">
* [[Sniffer_configuration]] - Full parameter reference
[mysqld]
* [[Database_troubleshooting]] - SQL queue, performance issues
# Use 50-70% of available RAM for caching
* [[Scaling]] - Performance tuning
innodb_buffer_pool_size = 4G
* [[Capture_rules]] - Selective recording


# Flush logs to OS every second (faster, safe for VoIPmonitor)
innodb_flush_log_at_trx_commit = 2


# Enable per-table filespace for easy space reclamation
innodb_file_per_table = 1
</syntaxhighlight>


For comprehensive tuning, see [[Scaling|Scaling and Performance Guide]].
== AI Summary for RAG ==


== See Also ==
'''Summary:''' VoIPmonitor has two independent data retention systems: (1) Cleanspool for PCAP files using <code>maxpoolsize</code>/<code>maxpooldays</code> parameters, running every 5 minutes with in-memory file index; (2) Database cleaning using <code>cleandatabase</code> with instant partition dropping. Key space-saving option: <code>savertp = header</code> reduces storage by 90% while keeping quality metrics. Emergency cleanup (<code>autocleanspoolminpercent=1</code>, <code>autocleanmingb=5</code>) activates when disk nearly full and ignores normal limits. GUI settings override config file when <code>mysqlloadconfig=yes</code>. Size-based database cleaning requires BOTH <code>cleandatabase_size</code> AND <code>cleandatabase_size_force=true</code>. In multi-sensor environments, only one sensor should manage partitions (<code>disable_partition_operations=yes</code> on others).
 
* [[Sniffer_configuration|Sniffer Configuration Reference]]
* [[Scaling|Scaling and Performance Guide]]
* [[SQL_queue_is_growing_in_a_peaktime|SQL Queue Troubleshooting]]
* [[Sniffer_troubleshooting|Sniffer Troubleshooting]]
 
== AI Summary for RAG ==
'''Summary:''' VoIPmonitor has multiple data retention mechanisms: (1) Filesystem cleaning for PCAP files using <code>maxpoolsize</code>/<code>maxpooldays</code> parameters, running every 5 minutes to delete oldest data first; (2) Database cleaning using <code>cleandatabase</code> parameter with daily partitioning for instant partition drops; (3) Custom autocleaning in GUI for one-time targeted cleanup of specific recordings based on filters (IP, phone number, duration, etc.). Key behavior: directories within <code>maxpoolrtpdays</code> window contain both SIP and RTP (large), while directories within <code>maxpoolsipdays</code> but beyond <code>maxpoolrtpdays</code> contain only SIP (smaller) - this is expected. Custom autocleaning is useful for cleaning up old recordings for specific IPs after configuring capture rules to stop future recording (capture rules only affect new calls, existing recordings must be cleaned separately). Cleanspool operates on the filesystem, NOT the database. The cleanspool process maintains its own index in <code>.cleanspool_cache</code> file and deletes files based solely on this filesystem index and <code>maxpoolsize</code>/<code>maxpooldays</code> limits. Files are deleted even if they are not in the database. The <code>maxpool_clean_obsolete</code> parameter (default: no) controls whether cleanspool deletes files not found in the cache. Troubleshooting covers disk full scenarios (check with <code>du -h --max-depth=1</code>, increase <code>maxpoolsize</code>), space not reclaimed issues (<code>innodb_file_per_table</code> setting), and database health monitoring (SQLq/SQLf metrics). Additional topics: relocating spooldir to different partition, syncing GUI <code>SNIFFER_DATA_PATH</code> configuration, size-based database cleaning via <code>cleandatabase_size</code> and <code>cleandatabase_size_force</code>, saving only RTP headers with <code>savertp=header</code> (up to 90% storage reduction), critical architecture consideration for <code>packetbuffer_sender</code> mode (apply settings on central server, not sensors). Emergency cleanup triggers: <code>autocleanspoolminpercent</code> (default 1%, triggers at 99% disk usage) and <code>autocleanmingb</code> (default 5GB, triggers when free space below 5GB) can override retention settings, causing premature file deletion even when <code>maxpooldays</code> is set correctly.


'''Keywords:''' data retention, cleaning, delete old calls, disk space, spooldir, maxpoolsize, maxpooldays, maxpoolsipdays, maxpoolrtpdays, cleandatabase, partitioning, cleandatabase_size, cleandatabase_size_force, reindexfiles, innodb_file_per_table, SQLq, SQLf, directory size difference, SIP vs RTP retention, relocate spooldir, SNIFFER_DATA_PATH, MySQL Error 28, savertp header, packetbuffer_sender, client server mode, distributed architecture, autocleanspoolminpercent, autocleanmingb, emergency cleanup, retention not working, premature deletion, cleanspool_cache, maxpool_clean_obsolete, filesystem index, database vs filesystem, orphaned files, custom autocleaning, one-time cleanup, targeted cleanup, IP filter cleanup, clean specific IP recordings
'''Keywords:''' data retention, maxpoolsize, maxpooldays, maxpoolrtpsize, cleandatabase, cleandatabase_size, cleandatabase_size_force, cleanspool, autocleanspoolminpercent, autocleanmingb, tar_move, tiered storage, savertp header, innodb_file_per_table, partition dropping, emergency cleanup, mysqlloadconfig, disable_partition_operations


'''Key Questions:'''
'''Key Questions:'''
* How do I automatically delete old PCAP files?
* How does VoIPmonitor cleanspool work?
* What is the difference between maxpoolsize and maxpooldays?
* How to configure PCAP file retention?
* My spool directory is full but old files are not deleted - how to fix?
* How to limit database size?
* How do I configure database retention with cleandatabase?
* Why are files being deleted faster than expected?
* Why is disk space not reclaimed after MySQL cleanup?
* How to fix MySQL Error 28 no space left?
* What do SQLq and SQLf metrics mean?
* How to reduce storage usage (savertp header)?
* Why are recent directories much larger than old ones in my spool?
* How to configure tiered storage?
* Directory size difference with maxpoolsipdays and maxpoolrtpdays
* Why is disk space not reclaimed after cleanup?
* How do I move the spooldir to a larger partition?
* How to manage partitions in multi-sensor environment?
* What is cleandatabase_size and cleandatabase_size_force?
* How do I fix MySQL Error 28 "No space left on device"?
* How do I stop audio recording and save only RTP headers?
* Where should I apply savertp=header in client-server mode with packetbuffer_sender?
* How do I reduce storage by saving only RTP headers instead of full audio?
* Why are my PCAP files being deleted even though maxpooldays is set to 60?
* What are autocleanspoolminpercent and autocleanmingb?
* How do emergency cleanup triggers override retention settings?
* How do I fix premature file deletion in spool directory?
* Does cleanspool delete files not in the database?
* What is maxpool_clean_obsolete?
* How does cleanspool filesystem index differ from database records?
* Will cleanspool delete files that are not indexed in the database?
* How do I clean up old recordings for a specific IP address?
* What is custom autocleaning in VoIPmonitor GUI?
* How do I delete old recordings after configuring capture rules to stop recording?
* Can I clean up recordings based on specific filters like IP or phone number?
* What is the difference between custom autocleaning and global retention settings?

Latest revision as of 12:10, 22 January 2026


This guide explains how VoIPmonitor manages data retention for PCAP files and database records.

Overview

VoIPmonitor generates two types of data requiring periodic cleanup:

Data Type Storage Cleanup Mechanism Key Parameters
PCAP Files Filesystem (spool directory) Cleanspool process (every 5 min) maxpoolsize, maxpooldays
CDR Records MySQL database Partition dropping (instant) cleandatabase

ℹ️ Note: These are independent systems - filesystem cleanup does not affect database records and vice versa.

Quick Reference

Purpose Parameter Example
Limit total PCAP size maxpoolsize maxpoolsize = 512000 (500 GB)
Limit RTP specifically maxpoolrtpsize maxpoolrtpsize = 102400 (100 GB)
Limit PCAP age maxpooldays maxpooldays = 30
CDR retention (days) cleandatabase cleandatabase = 30
CDR retention (size) cleandatabase_size cleandatabase_size = 512000 (+ cleandatabase_size_force = true)
Save only RTP stats savertp savertp = header (saves ~90% space)

Filesystem Cleaning (PCAP Files)

How Cleanspool Works

The sniffer maintains a file index in memory. Every 5 minutes, the cleanspool thread checks retention limits and deletes the oldest files when limits are exceeded.

The .cleanspool_cache files in hourly directories serve as persistent storage for fast restart - they allow quick index reload without scanning the entire filesystem.

ℹ️ Note: Legacy Indexing: Old VoIPmonitor versions stored metadata in MySQL files table. This can be enabled with cleanspool_use_files = yes (deprecated, do not use).

Retention Configuration

Limits can be set by size (MB) or age (days). When both are configured, the first limit reached triggers cleanup.

Global Limits

Parameter Default Description
maxpoolsize 102400 (100 GB) Maximum total size for all PCAP data
maxpooldays (unset) Maximum age in days for all PCAP data

Per-Type Limits

Data Type Size Parameter Days Parameter
SIP signaling maxpoolsipsize maxpoolsipdays
RTP audio maxpoolrtpsize maxpoolrtpdays
Quality graphs maxpoolgraphsize maxpoolgraphdays
Converted audio (WAV/OGG) maxpoolaudiosize maxpoolaudiodays

Recommended Configuration

Limit RTP (largest files) while keeping SIP longer for troubleshooting: 

# /etc/voipmonitor.conf
maxpoolrtpsize = 102400   # 100 GB limit for RTP
maxpoolsize = 512000      # 500 GB overall limit

Secondary Storage Limits (spooldir_2)

When using spooldir_2 for secondary storage (configured via capture rules with "Store pcaps to second spooldir" enabled), use _2 suffix parameters for cleaning:

maxpoolsize_2, maxpooldays_2, maxpoolrtpsize_2, etc.

ℹ️ Note: spooldir_2 is independent from tar_move. The _2 suffix parameters apply only to the secondary spooldir configured via capture rules, not to tar_move destination storage.

Emergency Cleanup

⚠️ Warning: Emergency cleanup activates when disk is nearly full and ignores all maxpool* settings.

Parameter Default Triggers When
autocleanspoolminpercent 1 Disk usage reaches 99%
autocleanmingb 5 Free space below 5 GB

When triggered, oldest data is deleted aggressively until thresholds are cleared. The cleanspool_enable_fromto time window is ignored.

Control Parameters

Parameter Default Description
cleanspool yes Enable/disable spool cleaning
cleanspool_enable_fromto 0-24 Restrict cleaning to hours (e.g., 1-5 for 1-5 AM)
maxpool_clean_obsolete no Delete files not in index (use with caution)
all_unlink_log no Log all file deletions

Reducing Data at Source

Save RTP Headers Only

If you only need quality metrics (MOS, jitter, packet loss) without audio playback: 

savertp = header

This reduces storage by up to 90% while preserving all quality statistics.


Disable RTP Entirely

If SIP signaling retention is the priority and you do not need RTP data at all: 

savertp = no

This disables RTP packet storage entirely, reducing PCAP size by approximately 20x. Use this when:

  • Storage limits are insufficient for even header-only RTP
  • SIP troubleshooting is the primary need (no audio analysis required)
  • You need to retain SIP signaling for much longer periods

⚠️ Warning:

Selective Recording

To record full audio only for specific calls:

  1. Set savertp = header globally
  2. Create capture rules with recordRTP=ON for exceptions

See Capture_rules for details.

Database Cleaning (CDR Records)

Partitioning Method

VoIPmonitor uses daily partitioning. Dropping old partitions is instant (milliseconds) regardless of row count. 

# Keep CDR records for 30 days
cleandatabase = 30

Database Cleaning Parameters

Parameter Default Description
cleandatabase 0 (disabled) Global retention for CDR, register_state, register_failed, sip_msg
cleandatabase_cdr 0 CDR table (includes message table)
cleandatabase_rtp_stat 2 RTP statistics table
cleandatabase_register_state 0 Registration state
cleandatabase_register_failed 0 Failed registrations
cleandatabase_register_time_info 0 Registration timing (NOT covered by global)
cleandatabase_sip_msg 0 SIP messages (OPTIONS/SUBSCRIBE/NOTIFY)
cleandatabase_ss7 0 SS7 records
partition_operations_enable_fromto 1-5 Time window for partition operations (24h format)

⚠️ Warning: register_time_info is NOT covered by global cleandatabase. Set cleandatabase_register_time_info explicitly.

Size-Based Cleaning

To limit database by size instead of time: 

cleandatabase_size = 512000        # 500 GB limit in MB
cleandatabase_size_force = true    # Required to enable

Multi-Sensor Environments

When multiple sensors share a database, only ONE sensor should manage partitions: 

# On all sensors EXCEPT one:
disable_partition_operations = yes

# On the designated sensor:
partition_operations_enable_fromto = 4-6

Limits

  • MySQL/MariaDB partition limit: ~8000 partitions per table (~22 years with daily partitioning)
  • Aurora DB partition limit: ~800 partitions per table - much lower than standard MySQL
  • CDR record limit: No practical limit - uses BIGINT

Aurora DB / Limited Partition Environments

For databases with low partition limits (like Amazon Aurora), you can pre-create partitions before starting the sniffer and prevent runtime partition creation: 

# Pre-create partitions up to a specific date before starting
create_new_partitions_until = 2026-01-31

# Disable only partition creation (dropping still works for cleanup)
disable_partition_operations_create = yes

# Or disable only partition dropping (creation still works)
# disable_partition_operations_drop = yes

# Number of partitions to create ahead (default 2)
create_new_partitions = 2

ℹ️ Note: For Aurora DB with ~800 partition limit: with daily partitions, this gives ~2 years of data. Plan retention accordingly using cleandatabase.

Advanced Topics

Spool Directory Location

Default: /var/spool/voipmonitor

Structure: YYYY-MM-DD/HH/MM/{SIP|RTP|GRAPH|AUDIO}/files...

Relocating the Spool

# 1. Create new directory
mkdir -p /mnt/storage/voipmonitor
chown voipmonitor:voipmonitor /mnt/storage/voipmonitor

# 2. Update sniffer config
# /etc/voipmonitor.conf:
# spooldir = /mnt/storage/voipmonitor

# 3. Update GUI config (config/configuration.php):
# define('SNIFFER_DATA_PATH', '/mnt/storage/voipmonitor');

# 4. Restart
systemctl restart voipmonitor

Tiered Storage (tar_move)

Extend retention using secondary storage: 

spooldir = /var/spool/voipmonitor
tar_move = yes
tar_move_destination_path = /mnt/archive/voipmonitor

Files in secondary storage remain accessible via GUI.

S3 Cloud Storage

Use rclone instead of s3fs to avoid GUI unresponsiveness: 

rclone mount bucket-name /mnt/s3-archive \
  --allow-other --dir-cache-time 30s --vfs-cache-mode off

Custom Autocleaning (GUI)

For one-time cleanup of specific recordings (by IP, phone number, etc.):

  1. Navigate to Settings > Custom Autocleaning
  2. Create rule with filters
  3. Apply and remove rule after completion

Troubleshooting

Files Disappearing Faster Than Expected

1. Check if emergency cleanup is active
df -h /var/spool/voipmonitor
# If >95% full, emergency cleanup is running
2. Check GUI configuration override

When mysqlloadconfig = yes (default), GUI settings override config file. Check: Settings > Sensors > wrench icon.

3. Set appropriate limits

Set maxpoolsize to 90-95% of disk capacity to leave buffer for growth.

Disk Space Not Reclaimed After Database Cleanup

Check innodb_file_per_table: 

SHOW GLOBAL VARIABLES LIKE 'innodb_file_per_table';

If OFF, enable in my.cnf: 

[mysqld]
innodb_file_per_table = 1

MySQL Error 28: No Space Left

Enable size-based cleaning: 

cleandatabase_size = 512000
cleandatabase_size_force = true

Other causes: Inode exhaustion (df -i), MySQL tmpdir full.

Verify Database Cleaning

SELECT PARTITION_NAME, TABLE_ROWS
FROM information_schema.PARTITIONS
WHERE TABLE_NAME = 'cdr'
ORDER BY PARTITION_ORDINAL_POSITION DESC
LIMIT 10;

If partition count matches your cleandatabase setting, cleaning IS working.

For SQL queue issues and database performance, see Database_troubleshooting.

See Also


AI Summary for RAG

Summary: VoIPmonitor has two independent data retention systems: (1) Cleanspool for PCAP files using maxpoolsize/maxpooldays parameters, running every 5 minutes with in-memory file index; (2) Database cleaning using cleandatabase with instant partition dropping. Key space-saving option: savertp = header reduces storage by 90% while keeping quality metrics. Emergency cleanup (autocleanspoolminpercent=1, autocleanmingb=5) activates when disk nearly full and ignores normal limits. GUI settings override config file when mysqlloadconfig=yes. Size-based database cleaning requires BOTH cleandatabase_size AND cleandatabase_size_force=true. In multi-sensor environments, only one sensor should manage partitions (disable_partition_operations=yes on others).

Keywords: data retention, maxpoolsize, maxpooldays, maxpoolrtpsize, cleandatabase, cleandatabase_size, cleandatabase_size_force, cleanspool, autocleanspoolminpercent, autocleanmingb, tar_move, tiered storage, savertp header, innodb_file_per_table, partition dropping, emergency cleanup, mysqlloadconfig, disable_partition_operations

Key Questions:

  • How does VoIPmonitor cleanspool work?
  • How to configure PCAP file retention?
  • How to limit database size?
  • Why are files being deleted faster than expected?
  • How to fix MySQL Error 28 no space left?
  • How to reduce storage usage (savertp header)?
  • How to configure tiered storage?
  • Why is disk space not reclaimed after cleanup?
  • How to manage partitions in multi-sensor environment?
Retrieved from "https://www.voipmonitor.org/doc/index.php?title=Data_Cleaning&oldid=11005"