Redundant database: Difference between revisions

From VoIPmonitor.org
(Review: oprava překlepů v odkazech (instancies->instances, aktualizace odkazu na Backing_Up_GUI_Configuration))
(Restore content from rev 8644 - previous edit incorrectly overwrote with Active_calls content)
Tag: Manual revert
Line 1: Line 1:
{{DISPLAYTITLE:Active Calls (Real-time Monitoring)}}
{{DISPLAYTITLE:Database Replication using a Dedicated Sniffer Instance}}
[[Category:GUI manual]]
[[Category:GUI manual]]


This page describes the Active Calls view, which provides real-time monitoring of ongoing calls in the VoIPmonitor GUI.
This guide explains how to use a dedicated VoIPmonitor sensor instance to perform real-time replication of the CDR database to a secondary MySQL/MariaDB server. This provides a native, application-level method for creating backups or read-only replicas.


== Overview ==
== Overview & Use Case ==
VoIPmonitor offers multiple built-in mechanisms to replicate or synchronize Call Detail Records (CDRs) between instances. These methods enable datacenter failover, disaster recovery, hot-standby setups, and centralized reporting.


The Active Calls view displays current calls in real time and allows you to listen to G.711 calls live (requires sniffer version 7 or newer). The view refreshes automatically according to the configured refresh interval (default: 2 seconds).
=== Available Synchronization Methods ===


=== Communication with Sniffer ===
There are three primary approaches for synchronizing CDRs between VoIPmonitor instances:


Live call data is fetched from the VoIPmonitor sniffer instance through the manager TCP port 5029. If calls are not displayed, verify that the web server can reach this port:
;1. Database Backup Mode (Read-Only Replica)
:Use a dedicated sniffer instance to read from the primary database and write to a secondary database. This creates a one-way, read-only replica. This method is ideal for online GUI migration, hot-standby setups, or centralized reporting.


<kroki lang="mermaid">
;2. Master-Master Database Replication
flowchart LR
:Configure MySQL/MariaDB in a bidirectional replication topology. Both instances can write CDRs, and changes automatically propagate between them. This provides full high-availability with automatic failover and read load balancing. See [[Mysql_master-master_replication_hints|Master-Master Replication]] for detailed setup.
    subgraph GUI["Web GUI"]
 
        Browser["Browser"]
;3. Dual Write Method
        PHP["PHP Backend"]
:Configure the active sniffer to write CDRs simultaneously to both a local and a remote database. This provides real-time synchronization without requiring MySQL replication setup. '''Note:''' This method only synchronizes CDRs; it does not synchronize GUI settings (users, sensors, capture rules, etc.).
    end
 
    subgraph Sniffer["VoIPmonitor Sniffer"]
This article primarily covers '''Method 1 (Database Backup Mode)'''. Method 2 is covered in the [[Mysql_master-master_replication_hints|Master-Master Replication]] guide. Method 3 is described below.
        Manager["Manager API<br/>Port 5029"]
 
        Capture["Packet Capture"]
The database backup mode method is particularly useful for:
    end
* '''Online GUI Migration''': Migrating the GUI database from an old server to a new server with minimal or no downtime. The sensor replicates CDR data incrementally while users continue using the old GUI. Once replication completes, you can switch users to the new GUI without service interruption.
    Browser -->|HTTP/HTTPS| PHP
* Creating a read-only replica database for reporting or analysis without impacting the performance of the primary database.
    PHP -->|TCP 5029| Manager
* Setting up a hot-standby database for disaster recovery.
    Manager -->|Active Call Data| PHP
* Consolidating data from multiple, smaller VoIPmonitor databases into one central database.
    Capture -->|Live Data| Manager
 
The process incrementally syncs data starting from a specified date, ensuring that the destination database stays up-to-date in near real-time.
 
=== Architecture Overview ===
 
<kroki lang="plantuml">
@startuml
skinparam shadowing false
skinparam defaultFontName Arial
skinparam rectangle {
  BorderColor #4A90E2
  BackgroundColor #FFFFFF
}
 
rectangle "OLD Server" as OLD {
  rectangle "VoIPmonitor\nSniffer" as SNIFF
  database "MySQL\n(Source DB)" as SRC_DB
}
 
rectangle "NEW Server" as NEW {
  rectangle "Migration\nInstance" as MIG
  database "MySQL\n(Destination DB)" as DST_DB
}
 
SNIFF -down-> SRC_DB : writes CDRs
MIG -left-> SRC_DB : reads CDRs\n(database_backup_from_*)
MIG -down-> DST_DB : writes CDRs\n(mysql*)
 
note bottom of MIG
  Migration instance runs with:
  - No packet capture (interface disabled)
  - Different managerport (5030)
  - database_backup_from_date = 2024-01-01
end note
@enduml
</kroki>
</kroki>
== Online GUI Migration Workflow ==
When migrating the GUI to a new server, you have two primary methods:
=== Method 1: Dump/Restore (With Downtime) ===
If a service interruption is acceptable:
# Stop the sniffer on the old server
# Perform a database dump:
#:<syntaxhighlight lang="bash">mysqldump -u root -p voipmonitor > backup.sql</syntaxhighlight>
# Transfer and restore on new server:
#:<syntaxhighlight lang="bash">mysql -u root -p voipmonitor < backup.sql</syntaxhighlight>
# Repoint the GUI to the new database
# Install and configure the new GUI instance
This method is simple but requires downtime and may take hours for terabyte-sized databases.
=== Method 2: Online Migration (Minimal/No Downtime) ===
Using the sensor's database backup mode:
# Install VoIPmonitor GUI on the new server with a fresh empty database
# On the old GUI: Go to '''Tools → Backup & Restore → Backup configuration TABLES''' and download the backup file
# On the new GUI: Go to '''Tools → Backup & Restore → Restore configuration tables''' and upload the backup file
#:'''IMPORTANT:''' Complete this step BEFORE starting the migration instance
# Create a <code>voipmonitor-migrate.conf</code> file by copying the old <code>voipmonitor.conf</code> and adding migration options (see Configuration section below)
# Configure and run the migration instance in "database backup mode" on the new server
# The sensor connects to the old database and replicates CDR data incrementally
# Once replication catches up, switch users to the new GUI
# Stop migration instance and decommission the old server
For assistance with this method, provide remote SSH access to both the old and new hosts to the support team.
== Comparison: Dump/Restore vs Online Migration ==
{| class="wikitable"
|-
! Aspect !! Dump/Restore !! Online Migration
|-
| '''Downtime''' || Required (hours for large DBs) || Minimal/None
|-
| '''Complexity''' || Low - Manual SQL dump || Medium - Sensor configuration
|-
| '''Data Loss Risk''' || Moderate (if dump fails) || Low - Incremental sync
|-
| '''Best For''' || Small databases || Large databases, zero-downtime requirements
|}
== Configuration ==
To set up database replication, you will run a new, dedicated VoIPmonitor sensor instance on the server that hosts your '''secondary (destination)''' database. This instance requires its own <code>voipmonitor.conf</code> file.
=== Creating the Migration Configuration File ===
The migration configuration file should be named consistently to avoid confusion (e.g., <code>voipmonitor-migrate.conf</code>).
'''Important:''' Start with a copy of the existing configuration. The easiest way to create this configuration is to copy the existing <code>voipmonitor.conf</code> from the old server and add the migration-specific options. This ensures all existing settings are preserved.


<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
telnet localhost 5029
# On the NEW server, copy the old config:
scp root@old-server:/etc/voipmonitor.conf /etc/voipmonitor-migrate.conf
</syntaxhighlight>
</syntaxhighlight>


=== Direct URL Access ===
Then edit <code>/etc/voipmonitor-migrate.conf</code> to modify the critical settings for migration mode.


You can open the Active Calls view directly using this URL format:
=== Required Modifications ===


<syntaxhighlight lang="text">
If you plan to run the migration instance while the original sniffer continues to capture traffic (zero-downtime migration), you '''MUST''' make the following changes to avoid conflicts:
http://your-server/admin.php?activecalls=1
 
</syntaxhighlight>
<syntaxhighlight lang="ini">
# --- 1. CRITICAL: Change managerport to avoid conflicts ---
# The original sniffer typically uses managerport = 5029
managerport = 5030
 
# --- 2. CRITICAL: Disable all packet capture bindings ---
# These prevent the migration instance from competing for network interfaces
# Comment out or delete ALL interface/bind options:
# interface = eth0      # DISABLE - do not capture packets
# interface = eth1      # DISABLE - do not capture packets
# interface = any        # DISABLE - do not capture packets
 
# --- 3. Destination Database (where to write data) ---
# Update to point to the NEW server's local database
mysqlhost                      = 127.0.0.1
mysqldb                        = voipmonitor
mysqlusername                  = root
mysqlpassword                  = new_db_password
cdr_partition                  = yes
 
# --- 4. Source Database (where to read data from) ---
# These parameters are ADDED to enable replication from the old database
database_backup_from_mysqlhost    = 192.168.0.1
database_backup_from_mysqldb      = voipmonitor
database_backup_from_mysqlusername = root
database_backup_from_mysqlpassword = old_db_password
 
# --- 5. Replication Control ---
# Choose the date from which replication should begin
database_backup_from_date          = 2024-01-01
 
# Performance tuning for replication speed
database_backup_insert_threads    = 3
database_backup_pause              = 3


== Filtering Calls ==
# Performance Impact Explanation:
# - database_backup_pause: Higher values (e.g., 10-60 seconds) create MORE pause between batches,
#  resulting in LOWER load on the SOURCE database. Use high values if the source database
#  is also handling live traffic and cannot withstand additional replication load.
# - database_backup_insert_threads: Higher values use MORE threads for inserting data,
#  resulting in HIGHER load on both source and destination databases. Lower values (1-2)
#  are gentler on the system but replication is slower.


The filter box adapts to the type of input provided. Calls can be filtered by:
# --- 6. Optional: Migrate Static Data ---
# Set to 'yes' if migrating static data and you want to write NEW data to the new database.
# This is useful when you want the new server to continue accepting data after migration.
database_backup_desc_dir = yes
</syntaxhighlight>


* National or international (via combo box)
{{Note|<code>database_backup_desc_dir = yes</code> is an optional parameter that should be used when migrating static data (historical CDRs) and you plan to write new data to the destination database. This tells the migration instance to write both migrated data AND future new data to the destination database.}}
* Sensor
* IP address or IP prefix
* Phone number or prefix
* Call-ID


=== Filter Syntax Examples ===
=== Configuration Parameters Reference ===


{| class="wikitable"
{| class="wikitable"
|-
|-
! Input !! Description
! Parameter !! Description
|-
| <code>managerport</code> || TCP port for the sensor's manager API. Must be different from the production sniffer (default: 5029).
|-
| <code>interface</code> || Network interface(s) to capture. Must be disabled/commented out for migration instance.
|-
| <code>mysqlhost</code> || IP/hostname of the '''destination''' database (new server).
|-
|-
| <code>192.168.%</code> || Filters calls with source or destination IP starting with 192.168.x.x
| <code>database_backup_from_mysqlhost</code> || IP/hostname of the '''source''' database (old server).
|-
|-
| <code>00</code> || Filters calls where the number starts with 00
| <code>database_backup_from_date</code> || Start date for incremental sync (format: YYYY-MM-DD).
|-
|-
| <code>abc123</code> || Searches for exact Call-ID match
| <code>database_backup_insert_threads</code> || Number of parallel threads for inserting data (default: 1). Higher values = higher load on both source and destination databases. Lower values (1-2) = gentler on the system but slower replication.
|-
|-
| <code>%abc123%</code> || Searches for Call-ID containing "abc123" (substring match)
| <code>database_backup_pause</code> || Pause in seconds between sync batches (default: 0). Higher values = lower load on the SOURCE database. Use high values (10-60 seconds) if the source database handles live traffic.
|}
|}


Note: By default, Call-ID search matches the full Call-ID string. To search for a substring, add the <code>%</code> wildcard character to the start and/or end of the search string.
=== Why These Changes Are Required ===
 
;'''managerport'''
:VoIPmonitor uses this port for internal communication and GUI connectivity. If both instances use the same port, they will conflict.
 
;'''interface options'''
:The migration instance should NOT capture packets - its only purpose is database replication. Disabling packet capture prevents:
:* CPU competition with the production sniffer
:* Duplicate packets being processed by both instances
:* Network interface conflicts
 
=== GUI Configuration Backup ===
 
Before starting the migration instance, you must migrate the GUI configuration tables (users, sensors, capture rules, alerts). This is done via the GUI's '''Tools → Backup & Restore''' feature.
 
'''IMPORTANT:''' Complete this step BEFORE starting the migration instance. See [[Backup_and_restore_GUI_tables|Backup and restore GUI tables]] for detailed instructions.
 
== Running the Replication Instance ==
 
Once your configuration file is ready, you can start the sensor in this special mode.
 
=== Manual Test Run ===
It is highly recommended to first run the process manually to ensure all settings are correct and there are no connection errors.
 
<syntaxhighlight lang="bash">
voipmonitor --config-file /etc/voipmonitor-migrate.conf -k -v 1
</syntaxhighlight>
 
;Command-line options:
* <code>-k</code>: Prevents the process from forking into the background (keeps it in foreground).
* <code>-v 1</code>: Sets verbosity to level 1, which will show status information.
 
Watch the output for any database connection errors or other warnings. A successful start will show messages indicating CDR data is being read from the source and written to the destination.
 
=== Running as a Service ===
To run the migration instance permanently as a background service, you will need to create a separate <code>systemd</code> or <code>init.d</code> service file for it. This process is covered in the [[Multiple_sniffer_instancies|Multiple Sniffer Instances]] guide. The key is to ensure the new service unit file uses the <code>-c /etc/voipmonitor-migrate.conf</code> argument to load your specific configuration.
 
Example <code>systemd</code> unit file (<code>/etc/systemd/system/voipmonitor-migrate.service</code>):
 
<syntaxhighlight lang="ini">
[Unit]
Description=VoIPmonitor Database Migration Instance
After=network.target mysql.service
 
[Service]
Type=simple
ExecStart=/usr/local/sbin/voipmonitor -c /etc/voipmonitor-migrate.conf
Restart=on-failure
 
[Install]
WantedBy=multi-user.target
</syntaxhighlight>


== Bottom Graphs ==
Enable and start:
<syntaxhighlight lang="bash">
systemctl daemon-reload
systemctl enable voipmonitor-migrate
systemctl start voipmonitor-migrate
</syntaxhighlight>


The bottom section of the Active Calls view displays graphs showing the top callers grouped by:
== Dual Write Method (Real-Time Synchronization) ==
* Caller IP address
* Called IP address


[[File:activecalls.png]]
The dual write method is a simpler alternative to MySQL replication for synchronizing CDRs between two active VoIPmonitor instances. Instead of configuring database replication at the MySQL level, you configure the sniffer to write each CDR to both a local (primary) and a remote (secondary) database simultaneously.


== Customizing Column Visibility ==
=== Overview ===


The Active Calls view allows you to show or hide columns based on your preferences. This is useful when:
In this setup, the sniffer writes each CDR record to two databases:
* '''Primary Database''' (local): The main database for the instance
* '''Secondary Database''' (remote): The secondary instance's database


* You want to reduce visual clutter and focus on specific information
This provides real-time synchronization without the complexity of MySQL replication configuration.
* Certain columns contain data that is causing performance issues (e.g., parsing errors in the Proxy column)
* Your display has limited screen space


To hide or show columns:
=== Use Cases ===


# Locate the column header area at the top of the table
* '''Active-Active Datacenter Deployment''': Two separate datacenters both capturing traffic, with each sniffer writing to the other's database
# Click on the column visibility icon (typically a column or grid icon) or look for a "Show columns" dropdown menu
* '''Real-time Backup''': Every CDR is immediately backed up to a remote database
# Uncheck columns you want to hide, check columns you want to show
* '''Centralized Reporting''': Multiple sniffers write to a central reporting database
# The view will update immediately with your selection


Your column visibility preferences are stored in the GUI configuration (saved in the MySQL <code>custom_config</code> table) and persist across sessions.
=== Configuration ===


=== Troubleshooting Performance Issues ===
To enable dual write mode, add the secondary database connection parameters to your <code>voipmonitor.conf</code>:


If the Active Calls view becomes unresponsive or slow, particularly when displaying certain columns like "Proxy":
<syntaxhighlight lang="ini">
# Primary Database Connection (Local)
mysqlhost      = 127.0.0.1
mysqldb        = voipmonitor
mysqlusername  = root
mysqlpassword  = primary_password


;Temporary workaround:
# Secondary Database Connection (Remote) - These parameters enable dual write
:Hide the problematic column using the column visibility method above.
mysql2host      = 192.168.1.100
mysql2db        = voipmonitor
mysql2username  = root
mysql2password  = secondary_password
</syntaxhighlight>


;If the GUI is completely unresponsive:
=== Important Notes ===
:* Connect directly to the GUI database
:* Check the <code>custom_config</code> table for column visibility settings
:* Clear or modify the relevant configuration entries
:* Contact support with SSH access to the GUI host for investigation


;Information to provide for support:
;'''GUI Settings Not Synchronized'''
:* A SIP PCAP dump of a problematic call (to identify data causing parsing errors)
:The dual write method only replicates CDR data. GUI configuration tables (users, sensors, capture rules, alerts) are NOT synchronized. You must manually configure these on each instance.
:* A mysqldump of the <code>voipmonitor.custom_config</code> table (to review current column configuration)


== Programmatic Access to Active Calls ==
;'''Write Performance Impact'''
:Writing to two databases adds overhead. Ensure your network can handle the additional database traffic. Consider using a dedicated database server for the secondary if performance becomes an issue.


In addition to viewing active calls in the GUI, you can programmatically retrieve active call data for automation and integration purposes.
;'''Connection Failure Handling'''
:If the secondary database becomes unreachable, the sniffer will log connection errors but continue writing to the primary database. CDRs written during the outage will NOT be retroactively synced.


=== GUI API (Recommended for Centralized Systems) ===
;'''Suitable for Simple Scenarios'''
:For complex high-availability requirements with bidirectional write support, consider [[Mysql_master-master_replication_hints|Master-Master replication]] instead.


The VoIPmonitor GUI provides a web API endpoint <code>listActiveCalls</code> that retrieves active calls from all connected sensors/probes. This is the preferred method for centralized monitoring environments.
=== Comparison: Dual Write vs Master-Master Replication ===


*Method:* HTTP POST or GET to <code>/php/api.php?task=listActiveCalls</code>
{| class="wikitable"
*Documentation:* [[WEB_API#listActiveCalls]]
|-
! Aspect !! Dual Write Method !! Master-Master Replication
|-
| '''Configuration Complexity''' || Low - Add mysql2* parameters || Medium - MySQL replication setup
|-
| '''Bidirectional Sync''' || No - A writes to B (one-way) || Yes - A and B sync with each other
|-
| '''GUI Settings Sync''' || No || Yes (via MySQL replication)
|-
| '''Automatic Failover''' || No || Yes (with load balancer)
|-
| '''Best For''' || Simple one-way backup || Full high-availability setup
|}
 
== Migrating PCAP (Packet Capture) Files ==
The database migration process described above migrates only the '''CDR (Call Detail Records)'''. It does not migrate PCAP files. You must migrate PCAP files separately.
 
=== Using rsync to Copy PCAPs ===
 
Use <code>rsync</code> over SSH to copy the <code>spooldir</code> (default: <code>/var/spool/voipmonitor</code>) from the old server to the new server.


Example:
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
curl -X POST 'http://yourserver/php/api.php?task=listActiveCalls&user=USER&password=PASSWORD&params={"sensorId":"1"}'
# Run on NEW SERVER pulling from OLD
rsync -avz root@old-server:/var/spool/voipmonitor/ /var/spool/voipmonitor/
</syntaxhighlight>
</syntaxhighlight>


=== Sniffer Manager API (For Individual Sensors) ===
;<code>-a</code>: Archive mode (preserves permissions, times, symbolic links)
;<code>-v</code>: Verbose output
;<code>-z</code>: Compress during transfer (saves bandwidth)


For direct access to a single sensor instance, use the sniffer service's manager API on port 5029. The <code>listcalls</code> command retrieves active call data from the specific sensor.
=== Timezone Considerations (CRITICAL) ===


*Method:* TCP connection to port 5029
;'''OS Timezone Must Match'''
*Documentation:* [[Encryption_in_manager_api_customer#How_to_use_the_API_-_examples|Manager API]]
:If you are copying PCAP files from an old probe to a new probe that will continue capturing, ensure both systems have the <strong>same OS timezone</strong> configured.


Example:
PCAP file paths are based on directory structure: <code>spooldir/YYYY-MM-DD/HH/MM/</code>. The directory structure uses the system's local timezone. If the old probe was in timezone A and the new probe is in timezone B, the new probe will create PCAP files in different directory paths than the old files. This can cause the GUI to fail to find or link historical PCAPs.
 
The database stores relative paths, so as long as the directory structure is preserved and the timezone remains the same, the GUI will correctly reference the old PCAPs.
 
To check/set timezone:
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
echo 'listcalls' | nc localhost 5029
# Check current timezone
timedatectl
 
# Set timezone (example for Europe/Prague)
timedatectl set-timezone Europe/Prague
</syntaxhighlight>
</syntaxhighlight>


The manager API supports filtering by caller number, called number, IP addresses, and other criteria. See the [[Encryption_in_manager_api_customer#Example_with_filter_used|Manager API documentation]] for advanced filtering examples.
=== Architecture-Specific Considerations ===
 
=== Choosing the Right Method ===


*Use the GUI <code>listActiveCalls</code> API when:*
;'''Client-Server Mode with Local Processing'''
* You have multiple sensors/probes and need centralized access
:In the new architecture where probes store their own PCAPs (<code>packetbuffer_sender=no</code>), you have these options:
* You want to query all connected sensors from a single endpoint
:* Keep the PCAPs on the OLD probe hardware if it is becoming one of the new probes
* You are building external integrations that need filtered results
:* Archive historical PCAPs to external storage
:* Copy to the new probe '''only if timezone matches'''


*Use the sniffer manager API when:*
;'''Centralized PCAP Storage'''
* You need direct access to a specific sensor instance
:If you are centralizing PCAPs (<code>packetbuffer_sender=yes</code>), copy all old PCAPs to the Central Server's spool directory. Ensure the central server's OS timezone matches all probes' timezones.
* You are running scripts locally on the sensor host
* You require low-latency queries without GUI involvement


== AI Summary for RAG ==
== AI Summary for RAG ==
'''Summary:''' Active Calls provides real-time monitoring of ongoing VoIP calls with live listening capability for G.711 codec. Calls are fetched via manager port 5029 and can be filtered by IP, number, sensor, or Call-ID. Column visibility can be customized and settings persist in the database. Programmatic access is available through two methods: the GUI's listActiveCalls API for centralized monitoring of multiple sensors, and the sniffer's manager API (listcalls command) for direct sensor access.
'''Summary:''' This article describes three methods for synchronizing VoIPmonitor databases between instances: (1) Database Backup Mode - using a dedicated sensor in "database backup mode" to perform application-level replication for read-only replicas (described in detail here), (2) Master-Master Database Replication - bidirectional MySQL/MariaDB replication for full high-availability (covered in [[Mysql_master-master_replication_hints|Master-Master Replication]]), and (3) Dual Write Method - configuring the sniffer to write CDRs simultaneously to local and remote databases using mysql2* parameters. The article primarily covers Method 1 with configuration details for setting up destination (mysqlhost) and source (database_backup_from_mysqlhost) databases, including database_backup_from_date for sync control. To control database load during replication, use database_backup_pause (higher values = lower load on source database) and database_backup_insert_threads (higher values = higher load on both databases). PCAP files must be migrated separately using rsync, with critical attention to timezone matching between servers.


'''Keywords:''' active calls, real-time monitoring, live calls, manager port 5029, call filter, column visibility, custom_config, G.711 listening, listActiveCalls API, programmatic access, automation, manager API, listcalls command
'''Keywords:''' database replication, backup, mysql backup, mariadb replication, replica, read-only, standby, disaster recovery, GUI migration, migrate database, online migration, zero downtime, database_backup_from_date, database_backup_from_mysqlhost, voipmonitor.conf, high availability, PCAP migration, spooldir, rsync, timezone, packetbuffer_sender, client-server mode, centralized storage, dual write, mysql2host, master-master replication, synchronize, datacenter, performance tuning, database_backup_pause, database_backup_insert_threads, performance impact, load control


'''Key Questions:'''
'''Key Questions:'''
* How do I view active/live calls in VoIPmonitor?
* How can I synchronize CDRs between two VoIPmonitor instances?
* Why are active calls not showing in the GUI?
* What methods are available for database replication in VoIPmonitor?
* How do I filter calls by IP address or phone number?
* What is the difference between database backup mode, dual write, and master-master replication?
* How do I hide or show columns in the Active Calls view?
* How can I migrate the GUI database with minimal downtime?
* What port is used for fetching live call data?
* What is the difference between dump/restore and online migration?
* How can I programmatically check if a call to a specific phone number is currently active?
* How can I create a real-time backup of my VoIPmonitor database?
* What is the difference between listActiveCalls API and the sniffer's manager API?
* How do I configure dual write mode (mysql2*)?
* How do I retrieve active calls from all connected sensors?
* What configuration is needed to mirror the VoIPmonitor database?
* How do I query active calls from a single sensor directly?
* Can I use VoIPmonitor to create a read-only replica for reporting?
* How does database_backup_from_date work?
* How do I migrate PCAP files to a new server?
* Why is timezone matching important when migrating PCAP files?
* How can I use rsync to copy VoIPmonitor spooldir?
* How can I control the database load during replication?
* What is database_backup_pause and how does it affect performance?
* What is database_backup_insert_threads and how does it affect performance?

Revision as of 13:01, 6 January 2026


This guide explains how to use a dedicated VoIPmonitor sensor instance to perform real-time replication of the CDR database to a secondary MySQL/MariaDB server. This provides a native, application-level method for creating backups or read-only replicas.

Overview & Use Case

VoIPmonitor offers multiple built-in mechanisms to replicate or synchronize Call Detail Records (CDRs) between instances. These methods enable datacenter failover, disaster recovery, hot-standby setups, and centralized reporting.

Available Synchronization Methods

There are three primary approaches for synchronizing CDRs between VoIPmonitor instances:

1. Database Backup Mode (Read-Only Replica)
Use a dedicated sniffer instance to read from the primary database and write to a secondary database. This creates a one-way, read-only replica. This method is ideal for online GUI migration, hot-standby setups, or centralized reporting.
2. Master-Master Database Replication
Configure MySQL/MariaDB in a bidirectional replication topology. Both instances can write CDRs, and changes automatically propagate between them. This provides full high-availability with automatic failover and read load balancing. See Master-Master Replication for detailed setup.
3. Dual Write Method
Configure the active sniffer to write CDRs simultaneously to both a local and a remote database. This provides real-time synchronization without requiring MySQL replication setup. Note: This method only synchronizes CDRs; it does not synchronize GUI settings (users, sensors, capture rules, etc.).

This article primarily covers Method 1 (Database Backup Mode). Method 2 is covered in the Master-Master Replication guide. Method 3 is described below.

The database backup mode method is particularly useful for:

  • Online GUI Migration: Migrating the GUI database from an old server to a new server with minimal or no downtime. The sensor replicates CDR data incrementally while users continue using the old GUI. Once replication completes, you can switch users to the new GUI without service interruption.
  • Creating a read-only replica database for reporting or analysis without impacting the performance of the primary database.
  • Setting up a hot-standby database for disaster recovery.
  • Consolidating data from multiple, smaller VoIPmonitor databases into one central database.

The process incrementally syncs data starting from a specified date, ensuring that the destination database stays up-to-date in near real-time.

Architecture Overview

Online GUI Migration Workflow

When migrating the GUI to a new server, you have two primary methods:

Method 1: Dump/Restore (With Downtime)

If a service interruption is acceptable:

  1. Stop the sniffer on the old server
  2. Perform a database dump: 
    mysqldump -u root -p voipmonitor > backup.sql
  3. Transfer and restore on new server: 
    mysql -u root -p voipmonitor < backup.sql
  4. Repoint the GUI to the new database
  5. Install and configure the new GUI instance

This method is simple but requires downtime and may take hours for terabyte-sized databases.

Method 2: Online Migration (Minimal/No Downtime)

Using the sensor's database backup mode:

  1. Install VoIPmonitor GUI on the new server with a fresh empty database
  2. On the old GUI: Go to Tools → Backup & Restore → Backup configuration TABLES and download the backup file
  3. On the new GUI: Go to Tools → Backup & Restore → Restore configuration tables and upload the backup file
    IMPORTANT: Complete this step BEFORE starting the migration instance
  4. Create a voipmonitor-migrate.conf file by copying the old voipmonitor.conf and adding migration options (see Configuration section below)
  5. Configure and run the migration instance in "database backup mode" on the new server
  6. The sensor connects to the old database and replicates CDR data incrementally
  7. Once replication catches up, switch users to the new GUI
  8. Stop migration instance and decommission the old server

For assistance with this method, provide remote SSH access to both the old and new hosts to the support team.

Comparison: Dump/Restore vs Online Migration

Aspect Dump/Restore Online Migration
Downtime Required (hours for large DBs) Minimal/None
Complexity Low - Manual SQL dump Medium - Sensor configuration
Data Loss Risk Moderate (if dump fails) Low - Incremental sync
Best For Small databases Large databases, zero-downtime requirements

Configuration

To set up database replication, you will run a new, dedicated VoIPmonitor sensor instance on the server that hosts your secondary (destination) database. This instance requires its own voipmonitor.conf file.

Creating the Migration Configuration File

The migration configuration file should be named consistently to avoid confusion (e.g., voipmonitor-migrate.conf).

Important: Start with a copy of the existing configuration. The easiest way to create this configuration is to copy the existing voipmonitor.conf from the old server and add the migration-specific options. This ensures all existing settings are preserved. 

# On the NEW server, copy the old config:
scp root@old-server:/etc/voipmonitor.conf /etc/voipmonitor-migrate.conf

Then edit /etc/voipmonitor-migrate.conf to modify the critical settings for migration mode.

Required Modifications

If you plan to run the migration instance while the original sniffer continues to capture traffic (zero-downtime migration), you MUST make the following changes to avoid conflicts: 

# --- 1. CRITICAL: Change managerport to avoid conflicts ---
# The original sniffer typically uses managerport = 5029
managerport = 5030

# --- 2. CRITICAL: Disable all packet capture bindings ---
# These prevent the migration instance from competing for network interfaces
# Comment out or delete ALL interface/bind options:
# interface = eth0       # DISABLE - do not capture packets
# interface = eth1       # DISABLE - do not capture packets
# interface = any        # DISABLE - do not capture packets

# --- 3. Destination Database (where to write data) ---
# Update to point to the NEW server's local database
mysqlhost                      = 127.0.0.1
mysqldb                        = voipmonitor
mysqlusername                  = root
mysqlpassword                  = new_db_password
cdr_partition                  = yes

# --- 4. Source Database (where to read data from) ---
# These parameters are ADDED to enable replication from the old database
database_backup_from_mysqlhost     = 192.168.0.1
database_backup_from_mysqldb       = voipmonitor
database_backup_from_mysqlusername = root
database_backup_from_mysqlpassword = old_db_password

# --- 5. Replication Control ---
# Choose the date from which replication should begin
database_backup_from_date          = 2024-01-01

# Performance tuning for replication speed
database_backup_insert_threads     = 3
database_backup_pause              = 3

# Performance Impact Explanation:
# - database_backup_pause: Higher values (e.g., 10-60 seconds) create MORE pause between batches,
#   resulting in LOWER load on the SOURCE database. Use high values if the source database
#   is also handling live traffic and cannot withstand additional replication load.
# - database_backup_insert_threads: Higher values use MORE threads for inserting data,
#   resulting in HIGHER load on both source and destination databases. Lower values (1-2)
#   are gentler on the system but replication is slower.

# --- 6. Optional: Migrate Static Data ---
# Set to 'yes' if migrating static data and you want to write NEW data to the new database.
# This is useful when you want the new server to continue accepting data after migration.
database_backup_desc_dir = yes

ℹ️ Note:

Configuration Parameters Reference

Parameter Description
managerport TCP port for the sensor's manager API. Must be different from the production sniffer (default: 5029).
interface Network interface(s) to capture. Must be disabled/commented out for migration instance.
mysqlhost IP/hostname of the destination database (new server).
database_backup_from_mysqlhost IP/hostname of the source database (old server).
database_backup_from_date Start date for incremental sync (format: YYYY-MM-DD).
database_backup_insert_threads Number of parallel threads for inserting data (default: 1). Higher values = higher load on both source and destination databases. Lower values (1-2) = gentler on the system but slower replication.
database_backup_pause Pause in seconds between sync batches (default: 0). Higher values = lower load on the SOURCE database. Use high values (10-60 seconds) if the source database handles live traffic.

Why These Changes Are Required

managerport
VoIPmonitor uses this port for internal communication and GUI connectivity. If both instances use the same port, they will conflict.
interface options
The migration instance should NOT capture packets - its only purpose is database replication. Disabling packet capture prevents:
  • CPU competition with the production sniffer
  • Duplicate packets being processed by both instances
  • Network interface conflicts

GUI Configuration Backup

Before starting the migration instance, you must migrate the GUI configuration tables (users, sensors, capture rules, alerts). This is done via the GUI's Tools → Backup & Restore feature.

IMPORTANT: Complete this step BEFORE starting the migration instance. See Backup and restore GUI tables for detailed instructions.

Running the Replication Instance

Once your configuration file is ready, you can start the sensor in this special mode.

Manual Test Run

It is highly recommended to first run the process manually to ensure all settings are correct and there are no connection errors. 

voipmonitor --config-file /etc/voipmonitor-migrate.conf -k -v 1
Command-line options
  • -k: Prevents the process from forking into the background (keeps it in foreground).
  • -v 1: Sets verbosity to level 1, which will show status information.

Watch the output for any database connection errors or other warnings. A successful start will show messages indicating CDR data is being read from the source and written to the destination.

Running as a Service

To run the migration instance permanently as a background service, you will need to create a separate systemd or init.d service file for it. This process is covered in the Multiple Sniffer Instances guide. The key is to ensure the new service unit file uses the -c /etc/voipmonitor-migrate.conf argument to load your specific configuration.

Example systemd unit file (/etc/systemd/system/voipmonitor-migrate.service): 

[Unit]
Description=VoIPmonitor Database Migration Instance
After=network.target mysql.service

[Service]
Type=simple
ExecStart=/usr/local/sbin/voipmonitor -c /etc/voipmonitor-migrate.conf
Restart=on-failure

[Install]
WantedBy=multi-user.target

Enable and start: 

systemctl daemon-reload
systemctl enable voipmonitor-migrate
systemctl start voipmonitor-migrate

Dual Write Method (Real-Time Synchronization)

The dual write method is a simpler alternative to MySQL replication for synchronizing CDRs between two active VoIPmonitor instances. Instead of configuring database replication at the MySQL level, you configure the sniffer to write each CDR to both a local (primary) and a remote (secondary) database simultaneously.

Overview

In this setup, the sniffer writes each CDR record to two databases:

  • Primary Database (local): The main database for the instance
  • Secondary Database (remote): The secondary instance's database

This provides real-time synchronization without the complexity of MySQL replication configuration.

Use Cases

  • Active-Active Datacenter Deployment: Two separate datacenters both capturing traffic, with each sniffer writing to the other's database
  • Real-time Backup: Every CDR is immediately backed up to a remote database
  • Centralized Reporting: Multiple sniffers write to a central reporting database

Configuration

To enable dual write mode, add the secondary database connection parameters to your voipmonitor.conf: 

# Primary Database Connection (Local)
mysqlhost      = 127.0.0.1
mysqldb        = voipmonitor
mysqlusername  = root
mysqlpassword  = primary_password

# Secondary Database Connection (Remote) - These parameters enable dual write
mysql2host      = 192.168.1.100
mysql2db        = voipmonitor
mysql2username  = root
mysql2password  = secondary_password

Important Notes

GUI Settings Not Synchronized
The dual write method only replicates CDR data. GUI configuration tables (users, sensors, capture rules, alerts) are NOT synchronized. You must manually configure these on each instance.
Write Performance Impact
Writing to two databases adds overhead. Ensure your network can handle the additional database traffic. Consider using a dedicated database server for the secondary if performance becomes an issue.
Connection Failure Handling
If the secondary database becomes unreachable, the sniffer will log connection errors but continue writing to the primary database. CDRs written during the outage will NOT be retroactively synced.
Suitable for Simple Scenarios
For complex high-availability requirements with bidirectional write support, consider Master-Master replication instead.

Comparison: Dual Write vs Master-Master Replication

Aspect Dual Write Method Master-Master Replication
Configuration Complexity Low - Add mysql2* parameters Medium - MySQL replication setup
Bidirectional Sync No - A writes to B (one-way) Yes - A and B sync with each other
GUI Settings Sync No Yes (via MySQL replication)
Automatic Failover No Yes (with load balancer)
Best For Simple one-way backup Full high-availability setup

Migrating PCAP (Packet Capture) Files

The database migration process described above migrates only the CDR (Call Detail Records). It does not migrate PCAP files. You must migrate PCAP files separately.

Using rsync to Copy PCAPs

Use rsync over SSH to copy the spooldir (default: /var/spool/voipmonitor) from the old server to the new server. 

# Run on NEW SERVER pulling from OLD
rsync -avz root@old-server:/var/spool/voipmonitor/ /var/spool/voipmonitor/
-a
Archive mode (preserves permissions, times, symbolic links)
-v
Verbose output
-z
Compress during transfer (saves bandwidth)

Timezone Considerations (CRITICAL)

OS Timezone Must Match
If you are copying PCAP files from an old probe to a new probe that will continue capturing, ensure both systems have the same OS timezone configured.

PCAP file paths are based on directory structure: spooldir/YYYY-MM-DD/HH/MM/. The directory structure uses the system's local timezone. If the old probe was in timezone A and the new probe is in timezone B, the new probe will create PCAP files in different directory paths than the old files. This can cause the GUI to fail to find or link historical PCAPs.

The database stores relative paths, so as long as the directory structure is preserved and the timezone remains the same, the GUI will correctly reference the old PCAPs.

To check/set timezone: 

# Check current timezone
timedatectl

# Set timezone (example for Europe/Prague)
timedatectl set-timezone Europe/Prague

Architecture-Specific Considerations

Client-Server Mode with Local Processing
In the new architecture where probes store their own PCAPs (packetbuffer_sender=no), you have these options:
  • Keep the PCAPs on the OLD probe hardware if it is becoming one of the new probes
  • Archive historical PCAPs to external storage
  • Copy to the new probe only if timezone matches
Centralized PCAP Storage
If you are centralizing PCAPs (packetbuffer_sender=yes), copy all old PCAPs to the Central Server's spool directory. Ensure the central server's OS timezone matches all probes' timezones.

AI Summary for RAG

Summary: This article describes three methods for synchronizing VoIPmonitor databases between instances: (1) Database Backup Mode - using a dedicated sensor in "database backup mode" to perform application-level replication for read-only replicas (described in detail here), (2) Master-Master Database Replication - bidirectional MySQL/MariaDB replication for full high-availability (covered in Master-Master Replication), and (3) Dual Write Method - configuring the sniffer to write CDRs simultaneously to local and remote databases using mysql2* parameters. The article primarily covers Method 1 with configuration details for setting up destination (mysqlhost) and source (database_backup_from_mysqlhost) databases, including database_backup_from_date for sync control. To control database load during replication, use database_backup_pause (higher values = lower load on source database) and database_backup_insert_threads (higher values = higher load on both databases). PCAP files must be migrated separately using rsync, with critical attention to timezone matching between servers.

Keywords: database replication, backup, mysql backup, mariadb replication, replica, read-only, standby, disaster recovery, GUI migration, migrate database, online migration, zero downtime, database_backup_from_date, database_backup_from_mysqlhost, voipmonitor.conf, high availability, PCAP migration, spooldir, rsync, timezone, packetbuffer_sender, client-server mode, centralized storage, dual write, mysql2host, master-master replication, synchronize, datacenter, performance tuning, database_backup_pause, database_backup_insert_threads, performance impact, load control

Key Questions:

  • How can I synchronize CDRs between two VoIPmonitor instances?
  • What methods are available for database replication in VoIPmonitor?
  • What is the difference between database backup mode, dual write, and master-master replication?
  • How can I migrate the GUI database with minimal downtime?
  • What is the difference between dump/restore and online migration?
  • How can I create a real-time backup of my VoIPmonitor database?
  • How do I configure dual write mode (mysql2*)?
  • What configuration is needed to mirror the VoIPmonitor database?
  • Can I use VoIPmonitor to create a read-only replica for reporting?
  • How does database_backup_from_date work?
  • How do I migrate PCAP files to a new server?
  • Why is timezone matching important when migrating PCAP files?
  • How can I use rsync to copy VoIPmonitor spooldir?
  • How can I control the database load during replication?
  • What is database_backup_pause and how does it affect performance?
  • What is database_backup_insert_threads and how does it affect performance?
Retrieved from "https://www.voipmonitor.org/doc/index.php?title=Redundant_database&oldid=8929"