Database structure: Difference between revisions

From VoIPmonitor.org
(Add system table documentation with manager_key troubleshooting)
(VS-1789: Document nth_occurrence column in cdr_custom_headers table)
 
(12 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{DISPLAYTITLE:CDR Database Table Schema}}
{{DISPLAYTITLE:CDR Database Table Schema}}


'''This document provides an overview of the most important tables and columns in the VoIPmonitor database schema. It is intended as a reference for administrators, developers, or anyone directly querying the database.'''
'''Database schema reference for VoIPmonitor. The <code>cdr</code> table stores one record per call; related tables store RTP statistics, custom headers, and metadata.'''


The core of the VoIPmonitor database is the <code>cdr</code> (Call Detail Record) table, which stores one primary record for each monitored call. Many other tables are linked to <code>cdr</code> to store additional, more detailed information.
= Schema Overview =
 
== Database Schema Overview ==
 
=== Listing CDR Tables ===
 
To list all CDR-related tables in the database, use the following SQL command:


<syntaxhighlight lang="sql">
<syntaxhighlight lang="sql">
-- List all CDR-related tables
SHOW TABLES LIKE 'cdr%';
SHOW TABLES LIKE 'cdr%';
</syntaxhighlight>
</syntaxhighlight>
This returns all tables with names starting with <code>cdr</code>, including the main <code>cdr</code> table and all related tables such as <code>cdr_next</code>, <code>cdr_rtp</code>, <code>cdr_proxy</code>, and custom header tables (<code>cdr_next_2</code>, etc.).
The following diagram shows the relationships between the main VoIPmonitor database tables:


<kroki lang="plantuml">
<kroki lang="plantuml">
Line 28: Line 19:
   * ID : bigint <<PK>>
   * ID : bigint <<PK>>
   --
   --
   id_sensor : int
   id_sensor, calldate, callend
  calldate : datetime
   caller, called, duration
  callend : datetime
   sipcallerip, sipcalledip
   caller : varchar
   a_lost, b_lost, a_mos_f1_mult10
  called : varchar
   lastSIPresponse_id <<FK>>
   sipcallerip : int
   a_ua_id, b_ua_id <<FK>>
  sipcalledip : int
   duration : int
   connect_duration : int
  a_ua_id : int <<FK>>
   b_ua_id : int <<FK>>
  lastSIPresponse_id : int <<FK>>
  a_lost / b_lost : int
  a_mos_f1_mult10 : tinyint
  ...
}
}


entity "cdr_next" as cdr_next {
entity "cdr_next" as cdr_next {
   * cdr_ID : bigint <<PK,FK>>
   * cdr_ID <<PK,FK>>
   --
   --
   fbasename : varchar
   fbasename, match_header
  match_header : varchar
  ...
}
}


entity "cdr_rtp" as cdr_rtp {
entity "cdr_rtp" as cdr_rtp {
   * ID : bigint <<PK>>
   cdr_ID <<FK>>
   --
   --
  cdr_ID : bigint <<FK>>
   saddr, daddr, ssrc
   saddr : int
   received, lost, payload
  daddr : int
  ssrc : bigint
   received : int
  lost : int
  ...
}
}


entity "cdr_dtmf" as cdr_dtmf {
entity "cdr_proxy" as cdr_proxy {
   * ID : bigint <<PK>>
   cdr_ID <<FK>>
   --
   --
   cdr_ID : bigint <<FK>>
   dst (proxy IP)
  dtmf : char
  ts : bigint
  ...
}
}


entity "cdr_proxy" as cdr_proxy {
entity "cdr_sdp" as cdr_sdp {
   * ID : bigint <<PK>>
   cdr_id <<FK>>
   --
   --
   cdr_ID : bigint <<FK>>
   ip, port, is_caller
  dst : int
  ...
}
}


entity "cdr_sip_response" as cdr_sip_response {
entity "cdr_siphistory" as cdr_siphistory {
   * ID : int <<PK>>
   cdr_ID <<FK>>
   --
   --
   lastSIPresponse : varchar
   method, calldate, cseq
}
}


entity "cdr_ua" as cdr_ua {
entity "cdr_ua" as cdr_ua {
   * ID : int <<PK>>
   * ID <<PK>>
   --
   --
   ua : varchar
   ua (User-Agent string)
}
 
entity "cdr_sip_response" as cdr_sip_response {
  * ID <<PK>>
  --
  lastSIPresponse
}
}


cdr ||--|| cdr_next : "1:1"
cdr ||--|| cdr_next : "1:1"
cdr ||--o{ cdr_rtp : "1:N"
cdr ||--o{ cdr_rtp : "1:N"
cdr ||--o{ cdr_dtmf : "1:N"
cdr ||--o{ cdr_proxy : "1:N"
cdr ||--o{ cdr_proxy : "1:N"
cdr ||--o{ cdr_sdp : "1:N"
cdr ||--o{ cdr_siphistory : "1:N"
cdr }o--|| cdr_ua : "N:1"
cdr }o--|| cdr_sip_response : "N:1"
cdr }o--|| cdr_sip_response : "N:1"
cdr }o--|| cdr_ua : "N:1 (a_ua_id)"
cdr }o--|| cdr_ua : "N:1 (b_ua_id)"
@enduml
@enduml
</kroki>
</kroki>


== Understanding Column Naming Conventions ==
== Column Naming Conventions ==
 
Before diving into the columns, it's important to understand these common naming patterns:


{| class="wikitable"
{| class="wikitable"
Line 112: Line 86:
! Pattern !! Meaning !! Example
! Pattern !! Meaning !! Example
|-
|-
| <code>a_</code> prefix || Caller's media stream (A-leg, RTP sent from caller) || <code>a_lost</code>, <code>a_mos_f1_mult10</code>
| <code>a_</code> || Caller's stream (A-leg) || <code>a_lost</code>, <code>a_mos_f1_mult10</code>
|-
|-
| <code>b_</code> prefix || Callee's media stream (B-leg, RTP sent from callee) || <code>b_lost</code>, <code>b_mos_f1_mult10</code>
| <code>b_</code> || Callee's stream (B-leg) || <code>b_lost</code>, <code>b_mos_f1_mult10</code>
|-
|-
| <code>_mult10</code> || Divide stored value by 10 to get real value || <code>jitter_mult10 = 15</code> means jitter = 1.5
| <code>_mult10</code> || Divide by 10 || <code>jitter_mult10 = 15</code> jitter = 1.5
|-
|-
| <code>_mult100</code> || Divide stored value by 100 to get real value || Used for percentage values
| <code>_mult100</code> || Divide by 100 || Percentage values
|-
|-
| <code>_mult1000</code> || Divide stored value by 1000 to get real value || <code>packet_loss_perc_mult1000</code>
| <code>_mult1000</code> || Divide by 1000 || <code>packet_loss_perc_mult1000</code>
|}
|}


== The <code>cdr</code> Table ==
= The <code>cdr</code> Table =


This is the main table containing the primary information for every call.
== Call Information & Timing ==
 
=== General Call Information ===


{| class="wikitable"
{| class="wikitable"
|-
|-
! Column !! Type !! Description
! Column !! Description
|-
|-
| <code>ID</code> || bigint || The unique primary key for the record
| <code>ID</code> || Primary key
|-
|-
| <code>id_sensor</code> || int || The ID of the sensor that processed this call (corresponds to <code>id_sensor</code> in <code>voipmonitor.conf</code>)
| <code>id_sensor</code> || Sensor ID (from <code>voipmonitor.conf</code>)
|-
|-
| <code>calldate</code> || datetime || The start time of the call, measured from the first <code>INVITE</code> packet
| <code>calldate</code> / <code>callend</code> || Start/end timestamps
|-
|-
| <code>callend</code> || datetime || The end time of the call, measured from the last packet associated with the call
| <code>duration</code> || Total length in seconds
|-
|-
| <code>bye</code> || tinyint || Indicates how the call was terminated (<code>1</code> = normal termination with <code>BYE</code>)
| <code>connect_duration</code> || Connected time (after 200 OK)
|-
|-
| <code>whohanged</code> || enum || Indicates which party sent the <code>BYE</code> message first (caller or callee)
| <code>progress_time</code> || Time to first provisional response ([[Glossary#Post_Dial_Delay_.28PDD.29|PDD]])
|-
|-
| <code>lastSIPresponse_id</code> || int || Foreign key to <code>cdr_sip_response</code> table for the last final SIP response
| <code>bye</code> || <code>1</code> = normal termination with BYE
|-
| <code>whohanged</code> || Which party sent BYE (caller/callee)
|-
| <code>lastSIPresponse_id</code> || FK to <code>cdr_sip_response</code>
|}
|}


=== Call Timing & Duration ===
== Participant Information ==


{| class="wikitable"
{| class="wikitable"
|-
|-
! Column !! Type !! Description
! Column !! Description
|-
|-
| <code>duration</code> || int || Total call length in seconds (<code>callend - calldate</code>)
| <code>caller</code> / <code>called</code> || Numbers from From/To headers
|-
|-
| <code>connect_duration</code> || int || Connected portion length in seconds (from first <code>200 OK</code> to <code>callend</code>)
| <code>caller_domain</code> / <code>called_domain</code> || Domain parts
|-
|-
| <code>progress_time</code> || int || Time from <code>INVITE</code> to first provisional response (e.g., <code>180 Ringing</code>). Also known as [[Glossary#Post_Dial_Delay_.28PDD.29|Post-Dial Delay (PDD)]]
| <code>sipcallerip</code> / <code>sipcalledip</code> || IPs stored as integers (use <code>INET_NTOA()</code>)
|-
|-
| <code>first_rtp_time</code> || int || Time from call start to first RTP packet detection
| <code>a_ua_id</code> / <code>b_ua_id</code> || FK to <code>cdr_ua</code> (User-Agent)
|}
|}


=== Participant Information ===
== Quality Metrics ==


=== Packet Loss ===
{| class="wikitable"
{| class="wikitable"
|-
|-
! Column !! Type !! Description
! Column !! Description
|-
|-
| <code>caller</code> / <code>called</code> || varchar || User part of the number from <code>From</code> and <code>To</code> SIP headers
| <code>a_lost</code> / <code>b_lost</code> || Total RTP packets lost
|-
|-
| <code>caller_domain</code> / <code>called_domain</code> || varchar || Domain part from <code>From</code> and <code>To</code> SIP headers
| <code>packet_loss_perc_mult1000</code> || Max loss percentage × 1000
|-
|-
| <code>caller_reverse</code> / <code>called_reverse</code> || varchar || Reversed strings for efficient <code>LIKE '%search'</code> queries
| <code>a_sl1</code> - <code>a_sl10</code> || Distribution of consecutive loss events
|-
| <code>callername</code> || varchar || Display name from the <code>From</code> SIP header
|-
| <code>sipcallerip</code> / <code>sipcalledip</code> || int unsigned || Source/destination IP addresses stored as integers (use <code>INET_NTOA()</code> to convert)
|-
| <code>a_ua_id</code> / <code>b_ua_id</code> || int || Foreign keys to <code>cdr_ua</code> table for User-Agent strings
|}
|}


{{Note|
=== Jitter & Delay ===
'''How User-Agent is extracted:'''
* '''Caller (<code>a_ua_id</code>):''' From the <code>User-Agent</code> header in the SIP INVITE request
* '''Callee (<code>b_ua_id</code>):''' From the <code>User-Agent</code> header in the SIP response (typically <code>200 OK</code>)
* Blank values indicate the <code>User-Agent</code> header was not present
}}
 
'''Example: Converting IP addresses in SQL queries:'''
<syntaxhighlight lang="sql">
-- Get calls with readable IP addresses
SELECT
    ID,
    caller,
    called,
    INET_NTOA(sipcallerip) AS caller_ip,
    INET_NTOA(sipcalledip) AS called_ip,
    duration
FROM cdr
WHERE calldate >= '2024-01-01'
LIMIT 10;
</syntaxhighlight>
 
=== Network Quality Metrics ===
 
==== Packet Loss ====
 
{| class="wikitable"
{| class="wikitable"
|-
|-
! Column !! Description
! Column !! Description
|-
|-
| <code>a_lost</code> / <code>b_lost</code> || Total RTP packets lost (detected by sequence number gaps)
| <code>a_avgjitter_mult10</code> || Average jitter × 10
|-
|-
| <code>lost</code> || Higher value between <code>a_lost</code> and <code>b_lost</code>
| <code>a_maxjitter</code> || Maximum jitter observed
|-
|-
| <code>packet_loss_perc_mult1000</code> || Maximum packet loss percentage, multiplied by 1000
| <code>a_d50</code> - <code>a_d300</code> || PDV distribution buckets (0-50ms, 50-70ms, etc.)
|-
| <code>a_sl1</code> - <code>a_sl10</code> / <code>b_sl1</code> - <code>b_sl10</code> || Distribution of consecutive packet loss events (sl1 = single packet lost, sl5 = 5 packets in a row, etc.)
|}
|}


==== Jitter & Delay (PDV) ====
=== MOS (Mean Opinion Score) ===
 
{| class="wikitable"
{| class="wikitable"
|-
|-
! Column !! Description
! Column !! Description
|-
|-
| <code>a_avgjitter_mult10</code> / <code>b_avgjitter_mult10</code> || Average jitter per RFC 3550, multiplied by 10
| <code>a_mos_f1_mult10</code> || [[Comprehensive_Guide_to_VoIP_Voice_Quality#MOS_Calculation_in_VoIPmonitor|MOS]] with 50ms fixed buffer × 10
|-
|-
| <code>a_maxjitter</code> / <code>b_maxjitter</code> || Maximum jitter value observed during the call
| <code>a_mos_f2_mult10</code> || MOS with 200ms fixed buffer × 10
|-
|-
| <code>jitter_mult10</code> || Higher <code>maxjitter</code> value between streams, multiplied by 10
| <code>a_mos_adapt_mult10</code> || MOS with adaptive buffer × 10
|-
|-
| <code>a_d50</code> - <code>a_d300</code> / <code>b_d50</code> - <code>b_d300</code> || Packet Delay Variation (PDV) distribution buckets (d50 = 0-50ms, d70 = 50-70ms, etc.)
| <code>mos_min_mult10</code> || Lowest MOS between A/B streams × 10
|}
|}


==== MOS (Mean Opinion Score) ====
=== RTCP-Reported Metrics ===
These come from endpoint RTCP reports, not calculated by the sniffer. See [[Call_Detail_Record_-_CDR#Understanding_Sniffer_vs_RTCP_Packet_Loss|Sniffer vs RTCP Packet Loss]].


{| class="wikitable"
{| class="wikitable"
Line 242: Line 186:
! Column !! Description
! Column !! Description
|-
|-
| <code>a_mos_f1_mult10</code> / <code>b_mos_f1_mult10</code> || [[Comprehensive_Guide_to_VoIP_Voice_Quality#MOS_Calculation_in_VoIPmonitor|Parametric MOS]] with 50ms fixed jitter buffer, multiplied by 10
| <code>a_rtcp_loss</code> || Packets lost (reported by endpoint)
|-
|-
| <code>a_mos_f2_mult10</code> / <code>b_mos_f2_mult10</code> || MOS with 200ms fixed jitter buffer, multiplied by 10
| <code>a_rtcp_avgfr_mult10</code> || Average fraction loss × 10
|-
|-
| <code>a_mos_adapt_mult10</code> / <code>b_mos_adapt_mult10</code> || MOS with adaptive jitter buffer (up to 500ms), multiplied by 10
| <code>a_rtcp_avgjitter_mult10</code> || Average jitter × 10
|-
|-
| <code>mos_min_mult10</code> || Lowest MOS score between A and B streams, multiplied by 10
| <code>a_rtcp_avgrtd_mult10</code> || '''Roundtrip delay (latency)''' in ms × 10
|}
|}


==== RTCP Reported Metrics ====
= Related Tables =


These metrics come from RTCP Receiver Reports sent by the endpoints, not calculated by the sniffer. For a detailed explanation of the difference between sniffer-calculated and RTCP-reported metrics, see [[Call_Detail_Record_-_CDR#Understanding_Sniffer_vs_RTCP_Packet_Loss|Understanding Sniffer vs RTCP Packet Loss]].
== <code>cdr_next</code> (1:1 with cdr) ==
 
Extension table linked via <code>cdr_ID</code>.


{| class="wikitable"
{| class="wikitable"
Line 259: Line 205:
! Column !! Description
! Column !! Description
|-
|-
| <code>a_rtcp_loss</code> / <code>b_rtcp_loss</code> || Total packets lost as reported by endpoint via RTCP
| <code>fbasename</code> || Derived from SIP Call-ID; used to link PCAP files. Special characters replaced with underscores (<code>convertchar</code> in config)
|-
|-
| <code>a_rtcp_avgfr_mult10</code> / <code>b_rtcp_avgfr_mult10</code> || Average fraction loss from RTCP, multiplied by 10
| <code>match_header</code> || Content of custom header for linking call legs (<code>matchheader</code> in config)
|-
| <code>a_rtcp_avgjitter_mult10</code> / <code>b_rtcp_avgjitter_mult10</code> || Average jitter from RTCP, multiplied by 10
|}
|}


=== Other Columns ===
== <code>cdr_rtp</code> (Per-Stream RTP Stats) ==
 
One row per RTP stream. Multiple streams per call.


{| class="wikitable"
{| class="wikitable"
Line 272: Line 218:
! Column !! Description
! Column !! Description
|-
|-
| <code>dscp</code> || DSCP values from SIP and RTP packets for QoS analysis
| <code>cdr_ID</code> || FK to <code>cdr</code>
|-
|-
| <code>payload</code> || Codec payload type number used during the call
| <code>ssrc</code> || RTP Synchronization Source identifier
|}
|-
 
| <code>saddr</code> / <code>daddr</code> || Source/destination IP (as integer)
== Important Related Tables ==
 
=== <code>cdr_next</code> ===
 
A 1-to-1 extension of the <code>cdr</code> table for additional data, keeping the main table smaller. This table is linked to <code>cdr</code> via the <code>cdr_ID</code> foreign key (where <code>cdr_next.cdr_ID = cdr.ID</code>).
 
{| class="wikitable"
|-
|-
! Column !! Description
| <code>sport</code> / <code>dport</code> || Source/destination port
|-
|-
| <code>fbasename</code> || String derived from SIP <code>Call-ID</code>, used to link PCAP files to CDR. Special characters in the Call-ID are replaced with underscores. This replacement behavior is controlled by the <code>convertchar = :</code> option in <code>voipmonitor.conf</code>
| <code>payload</code> || Codec type. '''Video streams: payload ≥ 10000''' (offset applied)
|-
|-
| <code>match_header</code> || Content of custom header (from <code>matchheader</code> in <code>voipmonitor.conf</code>) for linking call legs
| <code>received</code> / <code>lost</code> || Packet counts
|}
|}


'''Example: Finding PCAP filename for a call:'''
{{Tip|To query video streams only: <code>WHERE payload >= 10000</code>}}
<syntaxhighlight lang="sql">
SELECT c.ID, c.caller, c.called, cn.fbasename
FROM cdr c
JOIN cdr_next cn ON c.ID = cn.cdr_ID
WHERE c.ID = 12345;
</syntaxhighlight>
 
{{Note|
'''CDR Custom Headers Storage:'''
CDR-level custom SIP headers (one value per call) are stored dynamically in the <code>cdr_next_X</code> tables, not in the main <code>cdr</code> table. To find where a specific header is stored, you must query the <code>cdr_custom_headers</code> mapping table.
}}


=== <code>cdr_custom_headers</code> ===
== <code>cdr_sdp</code> (SDP-Negotiated Ports) ==


Lookup table that maps custom SIP header names to their storage locations in the <code>cdr_next_X</code> tables. You must query this table first to find the correct table and column for your custom header.
Stores media endpoints declared in SDP. Requires <code>save_sdp_ipport = yes</code>.


{| class="wikitable"
{| class="wikitable"
Line 313: Line 241:
! Column !! Description
! Column !! Description
|-
|-
| <code>header_field</code> || The name of the custom SIP header (e.g., <code>'X-IntNum'</code>)
| <code>ip</code> / <code>port</code> || Media IP and port from SDP
|-
| <code>dynamic_table</code> || The table number (e.g., <code>2</code> means <code>cdr_next_2</code>)
|-
|-
| <code>dynamic_column</code> || The column number (e.g., <code>1</code> means <code>custom_header_1</code>)
| <code>is_caller</code> || 1 = caller side, 0 = callee side
|}
|}


{{Tip|
{{Note|<code>cdr_sdp</code> stores what endpoints '''agreed to use''' in SDP; <code>cdr_rtp</code> stores what was '''actually sent'''.}}
The system automatically creates new <code>cdr_next_X</code> tables when needed and updates this mapping table. Each header gets its own dedicated column, not each call.
 
}}
== <code>cdr_proxy</code> (Proxy Chain) ==


=== Finding CDR Custom Header Storage ===
IP addresses of SIP proxies the call traversed.


Because CDR custom headers are stored in dynamically created tables, you must use the <code>cdr_custom_headers</code> table to find the correct table and column.
== <code>cdr_siphistory</code> (SIP Method History) ==


<syntaxhighlight lang="sql">
Stores SIP requests/responses including REFER, BYE, CANCEL, OPTIONS, etc. Requires <code>save_sip_history = all</code>.
-- Step 1: Find the dynamic table and column for your custom header
 
SELECT dynamic_table, dynamic_column
== <code>cdr_ua</code> / <code>cdr_sip_response</code> (Lookup Tables) ==
FROM cdr_custom_headers
WHERE header_field = 'X-IntNum';
</syntaxhighlight>


This query returns:
Lookup tables for User-Agent strings and SIP response texts.
* <code>dynamic_table</code>: The table number (e.g., <code>2</code> means <code>cdr_next_2</code>)
* <code>dynamic_column</code>: The column number (e.g., <code>1</code> means <code>custom_header_1</code>)


=== Querying CDRs with Custom Header Values ===
= Common SQL Queries =


Once you have the table and column information, you can query the <code>cdr_next_X</code> table for calls containing the desired header value.
== Basic CDR Query with IP Conversion ==


<syntaxhighlight lang="sql">
<syntaxhighlight lang="sql">
-- Example: Query cdr_next_2 for custom_header_1
SELECT
SELECT
     cdr.id,
     ID, caller, called,
    cdr.calldate,
     INET_NTOA(sipcallerip) AS caller_ip,
     cdr.caller,
     INET_NTOA(sipcalledip) AS called_ip,
     cdr.called,
     duration
     cdr_next_2.custom_header_1 AS X_IntNum
FROM cdr
FROM cdr
JOIN cdr_next_2 ON cdr.id = cdr_next_2.cdr_ID
WHERE calldate >= '2024-01-01'
WHERE cdr_next_2.custom_header_1 IS NOT NULL
  AND sipcallerip = INET_ATON('192.168.1.50')
ORDER BY cdr.calldate DESC
LIMIT 10;
LIMIT 10;
</syntaxhighlight>
</syntaxhighlight>


=== Complete Example Workflow ===
== Get PCAP Filename (fbasename) ==


<syntaxhighlight lang="sql">
<syntaxhighlight lang="sql">
-- Example: Find CDRs where X-Customer-ID = '12345'
SELECT c.ID, c.caller, c.called, cn.fbasename
FROM cdr c
JOIN cdr_next cn ON c.ID = cn.cdr_ID
WHERE c.ID = 12345;
</syntaxhighlight>


-- Step 1: Find where X-Customer-ID is stored
== Get User-Agent Strings ==
SELECT dynamic_table, dynamic_column
FROM cdr_custom_headers
WHERE header_field = 'X-Customer-ID';


-- Result example: dynamic_table=2, dynamic_column=1
<syntaxhighlight lang="sql">
-- This means the header is stored in cdr_next_2.custom_header_1
SELECT c.ID, ua_a.ua AS caller_ua, ua_b.ua AS callee_ua
 
-- Step 2: Query CDRs matching the custom header value
SELECT
    c.ID,
    c.calldate,
    c.caller,
    c.called,
    c.duration,
    cn.custom_header_1 AS customer_id
FROM cdr c
FROM cdr c
JOIN cdr_next_2 cn ON c.ID = cn.cdr_ID
LEFT JOIN cdr_ua ua_a ON c.a_ua_id = ua_a.ID
WHERE cn.custom_header_1 = '12345'
LEFT JOIN cdr_ua ua_b ON c.b_ua_id = ua_b.ID
ORDER BY c.calldate DESC
WHERE c.calldate >= CURDATE() - INTERVAL 1 DAY
LIMIT 100;
LIMIT 10;
</syntaxhighlight>
</syntaxhighlight>


{{Tip|
== Find Calls Through Specific Proxy ==
You can use variables or dynamic SQL to combine both steps into a single query when working programmatically.
}}


=== <code>cdr_rtp</code> ===
Stores detailed statistics for each individual RTP stream within a call. Multiple rows per call (one per stream).
{| class="wikitable"
|-
! Column !! Description
|-
| <code>cdr_ID</code> || Foreign key to <code>cdr</code> table (links to the primary call record)
|-
| <code>ssrc</code> || RTP Synchronization Source identifier (unique identifier for the stream)
|-
| <code>saddr</code> || Source IP address (stored as integer, use <code>INET_NTOA()</code> to convert)
|-
| <code>daddr</code> || Destination IP address (stored as integer, use <code>INET_NTOA()</code> to convert)
|-
| <code>payload</code> || Codec payload type number. Video streams are stored with an offset of <strong>+10000</strong> to distinguish them from audio streams
|-
| <code>received</code> || Number of RTP packets received
|-
| <code>lost</code> || Number of RTP packets lost (detected by sequence number gaps)
|-
| <code>jitter</code> || Jitter value (varies by schema version)
|}
{{Tip|
<strong>Identifying Video Streams:</strong> Video codecs (H.264, VP8, etc.) use dynamic payload types in RTP (typically 96-127). VoIPmonitor adds a <code>+10000</code> offset to video payload values in the database. To query only video streams, use <code>WHERE payload >= 10000</code>.}}
'''Example: Querying video stream quality:'''
<syntaxhighlight lang="sql">
<syntaxhighlight lang="sql">
-- Find calls with high packet loss on video RTP streams
SELECT c.ID, c.calldate, c.caller, c.called,
SELECT
      INET_NTOA(cp.dst) AS proxy_ip
    cdr.ID,
FROM cdr c
    cdr.calldate,
JOIN cdr_proxy cp ON c.ID = cp.cdr_ID
    cdr.caller,
WHERE cp.dst = INET_ATON('192.168.1.100')
    cdr.called,
LIMIT 100;
    INET_NTOA(cdr_rtp.saddr) AS source_ip,
    INET_NTOA(cdr_rtp.daddr) AS dest_ip,
    cdr_rtp.payload,
    cdr_rtp.received,
    cdr_rtp.lost,
    (cdr_rtp.lost / (cdr_rtp.received + cdr_rtp.lost)) * 100 AS loss_percentage
FROM cdr
JOIN cdr_rtp ON cdr.ID = cdr_rtp.cdr_ID
-- Filter for video streams (payload >= 10000)
WHERE cdr_rtp.payload >= 10000
  AND cdr.calldate >= '2024-01-01'
ORDER BY loss_percentage DESC
LIMIT 20;
</syntaxhighlight>
</syntaxhighlight>


'''Example: Finding all RTP streams in a call:'''
== Query Video Stream Quality ==
 
<syntaxhighlight lang="sql">
<syntaxhighlight lang="sql">
-- List all RTP streams (audio and video) for a specific call
SELECT cdr_ID, payload,
SELECT
      INET_NTOA(saddr) AS src_ip,
    payload,
      received, lost,
    CASE
      (lost / (received + lost)) * 100 AS loss_pct
        WHEN payload >= 10000 THEN 'Video'
        ELSE 'Audio'
    END AS stream_type,
    INET_NTOA(saddr) AS source_ip,
    INET_NTOA(daddr) AS dest_ip,
    received,
    lost,
    (lost / (received + lost)) * 100 AS loss_perc
FROM cdr_rtp
FROM cdr_rtp
WHERE cdr_ID = 12345;
WHERE payload >= 10000  -- Video streams only
  AND cdr_ID = 12345;
</syntaxhighlight>
</syntaxhighlight>


=== <code>cdr_dtmf</code> ===
== Find Calls with Specific SIP Method (REFER) ==
 
Stores detected DTMF key presses. Requires <code>dtmf2db = yes</code> in <code>voipmonitor.conf</code>.
 
=== <code>cdr_proxy</code> ===


Stores IP addresses of all SIP proxies the call traversed between caller and callee. Each proxy hop creates one row, allowing you to trace the full call path including intermediate servers.
{| class="wikitable"
|-
! Column !! Description
|-
| <code>cdr_ID</code> || Foreign key to <code>cdr</code> table (links to the primary call record)
|-
| <code>dst</code> || Proxy IP address (stored as integer, use <code>INET_NTOA()</code> to convert)
|}
{{Tip|
<strong>Proxy Chain Tracing:</strong> Multiple proxies may be involved in a single call (e.g., caller → SBC → gateway → callee). This table records each intermediate IP, allowing you to troubleshoot call routing issues or filter calls by specific proxy servers.}}
'''Example: Finding all proxy IPs for a specific call:'''
<syntaxhighlight lang="sql">
<syntaxhighlight lang="sql">
-- List all proxies a call passed through
SELECT cdr_ID, callid, calldate
SELECT
FROM cdr_siphistory
    cdr_ID,
WHERE method = 'REFER'
    INET_NTOA(dst) AS proxy_ip
  AND calldate > '2024-01-01'
FROM cdr_proxy
GROUP BY cdr_ID;
WHERE cdr_ID = 12345;
</syntaxhighlight>
</syntaxhighlight>


'''Example: Finding calls that used a specific proxy:'''
= Custom Headers =
<syntaxhighlight lang="sql">
-- Find all calls that went through a specific proxy server
SELECT
    c.ID,
    c.calldate,
    c.caller,
    c.called,
    INET_NTOA(c.sipcallerip) AS caller_ip,
    INET_NTOA(c.sipcalledip) AS called_ip,
    INET_NTOA(cp.dst) AS proxy_ip
FROM cdr c
JOIN cdr_proxy cp ON c.ID = cp.cdr_ID
WHERE cp.dst = INET_ATON('192.168.1.100')
ORDER BY c.calldate DESC
LIMIT 100;
</syntaxhighlight>


=== <code>cdr_sip_response</code> ===
== CDR-Level Custom Headers ==


Lookup table for unique SIP response texts (e.g., "404 Not Found"). The <code>cdr</code> table references by ID to save space.
CDR custom headers are stored in <code>cdr_next_X</code> tables. Use <code>cdr_custom_headers</code> to find the location:


=== <code>cdr_ua</code> ===
<syntaxhighlight lang="sql">
 
-- Step 1: Find table/column for header
Lookup table for <code>User-Agent</code> strings of SIP devices.
SELECT dynamic_table, dynamic_column
FROM cdr_custom_headers
WHERE header_field = 'X-Customer-ID';
-- Result: dynamic_table=2, dynamic_column=1 → cdr_next_2.custom_header_1


'''Example: Getting User-Agent strings for calls:'''
-- Step 2: Query the data
<syntaxhighlight lang="sql">
SELECT c.ID, c.calldate, cn.custom_header_1 AS customer_id
SELECT
    c.ID,
    c.caller,
    c.called,
    ua_a.ua AS caller_user_agent,
    ua_b.ua AS callee_user_agent
FROM cdr c
FROM cdr c
LEFT JOIN cdr_ua ua_a ON c.a_ua_id = ua_a.ID
JOIN cdr_next_2 cn ON c.ID = cn.cdr_ID
LEFT JOIN cdr_ua ua_b ON c.b_ua_id = ua_b.ID
WHERE cn.custom_header_1 = '12345';
WHERE c.calldate >= CURDATE() - INTERVAL 1 DAY
LIMIT 10;
</syntaxhighlight>
</syntaxhighlight>


=== <code>cdr_siphistory</code> ===
{{Note|The GUI's [[Reports|Reports]] module does not support grouping by custom headers. Use direct SQL queries.}}
=== <code>cdr_custom_headers</code> Table Structure ===


Stores detailed SIP request and response history for each call, including subdialog methods such as `REFER`, `BYE`, `CANCEL`, `OPTIONS`, `SUBSCRIBE`, `NOTIFY`, and `MESSAGE`. This table enables filtering CDRs by specific SIP methods via the GUI filter dropdown. Requires <code>save_sip_history = all</code> in <code>voipmonitor.conf</code>.
The <code>cdr_custom_headers</code> table stores the configuration for custom header extraction:


{| class="wikitable"
{| class="wikitable"
Line 538: Line 358:
! Column !! Description
! Column !! Description
|-
|-
| <code>cdr_ID</code> || Foreign key to <code>cdr</code> table (links to the primary call record)
| <code>header_field</code> || SIP header name to capture
|-
|-
| <code>method</code> || SIP request method (e.g., INVITE, REFER, BYE, CANCEL, OPTIONS, etc.)
| <code>dynamic_table</code> || Target table number (cdr_next_X)
|-
|-
| <code>calldate</code> || Timestamp of the SIP message
| <code>dynamic_column</code> || Target column number (custom_header_X)
|-
|-
| <code>callid</code> || SIP Call-ID header value
| <code>type</code> || Extraction type (first/last/nth)
|-
|-
| <code>cseq</code> || SIP CSeq (Command Sequence) number
| <code>nth_occurrence</code> || '''(New in 35.x)''' Specific occurrence number to extract when type is "nth" (e.g., 2 for second occurrence)
|}
|}


'''Prerequisite Configuration:'''
{{Note|The <code>nth_occurrence</code> column uses packet timestamps for accurate ordering, ensuring correct results even when packet time-based reordering is disabled in the sniffer.}}
Configure in <code>/etc/voipmonitor.conf</code>:
<syntaxhighlight lang="ini">
save_sip_history = all
</syntaxhighlight>
Restart the voipmonitor sniffer after changing this configuration.
 
'''Example: Find all calls containing a SIP REFER method:'''
<syntaxhighlight lang="sql">
SELECT
    cdr_ID,
    callid,
    calldate
FROM cdr_siphistory
WHERE calldate > '2024-01-01 00:00:00'
  AND method = 'REFER'
GROUP BY cdr_ID;
</syntaxhighlight>
 
'''Example: Find calls with OPTIONS or SUBSCRIBE messages (for troubleshooting qualify pings):'''
<syntaxhighlight lang="sql">
SELECT
    cdr_ID,
    callid,
    method,
    calldate
FROM cdr_siphistory
WHERE calldate > CURDATE() - INTERVAL 1 DAY
  AND method IN ('OPTIONS', 'SUBSCRIBE')
ORDER BY calldate DESC
LIMIT 20;
</syntaxhighlight>
 
== SIP Message Custom Headers ==
== SIP Message Custom Headers ==


VoIPmonitor can capture custom SIP headers from individual SIP messages (not just CDR-level headers). These are stored dynamically in the <code>sip_msg_next_X</code> tables, where each custom header gets its own dedicated column.
Per-message headers are stored in <code>sip_msg_next_X</code> tables. Use <code>sip_msg_custom_headers</code> to find the location.


{{Note|
= System Table =
'''CDR vs SIP Message Headers:'''
* <strong>CDR custom headers</strong> are stored in the <code>cdr_next</code> table for call-level data (one value per call)
* <strong>SIP message custom headers</strong> are stored in <code>sip_msg_next_X</code> tables for individual SIP messages (one value per message)
}}


=== Finding the Table and Column for a Custom Header ===
The <code>system</code> table stores configuration including <code>manager_key</code> for sensor operations.
 
Because custom SIP headers are stored in dynamically created tables, you must first query the <code>sip_msg_custom_headers</code> table to find which table and column contains your header.


<syntaxhighlight lang="sql">
<syntaxhighlight lang="sql">
-- Step 1: Find the dynamic table and column for your custom header
SELECT * FROM voipmonitor.`system` WHERE type='manager_key'\G
SELECT dynamic_table, dynamic_column
FROM sip_msg_custom_headers
WHERE header_field = 'X-My-Custom-Header';
</syntaxhighlight>
</syntaxhighlight>


This query returns:
{{Warning|1=If sniffer fails with "failed read rsa key", check that <code>mysqlloadconfig</code> is NOT set to <code>no</code> in <code>voipmonitor.conf</code>.}}
* <code>dynamic_table</code>: The table number (e.g., <code>1</code> means <code>sip_msg_next_1</code>)
* <code>dynamic_column</code>: The column number (e.g., <code>2</code> means <code>custom_header_2</code>)


=== Querying SIP Messages with Custom Header Values ===
== Synchronizing manager_key Between Databases ==


Once you have the table and column information, you can query the <code>sip_msg_next_X</code> table for SIP messages containing the desired header value.
When multiple GUI instances share sensors (e.g., after database migration), mismatched <code>manager_key</code> values cause PCAP download failures.


'''Symptoms:''' "Unable to download PCAP" errors, empty SIP history in one GUI.
'''Fix:'''
<syntaxhighlight lang="sql">
<syntaxhighlight lang="sql">
-- Example: Query sip_msg_next_1 for custom_header_2 = 'desired-value'
-- On active database: get the key
-- Step 2: Retrieve SIP messages matching the custom header value
SELECT content FROM voipmonitor.`system` WHERE type='manager_key';
SELECT
 
    sip_msg_ID,
-- On old database: update the key
    time,
UPDATE voipmonitor.`system`
    custom_header_2 AS header_value
SET content = '{"key":"<key_from_active>","iv":"<iv_from_active>"}'
FROM sip_msg_next_1
WHERE type = 'manager_key';
WHERE custom_header_2 = 'desired-value'
  AND time > NOW() - INTERVAL 15 MINUTE;
</syntaxhighlight>
</syntaxhighlight>


{{Tip|
Users must log out and log back in after the update.
You can also join this with other SIP message tables if needed. The SIP messages are linked to CDRs through the <code>sip_msg_ID</code> column and related message tables.
}}


=== Complete Example Workflow ===
= Troubleshooting =


<syntaxhighlight lang="sql">
== Missing <code>sport</code>/<code>dport</code> in cdr_rtp ==
-- Example: Find SIP messages with X-Customer-ID = '12345' in the last 1 hour


-- Step 1: Find where X-Customer-ID is stored
If RTP streams from overlapping calls are incorrectly merged, add the missing columns:
SELECT dynamic_table, dynamic_column
FROM sip_msg_custom_headers
WHERE header_field = 'X-Customer-ID';


-- Example result: dynamic_table=3, dynamic_column=1
<syntaxhighlight lang="sql">
 
ALTER TABLE cdr_rtp ADD `sport` smallint unsigned DEFAULT NULL AFTER `saddr`;
-- Step 2: Query the appropriate table
ALTER TABLE cdr_rtp ADD `dport` smallint unsigned DEFAULT NULL AFTER `daddr`;
SELECT
    sm.sip_msg_ID,
    sm.time,
    sm.custom_header_1 AS customer_id
FROM sip_msg_next_3 sm
WHERE sm.custom_header_1 = '12345'
  AND sm.time > NOW() - INTERVAL 1 HOUR
ORDER BY sm.time DESC;
</syntaxhighlight>
</syntaxhighlight>


=== Configuration Requirements ===
{{Warning|1=<code>ALTER TABLE</code> can lock large tables for hours. Run during maintenance windows.}}
 
To capture custom SIP headers, configure them in <code>/etc/voipmonitor.conf</code>:
 
<syntaxhighlight lang="ini">
[general]
# List custom headers to capture from SIP messages
sip_headers = X-Customer-ID, X-Branch-Code, X-Transaction-ID
</syntaxhighlight>


After modifying the configuration, restart the sensor:
To find other recommended schema changes:
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
service voipmonitor restart
journalctl -u voipmonitor | grep ALTER
</syntaxhighlight>
</syntaxhighlight>


The system will automatically create new columns in the appropriate <code>sip_msg_next_X</code> table and update the <code>sip_msg_custom_headers</code> mapping table.
= See Also =


=== <code>system</code> Table ===
* [[Call_Detail_Record_-_CDR|CDR View (GUI)]]
* [[Database_troubleshooting|Database Troubleshooting]]
* [[Sniffer_configuration#Database_Configuration|Database Configuration]]
* [[WEB_API|Web API Documentation]]


The <code>system</code> table stores system-wide configuration and keys used by VoIPmonitor.
;<code>manager_key</code>
:The <code>system</code> table contains a <code>manager_key</code> entry (identified by <code>type='manager_key'</code>) which is required for manager/sensor operations. If the sniffer fails to start with the error "failed read rsa key," this indicates the manager key cannot be loaded from the database.
:'''Important:''' Loading configuration from database (including the manager_key) is controlled by the <code>mysqlloadconfig</code> option in <code>voipmonitor.conf</code> (documented in [[Sniffer_configuration#Database_Configuration|Sniffer Configuration]]). If <code>mysqlloadconfig=no</code> is set, VoIPmonitor will NOT load the manager_key from the database, causing the "failed read rsa key" error on startup.
:'''Check manager_key in database:'''
<syntaxhighlight lang="sql">
-- Query the manager_key from the system table
SELECT * FROM voipmonitor.`system` WHERE type='manager_key'\G
</syntaxhighlight>


:'''Troubleshooting manager_key errors:'''
= AI Summary for RAG =
:# Check that <code>mysqlloadconfig</code> is NOT set to <code>no</code> in <code>/etc/voipmonitor.conf</code>
:# Query the <code>system</code> table to verify the manager_key exists using the SQL above
:# Check database connectivity permissions (VoIPmonitor needs read access to the <code>system</code> table)
:# Review syslog or journalctl for additional database error messages around the time of the startup failure
:# Ensure both probe and server have consistent configuration in <code>/etc/voipmonitor.conf</code> files


== AI Summary for RAG ==
'''Summary:''' VoIPmonitor database schema reference. Main table <code>cdr</code> stores one record per call with timing, participants, and quality metrics. Column naming: <code>a_</code>=caller, <code>b_</code>=callee, <code>_multNN</code>=divide by NN. Key tables: <code>cdr_next</code> (1:1, contains <code>fbasename</code> for PCAP linking), <code>cdr_rtp</code> (per-stream stats, video streams have payload≥10000), <code>cdr_proxy</code> (proxy chain IPs), <code>cdr_sdp</code> (SDP-negotiated ports), <code>cdr_siphistory</code> (SIP method history, requires <code>save_sip_history=all</code>). Custom headers stored in <code>cdr_next_X</code> tables; query <code>cdr_custom_headers</code> for mappings. System table stores <code>manager_key</code>; mismatch causes PCAP download failures.
'''Summary:''' Reference for VoIPmonitor database schema. The <code>cdr</code> table stores one record per call with timing, participant info, and quality metrics. Column naming: <code>a_</code> = caller stream, <code>b_</code> = callee stream, <code>_multNN</code> = divide by NN. Key metrics include packet loss (<code>a_lost</code>), jitter (<code>a_avgjitter_mult10</code>), MOS scores, and RTCP-reported metrics. Related tables: <code>cdr_next</code> (1-to-1 with <code>cdr</code>, linked via <code>cdr_ID</code> foreign key; contains <code>fbasename</code> from SIP Call-ID for PCAP file linking, where special characters are replaced with underscores via <code>convertchar</code> option in <code>voipmonitor.conf</code>), <code>cdr_rtp</code> (per-stream RTP stats), <code>cdr_proxy</code> (intermediate SIP proxy IPs), <code>cdr_ua</code> (User-Agent lookup). The <code>cdr_rtp</code> table stores individual RTP stream statistics; <strong>video streams are identified by <code>payload >= 10000</code></strong> (VoIPmonitor adds +10000 offset to distinguish video from audio). The <code>cdr_proxy</code> table stores proxy IP addresses a call traversed; query with <code>select INET_NTOA(dst) from cdr_proxy where cdr_ID=123;</code> or <code>join cdr_proxy on cdr.ID = cdr_proxy.cdr_ID</code> to find all calls going through a specific proxy. CDR custom headers require querying <code>cdr_custom_headers</code> to find dynamic table/column mappings, then querying <code>cdr_next_X</code> tables. SIP message custom headers use <code>sip_msg_custom_headers</code> and <code>sip_msg_next_X</code> tables. The <code>system</code> table stores system-wide configuration including <code>manager_key</code> (type='manager_key') which is required for manager/sensor operations. If the sniffer fails to start with "failed read rsa key" error, check that <code>mysqlloadconfig</code> in <code>voipmonitor.conf</code> is NOT set to <code>no</code> and query the manager_key with `SELECT * FROM voipmonitor.system WHERE type='manager_key'\G`. Use <code>SHOW TABLES LIKE 'cdr%';</code> to list all CDR-related tables.


'''Keywords:''' database, schema, cdr, mysql, mariadb, packet loss, jitter, mos, rtcp, a_lost, b_lost, fbasename, convertchar, user-agent, cdr_next, cdr_rtp, cdr_proxy, proxy ip, intermediate proxy, sip proxy chain, sip_msg, custom headers, cdr_custom_headers, cdr_next_X, sip_msg_custom_headers, sip_msg_next, video stream, payload, ssrc, saddr, daddr, system table, manager_key, manager_key type, failed read rsa key, mysqlloadconfig, mysqlloadconfig=no
'''Keywords:''' database, schema, cdr, mysql, cdr_next, fbasename, cdr_rtp, video payload, cdr_proxy, cdr_sdp, cdr_siphistory, INET_NTOA, INET_ATON, sipcallerip, custom headers, cdr_custom_headers, manager_key, packet loss, jitter, mos, rtcp, a_rtcp_avgrtd_mult10, latency


'''Key Questions:'''
'''Key Questions:'''
* Where is the manager_key stored in the VoIPmonitor database?
* What do a_ and b_ prefixes mean in cdr table columns?
* Why does the sniffer fail to start with "failed read rsa key" error?
* How do I convert sipcallerip to readable IP address?
* How do I fix the "failed read rsa key" error on startup?
* How do I get the PCAP filename (fbasename) for a call?
* What does mysqlloadconfig=no do in voipmonitor.conf?
* How do I query video streams in cdr_rtp (payload >= 10000)?
* How do I check if the manager_key exists in the database?
* How do I find calls that went through a specific proxy?
* What is the system table in VoIPmonitor?
* How do I find calls with REFER or other SIP methods?
* How do I query the manager_key from the system table?
* How do I query CDR custom headers?
* What do the a_ and b_ prefixes mean in the cdr table?
* Why does "failed read rsa key" error occur on startup?
* How do I convert sipcallerip to a readable IP address?
* How do I sync manager_key between multiple databases?
* How do I get the Call-ID for a call from the database?
* Why can't old GUI download PCAP files after database migration?
* Which table stores the User-Agent of phones?
* How do I query video streams or video call quality using cdr_rtp?
* How do I identify video streams in the database (payload filtering)?
* How do I find calls that went through a specific proxy server?
* How do I find all intermediate proxy IPs for a specific call?
* Which table stores SIP proxy IP addresses a call traversed?
* How do I find calls that transited through a specific IP address?
* How do I query CDRs with a custom header value?
* How do I find which cdr_next_X table stores my custom header?
* How do I query SIP messages with a custom header value?
* Why is my custom SIP header not in the cdr table?
* How do I find the database table and column for a custom SIP header added via web interface?
* A custom SIP header cannot be found in the main cdr table for querying - how do I query it?
* Where are CDR custom headers stored in the database?
* How do I use cdr_custom_headers table to find custom header data?

Latest revision as of 11:29, 19 January 2026


Database schema reference for VoIPmonitor. The cdr table stores one record per call; related tables store RTP statistics, custom headers, and metadata.

Schema Overview

-- List all CDR-related tables
SHOW TABLES LIKE 'cdr%';

Column Naming Conventions

Pattern Meaning Example
a_ Caller's stream (A-leg) a_lost, a_mos_f1_mult10
b_ Callee's stream (B-leg) b_lost, b_mos_f1_mult10
_mult10 Divide by 10 jitter_mult10 = 15 → jitter = 1.5
_mult100 Divide by 100 Percentage values
_mult1000 Divide by 1000 packet_loss_perc_mult1000

The cdr Table

Call Information & Timing

Column Description
ID Primary key
id_sensor Sensor ID (from voipmonitor.conf)
calldate / callend Start/end timestamps
duration Total length in seconds
connect_duration Connected time (after 200 OK)
progress_time Time to first provisional response (PDD)
bye 1 = normal termination with BYE
whohanged Which party sent BYE (caller/callee)
lastSIPresponse_id FK to cdr_sip_response

Participant Information

Column Description
caller / called Numbers from From/To headers
caller_domain / called_domain Domain parts
sipcallerip / sipcalledip IPs stored as integers (use INET_NTOA())
a_ua_id / b_ua_id FK to cdr_ua (User-Agent)

Quality Metrics

Packet Loss

Column Description
a_lost / b_lost Total RTP packets lost
packet_loss_perc_mult1000 Max loss percentage × 1000
a_sl1 - a_sl10 Distribution of consecutive loss events

Jitter & Delay

Column Description
a_avgjitter_mult10 Average jitter × 10
a_maxjitter Maximum jitter observed
a_d50 - a_d300 PDV distribution buckets (0-50ms, 50-70ms, etc.)

MOS (Mean Opinion Score)

Column Description
a_mos_f1_mult10 MOS with 50ms fixed buffer × 10
a_mos_f2_mult10 MOS with 200ms fixed buffer × 10
a_mos_adapt_mult10 MOS with adaptive buffer × 10
mos_min_mult10 Lowest MOS between A/B streams × 10

RTCP-Reported Metrics

These come from endpoint RTCP reports, not calculated by the sniffer. See Sniffer vs RTCP Packet Loss.

Column Description
a_rtcp_loss Packets lost (reported by endpoint)
a_rtcp_avgfr_mult10 Average fraction loss × 10
a_rtcp_avgjitter_mult10 Average jitter × 10
a_rtcp_avgrtd_mult10 Roundtrip delay (latency) in ms × 10

Related Tables

cdr_next (1:1 with cdr)

Extension table linked via cdr_ID.

Column Description
fbasename Derived from SIP Call-ID; used to link PCAP files. Special characters replaced with underscores (convertchar in config)
match_header Content of custom header for linking call legs (matchheader in config)

cdr_rtp (Per-Stream RTP Stats)

One row per RTP stream. Multiple streams per call.

Column Description
cdr_ID FK to cdr
ssrc RTP Synchronization Source identifier
saddr / daddr Source/destination IP (as integer)
sport / dport Source/destination port
payload Codec type. Video streams: payload ≥ 10000 (offset applied)
received / lost Packet counts

💡 Tip:

cdr_sdp (SDP-Negotiated Ports)

Stores media endpoints declared in SDP. Requires save_sdp_ipport = yes.

Column Description
ip / port Media IP and port from SDP
is_caller 1 = caller side, 0 = callee side

ℹ️ Note: cdr_sdp stores what endpoints agreed to use in SDP; cdr_rtp stores what was actually sent.

cdr_proxy (Proxy Chain)

IP addresses of SIP proxies the call traversed.

cdr_siphistory (SIP Method History)

Stores SIP requests/responses including REFER, BYE, CANCEL, OPTIONS, etc. Requires save_sip_history = all.

cdr_ua / cdr_sip_response (Lookup Tables)

Lookup tables for User-Agent strings and SIP response texts.

Common SQL Queries

Basic CDR Query with IP Conversion

SELECT
    ID, caller, called,
    INET_NTOA(sipcallerip) AS caller_ip,
    INET_NTOA(sipcalledip) AS called_ip,
    duration
FROM cdr
WHERE calldate >= '2024-01-01'
  AND sipcallerip = INET_ATON('192.168.1.50')
LIMIT 10;

Get PCAP Filename (fbasename)

SELECT c.ID, c.caller, c.called, cn.fbasename
FROM cdr c
JOIN cdr_next cn ON c.ID = cn.cdr_ID
WHERE c.ID = 12345;

Get User-Agent Strings

SELECT c.ID, ua_a.ua AS caller_ua, ua_b.ua AS callee_ua
FROM cdr c
LEFT JOIN cdr_ua ua_a ON c.a_ua_id = ua_a.ID
LEFT JOIN cdr_ua ua_b ON c.b_ua_id = ua_b.ID
WHERE c.calldate >= CURDATE() - INTERVAL 1 DAY
LIMIT 10;

Find Calls Through Specific Proxy

SELECT c.ID, c.calldate, c.caller, c.called,
       INET_NTOA(cp.dst) AS proxy_ip
FROM cdr c
JOIN cdr_proxy cp ON c.ID = cp.cdr_ID
WHERE cp.dst = INET_ATON('192.168.1.100')
LIMIT 100;

Query Video Stream Quality

SELECT cdr_ID, payload,
       INET_NTOA(saddr) AS src_ip,
       received, lost,
       (lost / (received + lost)) * 100 AS loss_pct
FROM cdr_rtp
WHERE payload >= 10000  -- Video streams only
  AND cdr_ID = 12345;

Find Calls with Specific SIP Method (REFER)

SELECT cdr_ID, callid, calldate
FROM cdr_siphistory
WHERE method = 'REFER'
  AND calldate > '2024-01-01'
GROUP BY cdr_ID;

Custom Headers

CDR-Level Custom Headers

CDR custom headers are stored in cdr_next_X tables. Use cdr_custom_headers to find the location:

-- Step 1: Find table/column for header
SELECT dynamic_table, dynamic_column
FROM cdr_custom_headers
WHERE header_field = 'X-Customer-ID';
-- Result: dynamic_table=2, dynamic_column=1 → cdr_next_2.custom_header_1

-- Step 2: Query the data
SELECT c.ID, c.calldate, cn.custom_header_1 AS customer_id
FROM cdr c
JOIN cdr_next_2 cn ON c.ID = cn.cdr_ID
WHERE cn.custom_header_1 = '12345';

ℹ️ Note: The GUI's Reports module does not support grouping by custom headers. Use direct SQL queries.

cdr_custom_headers Table Structure

The cdr_custom_headers table stores the configuration for custom header extraction:

Column Description
header_field SIP header name to capture
dynamic_table Target table number (cdr_next_X)
dynamic_column Target column number (custom_header_X)
type Extraction type (first/last/nth)
nth_occurrence (New in 35.x) Specific occurrence number to extract when type is "nth" (e.g., 2 for second occurrence)

ℹ️ Note: The nth_occurrence column uses packet timestamps for accurate ordering, ensuring correct results even when packet time-based reordering is disabled in the sniffer.

SIP Message Custom Headers

Per-message headers are stored in sip_msg_next_X tables. Use sip_msg_custom_headers to find the location.

System Table

The system table stores configuration including manager_key for sensor operations.

SELECT * FROM voipmonitor.`system` WHERE type='manager_key'\G

⚠️ Warning: If sniffer fails with "failed read rsa key", check that mysqlloadconfig is NOT set to no in voipmonitor.conf.

Synchronizing manager_key Between Databases

When multiple GUI instances share sensors (e.g., after database migration), mismatched manager_key values cause PCAP download failures.

Symptoms: "Unable to download PCAP" errors, empty SIP history in one GUI.

Fix:

-- On active database: get the key
SELECT content FROM voipmonitor.`system` WHERE type='manager_key';

-- On old database: update the key
UPDATE voipmonitor.`system`
SET content = '{"key":"<key_from_active>","iv":"<iv_from_active>"}'
WHERE type = 'manager_key';

Users must log out and log back in after the update.

Troubleshooting

Missing sport/dport in cdr_rtp

If RTP streams from overlapping calls are incorrectly merged, add the missing columns:

ALTER TABLE cdr_rtp ADD `sport` smallint unsigned DEFAULT NULL AFTER `saddr`;
ALTER TABLE cdr_rtp ADD `dport` smallint unsigned DEFAULT NULL AFTER `daddr`;

⚠️ Warning: ALTER TABLE can lock large tables for hours. Run during maintenance windows.

To find other recommended schema changes:

journalctl -u voipmonitor | grep ALTER

See Also


AI Summary for RAG

Summary: VoIPmonitor database schema reference. Main table cdr stores one record per call with timing, participants, and quality metrics. Column naming: a_=caller, b_=callee, _multNN=divide by NN. Key tables: cdr_next (1:1, contains fbasename for PCAP linking), cdr_rtp (per-stream stats, video streams have payload≥10000), cdr_proxy (proxy chain IPs), cdr_sdp (SDP-negotiated ports), cdr_siphistory (SIP method history, requires save_sip_history=all). Custom headers stored in cdr_next_X tables; query cdr_custom_headers for mappings. System table stores manager_key; mismatch causes PCAP download failures.

Keywords: database, schema, cdr, mysql, cdr_next, fbasename, cdr_rtp, video payload, cdr_proxy, cdr_sdp, cdr_siphistory, INET_NTOA, INET_ATON, sipcallerip, custom headers, cdr_custom_headers, manager_key, packet loss, jitter, mos, rtcp, a_rtcp_avgrtd_mult10, latency

Key Questions:

  • What do a_ and b_ prefixes mean in cdr table columns?
  • How do I convert sipcallerip to readable IP address?
  • How do I get the PCAP filename (fbasename) for a call?
  • How do I query video streams in cdr_rtp (payload >= 10000)?
  • How do I find calls that went through a specific proxy?
  • How do I find calls with REFER or other SIP methods?
  • How do I query CDR custom headers?
  • Why does "failed read rsa key" error occur on startup?
  • How do I sync manager_key between multiple databases?
  • Why can't old GUI download PCAP files after database migration?