Database structure: Difference between revisions

From VoIPmonitor.org
(Add detailed explanation of how User-Agent is extracted from SIP packets for caller (INVITE) and callee (SIP response))
(Improve formatting: add ER diagram, convert to wikitables, add SQL examples with syntax highlighting, add network topology diagram for RTCP vs sniffer loss, shorten AI Summary)
Line 3: Line 3:
'''This document provides an overview of the most important tables and columns in the VoIPmonitor database schema. It is intended as a reference for administrators, developers, or anyone directly querying the database.'''
'''This document provides an overview of the most important tables and columns in the VoIPmonitor database schema. It is intended as a reference for administrators, developers, or anyone directly querying the database.'''


The core of the VoIPmonitor database is the `cdr` (Call Detail Record) table, which stores one primary record for each monitored call. Many other tables are linked to `cdr` to store additional, more detailed information.
The core of the VoIPmonitor database is the <code>cdr</code> (Call Detail Record) table, which stores one primary record for each monitored call. Many other tables are linked to <code>cdr</code> to store additional, more detailed information.
 
== Database Schema Overview ==
 
The following diagram shows the relationships between the main VoIPmonitor database tables:
 
<kroki lang="plantuml">
@startuml
skinparam shadowing false
skinparam defaultFontName Arial
skinparam linetype ortho
 
entity "cdr" as cdr {
  * ID : bigint <<PK>>
  --
  id_sensor : int
  calldate : datetime
  callend : datetime
  caller : varchar
  called : varchar
  sipcallerip : int
  sipcalledip : int
  duration : int
  connect_duration : int
  a_ua_id : int <<FK>>
  b_ua_id : int <<FK>>
  lastSIPresponse_id : int <<FK>>
  a_lost / b_lost : int
  a_mos_f1_mult10 : tinyint
  ...
}
 
entity "cdr_next" as cdr_next {
  * cdr_ID : bigint <<PK,FK>>
  --
  fbasename : varchar
  match_header : varchar
  ...
}
 
entity "cdr_rtp" as cdr_rtp {
  * ID : bigint <<PK>>
  --
  cdr_ID : bigint <<FK>>
  saddr : int
  daddr : int
  ssrc : bigint
  received : int
  lost : int
  ...
}
 
entity "cdr_dtmf" as cdr_dtmf {
  * ID : bigint <<PK>>
  --
  cdr_ID : bigint <<FK>>
  dtmf : char
  ts : bigint
  ...
}
 
entity "cdr_proxy" as cdr_proxy {
  * ID : bigint <<PK>>
  --
  cdr_ID : bigint <<FK>>
  dst : int
  ...
}
 
entity "cdr_sip_response" as cdr_sip_response {
  * ID : int <<PK>>
  --
  lastSIPresponse : varchar
}
 
entity "cdr_ua" as cdr_ua {
  * ID : int <<PK>>
  --
  ua : varchar
}
 
cdr ||--|| cdr_next : "1:1"
cdr ||--o{ cdr_rtp : "1:N"
cdr ||--o{ cdr_dtmf : "1:N"
cdr ||--o{ cdr_proxy : "1:N"
cdr }o--|| cdr_sip_response : "N:1"
cdr }o--|| cdr_ua : "N:1 (a_ua_id)"
cdr }o--|| cdr_ua : "N:1 (b_ua_id)"
@enduml
</kroki>


== Understanding Column Naming Conventions ==
== Understanding Column Naming Conventions ==
Before diving into the columns, it's important to understand these common naming patterns:
Before diving into the columns, it's important to understand these common naming patterns:
*'''`a_` prefix:''' Refers to the "A-leg" or the '''caller's''' media stream (RTP sent from the caller).
*'''`b_` prefix:''' Refers to the "B-leg" or the '''callee's''' media stream (RTP sent from the callee).
*'''`_mult10` / `_mult100` / `_mult1000`:''' Indicates that the stored integer value must be '''divided''' by this number to get the real decimal value. For example, a `jitter_mult10` value of `15` represents a real jitter of `1.5`.


== The `cdr` Table ==
{| class="wikitable"
|-
! Pattern !! Meaning !! Example
|-
| <code>a_</code> prefix || Caller's media stream (A-leg, RTP sent from caller) || <code>a_lost</code>, <code>a_mos_f1_mult10</code>
|-
| <code>b_</code> prefix || Callee's media stream (B-leg, RTP sent from callee) || <code>b_lost</code>, <code>b_mos_f1_mult10</code>
|-
| <code>_mult10</code> || Divide stored value by 10 to get real value || <code>jitter_mult10 = 15</code> means jitter = 1.5
|-
| <code>_mult100</code> || Divide stored value by 100 to get real value || Used for percentage values
|-
| <code>_mult1000</code> || Divide stored value by 1000 to get real value || <code>packet_loss_perc_mult1000</code>
|}
 
== The <code>cdr</code> Table ==
 
This is the main table containing the primary information for every call.
This is the main table containing the primary information for every call.


=== General Call Information ===
=== General Call Information ===
;`ID`
 
:The unique primary key for the record.
{| class="wikitable"
;`id_sensor`
|-
:The ID of the sensor that processed this call, corresponding to `id_sensor` in `voipmonitor.conf`.
! Column !! Type !! Description
;`calldate`
|-
:The start time of the call, typically measured from the first `INVITE` packet.
| <code>ID</code> || bigint || The unique primary key for the record
;`callend`
|-
:The end time of the call, measured from the last packet associated with the call.
| <code>id_sensor</code> || int || The ID of the sensor that processed this call (corresponds to <code>id_sensor</code> in <code>voipmonitor.conf</code>)
;`bye`
|-
:An indicator of how the call was terminated. `1` typically means a normal termination with a `BYE` request.
| <code>calldate</code> || datetime || The start time of the call, measured from the first <code>INVITE</code> packet
;`whohanged`
|-
:Indicates which party sent the `BYE` message first (caller or callee).
| <code>callend</code> || datetime || The end time of the call, measured from the last packet associated with the call
;`lastSIPresponse_id`
|-
:A reference to the `cdr_sip_response` table, linking to the last final SIP response for the call.
| <code>bye</code> || tinyint || Indicates how the call was terminated (<code>1</code> = normal termination with <code>BYE</code>)
|-
| <code>whohanged</code> || enum || Indicates which party sent the <code>BYE</code> message first (caller or callee)
|-
| <code>lastSIPresponse_id</code> || int || Foreign key to <code>cdr_sip_response</code> table for the last final SIP response
|}


=== Call Timing & Duration ===
=== Call Timing & Duration ===
;`duration`
 
:The total length of the call in seconds (`callend` - `calldate`).
{| class="wikitable"
;`connect_duration`
|-
:The length of the connected portion of the call in seconds, measured from the first `200 OK` to the `callend`.
! Column !! Type !! Description
;`progress_time`
|-
:The time in seconds from the initial `INVITE` to the first provisional response (e.g., `180 Ringing`). This is often referred to as [[Glossary#Post_Dial_Delay_.28PDD.29|Post-Dial Delay (PDD)]].
| <code>duration</code> || int || Total call length in seconds (<code>callend - calldate</code>)
;`first_rtp_time`
|-
:The time in seconds from the start of the call to the detection of the very first RTP packet.
| <code>connect_duration</code> || int || Connected portion length in seconds (from first <code>200 OK</code> to <code>callend</code>)
|-
| <code>progress_time</code> || int || Time from <code>INVITE</code> to first provisional response (e.g., <code>180 Ringing</code>). Also known as [[Glossary#Post_Dial_Delay_.28PDD.29|Post-Dial Delay (PDD)]]
|-
| <code>first_rtp_time</code> || int || Time from call start to first RTP packet detection
|}


=== Participant Information ===
=== Participant Information ===
;`caller` / `called`
:The user part of the number from the `From` and `To` SIP headers, respectively.
;`caller_domain` / `called_domain`
:The domain part from the `From` and `To` SIP headers.
;`caller_reverse` / `called_reverse`
:Reversed versions of the `caller` and `called` fields, used internally for efficient `LIKE '%search'` queries in the database.
;`callername`
:The display name part from the `From` SIP header.
;`sipcallerip` / `sipcalledip`
:The source and destination IP addresses from the SIP signaling. These are stored as integers; use the `INET_NTOA()` or `INET6_NTOA()` SQL function to convert them to a string representation.
;`a_ua_id` / `b_ua_id`
:A reference to the `cdr_ua` table, linking to the User-Agent string of the caller's and callee's device.


{| style="width:100%; border:1px solid #ccc; background-color:#f9f9f9;"
{| class="wikitable"
|-
|-
| '''NOTE:''' How User-Agent is extracted from SIP packets
! Column !! Type !! Description
|
|-
* '''Caller (a_ua_id):''' Extracted from the <code>User-Agent</code> header in the <strong>SIP INVITE request</strong> sent by the caller
| <code>caller</code> / <code>called</code> || varchar || User part of the number from <code>From</code> and <code>To</code> SIP headers
* '''Callee (b_ua_id):''' Extracted from the <code>User-Agent</code> header in the <strong>SIP response</strong> (typically <code>200 OK</code>) sent by the called party
|-
* If the field is blank, it indicates that the <code>User-Agent</code> header was not present in the corresponding SIP packet
| <code>caller_domain</code> / <code>called_domain</code> || varchar || Domain part from <code>From</code> and <code>To</code> SIP headers
|-
| <code>caller_reverse</code> / <code>called_reverse</code> || varchar || Reversed strings for efficient <code>LIKE '%search'</code> queries
|-
| <code>callername</code> || varchar || Display name from the <code>From</code> SIP header
|-
| <code>sipcallerip</code> / <code>sipcalledip</code> || int unsigned || Source/destination IP addresses stored as integers (use <code>INET_NTOA()</code> to convert)
|-
| <code>a_ua_id</code> / <code>b_ua_id</code> || int || Foreign keys to <code>cdr_ua</code> table for User-Agent strings
|}
|}
{{Note|
'''How User-Agent is extracted:'''
* '''Caller (<code>a_ua_id</code>):''' From the <code>User-Agent</code> header in the SIP INVITE request
* '''Callee (<code>b_ua_id</code>):''' From the <code>User-Agent</code> header in the SIP response (typically <code>200 OK</code>)
* Blank values indicate the <code>User-Agent</code> header was not present
}}
'''Example: Converting IP addresses in SQL queries:'''
<syntaxhighlight lang="sql">
-- Get calls with readable IP addresses
SELECT
    ID,
    caller,
    called,
    INET_NTOA(sipcallerip) AS caller_ip,
    INET_NTOA(sipcalledip) AS called_ip,
    duration
FROM cdr
WHERE calldate >= '2024-01-01'
LIMIT 10;
</syntaxhighlight>


=== Network Quality Metrics ===
=== Network Quality Metrics ===
==== Packet Loss ====
==== Packet Loss ====
;`a_lost` / `b_lost`
 
:The total number of RTP packets lost, detected by sequence number gaps.
{| class="wikitable"
;`lost`
|-
:The higher value between `a_lost` and `b_lost`.
! Column !! Description
;`packet_loss_perc_mult1000`
|-
:The maximum packet loss percentage from either stream, multiplied by 1000.
| <code>a_lost</code> / <code>b_lost</code> || Total RTP packets lost (detected by sequence number gaps)
;`a_sl1` - `a_sl10` / `b_sl1` - `b_sl10`
|-
:A distribution of consecutive packet loss events. `a_sl1` counts the number of times a single packet was lost, `a_sl5` counts the number of times 5 packets were lost in a row, etc.
| <code>lost</code> || Higher value between <code>a_lost</code> and <code>b_lost</code>
|-
| <code>packet_loss_perc_mult1000</code> || Maximum packet loss percentage, multiplied by 1000
|-
| <code>a_sl1</code> - <code>a_sl10</code> / <code>b_sl1</code> - <code>b_sl10</code> || Distribution of consecutive packet loss events (sl1 = single packet lost, sl5 = 5 packets in a row, etc.)
|}


==== Jitter & Delay (PDV) ====
==== Jitter & Delay (PDV) ====
;`a_avgjitter_mult10` / `b_avgjitter_mult10`
 
:The average jitter calculated according to RFC3550, multiplied by 10.
{| class="wikitable"
;`a_maxjitter` / `b_maxjitter`
|-
:The maximum jitter value observed during the call.
! Column !! Description
;`jitter_mult10`
|-
:The higher `maxjitter` value between the two streams, multiplied by 10.
| <code>a_avgjitter_mult10</code> / <code>b_avgjitter_mult10</code> || Average jitter per RFC 3550, multiplied by 10
;`a_d50` - `a_d300` / `b_d50` - `b_d300`
|-
:A distribution of Packet Delay Variation (PDV). `a_d50` counts packets with a delay between 0-50ms, `a_d70` counts packets with a delay between 50-70ms, etc.
| <code>a_maxjitter</code> / <code>b_maxjitter</code> || Maximum jitter value observed during the call
|-
| <code>jitter_mult10</code> || Higher <code>maxjitter</code> value between streams, multiplied by 10
|-
| <code>a_d50</code> - <code>a_d300</code> / <code>b_d50</code> - <code>b_d300</code> || Packet Delay Variation (PDV) distribution buckets (d50 = 0-50ms, d70 = 50-70ms, etc.)
|}


==== MOS (Mean Opinion Score) ====
==== MOS (Mean Opinion Score) ====
;`a_mos_f1_mult10` / `b_mos_f1_mult10`
 
:The calculated [[Glossary#VoIPmonitor.27s_Parametric_MOS|parametric MOS score]] based on a simulated 50ms fixed jitter buffer, multiplied by 10.
{| class="wikitable"
;`a_mos_f2_mult10` / `b_mos_f2_mult10`
|-
:MOS score based on a simulated 200ms fixed jitter buffer, multiplied by 10.
! Column !! Description
;`a_mos_adapt_mult10` / `b_mos_adapt_mult10`
|-
:MOS score based on a simulated adaptive jitter buffer, multiplied by 10.
| <code>a_mos_f1_mult10</code> / <code>b_mos_f1_mult10</code> || [[Comprehensive_Guide_to_VoIP_Voice_Quality#MOS_Calculation_in_VoIPmonitor|Parametric MOS]] with 50ms fixed jitter buffer, multiplied by 10
;`mos_min_mult10`
|-
:The lowest MOS score recorded between the A and B streams, multiplied by 10.
| <code>a_mos_f2_mult10</code> / <code>b_mos_f2_mult10</code> || MOS with 200ms fixed jitter buffer, multiplied by 10
|-
| <code>a_mos_adapt_mult10</code> / <code>b_mos_adapt_mult10</code> || MOS with adaptive jitter buffer (up to 500ms), multiplied by 10
|-
| <code>mos_min_mult10</code> || Lowest MOS score between A and B streams, multiplied by 10
|}


==== RTCP Reported Metrics ====
==== RTCP Reported Metrics ====
These metrics are sourced from RTCP reports sent by the endpoints themselves, as opposed to being calculated by the sniffer.
;`a_rtcp_loss` / `b_rtcp_loss`
:Total packets lost as reported by RTCP.
;`a_rtcp_avgfr_mult10` / `b_rtcp_avgfr_mult10`
:Average fraction loss reported by RTCP, multiplied by 10.
;`a_rtcp_avgjitter_mult10` / `b_rtcp_avgjitter_mult10`
:Average jitter reported by RTCP, multiplied by 10.


These metrics come from RTCP Receiver Reports sent by the endpoints, not calculated by the sniffer.
{| class="wikitable"
|-
! Column !! Description
|-
| <code>a_rtcp_loss</code> / <code>b_rtcp_loss</code> || Total packets lost as reported by endpoint via RTCP
|-
| <code>a_rtcp_avgfr_mult10</code> / <code>b_rtcp_avgfr_mult10</code> || Average fraction loss from RTCP, multiplied by 10
|-
| <code>a_rtcp_avgjitter_mult10</code> / <code>b_rtcp_avgjitter_mult10</code> || Average jitter from RTCP, multiplied by 10
|}
==== Understanding Sniffer vs RTCP Packet Loss ====


==== Understanding the Difference Between Sniffer and RTCP Packet Loss ====
VoIPmonitor provides two different packet loss metrics measured from different network locations:


VoIPmonitor provides two different packet loss metrics that measure the same concept (lost VoIP packets) from two fundamentally different locations in the network. Understanding the distinction is critical for troubleshooting call quality issues.
<kroki lang="plantuml">
@startuml
skinparam shadowing false
skinparam defaultFontName Arial


The Two Perspectives:
rectangle "Caller\nPhone" as caller
;'''Sniffer Loss (`a_lost` / `b_lost`):''' The VoIPmonitor sniffer watches the traffic at its monitoring point (e.g., on the PBX switch) and counts missing packets based on RTP sequence number gaps. This represents what the network path ''up to the monitoring point'' experienced.
rectangle "Network\nSegment A" as netA
;'''RTCP Loss (`a_rtcp_loss` / `b_rtcp_loss`):''' The receiving endpoint (phone, gateway, or SBC) sends RTP Control Protocol (RTCP) Receiver Reports back to the transmitter, listing the cumulative count of packets it failed to receive or decode. This represents what the ''actual end-user device'' experienced.
rectangle "VoIPmonitor\nSniffer" as sniffer
rectangle "Network\nSegment B" as netB
rectangle "Callee\nPhone" as callee


Common Scenarios and Interpretations:
caller --> netA : RTP
netA --> sniffer : RTP
sniffer --> netB : RTP
netB --> callee : RTP


;'''1. RTCP Loss is Higher Than Sniffer Loss'''
callee --> caller : RTCP Reports
This indicates a network issue exists in the segment '''after''' the VoIPmonitor sniffer. Typical causes:
* Poor Wi-Fi connectivity at the end-user's home/office
* Network issues in the last-mile connection
* Problems between the monitoring point (e.g., PBX) and the actual receiver device
In this case, the sniffer sees all packets on the wire (low loss), but some packets are lost downstream before reaching the user's phone.


;'''2. Sniffer Loss is Higher Than RTCP Loss'''
note bottom of sniffer
This is unusual and typically indicates:
  **Sniffer Loss (a_lost/b_lost)**
* The monitoring point itself is dropping packets (e.g., due to high CPU load or interface saturation)
  Measures loss up to this point
* The RTCP reports from the endpoint are not being captured or transmitted properly
  based on RTP sequence gaps
end note


;'''3. High RTCP Loss but MOS is Still Good'''
note bottom of callee
This can occur when:
  **RTCP Loss (a_rtcp_loss/b_rtcp_loss)**
* Packet Loss Concealment (PLC) is effectively hiding scattered losses
  Measures loss at endpoint
* The endpoint's jitter buffer is successfully absorbing jitter-related delays
  via RTCP Receiver Reports
* The MOS calculation (based on sniffer loss) doesn't fully represent the user's actual experience
end note
@enduml
</kroki>


Troubleshooting Guidance:
'''Interpretation Guide:'''
* '''Trust RTCP loss for user experience''': For calls where users report quality issues, `a_rtcp_loss`/`b_rtcp_loss` is more authoritative because it reflects what the device actually received.
* '''Use sniffer loss for network health''': Monitoring `a_lost`/`b_lost` is useful for capacity planning and detecting issues in the network segment you control.
* '''Downstream issues''': When RTCP loss is higher, investigate the network segment between the monitoring point and the end-user device (check Wi-Fi, local network, ISP issues).


{| class="wikitable"
|-
! Scenario !! Likely Cause
|-
| RTCP loss > Sniffer loss || Network issue '''after''' the sniffer (Wi-Fi, last-mile, endpoint network)
|-
| Sniffer loss > RTCP loss || Monitoring point dropping packets (CPU overload, interface saturation) or RTCP reports not captured
|-
| High RTCP loss, good MOS || Packet Loss Concealment (PLC) effectively hiding scattered losses
|}


'''Troubleshooting tips:'''
* Trust RTCP loss for user experience issues - it reflects what the device actually received
* Use sniffer loss for network health monitoring in the segment you control
* When RTCP loss is higher, investigate the network path between sniffer and endpoint


=== Other Information ===
=== Other Columns ===
;`dscp`
 
:Stores the DSCP values from SIP and RTP packets for Quality of Service (QoS) analysis.
{| class="wikitable"
;`payload`
|-
:The codec payload type number used during the call.
! Column !! Description
|-
| <code>dscp</code> || DSCP values from SIP and RTP packets for QoS analysis
|-
| <code>payload</code> || Codec payload type number used during the call
|}


== Important Related Tables ==
== Important Related Tables ==


; `cdr_next`
=== <code>cdr_next</code> ===
:A 1-to-1 extension of the `cdr` table that stores additional data to keep the main `cdr` table smaller and faster.
:*'''fbasename:''' A string derived from the SIP `Call-ID`, used to link PCAP files to the CDR.
:*'''match_header:''' Stores the content of a custom header (defined by `matchheader` in `voipmonitor.conf`) used for linking call legs.


; `cdr_rtp`
A 1-to-1 extension of the <code>cdr</code> table for additional data, keeping the main table smaller.
:Stores detailed statistics for each individual RTP stream within a call, used by the GUI to render stream information in the call history view.


; `cdr_dtmf`
{| class="wikitable"
:Stores detected DTMF key presses (if `dtmf2db` is enabled).
|-
! Column !! Description
|-
| <code>fbasename</code> || String derived from SIP <code>Call-ID</code>, used to link PCAP files to CDR
|-
| <code>match_header</code> || Content of custom header (from <code>matchheader</code> in <code>voipmonitor.conf</code>) for linking call legs
|}
 
'''Example: Finding PCAP filename for a call:'''
<syntaxhighlight lang="sql">
SELECT c.ID, c.caller, c.called, cn.fbasename
FROM cdr c
JOIN cdr_next cn ON c.ID = cn.cdr_ID
WHERE c.ID = 12345;
</syntaxhighlight>
 
=== <code>cdr_rtp</code> ===
 
Stores detailed statistics for each individual RTP stream within a call. Multiple rows per call (one per stream).
 
=== <code>cdr_dtmf</code> ===
 
Stores detected DTMF key presses. Requires <code>dtmf2db = yes</code> in <code>voipmonitor.conf</code>.
 
=== <code>cdr_proxy</code> ===
 
Stores IP addresses of all SIP proxies the call traversed.
 
=== <code>cdr_sip_response</code> ===
 
Lookup table for unique SIP response texts (e.g., "404 Not Found"). The <code>cdr</code> table references by ID to save space.


; `cdr_proxy`
=== <code>cdr_ua</code> ===
:Stores the IP addresses of all SIP proxies that the call traversed.


; `cdr_sip_response`
Lookup table for <code>User-Agent</code> strings of SIP devices.
:A lookup table storing the text of unique SIP responses (e.g., "404 Not Found"). The `cdr` table references this table by ID to save space.


; `cdr_ua`
'''Example: Getting User-Agent strings for calls:'''
:A lookup table storing the `User-Agent` strings of SIP devices.
<syntaxhighlight lang="sql">
SELECT
    c.ID,
    c.caller,
    c.called,
    ua_a.ua AS caller_user_agent,
    ua_b.ua AS callee_user_agent
FROM cdr c
LEFT JOIN cdr_ua ua_a ON c.a_ua_id = ua_a.ID
LEFT JOIN cdr_ua ua_b ON c.b_ua_id = ua_b.ID
WHERE c.calldate >= CURDATE() - INTERVAL 1 DAY
LIMIT 10;
</syntaxhighlight>


== AI Summary for RAG ==
== AI Summary for RAG ==
'''Summary:''' This document provides a reference for the VoIPmonitor database schema, focusing on the main `cdr` table and its most important related tables. It explains the column naming conventions, such as the `a_` (caller) and `b_` (callee) prefixes for RTP streams and the `_multNN` suffix indicating a value needs to be divided. The `cdr` table columns are logically grouped into categories: General Call Information (ID, calldate), Call Timing (duration, connect_duration, PDD), Participant Information (caller, called, IPs, User-Agents), and detailed Network Quality Metrics. The quality metrics section details columns for Packet Loss (lost counts, distribution), Jitter (average, max, PDV distribution), Mean Opinion Score (MOS), and RTCP-reported statistics. A dedicated section explains the conceptual difference between sniffer-calculated packet loss (based on RTP sequence gaps at the monitoring point) and RTCP-reported loss (from the endpoint device), including troubleshooting scenarios for when values differ. It also explains how User-Agent information is extracted: the caller's User-Agent (a_ua_id) comes from the SIP INVITE request User-Agent header, while the callee's User-Agent (b_ua_id) comes from the SIP response User-Agent header. Finally, it briefly describes the purpose of key related tables like `cdr_next` (for Call-ID), `cdr_rtp`, `cdr_dtmf`, `cdr_proxy`, and the lookup tables `cdr_sip_response` and `cdr_ua`.
'''Summary:''' Reference for VoIPmonitor database schema. The <code>cdr</code> table stores one record per call with timing, participant info, and quality metrics. Column naming: <code>a_</code> = caller stream, <code>b_</code> = callee stream, <code>_multNN</code> = divide by NN. Key metrics include packet loss (<code>a_lost</code>), jitter (<code>a_avgjitter_mult10</code>), and MOS scores. RTCP metrics (<code>a_rtcp_loss</code>) show endpoint-reported loss vs sniffer-calculated loss. Related tables: <code>cdr_next</code> (Call-ID via <code>fbasename</code>), <code>cdr_rtp</code> (per-stream stats), <code>cdr_ua</code> (User-Agent lookup).
'''Keywords:''' database, schema, table, column, cdr, cdr_next, cdr_rtp, database schema, sql, mysql, mariadb, calldate, duration, packet loss, jitter, mos, rtcp, pdd, a_lost, b_lost, a_avgjitter, mos_f1, a_mos_adapt, fbasename, user-agent, rtcp_loss, troubleshooting, downstream issues, endpoint reported loss, a_ua_id, b_ua_id, SIP INVITE, SIP response, User-Agent header
 
'''Keywords:''' database, schema, cdr, mysql, mariadb, packet loss, jitter, mos, rtcp, a_lost, b_lost, fbasename, user-agent, cdr_next, cdr_rtp
 
'''Key Questions:'''
'''Key Questions:'''
* What do the `a_` and `b_` prefixes mean in the `cdr` table?
* What do the a_ and b_ prefixes mean in the cdr table?
* How are jitter and packet loss stored in the VoIPmonitor database?
* How do I convert sipcallerip to a readable IP address?
* What is the difference between `duration` and `connect_duration`?
* What is the difference between sniffer loss and RTCP loss?
* How can I get the Call-ID for a specific call from the database?
* How do I get the Call-ID for a call from the database?
* Which table stores the User-Agent of the phone?
* Which table stores the User-Agent of phones?
* How do I convert the `sipcallerip` integer back to an IP address in SQL?
* What is the purpose of the `cdr_next` table?
* How are MOS scores stored in the database?
* What is the difference between sniffer loss (`a_lost`) and RTCP loss (`a_rtcp_loss`)?
* Why might RTCP loss be higher than sniffer loss?
* What does it mean when RTCP loss is high but MOS is still good?

Revision as of 19:09, 4 January 2026


This document provides an overview of the most important tables and columns in the VoIPmonitor database schema. It is intended as a reference for administrators, developers, or anyone directly querying the database.

The core of the VoIPmonitor database is the cdr (Call Detail Record) table, which stores one primary record for each monitored call. Many other tables are linked to cdr to store additional, more detailed information.

Database Schema Overview

The following diagram shows the relationships between the main VoIPmonitor database tables:

Understanding Column Naming Conventions

Before diving into the columns, it's important to understand these common naming patterns:

Pattern Meaning Example
a_ prefix Caller's media stream (A-leg, RTP sent from caller) a_lost, a_mos_f1_mult10
b_ prefix Callee's media stream (B-leg, RTP sent from callee) b_lost, b_mos_f1_mult10
_mult10 Divide stored value by 10 to get real value jitter_mult10 = 15 means jitter = 1.5
_mult100 Divide stored value by 100 to get real value Used for percentage values
_mult1000 Divide stored value by 1000 to get real value packet_loss_perc_mult1000

The cdr Table

This is the main table containing the primary information for every call.

General Call Information

Column Type Description
ID bigint The unique primary key for the record
id_sensor int The ID of the sensor that processed this call (corresponds to id_sensor in voipmonitor.conf)
calldate datetime The start time of the call, measured from the first INVITE packet
callend datetime The end time of the call, measured from the last packet associated with the call
bye tinyint Indicates how the call was terminated (1 = normal termination with BYE)
whohanged enum Indicates which party sent the BYE message first (caller or callee)
lastSIPresponse_id int Foreign key to cdr_sip_response table for the last final SIP response

Call Timing & Duration

Column Type Description
duration int Total call length in seconds (callend - calldate)
connect_duration int Connected portion length in seconds (from first 200 OK to callend)
progress_time int Time from INVITE to first provisional response (e.g., 180 Ringing). Also known as Post-Dial Delay (PDD)
first_rtp_time int Time from call start to first RTP packet detection

Participant Information

Column Type Description
caller / called varchar User part of the number from From and To SIP headers
caller_domain / called_domain varchar Domain part from From and To SIP headers
caller_reverse / called_reverse varchar Reversed strings for efficient LIKE '%search' queries
callername varchar Display name from the From SIP header
sipcallerip / sipcalledip int unsigned Source/destination IP addresses stored as integers (use INET_NTOA() to convert)
a_ua_id / b_ua_id int Foreign keys to cdr_ua table for User-Agent strings

ℹ️ Note: How User-Agent is extracted:

  • Caller (a_ua_id): From the User-Agent header in the SIP INVITE request
  • Callee (b_ua_id): From the User-Agent header in the SIP response (typically 200 OK)
  • Blank values indicate the User-Agent header was not present

Example: Converting IP addresses in SQL queries:

-- Get calls with readable IP addresses
SELECT
    ID,
    caller,
    called,
    INET_NTOA(sipcallerip) AS caller_ip,
    INET_NTOA(sipcalledip) AS called_ip,
    duration
FROM cdr
WHERE calldate >= '2024-01-01'
LIMIT 10;

Network Quality Metrics

Packet Loss

Column Description
a_lost / b_lost Total RTP packets lost (detected by sequence number gaps)
lost Higher value between a_lost and b_lost
packet_loss_perc_mult1000 Maximum packet loss percentage, multiplied by 1000
a_sl1 - a_sl10 / b_sl1 - b_sl10 Distribution of consecutive packet loss events (sl1 = single packet lost, sl5 = 5 packets in a row, etc.)

Jitter & Delay (PDV)

Column Description
a_avgjitter_mult10 / b_avgjitter_mult10 Average jitter per RFC 3550, multiplied by 10
a_maxjitter / b_maxjitter Maximum jitter value observed during the call
jitter_mult10 Higher maxjitter value between streams, multiplied by 10
a_d50 - a_d300 / b_d50 - b_d300 Packet Delay Variation (PDV) distribution buckets (d50 = 0-50ms, d70 = 50-70ms, etc.)

MOS (Mean Opinion Score)

Column Description
a_mos_f1_mult10 / b_mos_f1_mult10 Parametric MOS with 50ms fixed jitter buffer, multiplied by 10
a_mos_f2_mult10 / b_mos_f2_mult10 MOS with 200ms fixed jitter buffer, multiplied by 10
a_mos_adapt_mult10 / b_mos_adapt_mult10 MOS with adaptive jitter buffer (up to 500ms), multiplied by 10
mos_min_mult10 Lowest MOS score between A and B streams, multiplied by 10

RTCP Reported Metrics

These metrics come from RTCP Receiver Reports sent by the endpoints, not calculated by the sniffer.

Column Description
a_rtcp_loss / b_rtcp_loss Total packets lost as reported by endpoint via RTCP
a_rtcp_avgfr_mult10 / b_rtcp_avgfr_mult10 Average fraction loss from RTCP, multiplied by 10
a_rtcp_avgjitter_mult10 / b_rtcp_avgjitter_mult10 Average jitter from RTCP, multiplied by 10

Understanding Sniffer vs RTCP Packet Loss

VoIPmonitor provides two different packet loss metrics measured from different network locations:

Interpretation Guide:

Scenario Likely Cause
RTCP loss > Sniffer loss Network issue after the sniffer (Wi-Fi, last-mile, endpoint network)
Sniffer loss > RTCP loss Monitoring point dropping packets (CPU overload, interface saturation) or RTCP reports not captured
High RTCP loss, good MOS Packet Loss Concealment (PLC) effectively hiding scattered losses

Troubleshooting tips:

  • Trust RTCP loss for user experience issues - it reflects what the device actually received
  • Use sniffer loss for network health monitoring in the segment you control
  • When RTCP loss is higher, investigate the network path between sniffer and endpoint

Other Columns

Column Description
dscp DSCP values from SIP and RTP packets for QoS analysis
payload Codec payload type number used during the call

Important Related Tables

cdr_next

A 1-to-1 extension of the cdr table for additional data, keeping the main table smaller.

Column Description
fbasename String derived from SIP Call-ID, used to link PCAP files to CDR
match_header Content of custom header (from matchheader in voipmonitor.conf) for linking call legs

Example: Finding PCAP filename for a call:

SELECT c.ID, c.caller, c.called, cn.fbasename
FROM cdr c
JOIN cdr_next cn ON c.ID = cn.cdr_ID
WHERE c.ID = 12345;

cdr_rtp

Stores detailed statistics for each individual RTP stream within a call. Multiple rows per call (one per stream).

cdr_dtmf

Stores detected DTMF key presses. Requires dtmf2db = yes in voipmonitor.conf.

cdr_proxy

Stores IP addresses of all SIP proxies the call traversed.

cdr_sip_response

Lookup table for unique SIP response texts (e.g., "404 Not Found"). The cdr table references by ID to save space.

cdr_ua

Lookup table for User-Agent strings of SIP devices.

Example: Getting User-Agent strings for calls:

SELECT
    c.ID,
    c.caller,
    c.called,
    ua_a.ua AS caller_user_agent,
    ua_b.ua AS callee_user_agent
FROM cdr c
LEFT JOIN cdr_ua ua_a ON c.a_ua_id = ua_a.ID
LEFT JOIN cdr_ua ua_b ON c.b_ua_id = ua_b.ID
WHERE c.calldate >= CURDATE() - INTERVAL 1 DAY
LIMIT 10;

AI Summary for RAG

Summary: Reference for VoIPmonitor database schema. The cdr table stores one record per call with timing, participant info, and quality metrics. Column naming: a_ = caller stream, b_ = callee stream, _multNN = divide by NN. Key metrics include packet loss (a_lost), jitter (a_avgjitter_mult10), and MOS scores. RTCP metrics (a_rtcp_loss) show endpoint-reported loss vs sniffer-calculated loss. Related tables: cdr_next (Call-ID via fbasename), cdr_rtp (per-stream stats), cdr_ua (User-Agent lookup).

Keywords: database, schema, cdr, mysql, mariadb, packet loss, jitter, mos, rtcp, a_lost, b_lost, fbasename, user-agent, cdr_next, cdr_rtp

Key Questions:

  • What do the a_ and b_ prefixes mean in the cdr table?
  • How do I convert sipcallerip to a readable IP address?
  • What is the difference between sniffer loss and RTCP loss?
  • How do I get the Call-ID for a call from the database?
  • Which table stores the User-Agent of phones?