Alerts: Difference between revisions

From VoIPmonitor.org
(Add documentation for SIP failed Register (beta) alert type - detects registration floods from single IP using multiple usernames)
(Fix template syntax - remove curly braces from 'from' and 'to' field names)
 
(19 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{DISPLAYTITLE:Alerts & Reports}}
{{DISPLAYTITLE:Alerts & Reports}}
Category:GUI manual
[[Category:GUI manual]]


== Alerts & Reports ==
= Alerts & Reports =


Alerts & Reports generate email notifications based on QoS parameters or SIP error conditions. The system includes daily reports, ad hoc reports, and stores all generated items in history.
Email notifications triggered by QoS thresholds, SIP errors, or sensor health conditions. The system stores all alerts in history for review.
 
=== Overview ===
 
The alert system monitors call quality and SIP signaling in real-time, triggering notifications when configured thresholds are exceeded.


<kroki lang="plantuml">
<kroki lang="plantuml">
Line 14: Line 10:
skinparam shadowing false
skinparam shadowing false
skinparam defaultFontName Arial
skinparam defaultFontName Arial
 
rectangle "Sensor" as sensor
rectangle "VoIPmonitor\nSensor" as sensor
database "MySQL" as db
database "MySQL\nDatabase" as db
rectangle "Cron\n(1min)" as cron
rectangle "Cron Job\n(every minute)" as cron
rectangle "Alert\nProcessor" as processor
rectangle "Alert\nProcessor" as processor
rectangle "MTA\n(Postfix/Exim)" as mta
rectangle "MTA" as mta
actor "Admin" as admin
actor "Admin" as admin
 
sensor --> db : CDRs
sensor --> db : CDRs with\nQoS metrics
cron --> processor : Trigger
cron --> processor : Trigger
processor --> db : Query alerts\n& CDRs
processor --> db : Query
processor --> mta : Send email
processor --> mta : Email
mta --> admin : Alert notification
mta --> admin : Alert
@enduml
@enduml
</kroki>
</kroki>


=== Email Configuration Prerequisites ===
== Prerequisites ==


Emails are sent using PHP's <code>mail()</code> function, which relies on the server's Mail Transfer Agent (MTA) such as Exim, Postfix, or Sendmail. Configure your MTA according to your Linux distribution documentation.
=== Email Configuration ===


==== Setting the Email "From" Address ====
Alerts use PHP's <code>mail()</code> function via the server's MTA (Postfix/Exim/Sendmail).


To configure the "From" address that appears in outgoing alert emails:
{| class="wikitable"
 
|-
;Navigate to: GUI > Settings > System Configuration > Email / HTTP Referer
! Setting !! Location !! Description
;Locate the field: DEFAULT_EMAIL_FROM (Default "From" address for outgoing emails)
|-
;Set your desired email address (e.g., <code>alerts@yourcompany.com</code>)
| From Address || GUI > Settings > System Configuration > Email || <code>DEFAULT_EMAIL_FROM</code> - sender address for all alerts
 
|-
This setting applies to all automated emails sent by VoIPmonitor, including:
| Cron Job || <code>/etc/crontab</code> || Required for alert processing
* QoS alerts (RTP, SIP response, sensors)
|}
* Daily reports
* License notifications
 
==== Setting Up the Cron Job ====
 
Alert processing requires a cron job that runs every minute:


<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
# Add to /etc/crontab (adjust path based on your GUI installation)
# Add cron job (required)
echo "* * * * * root php /var/www/html/php/run.php cron" >> /etc/crontab
echo "* * * * * root php /var/www/html/php/run.php cron" >> /etc/crontab
# Reload crontab
killall -HUP cron  # Debian/Ubuntu
killall -HUP cron  # Debian/Ubuntu
# or
# or: killall -HUP crond  # RHEL/CentOS
killall -HUP crond  # CentOS/RHEL
</syntaxhighlight>
</syntaxhighlight>


=== Configure Alerts ===
== Alert Types ==
 
Email alerts can trigger on SIP protocol events or RTP QoS metrics. Access alerts configuration via '''GUI > Alerts'''.
 
[[File:alertgrid.png|frame|center|Alert configuration grid]]


==== Alert Types ====
Access via '''GUI > Alerts'''.


===== RTP Alerts =====
=== RTP Alerts ===


RTP alerts trigger based on voice quality metrics:
Trigger on voice quality metrics:
* '''MOS''' (Mean Opinion Score) - below threshold
* '''MOS''' - below threshold
* '''Packet loss''' - percentage exceeded
* '''Packet loss''' - percentage exceeded
* '''Jitter''' - variation exceeded
* '''Jitter''' - variation exceeded
* '''Delay''' (PDV) - latency exceeded
* '''Delay''' (PDV) - latency exceeded
* '''One-way calls''' - answered but one RTP stream missing
* '''One-way calls''' - one RTP stream missing
* '''Missing RTP''' - answered but both RTP streams missing
* '''Missing RTP''' - both RTP streams missing
 
Configure alerts to trigger when:
* Number of incidents exceeds a set value, OR
* Percentage of CDRs exceeds a threshold
 
[[File:alertrtpform.png|frame|center|RTP alert configuration form]]
 
===== RTP&CDR Alerts =====
 
RTP&CDR alerts combine RTP quality metrics with CDR-based conditions, including Post Dial Delay (PDD). These alerts are useful for monitoring call setup performance and detecting network latency issues.
 
'''Available Conditions:'''
 
In the '''filter-common tab''', you can configure conditions including:
* '''PDD (Post Dial Delay)''' - Time between sending INVITE and receiving final response. Configure with comparison operators like <code>PDD > 5</code> (in seconds) to alert on long call setup delays. This can also detect [[Sniffer_troubleshooting#Routing_Loops|routing loops]] where looping calls continuously retransmit INVITE without receiving responses (PDD will be very large).
 
In the '''base config tab''':
* Set the '''recipient email address''' for alert notifications
* Consider limiting the '''max-lines in body''' to prevent oversized emails when many CDRs match the alert condition
 
===== SIP Response Alerts =====
 
SIP response alerts trigger based on SIP response codes:
* '''Empty response field''': Matches all call attempts per configured filters
* '''Response code 0''': Matches unreplied INVITE requests (no response received). This is useful for detecting [[Sniffer_troubleshooting#Routing_Loops|routing loops]] where calls continuously loop and never receive any SIP response.
* '''Specific codes''': Match exact codes like 404, 503, etc.
 
[[File:alertsipform.png|frame|center|SIP response alert configuration form]]
 
===== Percentage Alerts and the "from all" Checkbox =====
 
SIP response alerts can trigger based either on the '''number of incidents''' or the '''percentage of CDRs''' exceeding a threshold.
 
When setting a percentage threshold (e.g., <code>>10%</code>):
 
* '''"from all" checkbox CHECKED''': The percentage is calculated from '''ALL CDRs in the database''' (not just those matching filters).
* '''"from all" checkbox UNCHECKED''': The percentage is calculated only from CDRs that match your '''common filters''' (IP groups, numbers, etc.). This is the correct setting when monitoring a specific IP group.
 
'''Example: Monitor 503 responses for a specific IP group'''
 
To alert when the percentage of SIP 503 responses from the "Datora" IP group exceeds 10%:
 
1. Navigate to '''GUI > Alerts > filter common''' subtab
2. In '''IP/Number Group''', select your group (e.g., "Datora")
3. '''UNCHECK "from all"''' - this ensures the percentage is calculated only from CDRs involving IPs in the Datora group
4. In the '''Base config''' subtab:
  * Set '''Type''' to '''SIP Response'''
  * Set '''Response code''' to <code>503</code>
  * Set '''Incidents threshold''' to <code>>10%</code>
5. Configure recipient emails and save
 
If you leave "from all" CHECKED, the alert would calculate the 503 percentage across ALL CDRs in your database, which defeats the purpose of monitoring a specific IP group.
 
===== Detecting 408 Request Timeout Failures =====
 
A '''408 Request Timeout''' response occurs when the caller sends multiple INVITE retransmissions and receives no final response. This is useful for alerting on calls that timeout after the UAS (User Agent Server) sends a provisional response like '''100 Trying''' but then fails to send any further responses.
 
'''Use Cases:'''
* Detect failing PBX or SBC (Session Border Controller) instances that accept calls but stop processing
* Monitor network failures where SIP messages stop flowing after initial dialog establishment
* Identify servers that become unresponsive mid-call setup
 
'''Configuration:'''
1. Navigate to '''GUI > Alerts'''
2. Create new alert with type '''SIP Response'''
3. Set '''Response code''' to <code>408</code>
4. Optionally add Common Filters (IP addresses, numbers) to narrow scope
5. Save the alert
 
'''Understanding the Difference Between Response Code 0 and 408:'''
* '''Response code 0'': Matches calls that received absolutely no response (not even a 100 Trying). These are network or reachability issues.
* '''Response code 408''': Matches calls that received at least one provisional response (like 100 Trying) but eventually timed out. These indicate a server or application layer problem where the UAS stopped responding after initial acknowledgment.
 
Note: When a call times out with a 408 response, the CDR stores <code>408</code> as the Last SIP Response. Alerting on 408 will catch all call setup timeouts, including those where a 100 Trying was initially received.
 
===== Sensors Alerts =====
 
Sensors alerts monitor the health of VoIPmonitor probes and sniffer instances. This is the most reliable method to check if remote sensors are online and actively monitoring traffic.
 
Unlike simple network port monitoring (which may show a port as open even if the process is frozen or unresponsive), sensors alerts verify that the sensor instance is actively communicating with the VoIPmonitor GUI server.
 
;Setup:
:# Configure sensors in '''Settings > Sensors'''
:# Create a sensors alert to be notified when a probe goes offline or becomes unresponsive
 
===== Sensor Health Monitoring Conditions =====
 
In addition to detecting offline sensors, you can configure sensors alert conditions to monitor sensor performance issues:
 
<br>
'''Conditions you can configure:'''
 
* '''Old CDR''' - Alerts when the sensor has not written CDRs to the database recently. This indicates the sensor is either not capturing calls, or there is a database insertion bottleneck preventing CDRs from being committed.
 
* '''Big SQL queue stat''' - Alerts when the SQL cache queue is growing. The SQL queue represents CDRs waiting in memory to be written to the database. A growing queue indicates the database cannot keep up with write operations.
 
<br>
'''SQL Queue Threshold Guidance:'''
 
The SQL queue is measured in the number of cache files waiting. A healthy sensor should maintain a low SQL count.
 
* '''Normal operation''': SQL queue should remain near 0 during all traffic conditions.
* '''Warning level''': SQL queue above 20 files indicates a significant delay between packet capture and database insertion.
* '''Critical level''': SQL queue above 100 files indicates severe database performance issues requiring immediate attention.


When configuring a "Big SQL queue stat" alert, setting the threshold to 20 files provides early warning before the problem escalates to critical levels.
Configure alerts to trigger when number of incidents OR percentage of CDRs exceeds threshold.


<br>
=== RTP&CDR Alerts ===
'''Configuring Alert Actions:'''


When a sensor health alert triggers, you can configure the following actions:
Combine RTP metrics with CDR conditions including '''PDD (Post Dial Delay)'''.


* '''Email notification''' - Send alerts to administrators via email.
'''Using Filter Templates:'''
* '''External script execution''' - Execute a custom script with arguments about the triggering sensor and condition. This enables integration with monitoring systems like Nagios or Zabbix, or automated remediation workflows.
# Create CDR filter in '''GUI > CDR'''
# Save as template
# In alert config, select from '''Filter template''' dropdown


The external script receives information about which specific sensor triggered the alert and which health condition was violated, allowing you to build automated responses tailored to the type of failure.
{{Tip|1=Use filter templates for complex conditions like <code>duration > 14400</code> (calls over 4 hours) or <code>absolute_timeout</code> (truncated recordings).}}


===== SIP REGISTER RRD Beta Alerts =====
=== SIP Response Alerts ===
 
The '''SIP REGISTER RRD beta''' alert type monitors SIP REGISTER response times and alerts when REGISTER packets do not receive a response within a specified threshold (in milliseconds). This is useful for detecting network latency issues, packet loss, or failing switches that cause SIP retransmissions.
 
This alert serves as an effective proxy to monitor for registration issues, as REGISTER retransmissions often indicate problems with network connectivity or unresponsive SIP servers.
 
;Configuration:
:# Navigate to '''GUI > Alerts'''
:# Create a new alert with type '''SIP REGISTER RRD beta'''
:# Set the response time threshold in milliseconds (e.g., alert if REGISTER does not receive a response within 2000ms)
:# Configure recipient email addresses
:# Save the alert configuration
 
The system monitors REGISTER packets and triggers an alert when responses exceed the configured threshold, indicating potential SIP registration failures or network issues.
 
===== SIP Failed Register Beta Alerts =====
 
The '''SIP failed Register (beta)''' alert type detects SIP registration floods from a single IP address using multiple different usernames. This is a common attack pattern used in brute-force or credential-stuffing attacks where the attacker tries many different usernames from one source IP to find valid credentials.
 
Unlike the basic "SIP REGISTER flood" alert (which counts total registration attempts regardless of success/failure or username), this alert specifically monitors '''failed''' registrations and aggregates them by source IP address to detect patterns that indicate credential-guessing attacks.
 
;Use Cases:
:* Detect credential-stuffing attacks (one IP trying many different usernames in brute-force attempts)
:* Identify botnets attempting account takeovers by cycling through username lists
:* Monitor for registration abuse patterns that may indicate dictionary attacks
:* Alert administrators when an IP shows signs of attempting unauthorized access to SIP accounts
 
;How It Works:
 
This alert triggers when the total number of '''failed''' SIP registrations from any single IP address exceeds a specified threshold within a configured time interval. By focusing on failed registrations and grouping by source IP, it catches floods that use a variety of usernames from the same source.
 
;Configuration:
:# Navigate to '''GUI > Alerts'''
:# Create a new alert with type '''SIP failed Register (beta)'''
:# Set the '''threshold''' - maximum number of failed registrations allowed from a single IP (e.g., 20 failed registrations)
:# Set the '''interval''' - time window in seconds to evaluate (e.g., 60 seconds to check for 20 failed registrations in one minute)
:# Configure recipient email addresses
:# Optionally add Common Filters (IP addresses, numbers) to narrow scope
:# Save the alert configuration
 
;Example Scenario:
 
An attacker attempts to brute-force SIP credentials by sending REGISTER requests with 50 different usernames from IP 203.0.113.50 within 60 seconds. All 50 attempts fail because the credentials are invalid.
 
If you configure the alert with threshold=20 and interval=60 seconds, the system will:
# Detect 50 failed registrations from 203.0.113.50 within 60 seconds
# Compare (50 failed) > (threshold 20) = TRUE
# Trigger an alert notifying administrators about the potential registration flood attack from IP 203.0.113.50
 
;Comparison with Other Registration Alerts:


{| class="wikitable"
{| class="wikitable"
|-
|-
! Alert Type !! What It Monitors !! Attack Detection
! Response Code !! Meaning
|-
|-
| '''SIP failed Register (beta)''' || Failed registrations grouped by IP || Brute-force, credential stuffing
| Empty || All call attempts per filters
|-
|-
| SIP REGISTER RRD beta || REGISTER response times || Network latency, packet loss
| '''0''' || No response received (routing loops)
|-
|-
| multiple register (beta) || Same account from multiple IPs || Compromised credentials, misuse
| '''408''' || Timeout after provisional response (server unresponsive)
|-
|-
| Realtime REGISTER flood || Total REGISTER attempts (any status) from IP || Flood/spam of any registration
| Specific || Exact codes (404, 503, etc.)
|}
|}


The '''SIP failed Register (beta)''' alert is specifically optimized to detect attacks that use many different usernames from a single IP, which is the hallmark of credential-guessing or dictionary attacks. Use this alert in combination with other anti-fraud rules like [[Anti-fraud|Anti-Fraud Rules]] for comprehensive registration attack detection.
==== "from all" Checkbox (Percentage Thresholds) ====
 
{{Warning|1=This setting is critical for IP group monitoring.}}
 
* '''CHECKED''': % calculated from ALL CDRs in database
* '''UNCHECKED''': % calculated only from filtered CDRs (correct for specific IP groups)


===== CDR Trends Alerts =====


The '''CDR trends''' alert type enables trend-based monitoring and alerting on aggregated CDR statistics, including ASR (Answer Seizure Ratio) and other metrics. This alert type compares current performance against historical baselines and triggers notifications when metrics deviate beyond configurable thresholds.


'''Use Cases:'''
==== SIP Response vs Last SIP Response ====
* Monitor ASR drops or increases over time windows
* Detect sudden changes in call volume patterns
* Compare current hour/day/week against historical data
* Identify quality degradation trends before they become critical


'''Configuration Parameters:'''
There are two different fields for matching SIP responses:


{| class="wikitable"
{| class="wikitable"
|-
|-
! Parameter !! Description !! Example Values
! Field !! Location !! Supports % Threshold !! Use Case
|-
| '''Type''' || The metric to monitor for trend changes || ASR (Answer Seizure Ratio), Call count, ACD, etc.
|-
| '''Offset''' || Historical baseline period to compare against || 1 week, 1 day, 1 month
|-
| '''Range''' || Current time window to evaluate || 1 hour, 1 day, 1 week
|-
|-
| '''Method''' || Calculation method for trend comparison || Deviation (detects % change), Threshold (absolute value)
| '''SIP response''' || GUI > Alerts > SIP Response Alerts || {{Yes}} || Match by numeric code (e.g., 487, 503)
|-
|-
| '''Limit Inc./Limit Dec.''' || Percentage threshold for triggering alerts || 10%, 15%, 20%
| '''Last sip response''' || GUI > Alerts > Filter common || {{No}} || Match by full text (e.g., "487 Request Terminated")
|-
| '''IP whitelist''' || Optional filter to limit scope to specific IPs/agents || Source IP addresses or user agents
|}
|}


'''Example Configuration - ASR Trend Alert:'''
{{Warning|1=The GUI '''cannot trigger alerts based on percentage of full textual response strings'''. If you need percentage-based triggering for SIP response codes, use the '''SIP response''' numeric field instead.}}


To receive an alert when ASR drops by 10% compared to the previous week:
The '''Last sip response''' field supports wildcard patterns (%, %Request Terminated%, %487%) but only triggers based on count thresholds, not percentages.
=== International Call Alerts (Called Number Prefixes) ===


1. Navigate to '''GUI > Alerts'''
Monitor calls to international destinations using '''prefix-based matching''' (dialing patterns like 00, +).
2. Create new alert with type '''CDR trends'''
3. Configure parameters:
* '''Type:''' ASR
* '''Offset:''' 1 week (compare current period to previous week)
* '''Range:''' 1 hour (evaluate hourly)
* '''Method:''' Deviation (percentage-based comparison)
* '''Limit Dec.:''' 10% (trigger when drop exceeds 10%)
* '''IP whitelist:''' (optional) specify specific test user agents or IP addresses
4. Set recipient email addresses
5. Save the alert configuration


'''How Deviation Method Works:'''
{{Note|1=This uses phone number prefix detection, NOT IP geolocation. For GeoIP-based detection, see [[Anti-fraud|Anti-Fraud Rules]].}}


When using the '''Deviation''' method, the system calculates:
'''Configuration:'''
<syntaxhighlight lang="text">
# '''GUI > Settings > Country prefixes''' - Define international prefixes (00, +), local country, minimum digits
Deviation % = ((Current Value - Historical Baseline) / Historical Baseline) * 100
# '''GUI > Alerts > Filter common''' - Configure:
</syntaxhighlight>
 
* '''Limit Inc.''' triggers when Deviation % > threshold (e.g., ASR increased by 15%)
* '''Limit Dec.''' triggers when Deviation % < -threshold (e.g., ASR decreased by 10%)
 
A 10% ASR drop means current ASR is 90% of the historical baseline.
 
'''Understanding Offset vs Range:'''
 
* '''Offset''' defines the historical reference period (e.g., "1 week" means "same hour last week")
* '''Range''' defines the current evaluation window (e.g., "1 hour" means "current hour's ASR")
 
For example, with Offset=1 week and Range=1 hour, the system compares ASR for "today 09:00-10:00" against "last week 09:00-10:00".
 
===== Multiple Register Beta Alerts =====
 
The '''multiple register (beta)''' alert type detects SIP accounts that are registered from multiple different IP addresses. This is useful for identifying potential security issues, configuration errors, or unauthorized use of SIP credentials.
 
This alert specifically finds phone numbers or SIP accounts that have registered from more than one distinct IP address within the monitored timeframe.
 
;Use Cases:
:* Detect SIP account compromise (credential theft leading to registrations from unauthorized IPs)
:* Identify configuration issues where phones are registering from multiple networks unexpectedly
:* Monitor for roaming behavior when multi-IP registration is not expected
:* Audit SIP account usage across distributed environments
 
;Configuration:
:# Navigate to '''GUI > Alerts'''
:# Create a new alert with type '''multiple register (beta)'''
:# Configure recipient email addresses
:# Optionally add Common Filters (IP addresses, numbers) to narrow scope - '''leave filters empty to check ALL SIP numbers/accounts across all customers'''
:# Save the alert configuration
 
;Alert Scope:
:* '''With filters''': Monitors only the specific IP addresses, numbers, or groups defined in the Common Filters section
:* '''Without filters''': Monitors all SIP numbers/accounts across all customers in your system
 
The alert will trigger whenever it detects a SIP account that has registered from multiple distinct IP addresses, providing details about the affected account(s) and the IP addresses observed.
 
[[File:alertgrid.png|frame|center|Alert configuration grid]]
 
==== Common Filters ====
 
All alert types support the following filters:


{| class="wikitable"
{| class="wikitable"
|-
|-
! Filter !! Description
! Setting !! Description
|-
| IP/Number Group || Apply alert to predefined groups (from '''Groups''' menu)
|-
|-
| IP Addresses || Individual IPs or ranges (one per line)
| Called number prefixes || Which prefixes trigger alert (uncheck ALL for all international)
|-
|-
| Numbers || Individual phone numbers or prefixes (one per line)
| Exclude called number || Country codes to exclude (e.g., +44, 0044 for UK)
|-
|-
| Email Group || Send alerts to group-defined email addresses
| Strict for prefixes || Require international prefix (00/+)
|-
|-
| Emails || Individual recipient emails (one per line)
| NANPA || North American Numbering Plan
|-
| External script || Path to custom script to execute when alert triggers (see below)
|}
|}


[[File:alertgroup.png|frame|center|Alert filter configuration]]
=== Sensors Alerts ===
 
=== Using External Scripts for Alert Actions ===
 
Beyond email notifications, alerts can execute custom scripts when triggered. This enables integration with third-party systems (webhooks, Datadog, Slack, custom monitoring tools) without sending emails.
 
==== Configuration ====
 
1. Navigate to '''GUI > Alerts'''
2. Create or edit an alert (RTP, SIP Response, Sensors, etc.)
3. In the configuration form, locate the '''External script''' field
4. Enter the full path to your custom script (e.g., <code>/usr/local/bin/alert-webhook.sh</code>)
5. Save the alert configuration
 
The script will execute immediately when the alert triggers.


==== Script Arguments ====
Monitor sensor health and status:
* '''Offline detection''' - Sensor not communicating
* '''Old CDR''' - No recent CDRs written (capture or DB issue)
* '''Big SQL queue stat''' - Growing queue indicates DB bottleneck (warning: >20 files, critical: >100)


The custom script receives alert data as command-line arguments. The format is identical to anti-fraud scripts (see [[Anti-fraud|Anti-Fraud Rules]]):
=== SIP REGISTER Alerts ===


{| class="wikitable"
{| class="wikitable"
|-
|-
! Argument !! Description
! Alert Type !! Purpose !! Use Case
|-
|-
| <code>$1</code> || Alert ID (numeric identifier)
| '''SIP REGISTER RRD beta''' || Response time monitoring || Network latency, packet loss
|-
|-
| <code>$2</code> || Alert name/type
| '''SIP failed Register (beta)''' || Failed registrations by IP || Brute-force, credential stuffing
|-
|-
| <code>$3</code> || Unix timestamp of alert trigger
| '''multiple register (beta)''' || Same account from multiple IPs || Credential compromise detection
|-
| <code>$4</code> || JSON-encoded alert data
|}
|}


==== Alert Data Structure ====
{{Warning|1='''multiple register (beta)''' detects SIMULTANEOUS registrations from multiple IPs (security). For detecting IP changes when device moves networks, use CDR&RTP alert with external script.}}


The JSON in the fourth argument contains CDR IDs affected by the alert:


<syntaxhighlight lang="json">
{
  "cdr": [12345, 12346, 12347],
  "alert_type": "MOS below threshold",
  "threshold": 3.5,
  "actual_value": 2.8
}
</syntaxhighlight>


Use the <code>cdr</code> array to query additional information from the database if needed.
==== Alert Output Fields ====


==== Example: Send Webhook to Datadog ====
The '''multiple register (beta)''' and other SIP REGISTER alerts output the following fields in email notifications and GUI:
 
This bash script sends an alert notification to a Datadog webhook API:
 
<syntaxhighlight lang="bash">
#!/bin/bash
# /usr/local/bin/datadog-alert.sh
 
# Configuration
WEBHOOK_URL="https://webhook.site/your-custom-url"
DATADOG_API_KEY="your-datadog-api-key"
 
# Parse arguments
ALERT_ID="$1"
ALERT_NAME="$2"
TIMESTAMP="$3"
ALERT_DATA="$4"
 
# Convert Unix timestamp to readable date
DATE=$(date -d "@$TIMESTAMP" '+%Y-%m-%d %H:%M:%S')
 
# Extract relevant data from JSON
cdrCount=$(echo "$ALERT_DATA" | jq -r '.cdr | length')
threshold=$(echo "$ALERT_DATA" | jq -r '.threshold // empty')
actualValue=$(echo "$ALERT_DATA" | jq -r '.actual_value // empty')
 
# Build webhook payload
PAYLOAD=$(cat <<EOF
{
  "alert_id": "$ALERT_ID",
  "alert_name": "$ALERT_NAME",
  "triggered_at": "$DATE",
  "cdr_count": $cdrCount,
  "threshold": $threshold,
  "actual_value": $actualValue,
  "source": "voipmonitor"
}
EOF
)
 
# Send webhook
curl -X POST "$WEBHOOK_URL" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $DATADOG_API_KEY" \
  -d "$PAYLOAD"
</syntaxhighlight>
 
Make the script executable:
 
<syntaxhighlight lang="bash">
chmod +x /usr/local/bin/datadog-alert.sh
</syntaxhighlight>
 
==== Example: Send Slack Notification ====
 
<syntaxhighlight lang="bash">
#!/bin/bash
# /usr/local/bin/slack-alert.sh
 
SLACK_WEBHOOK="https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
 
ALERT_NAME="$2"
ALERT_DATA="$4"
cdrCount=$(echo "$ALERT_DATA" | jq -r '.cdr | length')
 
curl -X POST "$SLACK_WEBHOOK" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "VoIPmonitor Alert: '"$ALERT_NAME"'",
    "attachments": [{
      "color": "danger",
      "fields": [
        {"title": "CDRs affected", "value": "'"$cdrCount"'"}
      ]
    }]
  }'
</syntaxhighlight>
 
==== Example: Store Alert Details in File ====
 
<syntaxhighlight lang="bash">
#!/bin/bash
# /usr/local/bin/log-alert.sh
 
LOG_DIR="/var/log/voipmonitor-alerts"
mkdir -p "$LOG_DIR"
 
# Log all arguments for debugging
echo "=== Alert triggered at $(date) ===" >> "$LOG_DIR/alerts.log"
echo "Alert ID: $1" >> "$LOG_DIR/alerts.log"
echo "Alert name: $2" >> "$LOG_DIR/alerts.log"
echo "Timestamp: $3" >> "$LOG_DIR/alerts.log"
echo "Data: $4" >> "$LOG_DIR/alerts.log"
echo "" >> "$LOG_DIR/alerts.log"
</syntaxhighlight>
 
==== Example: Access Source IP Addresses ====
 
When querying the CDR database from an alert script, IP addresses are stored as decimal integers in the <code>cdr</code> table. To convert them to human-readable dotted-decimal format (e.g., <code>185.107.80.4</code>), use either PHP's <code>long2ip()</code> function or MySQL's <code>INET_NTOA()</code> function.
 
===== Using PHP's long2ip() (Recommended for Post-Processing) =====
 
If you fetch the raw integer value from the database and convert it in your script:
 
<syntaxhighlight lang="php">
#!/usr/bin/php
<?php
// Parse alert data
$alert = json_decode($argv[4]);
$cdrIds = implode(',', $alert->cdr);
 
// Query the CDR table - note: sipcallerip is a decimal integer
$query = "SELECT id, sipcallerip, sipcalledip
          FROM voipmonitor.cdr
          WHERE id IN ($cdrIds)";
$command = "mysql -h MYSQLHOST -u MYSQLUSER -pMYSQLPASS -N -e \"$query\"";
exec($command, $results);
 
// Process results and convert IP addresses
foreach ($results as $line) {
    list($id, $callerIP, $calledIP) = preg_split('/\t/', trim($line));
 
    // Convert decimal integer to dotted-decimal format
    $callerIPFormatted = long2ip($callerIP);
    $calledIPFormatted = long2ip($calledIP);
 
    echo "CDR ID $id: Caller IP $callerIPFormatted, Called IP $calledIPFormatted\n";
 
    // Example: long2ip(3110817796) returns "185.107.80.4"
}
?>
</syntaxhighlight>
 
===== Using MySQL's INET_NTOA() (Recommended for Database Queries) =====
 
If you prefer to handle conversion in the SQL query itself:
 
<syntaxhighlight lang="php">
#!/usr/bin/php
<?php
// Parse alert data
$alert = json_decode($argv[4]);
$cdrIds = implode(',', $alert->cdr);
 
// Query with IP conversion done in MySQL
$query = "SELECT id, INET_NTOA(sipcallerip) as caller_ip, INET_NTOA(sipcalledip) as called_ip
          FROM voipmonitor.cdr
          WHERE id IN ($cdrIds)";
$command = "mysql -h MYSQLHOST -u MYSQLUSER -pMYSQLPASS -N -e \"$query\"";
exec($command, $results);
 
// Process results - IPs are already formatted
foreach ($results as $line) {
    list($id, $callerIP, $calledIP) = preg_split('/\t/', trim($line));
    echo "CDR ID $id: Caller IP $callerIP, Called IP $calledIP\n";
}
?>
</syntaxhighlight>
 
===== Common IP Columns in CDR Table =====
 
The following columns contain IP addresses (all stored as decimal integers):


{| class="wikitable"
{| class="wikitable"
|-
|-
! Column !! Description
! Field !! Source !! Description
|-
|-
| <code>sipcallerip</code> || SIP signaling source IP
| '''username''' || SIP Contact header || The registered user identity
|-
|-
| <code>sipcalledip</code> || SIP signaling destination IP
| '''from''' fields || SIP From header || From-number, From-domain extracted from From header
|-
|-
| <code>rtpsrcipX</code> || RTP source IP for stream X (where X = 0-9)
| '''to''' fields || SIP To header || To-number, To-domain extracted from To header
|-
|-
| <code>rtpdstipX</code> || RTP destination IP for stream X (where X = 0-9)
| '''lookup name''' || Tools > Prefix Lookup || Custom label if phone number matches a configured prefix entry
|}
|}


===== Troubleshooting IP Format Issues =====
{{Note|1=The '''lookup name''' column displays custom labels from [[Tools#Prefix_Lookup|Prefix Lookup]] when a phone number matches a configured prefix. If no match exists, the field remains empty or shows the raw number.}}
=== CDR Trends Alerts ===


If your alert script receives IP addresses as large numbers (e.g., <code>3110817796</code> instead of <code>185.107.80.4</code>):
Monitor metric deviations from historical baselines (e.g., ASR drops).


1. Verify you are querying the <code>cdr</code> table directly (not using formatted variables)
2. Use <code>long2ip()</code> in PHP or <code>INET_NTOA()</code> in MySQL to convert the value
3. Check that the column is not already being converted by another layer of the application
For reference:
* <code>long2ip(3110817796)</code> returns <code>185.107.80.4</code>
* <code>long2ip(3232255785)</code> returns <code>192.168.1.101</code>
* <code>long2ip(2130706433)</code> returns <code>127.0.0.1</code>
==== Important Notes ====
* '''IP Address Format''': IP addresses in the <code>cdr</code> table are stored as decimal integers. Use <code>long2ip()</code> (PHP) or <code>INET_NTOA()</code> (MySQL) to convert to dotted-decimal format.
* '''Script execution time''': The alert processor waits for the script to complete. Keep scripts fast (under 5 seconds) or run them in the background if processing takes longer.
* '''Script permissions''': Ensure the script is executable by the web server user (typically <code>www-data</code> or <code>apache</code>).
* '''Error handling''': Script failures are logged but do not prevent email alerts from being sent.
* '''Querying CDRs''': The script receives CDR IDs in the JSON data. Query the <code>cdr</code> table to retrieve detailed information like caller numbers, call duration, etc.
* '''Security''': Validate input before using it in commands or database queries to prevent injection attacks.
=== Sent Alerts ===
All triggered alerts are saved in history and can be viewed via '''GUI > Alerts > Sent Alerts'''. The content matches what was sent via email.
[[File:alert-sentalerts.png|frame|center|Sent alerts history]]
==== Parameters Table ====
The parameters table shows QoS metrics with problematic values highlighted for quick identification.
[[File:alert-perameters.png|frame|center|Alert parameters with highlighted bad values]]
==== CDR Records Table ====
The CDR records table lists all calls that triggered the alert. Each row includes alert flags indicating which thresholds were exceeded:
* '''(M)''' - MOS below threshold
* '''(J)''' - Jitter exceeded
* '''(P)''' - Packet loss exceeded
* '''(D)''' - Delay exceeded
=== Anti-Fraud Alerts ===
VoIPmonitor includes specialized anti-fraud alert rules for detecting attacks and fraudulent activity. These include:
* Realtime concurrent calls monitoring
* SIP REGISTER flood/attack detection
* SIP PACKETS flood detection
* Country/Continent destination alerts
* CDR/REGISTER country change detection
For detailed configuration of anti-fraud rules and custom action scripts, see [[Anti-fraud|Anti-Fraud Rules]].
=== Alerts Based on Custom Reports ===
In addition to native alert types (RTP, SIP response, Sensors), VoIPmonitor supports generating alerts from custom reports. This workflow enables alerts based on criteria not available in native alert types, such as SIP header values captured via CDR custom headers.
==== Limitations of Custom Report Alerts ====
Custom report alerts are designed for filtering by SIP header values captured as CDR custom headers. They have the following limitations:
* '''No "Group By" functionality for threshold-based alerts:''' You cannot create an alert that triggers only when multiple events from the same caller ID or called number occur. For example, you cannot configure an alert that triggers when the same caller ID generates multiple SIP 486 responses, while ignoring single isolated failures from different callers.
* '''Scheduled reports vs. threshold alerts:''' CDR Summary reports can group data by caller number or called number, but these are scheduled reports that send data on a time-based schedule (e.g., daily), not threshold-based alerts that trigger when specific conditions are met.
* '''No alert-level aggregation:''' SIP Response alerts aggregate by total counts or percentages across all calls matching your filters, but cannot aggregate or group by specific caller/called numbers within those filtered results.
'''Workaround:'''
The closest available workflow is to create a **CDR Summary daily report** that:
1. Filters by the SIP response code (e.g., 486)
2. Groups by source number (caller) or destination number (called)
3. Sends a scheduled email (e.g., every 15 minutes or hourly)
This report will show which caller numbers have generated failures, but you must manually review the data to identify patterns. The report will be sent on schedule regardless of whether any failures occurred, and there is no way to configure it to only trigger when a specific threshold per unique caller is exceeded.
'''Feature Request:'''
Alerting based on grouped thresholds (e.g., "alert if the same caller ID generates >X SIP 486 responses") is a requested feature not currently available in VoIPmonitor. If you require this functionality, submit a feature request describing your specific use case.
==== Workflow Overview =====
1. [[Settings#CDR_Custom_Headers|Capture custom SIP headers]] in the database
2. Create a custom report filtered by the custom header values
3. Generate an alert from that report
4. Configure alert email options (e.g., limit email size)
==== Example: Alert on SIP Max-Forwards Header Value =====
This example shows how to receive an alert when the SIP <code>Max-Forwards</code> header value drops below 15.
'''Step 1: Configure Sniffer Capture'''
Add the header to your <code>/etc/voipmonitor.conf</code> configuration file:
<syntaxhighlight lang="ini">
# Capture Max-Forwards header
custom_headers = Max-Forwards
</syntaxhighlight>
Restart the sniffer to apply changes:
<syntaxhighlight lang="bash">
service voipmonitor restart
</syntaxhighlight>
'''Step 2: Configure Custom Header in GUI'''
1. Navigate to '''GUI > Settings > CDR Custom Headers'''
2. Select <code>Max-Forwards</code> from the available headers
3. Enable '''Show as Column''' to display it in CDR views
4. Save configuration
'''Step 3: Create Custom Report'''
1. Navigate to '''GUI > CDR Custom Headers''' or use the Report Generator
2. Create a filter for calls where <code>Max-Forwards</code> is less than 15
3. Since custom headers store string values, use a filter expression that matches the desired values:
<syntaxhighlight lang="text">
15 14 13 12 11 10 0_ _
</syntaxhighlight>
  Include additional space-separated values or use NULL to match other ranges as needed.
4. Run the report to verify it captures the expected calls
'''Step 4: Generate Alert from Report'''
You can create an alert based on this custom report using the Daily Reports feature:
1. Navigate to '''GUI > Reports > Configure Daily Reports'''
2. Click '''Add Daily Report'''
3. Configure the filter to target the custom header criteria (e.g., Max-Forwards < 15)
4. Set the schedule (e.g., run every hour)
5. Save the daily report configuration
'''Step 5: Limit Alert Email Size (Optional)'''
If the custom report generates many matching calls, the alert email can become large. To limit the email size:
1. Edit the daily report
2. Go to the '''Basic Data''' tab
3. Set the '''max-lines in body''' option to the desired limit (e.g., 100 lines)
==== Additional Use Cases =====
This workflow can be used for various custom monitoring scenarios:
* '''SIP headers beyond standard SIP response codes''' - Monitor any custom SIP header
* '''Complex filtering logic''' - Create reports based on multiple custom header filters
* '''Threshold monitoring for string fields''' - When numeric comparison is not available, use string matching
For more information on configuring custom headers, see [[Settings#CDR_Custom_Headers|CDR Custom Headers]].
=== Troubleshooting Email Alerts ===
If email alerts are not being sent, the issue is typically with the Mail Transfer Agent (MTA) rather than VoIPmonitor.
==== Step 1: Test Email Delivery from Command Line ====
Before investigating complex issues, verify your server can send emails:
<syntaxhighlight lang="bash">
# Test using the 'mail' command
echo "Test email body" | mail -s "Test Subject" your.email@example.com
</syntaxhighlight>
If this fails, the issue is with your MTA configuration, not VoIPmonitor.
==== Step 2: Check MTA Service Status ====
Ensure the MTA service is running:
<syntaxhighlight lang="bash">
# For Postfix (most common)
sudo systemctl status postfix
# For Exim (Debian default)
sudo systemctl status exim4
# For Sendmail
sudo systemctl status sendmail
</syntaxhighlight>
If the service is not running or not installed, install and configure it according to your Linux distribution's documentation.
==== Step 3: Check Mail Logs ====
Examine the MTA logs for specific error messages:
<syntaxhighlight lang="bash">
# Debian/Ubuntu
tail -f /var/log/mail.log
# RHEL/CentOS/AlmaLinux/Rocky
tail -f /var/log/maillog
</syntaxhighlight>
Common errors and their meanings:
{| class="wikitable"
{| class="wikitable"
|-
|-
! Error Message !! Cause !! Solution
! Parameter !! Description
|-
|-
| Connection refused || MTA not running or firewall blocking || Start MTA service, check firewall rules
| Type || Metric to monitor (ASR, ACD, etc.)
|-
|-
| Relay access denied || SMTP relay misconfiguration || See "Configuring SMTP Relay" below
| Offset || Historical baseline (1 week, 1 day)
|-
|-
| Authentication failed || Incorrect credentials || Verify credentials in sasl_passwd
| Range || Current evaluation window (1 hour)
|-
|-
| Host or domain name lookup failed || DNS issues || Check /etc/resolv.conf
| Method || Deviation (%) or Threshold (absolute)
|-
|-
| Greylisted || Temporary rejection || Wait and retry, or whitelist sender
| Limit Inc./Dec. || Trigger threshold percentage
|}
|}


==== Step 4: Check Mail Queue ====
== Common Filters ==


Emails may be stuck in the queue if delivery is failing:
All alert types support:
* '''IP/Number Group''' - Predefined groups from '''Groups''' menu
* '''IP Addresses''' / '''Numbers''' - Individual values (one per line)
* '''Email Group''' / '''Emails''' - Recipients
* '''Last sip response''' - Filter by response text (requires <code>save_sip_history = responses</code>)
* '''External script''' - Custom script path for integrations


<syntaxhighlight lang="bash">
{{Warning|1=Alerts use '''OR logic''' between conditions. AND logic is NOT supported. Workaround: create separate alerts and correlate manually.}}
# View the mail queue
mailq


# Force immediate delivery attempt
=== Caller vs Called Filtering ===
postqueue -f
</syntaxhighlight>


Deferred or failed messages in the queue contain error details explaining why delivery failed.
The Numbers filter matches against '''both caller and called fields'''. You cannot create alerts that trigger only when a number is the caller or only the called. Use IP Groups with Trunk/Server checkboxes for direction-based filtering. See [[Groups]].
 
==== Configuring SMTP Relay ===
 
If you encounter "Relay access denied" errors, your Postfix server cannot send emails through your external SMTP server. There are two solutions:
 
'''Solution 1: Configure External SMTP to Permit Relaying (Recommended for Trusted Networks)'''
 
If the VoIPmonitor server is in a trusted network, configure your external SMTP server to permit relaying from the VoIPmonitor server's IP address:
 
1. Access your external SMTP server configuration
2. Add the VoIPmonitor server's IP address to the allowed relay hosts (mynetworks)
3. Save configuration and reload: <code>postfix reload</code>
 
'''Solution 2: Configure Postfix SMTP Authentication (Recommended for Remote SMTP)'''
 
If using an external SMTP server that requires authentication, configure Postfix to authenticate using SASL:
 
1. Install SASL authentication packages:
<syntaxhighlight lang="bash">
# Debian/Ubuntu
sudo apt-get install libsasl2-modules
 
# RHEL/CentOS/AlmaLinux/Rocky
sudo yum install cyrus-sasl-plain
</syntaxhighlight>
 
2. Configure Postfix to use the external SMTP relay:
 
Edit <code>/etc/postfix/main.cf</code>:
<syntaxhighlight lang="ini">
# Use external SMTP as relay host
relayhost = smtp.yourprovider.com:587
 
# Enable SASL authentication
smtp_sasl_auth_enable = yes
 
# Use SASL password file
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
 
# Disable anonymous authentication (use only SASL)
smtp_sasl_security_options = noanonymous
 
# Enable TLS (recommended)
smtp_tls_security_level = encrypt
</syntaxhighlight>
 
3. Create the SASL password file with your SMTP credentials:
 
<syntaxhighlight lang="bash">
# Create the file (your SMTP username and password)
echo "[smtp.yourprovider.com]:587 username:password" | sudo tee /etc/postfix/sasl_passwd
 
# Secure the file (rw root only)
sudo chmod 600 /etc/postfix/sasl_passwd
 
# Create the Postfix hash database
sudo postmap /etc/postfix/sasl_passwd
 
# Reload Postfix
sudo systemctl reload postfix
</syntaxhighlight>
 
4. Test email delivery:
 
<syntaxhighlight lang="bash">
echo "Test email" | mail -s "SMTP Relay Test" your.email@example.com
</syntaxhighlight>
 
If successful, emails should be delivered through the authenticated SMTP relay.
 
==== Step 5: Verify Cronjob ====
 
Ensure the alert processing script runs every minute:
 
<syntaxhighlight lang="bash">
# Check current crontab
crontab -l
</syntaxhighlight>
 
You should see:
<syntaxhighlight lang="bash">
* * * * * root php /var/www/html/php/run.php cron
</syntaxhighlight>
 
If missing, add it:
<syntaxhighlight lang="bash">
crontab -e
# Add the line above, then reload cron
killall -HUP cron
</syntaxhighlight>
 
==== Step 6: Verify Alert Configuration in GUI ====
 
After confirming the MTA works:
# Navigate to '''GUI > Alerts'''
# Verify alert conditions are enabled
# Check that recipient email addresses are valid
# Go to '''GUI > Alerts > Sent Alerts''' to see if alerts were triggered
 
'''Diagnosis:'''
* Entries in "Sent Alerts" but no emails received → MTA issue
* No entries in "Sent Alerts" → Check alert conditions or cronjob
 
==== Step 7: Test PHP mail() Function ====
 
Isolate the issue by testing PHP directly:
 
<syntaxhighlight lang="bash">
php -r "mail('your.email@example.com', 'Test from PHP', 'This is a test email');"
</syntaxhighlight>
 
* If this works but VoIPmonitor alerts don't → Check GUI cronjob and alert configuration
* If this fails → MTA or PHP configuration issue
 
=== Troubleshooting Concurrent Calls Alerts Not Triggering ===
 
CDR-based concurrent calls alerts may not trigger as expected due to database queue delays or alert timing configuration. Unlike realtime concurrent calls alerts (see [[Anti-fraud|Anti-Fraud Rules]]), CDR-based alerts require CDRs to be written to the database before evaluation.
 
==== Check SQL Cache Files Queue ====
 
A growing SQL cache queue can prevent CDR-based alerts from triggering because the alert processor evaluates CDRs that have already been stored in the database, not calls still waiting in the queue.
 
* Navigate to '''GUI > Settings > Sensors'''
* Check the RRD chart for '''SQL cache files''' (SQLq/SQLf metric)
* '''If the queue is growing during peak times:'''
** Database cannot keep up with CDR insertion rates
** Alerts evaluate outdated data because recent CDRs have not been written yet
** See [[SQL_queue_is_growing_in_a_peaktime|Delay between active call and cdr view]] for solutions
 
==== CDR Timing vs "CDR not older than" Setting ====
 
CDR-based alerts include a ''CDR not older than'' parameter that filters which CDRs are considered for alert evaluation.
 
* '''Parameter location:''' In the concurrent calls alert configuration form
* '''Function:''' Only CDRs newer than this time window are evaluated
* '''Diagnosis:'''
** Verify that the time difference between ''Last CDR in database'' and ''Last CDR in processing queue'' (in Sensors status) is smaller than your ''CDR not older than'' value
** If the delay is larger, CDRs are being excluded from alert evaluation
** Common causes: Database overload, slow storage, insufficient MySQL configuration
* '''Solution:'''
** Increase the ''CDR not older than'' value to match your database performance
** See [[SQL_queue_is_growing_in_a_peaktime]] and [[Scaling]] for database tuning
** Check SQLq value should remain low (under 1000) during peak load
 
==== "Check Interval" Parameter and Low Thresholds ====
 
When testing concurrent calls alerts with very low thresholds (e.g., greater than 0 calls or 1 call), consider the ''Check interval'' parameter.
 
* '''Parameter location:''' In the concurrent calls alert configuration form
* '''Function:''' How often the alert condition is evaluated (time window for concurrent call calculation)
* '''Issue with low thresholds:**
** A call lasting 300 seconds (5 minutes) will show as concurrent for the entire duration
** If ''Check interval'' is shorter than typical call durations, you may see temporary concurrent counts that disappear between evaluations
* '''Recommendation for testing:**
** Increase the ''Check interval'' to a longer duration (e.g., 60 minutes) when testing with very low thresholds
** This ensures concurrent calls are counted over a longer time window, avoiding false negatives from short interval checks
 
==== Fraud Concurrent Calls vs Regular Concurrent Calls Alerts ====
 
VoIPmonitor provides two different alert types for concurrent calls monitoring, which operate on different data sources and have different capabilities:
 
{| class="wikitable"
|-
! Feature !! Fraud Concurrent Calls !! Regular Concurrent Calls
|-
| '''Data source''' || SIP INVITEs (realtime) || CDRs (after call ends)
|-
| '''Processing type''' || Realtime (packet inspection) || CDR-based (database query)
|-
| '''Aggregation ("BY" dropdown)''' || '''Source IP only''' (hard-coded) || Source IP, '''Destination IP''', Domain, Custom Headers
|-
| '''Domain filtering''' || '''Not available''' (major limitation) || Available via SQL filter
|-
| '''Timing''' || Immediate (no database delay) || Delayed (requires CDR insertion)
|-
| '''Where configured''' || '''GUI > Alerts > Anti Fraud''' || '''GUI > Alerts'''
|-
| '''Table name''' || `list_concurrent_calls` in anti-fraud section | Standard alerts table
|}


'''Key Differences:'''
== External Scripts ==


* '''Fraud concurrent calls''' detect concurrent INVITEs in realtime but the "BY" dropdown only supports '''Source IP aggregation'''. This is a hard-coded limitation of the realtime detection logic designed for detecting attacks from specific IPs. Use this for attack detection where you need immediate alerts.
Enable webhook integration (Datadog, Slack, custom systems).
* '''Regular concurrent calls alerts''' use stored CDRs and support multiple aggregation options including '''Destination IP (Called)''', Source IP, Domain, and custom headers via the Common Filters tab. Use this for capacity planning, trunk/capacity monitoring, or when you need to filter by destination.
* '''Destination IP monitoring''': If you need to alert when concurrent calls to a specific destination IP (e.g., carrier, trunk) exceed a threshold, use the regular '''Concurrent calls''' alert. The Fraud concurrent calls alert cannot filter or aggregate by destination IP.


Example use cases:
'''Configuration:''' Enter full absolute path in '''External script''' field (e.g., <code>/usr/local/bin/alert-webhook.sh</code>).


'''Arguments passed to script:'''
{| class="wikitable"
{| class="wikitable"
|-
|-
! Scenario !! Recommended Alert Type !! Configuration
! Arg !! Description
|-
|-
| Detect flooding/attack from specific source IP (immediate) || '''Fraud: realtime concurrent calls''' || GUI > Alerts > Anti Fraud, BY: Source IP
| <code>$1</code> || Alert ID
|-
|-
| Monitor trunk capacity limit (destination IP threshold) || '''Concurrent calls''' || GUI > Alerts, BY: Destination IP (Called)
| <code>$2</code> || Alert name
|-
|-
| Detect domain-specific concurrent call patterns || '''Concurrent calls''' || GUI > Alerts, Common Filters: Domain
| <code>$3</code> || Unix timestamp
|-
|-
| Detect attacks with immediate triggering || '''Fraud: realtime concurrent calls''' || GUI > Alerts > Anti Fraud
| <code>$4</code> || JSON data with CDR IDs
|}
|}


If you have configured a concurrent calls alert and need filtering by destination IP, domain, or custom headers, verify that you are using the regular '''Concurrent calls''' alert in '''GUI > Alerts''' and not the fraud variant.
'''Example - Slack notification:'''
 
=== Troubleshooting Alerts Not Triggering (General) ===
 
If alerts are not appearing in the '''Sent Alerts''' history at all, the problem is typically with the alert processor not evaluating alerts. This is different from MTA issues where alerts appear in history but emails are not sent.
 
==== Enable Detailed Alert Processing Logs ====
 
To debug why alerts are not being evaluated, enable detailed logging for the alert processor by adding the following line to your GUI configuration file:
 
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
# Edit the config file (adjust path based on your GUI installation)
#!/bin/bash
nano ./config/system_configuration.php
# /usr/local/bin/slack-alert.sh
SLACK_WEBHOOK="https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
curl -X POST "$SLACK_WEBHOOK" -H "Content-Type: application/json" \
  -d '{"text": "VoIPmonitor Alert: '"$2"'"}'
</syntaxhighlight>
</syntaxhighlight>


Add this line at the end of the file:
{{Note|1=IP addresses in CDR table are decimal integers. Use <code>long2ip()</code> (PHP) or <code>INET_NTOA()</code> (MySQL) for conversion.}}
<syntaxhighlight lang="php">
<?php
define('CRON_LOG_FILE', '/tmp/alert.log');
?>
</syntaxhighlight>


This enables logging that shows which alerts are being processed during each cron job run.
== Sent Alerts ==


==== Increase Parallel Processing Threads ====
View triggered alerts via '''GUI > Alerts > Sent Alerts'''. Shows:
* '''Parameters table''' - QoS metrics with highlighted bad values
* '''CDR records''' - Calls that triggered alert with flags: (M)OS, (J)itter, (P)acket loss, (D)elay


If you have many alerts or reports, the default number of parallel threads may cause timeout issues. Increase the parallel task limit:
== Custom Report Alerts ==


<syntaxhighlight lang="bash">
Alert on criteria not in native types (e.g., custom SIP headers).
# Edit the configuration file
nano ./config/configuration.php
</syntaxhighlight>


Add this line at the end of the file:
'''Workflow:'''
<syntaxhighlight lang="php">
# Capture header in <code>/etc/voipmonitor.conf</code>: <code>custom_headers = Max-Forwards</code>
<?php
# Enable in '''GUI > Settings > CDR Custom Headers'''
define('CRON_PARALLEL_TASKS', 8);
# Create filter in CDR view, save as template
?>
# Create Daily Report with filter in '''GUI > Reports > Configure Daily Reports'''
</syntaxhighlight>


The value of 8 is recommended for high-load environments. Adjust based on your alert/report volume and server capacity.
{{Note|1=Custom report alerts cannot group by caller/called for threshold detection (e.g., "alert if same caller has >X failures"). Use CDR Summary reports for aggregated data.}}


==== Monitor Alert Processing Logs ====
== Troubleshooting ==


After enabling logging, monitor the alert log file to see which alerts are being processed:
=== Email Not Sent ===


<syntaxhighlight lang="bash">
'''Diagnosis:'''
# Watch the log in real-time
* Entries in "Sent Alerts" but no email → MTA issue
tail -f /tmp/alert.log
* No entries in "Sent Alerts" → Alert conditions or cron issue
</syntaxhighlight>
 
The log shows entries like:
<syntaxhighlight lang="text">
begin alert [alert_name]
end alert [alert_name]
</syntaxhighlight>


'''Interpreting the logs:'''
* If you '''do not see''' your alert name in the logs → The alert processor is not evaluating it. Check your alert configuration, filters, and data availability.
* If you '''see''' the alert in logs but it does not trigger → The alert conditions are not being met. Check your thresholds, filter logic, and verify the CDR data matches your expectations.
* If logs are completely empty → The cron job may not be running or the GUI configuration files are not being loaded. Verify the cron job and file paths.
==== Alert Not Appearing in Logs ====
If your alert does not appear in `/tmp/alert.log`:
1. '''Verify the cron job is running:'''
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
# Check the cron job exists
# Test MTA
crontab -l
echo "Test" | mail -s "Test" your@email.com


# Manually test the cron script to see errors
# Check MTA status
php ./php/run.php cron
systemctl status postfix  # or exim4/sendmail
</syntaxhighlight>
 
2. '''Verify data exists in CDR:'''
<syntaxhighlight lang="bash">
# Check if the calls that should trigger the alert exist
# Navigate to GUI > CDR > Browse and filter for the timeframe
</syntaxhighlight>


3. '''Check alert configuration:'''
# Check logs
* Verify alert is enabled
tail -f /var/log/mail.log  # Debian/Ubuntu
* Verify filter logic matches your data (IP addresses, numbers, groups)
tail -f /var/log/maillog  # RHEL/CentOS
* Verify thresholds are reasonable for the actual QoS metrics
* Verify GUI license is not locked (Check '''GUI > Settings > License''')


=== "Crontab Log is Too Old" Warning - Database Performance Issues ===
# Check mail queue
 
mailq
The VoIPmonitor GUI displays a warning message "Crontab log is too old" when the last successful cron run timestamp exceeds the expected interval. While this often indicates a missing or misconfigured cron job, it can also occur when the database is overloaded and the cron script runs slowly.
 
==== Common Causes ====
 
# '''Missing or broken cron entry''' - The cron job does not exist in /etc/crontab or the command fails when executed
# '''Database overload''' - The cron job runs but completes slowly due to database performance bottlenecks, causing the "last run" timestamp to drift outside the expected window
 
==== Distinguishing the Causes ====
 
Use the following diagnostic workflow to determine if the issue is cron configuration vs. database performance:
 
'''Step 1: Verify the cron job is actually running'''
 
Check if the cron execution timestamp is updating (even if slowly):
 
<syntaxhighlight lang="bash"># Check the current cron timestamp from the database
mysql -u voipmonitor -p voipmonitor -e "SELECT name, last_run FROM scheduler LIMIT 1"
 
# The last_run timestamp should update at least every few minutes
# If it never updates, the cron is not running (see Step 2)
# If it updates but lags by more than 5-10 minutes, it's a performance issue (see Step 3)
</syntaxhighlight>
</syntaxhighlight>


'''Step 2: If cron is not running at all'''
'''Status 250 or "Queued mail for delivery"''' = Your server delivered successfully. If recipient didn't receive, issue is on their side (spam folder, quarantine, blacklisting).
 
'''Mail Queue Not Delivering:'''
Follow the standard cron setup instructions in the "Setting Up the Cron Job" section above. Common issues:
If emails accumulate in the queue but are not being sent:
 
* Cron entry missing from /etc/crontab
* Incorrect PHP path (use full path like /usr/bin/php instead of php)
* PHP CLI missing IonCube loader (check with `php -r 'echo extension_loaded("ionCube Loader")?"yes":"no";'`)
* Wrong file permissions or incorrect web directory path
* '''PHP CLI version mismatch''' - System CLI PHP differs from web server PHP


===== PHP CLI Version Mismatch Fix =====
Even if the cron job exists and IonCube Loader is installed for both web and CLI, the CLI may be using a different PHP version than the web server, causing the cron script to fail. This commonly occurs when multiple PHP versions are installed on the system.
'''Symptoms:'''
* Cron job exists in /etc/crontab
* PHP CLI has IonCube loader installed
* The GUI shows "Crontab log is too old" warning
* Manual command execution succeeds when using the correct PHP version
'''Diagnosis:'''
1. Check the target PHP version required by the GUI:
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
cat /var/www/html/ioncube_phpver
# Verify queue manager is running
# Output may be: 81 (for PHP 8.1), 82 (for PHP 8.2), etc.
ps aux | grep qmgr
</syntaxhighlight>


2. Check the current CLI PHP version:
# Restart Postfix
<syntaxhighlight lang="bash">
systemctl restart postfix
php -v
# Example output: PHP 8.2.26 (the default CLI version may not match the GUI requirement)
</syntaxhighlight>


3. List available PHP CLI versions:
# Force immediate delivery of queued emails
<syntaxhighlight lang="bash">
postfix flush
ls /usr/bin/php*
# You may see: php8.1, php8.2, php8.3
</syntaxhighlight>
</syntaxhighlight>
=== Alerts Not Triggering ===


'''Solution: Set CLI PHP Version to Match Web Server'''
'''Enable debug logging:'''
 
<syntaxhighlight lang="php">
Use the `update-alternatives` command to set the default CLI PHP version to match the web server:
// Add to ./config/system_configuration.php
 
define('CRON_LOG_FILE', '/tmp/alert.log');
<syntaxhighlight lang="bash">
# If ioncube_phpver shows "81", set CLI to PHP 8.1
sudo update-alternatives --set php /usr/bin/php8.1
 
# If ioncube_phpver shows "82", set CLI to PHP 8.2
sudo update-alternatives --set php /usr/bin/php8.2
 
# Verify the change
php -v
which php
# Should now point to the correct version
</syntaxhighlight>
</syntaxhighlight>
'''Verify the Fix:'''
Test the cron command manually and check IonCube is loaded with the new version:


<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
cd /var/www/html
# Monitor processing
php php/run.php cron
tail -f /tmp/alert.log
 
# Verify IonCube is loaded
php -r 'echo extension_loaded("ionCube Loader")?"yes":"no";'
# Should output: yes
</syntaxhighlight>
</syntaxhighlight>


After a few minutes, the "Crontab log is too old" warning in the GUI should disappear, confirming the cron job is now running successfully.
'''Common causes:'''
* Cron not running - verify with <code>crontab -l</code>
* PHP CLI version mismatch - use <code>update-alternatives --set php /usr/bin/php8.x</code>
* SQL queue growing - DB can't keep up (see [[Scaling]])
* Alert disabled or filter mismatch


'''Step 3: If cron runs but slowly (database performance issue)'''
=== Concurrent Calls Alerts ===


When the cron job runs but takes a long time to complete, the issue is database overload. Diagnose using the Sensors statistics:
{| class="wikitable"
|-
! Type !! Data Source !! Aggregation !! Timing
|-
| '''Fraud concurrent calls''' || SIP INVITEs (realtime) || Source IP only || Immediate
|-
| '''Regular concurrent calls''' || CDRs (database) || Source/Dest IP, Domain, Custom || Delayed
|}


# Navigate to '''GUI > Settings > Sensors'''
Use regular concurrent calls for destination IP monitoring (trunk capacity).
# Click on the sensor status to view detailed statistics
'''Investigating Fraud: Realtime Concurrent Calls Alerts'''
# Compare the following timestamps:
**'''Last CDR in database'''** - The timestamp of the most recently completed call stored in MySQL
**'''Last CDR in processing queue'''** - The timestamp of the most recent call reached by the sniffer


If there is a significant delay (minutes or more) between these two timestamps during peak traffic, the database cannot keep up with CDR insertion. This causes alert/reports processing (run.php cron) to also run slowly.
Since this alert type triggers before CDRs are written, use the following procedure to investigate the calls that triggered the alert:
# Navigate to '''GUI → CDR'''
# Use the filter form to add the '''is international''' filter
# Set the '''from''' and '''to''' date range to match the time the alert was sent
# Go to the bottom of the CDR view and enable grouping by '''country'''
# Analyze the traffic by country to identify the source of the fraudulent activity
=== External Script Not Running ===


=== Solutions for Database Performance Issues ===
# Use '''preview button''' to test alert triggers
# Verify absolute path (not relative)
# Check permissions: <code>chmod 755 /path/to/script.sh</code>
# Include shebang: <code>#!/bin/bash</code>
# Use full command paths (e.g., <code>/usr/bin/curl</code>)
# For URLs, create script with curl/wget - cannot put URL directly in field


**1. Check MySQL configuration**
=== "Crontab Log Too Old" Warning ===


Ensure your MySQL/MariaDB configuration follows the recommended settings for your call volume. Key parameters:
'''Causes:'''
# Cron not running → Add cron entry
# PHP CLI version mismatch → <code>update-alternatives --set php /usr/bin/php8.x</code>
# Database overload → Check SQLq in '''GUI > Settings > Sensors''', see [[Scaling]]


* <code>innodb_flush_log_at_trx_commit</code> - Set to 2 for better performance (or 0 in extreme high-CPS environments)
== See Also ==
* <code>innodb_buffer_pool_size</code> - Allocate 70-80% of available RAM for high-volume deployments
* <code>innodb_io_capacity</code> - Match your storage system capabilities (e.g., 1000000 for NVMe SSDs)


See [[Scaling]] and [[High-Performance_VoIPmonitor_and_MySQL_Setup_Manual]] for detailed tuning guides.
* [[Anti-fraud|Anti-Fraud Rules]] - Realtime fraud detection
* [[Reports]] - Daily reports and report generator
* [[Groups]] - IP and number groups for filtering


**2. Increase database write threads**


In <code>/etc/voipmonitor.conf</code>, increase the number of threads used for writing CDRs:


<syntaxhighlight lang="ini">
mysqlstore_max_threads_cdr = 8  # Default is 4, increase based on workload
</syntaxhighlight>
**3. Monitor SQL queue statistics**
In the expanded status view (GUI > Settings > Sensors > status), check the SQLq value:
* '''SQLq (SQL queue) growing steadily''' - Database is a bottleneck, calls are waiting in memory
* '''SQLq remains low (under 1000)''' - Database is keeping up, may need other tuning
See [[SQL_queue_is_growing_in_a_peaktime]] for more information.
**4. Reduce alert/report processing load**
Too many alert rules or complex reports can exacerbate the problem:
* Review and disable unnecessary alerts in '''GUI > Alerts'''
* Reduce the frequency of daily reports (edit in '''GUI > Reports''')
* Increase parallel processing tasks: In <code>/var/www/html/configuration.php</code>, set <code>define('CRON_PARALLEL_TASKS', 8);</code> (requires increasing PHP memory limits)
**5. Check database query performance**
Identify slow queries:
<syntaxhighlight lang="bash"># Enable slow query logging in my.cnf
slow_query_log = 1
long_query_time = 2
# After waiting for a cron cycle, check the slow query log
tail -f /var/log/mysql/slow.log
</syntaxhighlight>
Look for queries taking more than a few seconds. Common culprits:
* Missing indexes on frequently filtered columns (caller, callee, sipcallerip, etc.)
* Complex alert conditions joining large tables
* Daily reports scanning millions of rows without date range limitations
**6. Scale database architecture**
For very high call volumes (4000+ concurrent calls), consider:
* Separate database server from sensor hosts
* Use MariaDB with LZ4 page compression
* Implement database replication for read queries
* Use hourly table partitioning for improved write performance
See [[High-Performance_VoIPmonitor_and_MySQL_Setup_Manual]] for architecture recommendations.
=== Verification ===
After applying fixes:
1. '''Monitor the "Crontab log is too old" timestamp in the GUI'''
  * The timestamp should update every 1-3 minutes during normal operation
  * If it still lags by 10+ minutes, further tuning is required
2. '''Check sensor statistics (GUI > Settings > Sensors)'''
  * The delay between "Last CDR in database" and "Last CDR in processing queue" should be under 1-2 minutes during peak load
  * SQLq should remain below 1000 and not grow continuously
3. '''Test alert processing manually'''
<syntaxhighlight lang="bash">
# Run the cron script manually and measure execution time
time php /var/www/html/php/run.php cron
# Should complete within 10-30 seconds in most environments
# If it takes longer than 60-120 seconds, database tuning is needed
</syntaxhighlight>


=== See Also ===


* [[Anti-fraud|Anti-Fraud Rules]] - Detailed fraud detection configuration
* [[Reports|Reports]] - Daily reports and report generator
* [[Sniffer_troubleshooting|Sniffer Troubleshooting]] - General troubleshooting


== AI Summary for RAG ==
== AI Summary for RAG ==


'''Summary:''' VoIPmonitor Alerts & Reports system for email notifications on QoS and SIP issues. Covers RTP alerts (MOS, jitter, packet loss), SIP response alerts (including detecting 408 Request Timeout from 100 Trying scenarios), sensors health monitoring, SIP REGISTER RRD beta alerts for monitoring registration response times, SIP failed Register (beta) alerts for detecting SIP registration floods from single IP addresses using multiple different usernames (useful for detecting credential-stuffing and brute-force attacks), multiple register (beta) alerts for detecting SIP accounts registered from multiple IP addresses (useful for detecting credential theft or unauthorized access), RPT&CDR alerts for Post Dial Delay (PDD) monitoring with filter-common tab conditions (e.g., PDD > 5 in seconds) and base config tab for email recipient and max-lines in body settings, creating alerts from custom reports based on CDR custom headers, and CDR trends alerts for trend-based monitoring and alerting on aggregated CDR statistics including ASR (Answer Seizure Ratio). CDR trends include configuration parameters: Type (ASR, call count, ACD), Offset (historical baseline: 1 week/day/month), Range (current window: 1 hour/day/week), Method (Deviation for percentage change or Threshold for absolute value), Limit Inc./Limit Dec. (percentage threshold for triggering alerts), and IP whitelist (optional filter for specific IPs/agents). Example: Set Type=ASR, Offset=1 week, Range=1 hour, Method=Deviation, Limit Dec.=10% to alert when ASR drops by 10% compared to same hour last week. PERCENTAGE ALERTS CRITICAL: When setting percentage thresholds like >10%, the "from all" checkbox controls the calculation base: CHECKED calculates from ALL CDRs in database, UNCHECKED calculates only from CDRs matching common filters (IP groups, numbers). Always UNCHECK "from all" when monitoring specific IP groups. Includes email troubleshooting for MTA configuration, setting the "From" address for outbound emails (GUI > Settings > System Configuration > Email / HTTP Referer > DEFAULT_EMAIL_FROM field), detailed debugging for alerts not triggering using CRON_LOG_FILE and CRON_PARALLEL_TASKS, external scripts for webhook integration (Datadog, Slack, third-party monitoring), and accessing source IP addresses from alert scripts with proper decimal-to-dotted-decimal conversion using PHP's long2ip() or MySQL's INET_NTOA(). TROUBLESHOOTING concurrent calls alerts: CDR-based alerts may not trigger due to SQL queue delays (check SQL cache files RRD chart in GUI > Settings > Sensors), verify "CDR not older than" parameter is larger than delay between "Last CDR in database" and "Last CDR in processing queue", increase "Check interval" for low threshold testing (e.g., >0 calls use 60 minutes), and understand fraud concurrent calls vs regular concurrent calls: fraud uses INVITEs realtime but lacks domain filtering, regular uses CDRs with domain support but has database delay.
'''Summary:''' VoIPmonitor Alerts system provides email notifications for QoS thresholds (RTP: MOS, jitter, packet loss), SIP response codes (0=no response, 408=timeout), sensor health, and registration monitoring. Alert types include RTP, RTP&CDR (with filter templates for duration/absolute_timeout), SIP Response (use "from all" unchecked for IP group percentages), International Calls (prefix-based, NOT GeoIP), Sensors, SIP REGISTER alerts (RRD beta for latency, failed Register beta for brute-force, multiple register beta for credential compromise), and CDR Trends (ASR deviation monitoring). External scripts enable webhook integrations. CRITICAL: Alerts use OR logic only - AND not supported. IP addresses stored as integers - use long2ip()/INET_NTOA() for conversion.


'''Keywords:''' alerts, email notifications, QoS, MOS, jitter, packet loss, SIP response, 408 Request Timeout, 100 Trying, response code 0, INVITE retransmissions, sensors monitoring, SIP REGISTER RRD beta, REGISTER retransmissions, registration monitoring, SIP failed Register beta, failed registrations, registration flood, credential stuffing, brute force attack, botnet, username list, dictionary attack, unauthorized SIP access, multiple register beta, multiple IP addresses, SIP account compromise, credential theft, security alerts, unauthorized SIP usage, RTP&CDR alerts, PDD, Post Dial Delay, call setup delay, network latency, filter-common tab, base config tab, max-lines in body, email recipient, DEFAULT_EMAIL_FROM, From address, email from header, GUI > Settings > System Configuration > Email / HTTP Referer, crontab, MTA, Postfix, Exim, troubleshooting, custom headers, custom reports, Max-Forwards, daily reports, CRON_LOG_FILE, CRON_PARALLEL_TASKS, alert not triggering, /tmp/alert.log, begin alert, end alert, configuration.php, system_configuration.php, external scripts, webhooks, Datadog, Slack, command-line arguments, JSON data, CDR array, IP address conversion, long2ip, INET_NTOA, decimal IP, dotted-decimal, sipcallerip, sipcalledip, rtpsrcip, rtpdstip, human-readable IP, IP format, 3110817796, concurrent calls not triggering, CDR not older than, Check interval, SQL queue, SQL cache files, SQLq, SQLf, growing queue, database delay, CDR timing, fraud concurrent calls, realtime concurrent calls, SIP INVITE concurrent alerts, domain filtering, domain-based alerts, CDR-based alerts, anti-fraud concurrent calls, low threshold testing, concurrent calls window, Last CDR in database, Last CDR in processing queue, CDR trends, ASR trend, Answer Seizure Ratio, trend-based alerts, ASR drop, ASR increase, Offset parameter, Range parameter, Deviation method, Limit Inc, Limit Dec, IP whitelist, historical baseline, current time window, percentage change, trend monitoring, ASR deviation, trend alerting, from all checkbox, percentage alert, percentage threshold, filtered vs all CDRs, IP group percentage, 503 response percentage, Datora, Crontab log is too old, cron job not running, PHP CLI version mismatch, update-alternatives, ioncube_phpver, web server PHP version, CLI PHP version, ioncube loader, cron fails to execute, PHP versions, php8.1, php8.2, php8.3
'''Keywords:''' alerts, email notifications, QoS, MOS, jitter, packet loss, SIP response, 408 timeout, sensors monitoring, SIP REGISTER, brute force, credential stuffing, international calls, called number prefixes, CDR trends, ASR, external scripts, webhooks, from all checkbox, OR logic, crontab, MTA, Postfix, CRON_LOG_FILE, concurrent calls, SQL queue


'''Key Questions:'''
'''Key Questions:'''
* How do I configure a percentage alert for IP groups?
* How do I configure email alerts in VoIPmonitor?
* What does the "from all" checkbox do in alerts?
* What alert types are available (RTP, SIP, Sensors)?
* How do I trigger an alert when percentage of 503 responses exceeds threshold for a specific IP group?
* How do I configure international call alerts with prefix filtering?
* How do I set percentage thresholds like >10% in alerts?
* What does the "from all" checkbox do in percentage alerts?
* Should the "from all" checkbox be checked when monitoring a specific IP group?
* How do I integrate alerts with webhooks (Slack, Datadog)?
* How do I detect SIP registration floods from a single IP address using multiple different usernames?
* Why are my alerts not triggering?
* What is the SIP failed Register (beta) alert type?
* How do I troubleshoot email delivery issues?
* How do I configure SIP failed Register (beta) alert to detect credential-stuffing attacks?
* What is the difference between fraud and regular concurrent calls alerts?
* How do I detect brute-force attempts against SIP registrations?
* How do I detect SIP registration attacks (brute-force)?
* How does SIP failed Register (beta) alert detect floods with many usernames from one IP?
* Do alerts support AND logic between conditions?
* What is the difference between SIP failed Register (beta) and SIP REGISTER RRD beta alerts?
* What is the difference between SIP failed Register (beta) and multiple register (beta) alerts?
* How do I detect SIP account takeover attempts using alerts?
* How do I configure alerts to monitor for registration abuse patterns?
* How do I set threshold and interval parameters for SIP failed Register (beta) alerts?
* How do I configure alerts based on ASR trend changes?
* How do I create an alert for a 10% ASR drop over the last hour?
* What is the CDR trends alert type?
* How do I use CDR trends for trend-based monitoring?
* How do I configure Offset and Range parameters in CDR trends alerts?
* What is the Deviation method in CDR trends alerts?
* How do I set Limit Inc and Limit Dec for trend alerts?
* How do I compare current ASR to previous week in alerts?
* How do I detect ASR drops or increases over time windows?
* What is the IP whitelist in CDR trends alerts?
* How do I create an alert for Answer Seizure Ratio trends?
* What is the difference between Offset and Range in trend alerts?
* How do I monitor call volume pattern changes with alerts?
* How do I use trend-based alerts for quality degradation detection?
* How do I create an email alert for high Post Dial Delay (PDD)?
* How do I configure the "From" address for alert emails?
* Where do I set the email from header in VoIPmonitor?
* How do I set up email alerts in VoIPmonitor?
* What types of alerts are available (RTP, SIP, Sensors, REGISTER RRD beta, multiple register beta)?
* How do I detect calls with 408 Request Timeout after 100 Trying?
* How do I create an alert for calls where 100 Trying is sent but no further response?
* What is the difference between response code 0 and 408?
* How do I monitor and alert on SIP REGISTER retransmissions?
* How do I detect registration response time issues?
* How can I use SIP REGISTER RRD beta alert for detecting switch problems?
* How do I detect SIP accounts registered from multiple IP addresses?
* How do I use multiple register (beta) alert to find accounts with multiple IP registrations?
* How do I configure multiple register (beta) alert to check all SIP numbers across all customers?
* How do I detect SIP account compromise or credential theft using alerts?
* How do I monitor for unauthorized SIP account usage?
* How do I configure crontab for alert processing?
* How do I monitor remote sensor health?
* Why are email alerts not being sent?
* How do I troubleshoot MTA email issues?
* How can I create alerts based on SIP headers like Max-Forwards?
* How do I use CDR custom headers for custom reports?
* How do I limit the size of alert emails from custom reports?
* Why are alerts not triggering or appearing in Sent Alerts?
* How do I enable detailed alert processing logs using CRON_LOG_FILE?
* How do I increase parallel alert processing threads with CRON_PARALLEL_TASKS?
* How do I monitor /tmp/alert.log for alert processing debug information?
* How do I configure external scripts for alerts?
* How do I send webhooks from VoIPmonitor alerts to Datadog?
* How do I send alerts to Slack from VoIPmonitor?
* What command-line arguments are passed to alert scripts?
* What JSON data structure is provided to external scripts?
* How do I access source IP addresses from alert scripts?
* Why are IP addresses in alert scripts showing as decimal numbers?
* How do I convert decimal IP addresses to human-readable format?
* What is long2ip() in PHP and how do I use it for IP addresses?
* What is INET_NTOA() in MySQL and how do I use it for IP conversion?
* How do I access sipcallerip, sipcalledip, rtpsrcip from alert scripts?
* How do I convert 3110817796 to 185.107.80.4 in an alert script?
* Why are concurrent calls alerts not triggering?
* How do I check SQL queue for alert timing issues?
* What is the "CDR not older than" parameter in concurrent calls alerts?
* How do I verify if database delays are preventing alerts from triggering?
* How do I monitor SQL cache files queue in GUI > Settings > Sensors?
* What is the "Check interval" parameter in concurrent calls alerts?
* How should I configure "Check interval" for low threshold testing (e.g., >0 calls)?
* What is the difference between fraud concurrent calls and regular concurrent calls alerts?
* Can fraud concurrent calls filter by domain in SIP headers?
* Do regular concurrent calls alerts support domain filtering?
* Why is my domain-based concurrent calls alert not working?
* What is SQLq/SQLf in the Sensors status?
* How does "Last CDR in database" vs "Last CDR in processing queue" affect alerts?
* What should the SQL queue value be during peak load?
* What does "Crontab log is too old" warning mean?
* Why does the cron job exist but fail to execute?
* How do I fix PHP CLI version mismatch for cron jobs?
* How do I check what PHP version the web server is using?
* What is the ioncube_phpver file?
* How do I use update-alternatives to set the CLI PHP version?
* How do I set default PHP CLI version to match the web server?
* How do I verify if IonCube Loader is loaded in PHP CLI?
* Why does the cron job fail even with IonCube installed?

Latest revision as of 10:34, 17 January 2026


Alerts & Reports

Email notifications triggered by QoS thresholds, SIP errors, or sensor health conditions. The system stores all alerts in history for review.

Prerequisites

Email Configuration

Alerts use PHP's mail() function via the server's MTA (Postfix/Exim/Sendmail).

Setting Location Description
From Address GUI > Settings > System Configuration > Email DEFAULT_EMAIL_FROM - sender address for all alerts
Cron Job /etc/crontab Required for alert processing 
# Add cron job (required)
echo "* * * * * root php /var/www/html/php/run.php cron" >> /etc/crontab
killall -HUP cron   # Debian/Ubuntu
# or: killall -HUP crond  # RHEL/CentOS

Alert Types

Access via GUI > Alerts.

RTP Alerts

Trigger on voice quality metrics:

  • MOS - below threshold
  • Packet loss - percentage exceeded
  • Jitter - variation exceeded
  • Delay (PDV) - latency exceeded
  • One-way calls - one RTP stream missing
  • Missing RTP - both RTP streams missing

Configure alerts to trigger when number of incidents OR percentage of CDRs exceeds threshold.

RTP&CDR Alerts

Combine RTP metrics with CDR conditions including PDD (Post Dial Delay).

Using Filter Templates:

  1. Create CDR filter in GUI > CDR
  2. Save as template
  3. In alert config, select from Filter template dropdown

💡 Tip: Use filter templates for complex conditions like duration > 14400 (calls over 4 hours) or absolute_timeout (truncated recordings).

SIP Response Alerts

Response Code Meaning
Empty All call attempts per filters
0 No response received (routing loops)
408 Timeout after provisional response (server unresponsive)
Specific Exact codes (404, 503, etc.)

"from all" Checkbox (Percentage Thresholds)

⚠️ Warning: This setting is critical for IP group monitoring.

  • CHECKED: % calculated from ALL CDRs in database
  • UNCHECKED: % calculated only from filtered CDRs (correct for specific IP groups)


SIP Response vs Last SIP Response

There are two different fields for matching SIP responses:

Field Location Supports % Threshold Use Case
SIP response GUI > Alerts > SIP Response Alerts ✓ Yes Match by numeric code (e.g., 487, 503)
Last sip response GUI > Alerts > Filter common ✗ No Match by full text (e.g., "487 Request Terminated")

⚠️ Warning: The GUI cannot trigger alerts based on percentage of full textual response strings. If you need percentage-based triggering for SIP response codes, use the SIP response numeric field instead.

The Last sip response field supports wildcard patterns (%, %Request Terminated%, %487%) but only triggers based on count thresholds, not percentages.

International Call Alerts (Called Number Prefixes)

Monitor calls to international destinations using prefix-based matching (dialing patterns like 00, +).

ℹ️ Note: This uses phone number prefix detection, NOT IP geolocation. For GeoIP-based detection, see Anti-Fraud Rules.

Configuration:

  1. GUI > Settings > Country prefixes - Define international prefixes (00, +), local country, minimum digits
  2. GUI > Alerts > Filter common - Configure:
Setting Description
Called number prefixes Which prefixes trigger alert (uncheck ALL for all international)
Exclude called number Country codes to exclude (e.g., +44, 0044 for UK)
Strict for prefixes Require international prefix (00/+)
NANPA North American Numbering Plan

Sensors Alerts

Monitor sensor health and status:

  • Offline detection - Sensor not communicating
  • Old CDR - No recent CDRs written (capture or DB issue)
  • Big SQL queue stat - Growing queue indicates DB bottleneck (warning: >20 files, critical: >100)

SIP REGISTER Alerts

Alert Type Purpose Use Case
SIP REGISTER RRD beta Response time monitoring Network latency, packet loss
SIP failed Register (beta) Failed registrations by IP Brute-force, credential stuffing
multiple register (beta) Same account from multiple IPs Credential compromise detection

⚠️ Warning: multiple register (beta) detects SIMULTANEOUS registrations from multiple IPs (security). For detecting IP changes when device moves networks, use CDR&RTP alert with external script.


Alert Output Fields

The multiple register (beta) and other SIP REGISTER alerts output the following fields in email notifications and GUI:

Field Source Description
username SIP Contact header The registered user identity
from fields SIP From header From-number, From-domain extracted from From header
to fields SIP To header To-number, To-domain extracted from To header
lookup name Tools > Prefix Lookup Custom label if phone number matches a configured prefix entry

ℹ️ Note: The lookup name column displays custom labels from Prefix Lookup when a phone number matches a configured prefix. If no match exists, the field remains empty or shows the raw number.

CDR Trends Alerts

Monitor metric deviations from historical baselines (e.g., ASR drops).

Parameter Description
Type Metric to monitor (ASR, ACD, etc.)
Offset Historical baseline (1 week, 1 day)
Range Current evaluation window (1 hour)
Method Deviation (%) or Threshold (absolute)
Limit Inc./Dec. Trigger threshold percentage

Common Filters

All alert types support:

  • IP/Number Group - Predefined groups from Groups menu
  • IP Addresses / Numbers - Individual values (one per line)
  • Email Group / Emails - Recipients
  • Last sip response - Filter by response text (requires save_sip_history = responses)
  • External script - Custom script path for integrations

⚠️ Warning: Alerts use OR logic between conditions. AND logic is NOT supported. Workaround: create separate alerts and correlate manually.

Caller vs Called Filtering

The Numbers filter matches against both caller and called fields. You cannot create alerts that trigger only when a number is the caller or only the called. Use IP Groups with Trunk/Server checkboxes for direction-based filtering. See Groups.

External Scripts

Enable webhook integration (Datadog, Slack, custom systems).

Configuration: Enter full absolute path in External script field (e.g., /usr/local/bin/alert-webhook.sh).

Arguments passed to script:

Arg Description
$1 Alert ID
$2 Alert name
$3 Unix timestamp
$4 JSON data with CDR IDs

Example - Slack notification: 

#!/bin/bash
# /usr/local/bin/slack-alert.sh
SLACK_WEBHOOK="https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
curl -X POST "$SLACK_WEBHOOK" -H "Content-Type: application/json" \
  -d '{"text": "VoIPmonitor Alert: '"$2"'"}'

ℹ️ Note: IP addresses in CDR table are decimal integers. Use long2ip() (PHP) or INET_NTOA() (MySQL) for conversion.

Sent Alerts

View triggered alerts via GUI > Alerts > Sent Alerts. Shows:

  • Parameters table - QoS metrics with highlighted bad values
  • CDR records - Calls that triggered alert with flags: (M)OS, (J)itter, (P)acket loss, (D)elay

Custom Report Alerts

Alert on criteria not in native types (e.g., custom SIP headers).

Workflow:

  1. Capture header in /etc/voipmonitor.conf: custom_headers = Max-Forwards
  2. Enable in GUI > Settings > CDR Custom Headers
  3. Create filter in CDR view, save as template
  4. Create Daily Report with filter in GUI > Reports > Configure Daily Reports

ℹ️ Note: Custom report alerts cannot group by caller/called for threshold detection (e.g., "alert if same caller has >X failures"). Use CDR Summary reports for aggregated data.

Troubleshooting

Email Not Sent

Diagnosis:

  • Entries in "Sent Alerts" but no email → MTA issue
  • No entries in "Sent Alerts" → Alert conditions or cron issue
# Test MTA
echo "Test" | mail -s "Test" your@email.com

# Check MTA status
systemctl status postfix  # or exim4/sendmail

# Check logs
tail -f /var/log/mail.log  # Debian/Ubuntu
tail -f /var/log/maillog   # RHEL/CentOS

# Check mail queue
mailq

Status 250 or "Queued mail for delivery" = Your server delivered successfully. If recipient didn't receive, issue is on their side (spam folder, quarantine, blacklisting). Mail Queue Not Delivering: If emails accumulate in the queue but are not being sent: 

# Verify queue manager is running
ps aux | grep qmgr

# Restart Postfix
systemctl restart postfix

# Force immediate delivery of queued emails
postfix flush

Alerts Not Triggering

Enable debug logging: 

// Add to ./config/system_configuration.php
define('CRON_LOG_FILE', '/tmp/alert.log');

# Monitor processing
tail -f /tmp/alert.log

Common causes:

  • Cron not running - verify with crontab -l
  • PHP CLI version mismatch - use update-alternatives --set php /usr/bin/php8.x
  • SQL queue growing - DB can't keep up (see Scaling)
  • Alert disabled or filter mismatch

Concurrent Calls Alerts

Type Data Source Aggregation Timing
Fraud concurrent calls SIP INVITEs (realtime) Source IP only Immediate
Regular concurrent calls CDRs (database) Source/Dest IP, Domain, Custom Delayed

Use regular concurrent calls for destination IP monitoring (trunk capacity). Investigating Fraud: Realtime Concurrent Calls Alerts

Since this alert type triggers before CDRs are written, use the following procedure to investigate the calls that triggered the alert:

  1. Navigate to GUI → CDR
  2. Use the filter form to add the is international filter
  3. Set the from and to date range to match the time the alert was sent
  4. Go to the bottom of the CDR view and enable grouping by country
  5. Analyze the traffic by country to identify the source of the fraudulent activity

External Script Not Running

  1. Use preview button to test alert triggers
  2. Verify absolute path (not relative)
  3. Check permissions: chmod 755 /path/to/script.sh
  4. Include shebang: #!/bin/bash
  5. Use full command paths (e.g., /usr/bin/curl)
  6. For URLs, create script with curl/wget - cannot put URL directly in field

"Crontab Log Too Old" Warning

Causes:

  1. Cron not running → Add cron entry
  2. PHP CLI version mismatch → update-alternatives --set php /usr/bin/php8.x
  3. Database overload → Check SQLq in GUI > Settings > Sensors, see Scaling

See Also




AI Summary for RAG

Summary: VoIPmonitor Alerts system provides email notifications for QoS thresholds (RTP: MOS, jitter, packet loss), SIP response codes (0=no response, 408=timeout), sensor health, and registration monitoring. Alert types include RTP, RTP&CDR (with filter templates for duration/absolute_timeout), SIP Response (use "from all" unchecked for IP group percentages), International Calls (prefix-based, NOT GeoIP), Sensors, SIP REGISTER alerts (RRD beta for latency, failed Register beta for brute-force, multiple register beta for credential compromise), and CDR Trends (ASR deviation monitoring). External scripts enable webhook integrations. CRITICAL: Alerts use OR logic only - AND not supported. IP addresses stored as integers - use long2ip()/INET_NTOA() for conversion.

Keywords: alerts, email notifications, QoS, MOS, jitter, packet loss, SIP response, 408 timeout, sensors monitoring, SIP REGISTER, brute force, credential stuffing, international calls, called number prefixes, CDR trends, ASR, external scripts, webhooks, from all checkbox, OR logic, crontab, MTA, Postfix, CRON_LOG_FILE, concurrent calls, SQL queue

Key Questions:

  • How do I configure email alerts in VoIPmonitor?
  • What alert types are available (RTP, SIP, Sensors)?
  • How do I configure international call alerts with prefix filtering?
  • What does the "from all" checkbox do in percentage alerts?
  • How do I integrate alerts with webhooks (Slack, Datadog)?
  • Why are my alerts not triggering?
  • How do I troubleshoot email delivery issues?
  • What is the difference between fraud and regular concurrent calls alerts?
  • How do I detect SIP registration attacks (brute-force)?
  • Do alerts support AND logic between conditions?
Retrieved from "https://www.voipmonitor.org/doc/index.php?title=Alerts&oldid=10738"