Collectd installation: Difference between revisions

From VoIPmonitor.org
Jump to navigation Jump to search
No edit summary
No edit summary
 
(One intermediate revision by one other user not shown)
Line 1: Line 1:
= Centos =
{{DISPLAYTITLE:System Performance Monitoring with Collectd}}


''' Httpd '''
'''This guide explains how to set up `collectd`, a powerful system statistics collection daemon, to monitor the long-term performance of your VoIPmonitor server and create automated alerts for critical events.'''
You need to have installed web server with enabled perl and CGI scripts processing (for be possible to look at results using web-browser). This how to doesn't cover web server configuation.


== What is Collectd and Why Use It? ==
'''Collectd''' is a lightweight daemon that periodically collects performance metrics from your server and stores them in a time-series database (by default, RRD files). While VoIPmonitor's own logs provide real-time status, `collectd` is invaluable for:


Install collectd packages from epel repositories:
* '''Long-Term Performance Graphing:''' Track CPU load, memory usage, disk space, network traffic, and even specific VoIPmonitor metrics over weeks, months, or years.
yum install epel-release
* '''Capacity Planning:''' Identify performance trends to anticipate when you might need to upgrade hardware.
yum install collectd
* '''Automated Alerting:''' Configure thresholds to automatically send email notifications when critical limits are reached (e.g., disk space is 90% full).
yum install collectd-web


This guide covers installing `collectd` and `collectd-web` (a simple web interface for viewing graphs) and configuring key plugins for monitoring a VoIPmonitor sensor.


== Step 1: Installation and Basic Setup ==
The installation process differs slightly between Debian-based and RHEL-based systems.


Atention: by default is web-collection accessible only from localhost, see config of httpd (search for 'Deny from' and 'Allow from' and change it:
=== For Debian / Ubuntu ===
vim /etc/httpd/conf.d/collectd.conf
;1. Install required packages:
service httpd restart
<pre>sudo apt-get update
sudo apt-get install collectd collectd-web rrdtool</pre>


You can now access collectd collections entering this url to a browser:
;2. Configure Apache2 for `collectd-web`:
http://server_ip/collectd/bin/index.cgi
The `collectd-web` interface is a set of CGI scripts. You need to enable the CGI module in Apache.
<pre>
sudo a2enmod cgi
sudo systemctl restart apache2
</pre>


;3. Create a web-accessible link:
<pre>sudo ln -s /usr/share/collectd/www/ /var/www/html/collectd</pre>


= Debian =
=== For CentOS / RHEL / AlmaLinux ===
;1. Enable the EPEL repository and install packages:
<pre>sudo yum install epel-release
sudo yum install collectd collectd-web
</pre>


== Apache2 ==
;2. Configure Apache (`httpd`):
The default `collectd-web` configuration only allows access from localhost. You need to edit it to allow access from your IP address or network.
;Edit the configuration file:
<pre>sudo nano /etc/httpd/conf.d/collectd.conf</pre>
;Find the `<Directory>` block and modify the `Require` directive. For example, to allow access from any IP on the `192.168.1.0/24` network:
<pre>
<Directory /usr/share/collectd/www/cgi-bin/>
    # ... other settings ...
    Require ip 127.0.0.1 192.168.1.0/24
</Directory>
</pre>


For ability checking results using browser you need web server with enabled CGI and perl scripts processing.
;3. Restart Apache:
<pre>sudo systemctl restart httpd</pre>


'''Enable apache2 modules'''
After these steps, you should be able to access the web interface at `http://your-server-ip/collectd/`.
perl
cgi
using command '''a2enmod''' .


== Step 2: Configuring Collectd Plugins ==
All configuration is done in `/etc/collectd/collectd.conf` or by adding new `.conf` files to `/etc/collectd/conf.d/`.


''' Add CGI handler into apache processing (in mime.conf uncomment)'''
=== A. Monitoring VoIPmonitor-Specific Metrics (Tail Plugin) ===
  AddHandler cgi-script .cgi
The `tail` plugin allows `collectd` to parse VoIPmonitor's syslog output in real-time and extract key performance indicators. This is the most powerful feature for VoIPmonitor diagnostics.


=== B. Monitoring Hardware Health (Sensors Plugin) ===
The `sensors` plugin reads temperature, voltage, and fan speed data from hardware sensors on your server's motherboard and CPU.
;First, install and configure `lm-sensors`:
<pre>
sudo apt-get install lm-sensors  # Debian/Ubuntu
sudo yum install lm_sensors    # RHEL/CentOS
sudo sensors-detect            # Follow the prompts to detect sensors
</pre>


=== C. Monitoring Disk Space (DF Plugin) ===
The `df` plugin tracks disk space usage, which is critical for the VoIPmonitor spool directory.


''' Define +ExecCGI option '''
=== Example `collectd.conf` Configuration ===
under directory where will be collctd's GUI link placed
Here is a consolidated configuration block you can add to `/etc/collectd/collectd.conf` to enable these three plugins.
<Directory "/var/www/html/abcdefg/collectd">
  Options +ExecCGI
</Directory>


== Debian7 ==
<pre>
Install collectd packages from debian main repositories:
# /etc/collectd/collectd.conf
apt-get install collectd libregexp-common-perl libconfig-general-perl librrds-perl


Create link from default www directory to collectd's doc.
# --------------------------------------------------------------------
ln -s /usr/share/doc/collectd-core/examples/collection3/ /var/www/collectd
# Plugin to parse VoIPmonitor's own performance logs
# --------------------------------------------------------------------
LoadPlugin tail
<Plugin "tail">
  <File "/var/log/syslog"> # Change to /var/log/messages for RHEL/CentOS
    Instance "voipmonitor"
    <Match>
      # Graphs the number of active calls
      Regex "calls\[([0-9]+),"
      DSType "GaugeAverage"
      Type "absolute"
      Instance "active_calls"
    </Match>
    <Match>
      # Graphs the CPU usage of the critical t0 packet capture thread
      Regex "t0CPU\[([0-9\.]+)%\]"
      DSType "GaugeAverage"
      Type "percent"
      Instance "cpu_thread_t0"
    </Match>
    <Match>
      # Graphs the fullness of the main memory buffer
      Regex "heap\[([0-9]+)\|"
      DSType "GaugeAverage"
      Type "percent"
      Instance "heap_usage"
    </Match>
  </File>
</Plugin>


Restart apache web-server (you should have cgid mod enabled "a2enmod cgid")
# --------------------------------------------------------------------
/etc/init.d/apache2 restart
# Plugin to monitor hardware temperatures and fan speeds
# --------------------------------------------------------------------
LoadPlugin sensors
<Plugin "sensors">
    SensorConfigFile "/etc/sensors3.conf"
</Plugin>


You can now access using browser:
# --------------------------------------------------------------------
http://server_ip/collectd/bin/index.cgi
# Plugin to monitor disk space usage
# --------------------------------------------------------------------
LoadPlugin df
<Plugin "df">
    # Specify the device or mount point you want to monitor
    MountPoint "/"
    MountPoint "/var/spool/voipmonitor"
   
    # Report values as a percentage instead of absolute numbers
    ValuesPercentage true
   
    # It's good practice to ignore filesystem types you don't care about
    FSType "tmpfs"
    FSType "sysfs"
    FSType "proc"
    IgnoreSelected true
</Plugin>
</pre>


== Debian8 ==
== Step 3: Setting Up Automated Alerts ==
Install collectd packages from debian main repositories:
You can configure `collectd` to send an email notification when a metric crosses a defined threshold.
apt-get install collectd libregexp-common-perl libconfig-general-perl librrds-perl


Create link from default www directory to collectd's collections.
=== Example: Low Disk Space Alert ===
ln -s /usr/share/doc/collectd-core/examples/collection3/ /var/www/html/collectd
This example will send an email to `admin@example.com` when the disk usage on the root (`/`) mount point exceeds 90%.


Restart apache
;Add the following to `/etc/collectd/collectd.conf`:
service apache2 restart
<pre>
# /etc/collectd/collectd.conf


You can now access using browser:
# --------------------------------------------------------------------
http://server_ip/collectd/bin/index.cgi
# NOTIFICATION PLUGINS
# --------------------------------------------------------------------


= Tail config example =
# First, load the plugins needed for alerting
LoadPlugin notify_email
LoadPlugin threshold


LoadPlugin tail
# Configure the email recipient and sender
<Plugin tail>
<Plugin "notify_email">
  <File "/var/log/messages">
  From "collectd@myserver.example.com"
    Instance "voipmonitor"
  Recipient "admin@example.com"
    Interval 30
  Subject "Collectd Alert: %s on %s"
    <Match>
</Plugin>
        Regex "calls.([0-9]+),"
        DSType "GaugeAverage"
        Type "count"
        Instance "Calls"
    </Match>
    <Match>
        Regex "PS\\[C:([0-9]+)"
        DSType "GaugeAverage"
        Type "count"
        Instance "PS-C"
    </Match>
    <Match>
        Regex "PS\\[.* R:([0-9]+)"
        DSType "GaugeAverage"
        Type "count"
        Instance "PS-R"
    </Match>
    <Match>
        Regex "PS\\[.* A:([0-9]+)"
        DSType "GaugeAverage"
        Type "count"
        Instance "PS-a"
    </Match>
    <Match>
        Regex "heap\\[([0-9]+)"
        DSType "GaugeAverage"
        Type "count"
        Instance "heap"
    </Match>
    <Match>
        Regex "VSZ\\[([0-9]+)"
        DSType "GaugeAverage"
        Type "count"
        Instance "rss"
    </Match>
  </File>
</Plugin>


= Sensor plugin configuration =
# Configure the threshold
<Plugin "threshold">
  # Target the 'df' plugin
  <Plugin "df">
    # Specify the mount point instance
    Instance "root"
    # Target the metric for used percentage
    <Type "percent_bytes">
      Instance "used"
      # Trigger a warning if the value goes above 90%
      WarningMax 90.00
    </Type>
  </Plugin>
</Plugin>
</pre>
After configuring, restart the `collectd` service (`sudo systemctl restart collectd`) for the changes to take effect.


* plugin contains pcie temperature, cpu temperature, etc.
== AI Summary for RAG ==
* install lm sensor package
'''Summary:''' This guide explains how to use `collectd` for long-term performance monitoring and automated alerting for a VoIPmonitor server. It clarifies that `collectd` is a system statistics daemon used to create historical graphs and send notifications, complementing VoIPmonitor's real-time logs. The article provides a step-by-step guide for installing `collectd` and its web interface, `collectd-web`, on both Debian/Ubuntu and CentOS/RHEL systems, including the necessary Apache configurations. The core of the guide focuses on configuring key `collectd` plugins: the `tail` plugin to parse VoIPmonitor's syslog output and graph critical metrics like active calls and `t0CPU` usage; the `sensors` plugin for hardware health; and the `df` plugin for monitoring disk space. It provides a consolidated, annotated `collectd.conf` example. Finally, it details how to set up automated email alerts using the `threshold` and `notify_email` plugins, with a practical example for sending a notification when disk usage exceeds a defined percentage.
* run sensors-detect binary to detect sensors
'''Keywords:''' collectd, monitoring, performance, graphs, alerting, notification, threshold, rrdtool, collectd-web, tail plugin, syslog, df plugin, disk space, sensors plugin, temperature, cpu, memory
* add an example to the config
'''Key Questions:'''
 
* How can I create long-term performance graphs for my VoIPmonitor server?
LoadPlugin sensors
* What is `collectd` and how can it help monitor VoIPmonitor?
<Plugin sensors>
* How to install `collectd` and `collectd-web` on Debian or CentOS?
        SensorConfigFile "/etc/sensors3.conf"
* How can I graph specific metrics from the VoIPmonitor syslog, like active calls or t0CPU?
</Plugin>
* How do I configure the `tail` plugin for collectd?
 
* How can I set up an email alert for when my disk space is almost full?
= Disk space plugin =
* How to configure the `threshold` and `notify_email` plugins in collectd?
 
* simple setting for percentage usage
 
LoadPlugin df
<Plugin df>
#      Device "/dev/hda1"
#      Device "192.168.0.2:/mnt/nfs"
#      MountPoint "/home"
#      FSType "ext3"
#      IgnoreSelected false
#      ReportByDevice false
#      ReportInodes false
        ValuesAbsolute false
        ValuesPercentage true
</Plugin>
 
= Scale stored value =
* enable these two plugins
 
LoadPlugin match_regex
LoadPlugin target_scale
 
* add a filter (example is for rss count)
 
<Chain "PreCache">
    <Rule>
        <Match "regex">
            Plugin "^tail$"
            TypeInstance "^rss$"
            Invert false
        </Match>
        <Target "scale">
            Factor 1048576
        </Target>
    </Rule>
</Chain>
 
= Threshold and notify setting =
* on RH you need to install collectd-notify_email.x86_64 package. On debian it's in collectd-core package.
* example for df setting from the example above
 
LoadPlugin "notify_email"
<Plugin "notify_email">
  From "collectd-datora@localhost"
  Recipient "milan.kocian@voipmonitor.org"
  Recipient "festr@voipmonitor.org"
</Plugin>
 
LoadPlugin "threshold"
<Plugin threshold>
  <Plugin "df">
    Instance "root"
    <Type "percent_bytes-used">
        WarningMax 75
    </Type>
  </Plugin>
</Plugin>
 
* for notify's logging in syslog (can be set together with notify_email)
<Plugin syslog>
#      LogLevel info
        NotifyLevel "OKAY"
</Plugin>

Latest revision as of 22:35, 30 June 2025


This guide explains how to set up `collectd`, a powerful system statistics collection daemon, to monitor the long-term performance of your VoIPmonitor server and create automated alerts for critical events.

What is Collectd and Why Use It?

Collectd is a lightweight daemon that periodically collects performance metrics from your server and stores them in a time-series database (by default, RRD files). While VoIPmonitor's own logs provide real-time status, `collectd` is invaluable for:

  • Long-Term Performance Graphing: Track CPU load, memory usage, disk space, network traffic, and even specific VoIPmonitor metrics over weeks, months, or years.
  • Capacity Planning: Identify performance trends to anticipate when you might need to upgrade hardware.
  • Automated Alerting: Configure thresholds to automatically send email notifications when critical limits are reached (e.g., disk space is 90% full).

This guide covers installing `collectd` and `collectd-web` (a simple web interface for viewing graphs) and configuring key plugins for monitoring a VoIPmonitor sensor.

Step 1: Installation and Basic Setup

The installation process differs slightly between Debian-based and RHEL-based systems.

For Debian / Ubuntu

1. Install required packages
sudo apt-get update
sudo apt-get install collectd collectd-web rrdtool
2. Configure Apache2 for `collectd-web`

The `collectd-web` interface is a set of CGI scripts. You need to enable the CGI module in Apache.

sudo a2enmod cgi
sudo systemctl restart apache2
3. Create a web-accessible link
sudo ln -s /usr/share/collectd/www/ /var/www/html/collectd

For CentOS / RHEL / AlmaLinux

1. Enable the EPEL repository and install packages
sudo yum install epel-release
sudo yum install collectd collectd-web
2. Configure Apache (`httpd`)

The default `collectd-web` configuration only allows access from localhost. You need to edit it to allow access from your IP address or network.

Edit the configuration file
sudo nano /etc/httpd/conf.d/collectd.conf
Find the `<Directory>` block and modify the `Require` directive. For example, to allow access from any IP on the `192.168.1.0/24` network
<Directory /usr/share/collectd/www/cgi-bin/>
    # ... other settings ...
    Require ip 127.0.0.1 192.168.1.0/24
</Directory>
3. Restart Apache
sudo systemctl restart httpd

After these steps, you should be able to access the web interface at `http://your-server-ip/collectd/`.

Step 2: Configuring Collectd Plugins

All configuration is done in `/etc/collectd/collectd.conf` or by adding new `.conf` files to `/etc/collectd/conf.d/`.

A. Monitoring VoIPmonitor-Specific Metrics (Tail Plugin)

The `tail` plugin allows `collectd` to parse VoIPmonitor's syslog output in real-time and extract key performance indicators. This is the most powerful feature for VoIPmonitor diagnostics.

B. Monitoring Hardware Health (Sensors Plugin)

The `sensors` plugin reads temperature, voltage, and fan speed data from hardware sensors on your server's motherboard and CPU.

First, install and configure `lm-sensors`
sudo apt-get install lm-sensors  # Debian/Ubuntu
sudo yum install lm_sensors     # RHEL/CentOS
sudo sensors-detect             # Follow the prompts to detect sensors

C. Monitoring Disk Space (DF Plugin)

The `df` plugin tracks disk space usage, which is critical for the VoIPmonitor spool directory.

Example `collectd.conf` Configuration

Here is a consolidated configuration block you can add to `/etc/collectd/collectd.conf` to enable these three plugins.

# /etc/collectd/collectd.conf

# --------------------------------------------------------------------
# Plugin to parse VoIPmonitor's own performance logs
# --------------------------------------------------------------------
LoadPlugin tail
<Plugin "tail">
  <File "/var/log/syslog"> # Change to /var/log/messages for RHEL/CentOS
    Instance "voipmonitor"
    <Match>
      # Graphs the number of active calls
      Regex "calls\[([0-9]+),"
      DSType "GaugeAverage"
      Type "absolute"
      Instance "active_calls"
    </Match>
    <Match>
      # Graphs the CPU usage of the critical t0 packet capture thread
      Regex "t0CPU\[([0-9\.]+)%\]"
      DSType "GaugeAverage"
      Type "percent"
      Instance "cpu_thread_t0"
    </Match>
    <Match>
      # Graphs the fullness of the main memory buffer
      Regex "heap\[([0-9]+)\|"
      DSType "GaugeAverage"
      Type "percent"
      Instance "heap_usage"
    </Match>
  </File>
</Plugin>

# --------------------------------------------------------------------
# Plugin to monitor hardware temperatures and fan speeds
# --------------------------------------------------------------------
LoadPlugin sensors
<Plugin "sensors">
    SensorConfigFile "/etc/sensors3.conf"
</Plugin>

# --------------------------------------------------------------------
# Plugin to monitor disk space usage
# --------------------------------------------------------------------
LoadPlugin df
<Plugin "df">
    # Specify the device or mount point you want to monitor
    MountPoint "/"
    MountPoint "/var/spool/voipmonitor"
    
    # Report values as a percentage instead of absolute numbers
    ValuesPercentage true
    
    # It's good practice to ignore filesystem types you don't care about
    FSType "tmpfs"
    FSType "sysfs"
    FSType "proc"
    IgnoreSelected true
</Plugin>

Step 3: Setting Up Automated Alerts

You can configure `collectd` to send an email notification when a metric crosses a defined threshold.

Example: Low Disk Space Alert

This example will send an email to `admin@example.com` when the disk usage on the root (`/`) mount point exceeds 90%.

Add the following to `/etc/collectd/collectd.conf`
# /etc/collectd/collectd.conf

# --------------------------------------------------------------------
# NOTIFICATION PLUGINS
# --------------------------------------------------------------------

# First, load the plugins needed for alerting
LoadPlugin notify_email
LoadPlugin threshold

# Configure the email recipient and sender
<Plugin "notify_email">
  From "collectd@myserver.example.com"
  Recipient "admin@example.com"
  Subject "Collectd Alert: %s on %s"
</Plugin>

# Configure the threshold
<Plugin "threshold">
  # Target the 'df' plugin
  <Plugin "df">
    # Specify the mount point instance
    Instance "root"
    # Target the metric for used percentage
    <Type "percent_bytes">
      Instance "used"
      # Trigger a warning if the value goes above 90%
      WarningMax 90.00
    </Type>
  </Plugin>
</Plugin>

After configuring, restart the `collectd` service (`sudo systemctl restart collectd`) for the changes to take effect.

AI Summary for RAG

Summary: This guide explains how to use `collectd` for long-term performance monitoring and automated alerting for a VoIPmonitor server. It clarifies that `collectd` is a system statistics daemon used to create historical graphs and send notifications, complementing VoIPmonitor's real-time logs. The article provides a step-by-step guide for installing `collectd` and its web interface, `collectd-web`, on both Debian/Ubuntu and CentOS/RHEL systems, including the necessary Apache configurations. The core of the guide focuses on configuring key `collectd` plugins: the `tail` plugin to parse VoIPmonitor's syslog output and graph critical metrics like active calls and `t0CPU` usage; the `sensors` plugin for hardware health; and the `df` plugin for monitoring disk space. It provides a consolidated, annotated `collectd.conf` example. Finally, it details how to set up automated email alerts using the `threshold` and `notify_email` plugins, with a practical example for sending a notification when disk usage exceeds a defined percentage. Keywords: collectd, monitoring, performance, graphs, alerting, notification, threshold, rrdtool, collectd-web, tail plugin, syslog, df plugin, disk space, sensors plugin, temperature, cpu, memory Key Questions:

  • How can I create long-term performance graphs for my VoIPmonitor server?
  • What is `collectd` and how can it help monitor VoIPmonitor?
  • How to install `collectd` and `collectd-web` on Debian or CentOS?
  • How can I graph specific metrics from the VoIPmonitor syslog, like active calls or t0CPU?
  • How do I configure the `tail` plugin for collectd?
  • How can I set up an email alert for when my disk space is almost full?
  • How to configure the `threshold` and `notify_email` plugins in collectd?