Whisper

This guide explains how to integrate OpenAI's Whisper, a powerful automatic speech recognition (ASR) system, with VoIPmonitor for both on-demand and automatic call transcription.

Introduction to Whisper Integration

VoIPmonitor integrates Whisper, an ASR system from OpenAI trained on 680,000 hours of multilingual and multitask supervised data. This large and diverse dataset leads to improved robustness to accents, background noise, and technical language.

There are two primary ways to use Whisper with VoIPmonitor:

1. On-Demand Transcription (in the GUI): The simplest method. A user can click a button on any call in the GUI to transcribe it. The processing happens on the GUI server.
2. Automatic Transcription (in the Sniffer): A more advanced setup where the sensor automatically transcribes all calls (or a subset) in the background immediately after they finish.

For both methods, you must choose one of two underlying Whisper engines to install and configure.

Choosing Your Whisper Engine

• OpenAI Whisper (Python): The official implementation from OpenAI. It is easier to install (pip install openai-whisper) but can be slower for CPU-based transcription. It uses PyTorch and requires ffmpeg for audio pre-processing. The official implementation does not behave deterministically (the same run can have different results), but this can be addressed with a custom script.
• whisper.cpp (C++): A high-performance C++ port of whisper.cpp. It is significantly faster for CPU transcription and is the recommended engine for server-side processing. It requires manual compilation but offers superior performance and optimizations like NVIDIA CUDA for GPU acceleration (up to 30x faster). Note that it requires audio input to be 16kHz, 1-channel (mono).

Path A: On-Demand Transcription in the GUI

This setup allows users to manually trigger transcription from the call detail page. The processing occurs on the web server where the GUI is hosted.

Quick Start: Using Pre-built Model (No Compilation Required)

If you want to enable on-demand transcription without compiling or installing packages, you can download a pre-built model from the VoIPmonitor server and use it directly.

Step 1: Download the Pre-built Model

Download the Whisper model file directly to the GUI's bin/ directory:

# Download the base model to the default GUI directory
wget https://download.voipmonitor.org/whisper/ggml-base.bin -O /var/www/html/bin/ggml-base.bin

# For Debian-based systems where the GUI is in /var/www/:
wget https://download.voipmonitor.org/whisper/ggml-base.bin -O /var/www/voipmonitor/bin/ggml-base.bin

Step 2: Set File Ownership

Ensure the web server user owns the model file:

# For Apache on Debian/Ubuntu (www-data user)
chown www-data:www-data /var/www/html/bin/ggml-base.bin

# For Apache on RedHat/CentOS (apache user)
chown apache:apache /var/www/html/bin/ggml-base.bin

# For GUI in /var/www/voipmonitor:
chown www-data:www-data /var/www/voipmonitor/bin/ggml-base.bin

Step 3: Verify

The "Transcribe" button should now appear on call detail pages in the GUI. No configuration changes are required when using the default /var/www/html/bin/ location.

Note: This method uses the whisper.cpp engine which is bundled with the GUI. The model file format (.bin) is compatible with the bundled engine.

Option 1: Using the whisper.cpp Engine (Recommended)

Step 1: Install whisper.cpp and Download a Model

First, you need to compile the whisper.cpp project and download a pre-trained model on your GUI server.

# Clone the repository
git clone https://github.com/ggerganov/whisper.cpp.git
cd whisper.cpp

# Compile the main application
make -j

# Download a model (e.g., 'base.en' for English-only, or 'small' for multilingual)
./models/download-ggml-model.sh base.en

This will create the main executable at ./main and download the model to the ./models/ directory. Note that whisper.cpp models (GGML format) are not binary compatible with the official OpenAI models (.pt format), but a conversion script is provided in the project.

Step 2: Configure the VoIPmonitor GUI

Edit your GUI's configuration file at /var/www/html/config/configuration.php and add the following definitions:

<?php
// /var/www/html/config/configuration.php

// Tell the GUI to use the whisper.cpp engine
define('WHISPER_NATIVE', true);

// Provide the absolute path to the model file you downloaded
define('WHISPER_MODEL', '/path/to/your/whisper.cpp/models/ggml-base.en.bin');

// Optional: Specify the number of threads for transcription
define('WHISPER_THREADS', 4);

No further setup is required. The GUI will now show a "Transcribe" button on call detail pages.

Option 2: Using the OpenAI Whisper Engine

Step 1: Install the Python Package and Dependencies

# Install the whisper library via pip
pip install openai-whisper

# Install ffmpeg, which is required for audio conversion
# For Debian/Ubuntu
sudo apt-get install ffmpeg
# For CentOS/RHEL/Fedora
sudo dnf install ffmpeg

Step 2: Prepare the Model and Configure the GUI

The Python library can download models automatically, but it's best practice to specify an absolute path in the configuration. First, trigger a download to a known location.

# This command will download the 'small' model to /opt/whisper_models/
# You can use any audio file; its content doesn't matter for the download.
whisper audio.wav --model=small --model_dir=/opt/whisper_models

Now, edit /var/www/html/config/configuration.php and provide the full path to the downloaded model file.

<?php
// /var/www/html/config/configuration.php

// Provide the absolute path to the downloaded .pt model file.
define('WHISPER_MODEL', '/opt/whisper_models/small.pt');

// Optional: Specify the number of threads
define('WHISPER_THREADS', 4);

Testing the GUI Integration

You can test the transcription process from the command line as the GUI would run it. This is useful for debugging paths and performance.

# Example test for a whisper.cpp setup
/var/www/html/bin/vm --audio-transcribe='/tmp/audio.wav {}' --json_config='[{"whisper_native":"yes"},{"whisper_model":"/path/to/your/whisper.cpp/models/ggml-small.bin"},{"whisper_threads":"2"}]' -v1,whisper

Troubleshooting Failed Model Downloads

If the Whisper transcription feature fails to automatically download the required model file (e.g., ggml-base.bin), follow these steps to diagnose the issue.

Step 1: Test Connectivity to the Download URL

From your GUI server, test if the server can reach the model download URL:

# Test connectivity to the pre-built model
curl -I https://download.voipmonitor.org/whisper/ggml-base.bin

# Or test with wget (this will show connection details)
wget --spider https://download.voipmonitor.org/whisper/ggml-base.bin

If the URL is reachable, you should see HTTP 200 OK or similar success response.

Step 2: Check Firewall Rules

If the connectivity test fails or times out, check if outbound connections are blocked by a firewall:

# Check if iptables is blocking outbound connections (as root)
sudo iptables -L -v -n | grep -E "OUTPUT|FORWARD"

# Check if firewalld is active and blocking outbound connections
sudo firewall-cmd --list-all

# Check UFW status (Ubuntu/Debian)
sudo ufw status

If outbound connections are restricted, you may need to allow connections on port 443 (HTTPS) from your server.

Step 3: Check Proxy Settings

If your network requires outbound connections through a proxy server, configure the appropriate environment variables:

# Set proxy environment variables (temporary, for current session)
export HTTP_PROXY="http://proxy.yourdomain.com:3128"
export HTTPS_PROXY="http://proxy.yourdomain.com:3128"
export http_proxy="http://proxy.yourdomain.com:3128"
export https_proxy="http://proxy.yourdomain.com:3128"

# Then retry the download or connection test
wget https://download.voipmonitor.org/whisper/ggml-base.bin

To make proxy settings permanent, add them to /etc/environment or your web server's configuration file.

Step 4: Check DNS Resolution

If you get connection errors like "Could not resolve host", verify DNS resolution is working:

# Test DNS resolution
nslookup download.voipmonitor.org
dig download.voipmonitor.org

# Check your DNS servers
cat /etc/resolv.conf

If DNS resolution fails, you may need to update your DNS servers in /etc/resolv.conf or your network configuration.

Workaround: Manual Download

If you cannot resolve the network issues immediately, you can manually download the model file using the instructions in the "Quick Start" section above:

wget https://download.voipmonitor.org/whisper/ggml-base.bin -O /var/www/html/bin/ggml-base.bin
chown www-data:www-data /var/www/html/bin/ggml-base.bin

⚠️ Warning: Manually downloading the model file is a workaround. The underlying network issue (firewall, proxy, DNS) should still be resolved to ensure future automatic downloads work correctly.

Path B: Automatic Transcription in the Sniffer

This setup automatically transcribes calls in the background on the sensor itself. This is a headless operation and requires configuration in voipmonitor.conf. Using whisper.cpp is strongly recommended for this server-side task due to its superior performance.

Step 1: Prepare Your Engine on the Sensor

You must have one of the Whisper engines installed on the sensor machine.

For whisper.cpp

Follow the installation steps from "Path A" to compile whisper.cpp. For advanced integration, you may need to build the shared libraries and install them system-wide (see Advanced section below).

For OpenAI Whisper

Follow the Python package installation steps from "Path A".

Step 2: Configure the Sniffer

Edit /etc/voipmonitor.conf on your sensor to enable and control automatic transcription. You have three main ways to integrate it.

Option 1: Using whisper.cpp (Recommended)

This uses the compiled main executable.

# /etc/voipmonitor.conf

# Enable the transcription feature
audio_transcribe = yes

# Tell the sniffer to use the high-performance C++ engine
whisper_native = yes

# --- CRITICAL ---
# You MUST provide the absolute path to the downloaded whisper.cpp model file
whisper_model = /path/to/your/whisper.cpp/models/ggml-small.bin

Option 2: Using OpenAI Whisper

This uses the Python library.

# /etc/voipmonitor.conf

# Enable the transcription feature
audio_transcribe = yes

# Use the Python engine (this is the default, but explicit is better)
whisper_native = no

# Specify the model name to use ('small' is a good default).
# The library will download it to ~/.cache/whisper/ if not found.
whisper_model = small

Option 3: Using whisper.cpp as a Loadable Module (Advanced)

This method allows you to update the whisper.cpp library without recompiling the entire sniffer. It requires a modified whisper.cpp build (see Advanced section).

# /etc/voipmonitor.conf

audio_transcribe = yes
whisper_native = yes
whisper_model = /path/to/your/whisper.cpp/models/ggml-small.bin

# Specify the path to the compiled shared library
whisper_native_lib = /path/to/your/whisper.cpp/libwhisper.so

Step 3: Fine-Tuning Transcription Parameters

The following parameters in voipmonitor.conf allow you to control the transcription process:

audio_transcribe = yes

(Default: no) Enables the audio transcription feature.

audio_transcribe_connect_duration_min = 10

(Default: 10) Only transcribes calls that were connected for at least this many seconds.

audio_transcribe_threads = 2

(Default: 2) The number of calls to transcribe concurrently.

audio_transcribe_queue_length_max = 100

(Default: 100) The maximum number of calls waiting in the transcription queue.

whisper_native = no

(Default: no) Set to yes to force the use of the whisper.cpp engine.

whisper_model = small

For OpenAI Whisper, this is the model name (tiny, base, small, etc.). For whisper.cpp, this must be the full, absolute path to the .bin model file.

whisper_language = auto

(Default: auto) Can be a specific language code (e.g., en, de), auto for detection, or by_number to guess based on the phone number's country code.

whisper_threads = 2

(Default: 2) The number of CPU threads to use for a single transcription job.

whisper_timeout = 300

(Default: 300) For OpenAI Whisper only. Maximum time in seconds for a single transcription.

whisper_deterministic_mode = yes

(Default: yes) For OpenAI Whisper only. Aims for more consistent, repeatable transcription results.

whisper_python = /usr/bin/python3

(Default: not set) For OpenAI Whisper only. Specifies the path to the Python binary if it's not in the system's PATH.

whisper_native_lib = /path/to/libwhisper.so

(Default: not set) For whisper.cpp only. Specifies the path to the shared library when using the loadable module method.

Advanced Topics

Compiling whisper.cpp with Libraries for Sniffer Integration

To compile the VoIPmonitor sniffer with built-in whisper.cpp support or to use it as a loadable library, you must build its shared and static libraries.

1. Build the libraries

cd /path/to/your/whisper.cpp
# Build the main executable, shared lib, and static lib
make -j
make libwhisper.so -j
make libwhisper.a -j

2. (Optional) Apply patch for loadable module

For the advanced "loadable module" integration (whisper_native_lib), a patch is required.

# Inside the whisper.cpp directory
patch < whisper.diff
make clean
make -j
make libwhisper.so -j

3. Install libraries and headers

For the sniffer's build process to find the whisper.cpp components, place them in standard system locations or create symbolic links.

# Create symbolic links to the compiled files in your whisper.cpp directory
ln -s $(pwd)/whisper.h /usr/local/include/whisper.h
ln -s $(pwd)/ggml.h /usr/local/include/ggml.h
ln -s $(pwd)/libwhisper.so /usr/local/lib64/libwhisper.so
ln -s $(pwd)/libwhisper.a /usr/local/lib64/libwhisper.a

CUDA Acceleration for whisper.cpp

To achieve a massive speed increase (up to 30x), you can compile whisper.cpp with NVIDIA CUDA support. This is highly recommended if you have a compatible NVIDIA GPU on your sensor or GUI server.

1. Install the NVIDIA CUDA Toolkit

Follow the official guide for your Linux distribution.

2. Set environment variables

Ensure the CUDA toolkit is in your system's path. You can add these lines to your ~/.bashrc file.

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

Verify with nvcc --version.

3. Re-compile whisper.cpp with the CUDA flag

cd /path/to/your/whisper.cpp
make clean
# Rebuild the executable and libraries with CUDA enabled
WHISPER_CUDA=1 make -j
WHISPER_CUDA=1 make libwhisper.so -j
WHISPER_CUDA=1 make libwhisper.a -j

VoIPmonitor will automatically detect and use the CUDA-enabled whisper.cpp binary or library.

AI Summary for RAG

Summary: This guide explains how to integrate OpenAI's Whisper ASR for call transcription in VoIPmonitor. It details two primary methods: on-demand transcription from the GUI (user-triggered) and automatic background transcription on the sniffer (server-side). For GUI on-demand transcription without installing packages or compiling, users can download a pre-built Whisper model directly from https://download.voipmonitor.org/whisper/ggml-base.bin to /var/www/html/bin/ and set ownership to the web server user (e.g., www-data:www-data). For more advanced setups, the guide compares the two available engines: the official Python OpenAI Whisper library and the high-performance C++ port, whisper.cpp, recommending whisper.cpp for server-side processing. It provides step-by-step instructions for compiling whisper.cpp from source, building its libraries, and installing the Python package via pip. It details the necessary configuration in both the GUI's configuration.php (e.g., WHISPER_NATIVE, WHISPER_MODEL) and the sniffer's voipmonitor.conf (e.g., audio_transcribe, whisper_native, whisper_native_lib). It also covers optional parameters for fine-tuning, such as setting language, thread counts, and minimum call duration. Finally, it includes a detailed section on enabling NVIDIA CUDA acceleration for whisper.cpp to achieve significant performance gains and explains advanced integration methods like using whisper.cpp as a loadable library.

Keywords: whisper, transcription, asr, speech to text, openai, whisper.cpp, audio_transcribe, whisper_native, whisper_model, whisper_native_lib, cuda, nvidia, gpu, acceleration, gui, sniffer, automatic transcription, on-demand, libwhisper.so, loadable module

Key Questions:

• How can I transcribe phone calls in VoIPmonitor?
• Can I enable on-demand transcription in the GUI without compiling or installing packages?
• How do I fix the "required Whisper model is missing" error?
• What is the simplest way to download the Whisper model for the GUI?
• What is the difference between OpenAI Whisper and whisper.cpp? Which one should I use?
• How do I configure on-demand call transcription in the GUI?
• How do I set up the sniffer for automatic, server-side transcription of all calls?
• What are the required parameters in voipmonitor.conf for Whisper?
• How can I speed up Whisper transcription using an NVIDIA GPU (CUDA)?
• How do I install and compile whisper.cpp, including its libraries (libwhisper.so)?
• What do the audio_transcribe, whisper_native, and whisper_native_lib options do?
• How do I use whisper.cpp as a loadable module in the sniffer?
• If the Whisper feature fails to download the model file, how do I troubleshoot the issue (firewall, proxy, DNS)?