Disaster Recovery: Difference between revisions
(Add disaster recovery guide for disk failure scenarios) |
(Rewrite: fixed syntax errors, improved structure, added workflow diagram, consolidated content) |
||
| Line 1: | Line 1: | ||
{{DISPLAYTITLE:Disaster Recovery}} | {{DISPLAYTITLE:Disaster Recovery}} | ||
''' | '''Recovery procedure after disk failure requiring OS and VoIPmonitor reinstallation.''' | ||
== Overview == | == Overview == | ||
When a disk fails and requires replacement, | When a disk fails and requires replacement, recovery involves: | ||
# '''Customer:''' Replace hardware, install OS, configure network, provide remote access | |||
# '''Support:''' Install VoIPmonitor, restore configuration, verify operation | |||
{{Note|Self-service installation is possible using [[Sniffer_installation|Sniffer Installation]] and [[GUI_installation|GUI Installation]] guides if you have VoIPmonitor expertise.}} | |||
<kroki lang="mermaid"> | |||
%%{init: {'flowchart': {'nodeSpacing': 15, 'rankSpacing': 25}}}%% | |||
flowchart LR | |||
A[Disk Failure] --> B[Replace Hardware] | |||
B --> C[Install OS] | |||
C --> D[Configure Network] | |||
D --> E[Set Timezone] | |||
E --> F[Provide Remote Access] | |||
F --> G[Support Installs VoIPmonitor] | |||
G --> H{Backups Available?} | |||
H -->|Yes| I[Restore Data] | |||
H -->|No| J[Start Fresh] | |||
I --> K[Verify Services] | |||
J --> K | |||
</kroki> | |||
== Step 1: Hardware & OS == | |||
Replace failed disks and install a supported OS: | |||
{| class="wikitable" | {| class="wikitable" | ||
|- | |- | ||
! | ! OS !! Status | ||
|- | |- | ||
| Ubuntu 24.04 LTS || '''Recommended''' | |||
| | |||
|- | |- | ||
| Debian 12/13 || Supported | |||
| | |- | ||
| Rocky Linux 9/10 || Supported | |||
|- | |||
| CentOS 7 || EOL - migrate to Rocky/Alma | |||
|} | |} | ||
{{Warning|1=Do not copy binaries from old disk if OS version differs. Always install VoIPmonitor fresh using the official installer.}} | |||
== Step 2: Network Configuration == | |||
Configure network interfaces to match original server: | |||
* Same IP address (for fixed configurations) | |||
* DNS servers and routing | |||
* SPAN/TAP port connectivity (for sniffer servers) | |||
== Step 3: Timezone Configuration == | |||
{{Warning|1='''CRITICAL:''' PCAP paths use local time (<code>/var/spool/voipmonitor/YYYY/MM/DD</code>). Timezone mismatch prevents GUI from finding historical recordings.}} | |||
{| | |||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> | ||
| Line 73: | Line 62: | ||
timedatectl | timedatectl | ||
# Set timezone ( | # Set timezone (must match original server) | ||
sudo timedatectl set-timezone Europe/Prague | sudo timedatectl set-timezone Europe/Prague | ||
# Verify | # Verify | ||
timedatectl | timedatectl | ||
</syntaxhighlight> | </syntaxhighlight> | ||
== Step 4: Provide Remote Access == | |||
Choose one method for VoIPmonitor support staff: | |||
=== Method A: Direct SSH === | |||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> | ||
# Create support user (optional) | # Create support user (optional but recommended) | ||
adduser supportuser | adduser supportuser | ||
usermod -aG sudo supportuser | usermod -aG sudo supportuser | ||
# | # Verify SSH is running | ||
systemctl status sshd | systemctl status sshd | ||
</syntaxhighlight> | </syntaxhighlight> | ||
=== Method B: VPN Access === | |||
Provide VPN credentials and server internal IP. | |||
=== Method C: Reverse SSH Tunnel === | |||
For servers behind firewalls without direct SSH access: | |||
<syntaxhighlight lang="bash"> | |||
# Run on your internal server (connects to your bastion host) | |||
# | |||
ssh -R 2222:localhost:22 -N support@bastion.yourdomain.com | ssh -R 2222:localhost:22 -N support@bastion.yourdomain.com | ||
# | # Support connects via: ssh -p 2222 user@bastion.yourdomain.com | ||
</syntaxhighlight> | |||
== Step 5: VoIPmonitor Installation == | |||
Support staff will install and configure based on deployment type: | |||
{| class="wikitable" | |||
|- | |||
! Deployment Type !! Tasks | |||
|- | |||
| '''Sniffer-Only''' || Install binary, configure interface, connect to central server, restore <code>voipmonitor.conf</code> | |||
|- | |||
| '''GUI/Database''' || Install MariaDB/MySQL, GUI, PHP dependencies, restore configuration | |||
|- | |||
| '''All-in-One''' || All of the above, reconstruct database from PCAPs if needed | |||
|} | |||
== | == Step 6: Data Recovery == | ||
1 | === Case 1: Database Backup Available === | ||
= | <syntaxhighlight lang="bash"> | ||
# Restore from mysqldump | |||
gunzip < backup.sql.gz | mysql -u root -p voipmonitor | |||
</syntaxhighlight> | |||
For GUI configuration restore, see [[Backing_Up_GUI_Configuration#Restore|Backup & Restore Guide]]. | |||
=== | === Case 2: PCAP Files Survived === | ||
If PCAPs can be recovered from separate storage or mounted old disk: | |||
<syntaxhighlight lang="bash"> | |||
# 1. Restore PCAP files to spooldir | |||
# 1. Restore PCAP files to | |||
rsync -avz /mnt/old_disk/var/spool/voipmonitor/ /var/spool/voipmonitor/ | rsync -avz /mnt/old_disk/var/spool/voipmonitor/ /var/spool/voipmonitor/ | ||
# 2. | # 2. Create fresh database | ||
mysql -e "CREATE DATABASE voipmonitor;" | mysql -e "CREATE DATABASE voipmonitor;" | ||
# 3. Restore GUI configuration first (if available) | # 3. Restore GUI configuration first (if available) | ||
# | # See: Backing_Up_GUI_Configuration | ||
# 4. | # 4. Reconstruct database from PCAPs | ||
voipmonitor --config-file=/etc/voipmonitor.conf --readpcapdir /var/spool/voipmonitor | voipmonitor --config-file=/etc/voipmonitor.conf --readpcapdir /var/spool/voipmonitor | ||
</syntaxhighlight> | |||
{{Note|1=Processing | {{Note|1=Processing PCAPs in restore mode can take '''significant time''' depending on data volume.}} | ||
=== Case 3: No Backups === | |||
* Historical data is '''permanently lost''' | |||
* | |||
* System must be reconfigured from scratch | * System must be reconfigured from scratch | ||
* | * Implement backup strategy (see Prevention below) | ||
== Step 7: Verification == | |||
<syntaxhighlight lang="bash"> | |||
# Check service status | |||
# Check | |||
systemctl status voipmonitor | systemctl status voipmonitor | ||
# View logs | |||
# View | |||
journalctl -u voipmonitor -f | journalctl -u voipmonitor -f | ||
# For GUI server | |||
systemctl status apache2 # or nginx | |||
</syntaxhighlight> | |||
Verify in GUI: | |||
* Sensors connected (distributed mode) | |||
* New calls being captured | |||
* Historical recordings accessible (if restored) | |||
== Prevention == | |||
Implement these measures to minimize future data loss: | |||
{ | {| class="wikitable" | ||
|- | |- | ||
! !! | ! Measure !! Description !! Reference | ||
|- | |- | ||
| '''GUI Backups''' || Daily automated backup of configuration, users, alerts || [[Backing_Up_GUI_Configuration]] | |||
| ''' | |||
| | |||
|- | |- | ||
| '''Separate PCAP Storage''' || Move <code>spooldir</code> to NAS/SAN/separate disk || [[Data_Cleaning#Moving_Spooldir|Data Cleaning]] | |||
| ''' | |||
| | |||
|- | |- | ||
| '''Database Replication''' || Master-master or dual-write for HA || [[Redundant_database]] | |||
| ''' | |||
| | |||
| | |||
| | |||
| | |||
|- | |- | ||
| '''Database Backups''' || Regular mysqldump to off-site storage || See below | |||
| ''' | |||
| | |||
|- | |- | ||
| '''Disk Health Monitoring''' || SMART monitoring for early warning || See below | |||
| ''' | |||
| | |||
|} | |} | ||
=== Database Backup Cron === | |||
<syntaxhighlight lang="bash"> | |||
2 | # /etc/cron.d/voipmonitor-backup | ||
0 2 * * * root mysqldump -u root -pPASSWORD voipmonitor | gzip > /backup/db_$(date +\%Y\%m\%d).sql.gz | |||
</syntaxhighlight> | |||
== | === Disk Health Monitoring === | ||
<syntaxhighlight lang="bash"> | |||
# Install smartmontools | |||
apt-get install smartmontools | |||
# Check disk SMART status | # Check disk SMART status | ||
smartctl -a /dev/sda | smartctl -a /dev/sda | ||
</syntaxhighlight> | |||
== See Also == | |||
== | |||
* [[Sniffer_installation|Sniffer Installation]] | |||
| | * [[GUI_installation|GUI Installation]] | ||
* [[Backing_Up_GUI_Configuration|Backup & Restore]] | |||
* [[Redundant_database|Database Redundancy]] | |||
* [[Recovering_corrupted_database_tables|Database Recovery]] | |||
| | |||
| | |||
== AI Summary for RAG == | == AI Summary for RAG == | ||
'''Summary:''' Disaster recovery | '''Summary:''' Disaster recovery procedure after VoIPmonitor server disk failure. Workflow: (1) Replace hardware; (2) Install supported OS (Ubuntu 24.04, Debian 12/13, Rocky Linux 9/10 recommended); (3) Configure network to match original; (4) Set timezone - CRITICAL because PCAP paths use local time format <code>/var/spool/voipmonitor/YYYY/MM/DD</code>; (5) Provide remote access to support (SSH, VPN, or reverse tunnel); (6) Support installs VoIPmonitor. Data recovery options: database backup restore via mysqldump, PCAP reconstruction using <code>voipmonitor --readpcapdir /var/spool/voipmonitor</code>, or start fresh if no backups. Prevention: automated GUI backups, separate PCAP storage on NAS/SAN, database replication, regular mysqldump, SMART disk monitoring. | ||
'''Keywords:''' disaster recovery, disk failure, data recovery, OS reinstall | '''Keywords:''' disaster recovery, disk failure, data recovery, OS reinstall, timezone configuration, PCAP restoration, database reconstruction, readpcapdir, spooldir, remote access, SSH tunnel, reverse tunnel, VPN, mysqldump, smartctl, backup strategy, NAS, SAN, high availability | ||
'''Key Questions:''' | '''Key Questions:''' | ||
* What do I do after a disk failure on my VoIPmonitor server? | * What do I do after a disk failure on my VoIPmonitor server? | ||
* How do I recover VoIPmonitor after | * How do I recover VoIPmonitor after complete disk failure? | ||
* | * Why is timezone critical after disk failure recovery? | ||
* | * How do I provide remote access to VoIPmonitor support? | ||
* How do I reconstruct VoIPmonitor database from PCAP files? | |||
* Can I recover PCAP files if the database is lost? | |||
* What backups should I have for disaster recovery? | |||
* How do I reconstruct | * How do I set up a reverse SSH tunnel for support access? | ||
* | |||
* What backups should I have | |||
* | |||
Latest revision as of 16:49, 8 January 2026
Recovery procedure after disk failure requiring OS and VoIPmonitor reinstallation.
Overview
When a disk fails and requires replacement, recovery involves:
- Customer: Replace hardware, install OS, configure network, provide remote access
- Support: Install VoIPmonitor, restore configuration, verify operation
ℹ️ Note: Self-service installation is possible using Sniffer Installation and GUI Installation guides if you have VoIPmonitor expertise.
Step 1: Hardware & OS
Replace failed disks and install a supported OS:
| OS | Status |
|---|---|
| Ubuntu 24.04 LTS | Recommended |
| Debian 12/13 | Supported |
| Rocky Linux 9/10 | Supported |
| CentOS 7 | EOL - migrate to Rocky/Alma |
⚠️ Warning: Do not copy binaries from old disk if OS version differs. Always install VoIPmonitor fresh using the official installer.
Step 2: Network Configuration
Configure network interfaces to match original server:
- Same IP address (for fixed configurations)
- DNS servers and routing
- SPAN/TAP port connectivity (for sniffer servers)
Step 3: Timezone Configuration
⚠️ Warning: CRITICAL: PCAP paths use local time (/var/spool/voipmonitor/YYYY/MM/DD). Timezone mismatch prevents GUI from finding historical recordings.
# Check current timezone
timedatectl
# Set timezone (must match original server)
sudo timedatectl set-timezone Europe/Prague
# Verify
timedatectl
Step 4: Provide Remote Access
Choose one method for VoIPmonitor support staff:
Method A: Direct SSH
# Create support user (optional but recommended)
adduser supportuser
usermod -aG sudo supportuser
# Verify SSH is running
systemctl status sshd
Method B: VPN Access
Provide VPN credentials and server internal IP.
Method C: Reverse SSH Tunnel
For servers behind firewalls without direct SSH access:
# Run on your internal server (connects to your bastion host)
ssh -R 2222:localhost:22 -N support@bastion.yourdomain.com
# Support connects via: ssh -p 2222 user@bastion.yourdomain.com
Step 5: VoIPmonitor Installation
Support staff will install and configure based on deployment type:
| Deployment Type | Tasks |
|---|---|
| Sniffer-Only | Install binary, configure interface, connect to central server, restore voipmonitor.conf
|
| GUI/Database | Install MariaDB/MySQL, GUI, PHP dependencies, restore configuration |
| All-in-One | All of the above, reconstruct database from PCAPs if needed |
Step 6: Data Recovery
Case 1: Database Backup Available
# Restore from mysqldump
gunzip < backup.sql.gz | mysql -u root -p voipmonitor
For GUI configuration restore, see Backup & Restore Guide.
Case 2: PCAP Files Survived
If PCAPs can be recovered from separate storage or mounted old disk:
# 1. Restore PCAP files to spooldir
rsync -avz /mnt/old_disk/var/spool/voipmonitor/ /var/spool/voipmonitor/
# 2. Create fresh database
mysql -e "CREATE DATABASE voipmonitor;"
# 3. Restore GUI configuration first (if available)
# See: Backing_Up_GUI_Configuration
# 4. Reconstruct database from PCAPs
voipmonitor --config-file=/etc/voipmonitor.conf --readpcapdir /var/spool/voipmonitor
ℹ️ Note: Processing PCAPs in restore mode can take significant time depending on data volume.
Case 3: No Backups
- Historical data is permanently lost
- System must be reconfigured from scratch
- Implement backup strategy (see Prevention below)
Step 7: Verification
# Check service status
systemctl status voipmonitor
# View logs
journalctl -u voipmonitor -f
# For GUI server
systemctl status apache2 # or nginx
Verify in GUI:
- Sensors connected (distributed mode)
- New calls being captured
- Historical recordings accessible (if restored)
Prevention
Implement these measures to minimize future data loss:
| Measure | Description | Reference |
|---|---|---|
| GUI Backups | Daily automated backup of configuration, users, alerts | Backing_Up_GUI_Configuration |
| Separate PCAP Storage | Move spooldir to NAS/SAN/separate disk |
Data Cleaning |
| Database Replication | Master-master or dual-write for HA | Redundant_database |
| Database Backups | Regular mysqldump to off-site storage | See below |
| Disk Health Monitoring | SMART monitoring for early warning | See below |
Database Backup Cron
# /etc/cron.d/voipmonitor-backup
0 2 * * * root mysqldump -u root -pPASSWORD voipmonitor | gzip > /backup/db_$(date +\%Y\%m\%d).sql.gz
Disk Health Monitoring
# Install smartmontools
apt-get install smartmontools
# Check disk SMART status
smartctl -a /dev/sda
See Also
AI Summary for RAG
Summary: Disaster recovery procedure after VoIPmonitor server disk failure. Workflow: (1) Replace hardware; (2) Install supported OS (Ubuntu 24.04, Debian 12/13, Rocky Linux 9/10 recommended); (3) Configure network to match original; (4) Set timezone - CRITICAL because PCAP paths use local time format /var/spool/voipmonitor/YYYY/MM/DD; (5) Provide remote access to support (SSH, VPN, or reverse tunnel); (6) Support installs VoIPmonitor. Data recovery options: database backup restore via mysqldump, PCAP reconstruction using voipmonitor --readpcapdir /var/spool/voipmonitor, or start fresh if no backups. Prevention: automated GUI backups, separate PCAP storage on NAS/SAN, database replication, regular mysqldump, SMART disk monitoring.
Keywords: disaster recovery, disk failure, data recovery, OS reinstall, timezone configuration, PCAP restoration, database reconstruction, readpcapdir, spooldir, remote access, SSH tunnel, reverse tunnel, VPN, mysqldump, smartctl, backup strategy, NAS, SAN, high availability
Key Questions:
- What do I do after a disk failure on my VoIPmonitor server?
- How do I recover VoIPmonitor after complete disk failure?
- Why is timezone critical after disk failure recovery?
- How do I provide remote access to VoIPmonitor support?
- How do I reconstruct VoIPmonitor database from PCAP files?
- Can I recover PCAP files if the database is lost?
- What backups should I have for disaster recovery?
- How do I set up a reverse SSH tunnel for support access?