Setting up a secondary High-Availability Asterisk server

By jbayer - Last updated: Wednesday, April 6, 2011 - Save & Share - 22 Comments

Congratulations!  You have now set up your first Asterisk-based VOIP server.

Next question (probably the first your boss will ask you):  What happens when the server crashes, or loses power, or needs to be upgraded, or…….;  well, you get the idea.  If you only have a single server, you lose your phones until it is rebooted.  Given that you probably installed this on a cheap, inexpensive piece of hardware, these questions are not out of the question, especially for a business.

We had the same questions, and have implemented a failover solution.  I started with a script I got from this web page, but was dissatisfied with it.  It seemed to be a bit of a kludge, in that it wasn’t a true HA solution, but was only a backup solution.

I did a massive reworking of the script, and the result downloadable from here (revised 6/15/11, now version 1.3):

  flip1405.sh (21.2 KiB, 1,827 hits)

This script is used to synchronize the /tftpboot with the /var/ftp/ directory:

  syncftp.sh (1.6 KiB, 0 hits)

The only dependencies for Flip1405.sh are nmap and arping

To use, first get your primary Asterisk server running smoothly.  When ready, you can either clone it using Clonezilla (or some other utility) or just install Asterisk on the second box.  Then follow these steps:

  1. Setup shared-keys so the servers can copy between each other without user intervention. Setup Login without password
  2. Set up ftp (I use vsftp), if needed.  We use it to download the configurations for the Aastra phones.  Usually we would use tftp, but since we have some phones outside our office we simply use ftp for everything.
  3. Install this script into /usr/local/sbin on the primary/active system.
  4. Run the script with the  -s option:
    /usr/local/sbin/flip1405.sh -s
  5. The second script:  syncftp.sh, is used to synchronize the /tftpboot directory with the /var/ftp directory.  If you need it, install the syncftp.sh script into /usr/local/sbin
  6. Open up the flip1405.sh in an editor, and check the various settings.  I would recommend that if you need to change them, to use a config file (/etc/sysconfig/flip1405) rather than editing each file.  The config file is NOT sync’d between the systems, so you  can have system specific options in the file.
  7. There is no need to specify the device, or whether the system is a master or slave.  The script determines this at run time.  The most important lines to set are the first three variables:   MASTERIP, SLAVEIP, and FLOAT.  These are:
    1. MASTERIP     IP address of the master system (arbitrary name)
    2. SLAVEIP        IP address of the slave system
    3. FLOAT          IP address that will float between the Master and Slave, and which your phones will access
  8. All other variables are set for an Elastix system.  Elastix is based on the following:
    Asterisk 1.6
    Centos 5.5
    FreePBX 2.6
  9. Make sure to set the bindaddr to the virtual IP address in /etc/asterisk/sip.conf or
    /etc/asterisk/sip_general_custom.conf 

    bindaddr=192.168.1.12 ;IP of Virtual interface
  10. Add the following line(s) to the root crontab (only install the syncftp.sh line if you are using it):
    * * * * *     /usr/local/sbin/flip1405.sh
    */5 * * * *	/usr/local/sbin/syncftp.sh
  11. If you like, you can add the following lines to /etc/crontab instead:

    * * * * *     root /usr/local/sbin/flip1405.sh
    */5 * * * *	root /usr/local/sbin/syncftp.sh
  12. If you want to automatically rotate the logs, add the following file:

/etc/logrotate.d/flip1405:

/var/log/flip1405.log {
 missingok
 notifempty
 sharedscripts
}

If you have phones outside your network, you will need to configure your firewall to direct the following ports to the FLOAT address:

tcp 21 (if you are using ftp)
tcp 3000
udp 3000
tcp 5060-5082
tcp 10000-20000
tcp 5800-5900
udp 5060-5082
udp 1000-20000
udp 5800-5900

Posted in Administration, VOIP • • Top Of Page
6,645 views

22 Responses to “Setting up a secondary High-Availability Asterisk server”

Comment from ioan
Time June 2, 2011 at 8:57 am

Hi – I have tried to understand the dbBackup code and it’s invocation.

For me it looks like at each execution (like at each minute) there is a dump for all tables from the master MySQL and a “restore” to the slave.

Is not this mechanism too archaic and with. For example in case there is no new data in the master there is nothing to prevent the sync between master and slave.

Am I missing something?

Best regards,
Ioan

Comment from jbayer
Time June 2, 2011 at 12:54 pm

On our production servers, I’ve made it once every 5 minutes.

Yes, this is a little inefficient, but this is supposed to be a simple script, without the headaches of implementing replication. This also allows the slave to take over as master without having to reconfigure the databases.

JBB

Comment from Shane
Time August 17, 2011 at 5:49 am

hi,

when ever i try run this script i get thsi

root@asterisk:/usr/local/sbin# /usr/local/sbin/flip1405.sh
/usr/local/sbin/flip1405.sh: 231: function: not found

usage: /usr/local/sbin/flip1405.sh options

Options:
-i Initialize MySql (create appropriate account for remote access)
-v Verbose (write to log file)
-s copy current file to the other system in the same location
-r force current system to release the virtualIP
-c check config (make sure all commands are available)

root@asterisk:/usr/local/sbin#

do you have any ideas?

Comment from Shane
Time August 17, 2011 at 6:59 am

hmm fixed the issue above but now i try to run the script and it just says this in the log

2011-08-17 23:42
Started as Master
2011-08-17 23:42 Status:
2011-08-17 23:42 FLOAT: 202.89.49.78
2011-08-17 23:42 PrimaryIP:
2011-08-17 23:42 VirtualIP: 1
2011-08-17 23:42 Not up
2011-08-17 23:42 3 Stopping all services

Comment from Shane
Time August 18, 2011 at 4:29 pm

Hey

So i have it working on the master but on the slave it doesnt seem to even check the script for bringing the float on the slave up

this is whats showing in the log

Master: 0
Device: eth0
2011-08-19 09:28
Started as Slave
2011-08-19 09:28 Uptime in minutes: 964
2011-08-19 09:28 check4DupVirtualIP: local: 202.89.49.82 remote: 202.89.49.81 virtual: 202.89.49.83
2011-08-19 09:28 Status:
2011-08-19 09:28 FLOAT: 202.89.49.83
2011-08-19 09:28 PrimaryIP:
2011-08-19 09:28 VirtualIP: 1
2011-08-19 09:28 Not up
2011-08-19 09:28 3 Stopping all services

Comment from jbayer
Time September 6, 2011 at 1:24 pm

Did you try to shut down the master and see if the slave would start?

It’s suspicious that it doesn’t list anything in the PrimaryIP in the log

JBB

Comment from Cullen
Time October 15, 2011 at 10:10 am

Is it possible to assign the master and float address to the same IP?

Comment from jbayer
Time October 19, 2011 at 2:32 pm

No. The whole idea is that the float address floats between the two systems. It can’t do that if it is the same as the master ip.

Comment from James
Time October 17, 2011 at 7:45 am

Hi,

I’d like to try this script on my Asterisk 1.8 server (with freepbx 2.9) but I’m stuck.

When typing /usr/local/sbin/flip1405.sh -s
I get an error saying: /bin/sh^M: bad interpreter: Nosuch file or directory

Any suggestions on this?

Thanks in advance,

James

Comment from jbayer
Time October 19, 2011 at 2:33 pm

How did you download it? If you downloaded it to a windows box, it might have gotten the dos line ending put in (cr, nl) instead of the nl.

Comment from James
Time October 17, 2011 at 9:59 am

Solved my issue by doing:

cp flip1405.sh flip1405.sh.dos &&
cat flip1405.dos | tr -d ‘\r’ > flip1405.sh

Comment from jbayer
Time October 19, 2011 at 2:33 pm

Glad you were able to get it to work. Let me know how it works for you.

Comment from Chris
Time November 25, 2011 at 5:30 pm

Your script is great. I am new to Asterisk and Linux in general so I am stumbling through everything. I do have one question.

We are Device and User mode. What happens to the Logged in Users if the Primary server fails and the Secondary server automatically goes online? Will the users have to login to their phones again to make and receive phone calls?

Comment from jbayer
Time November 29, 2011 at 8:28 am

I don’t think so. Everything is stored in the database, and the database is copied from the master to the slave pretty frequently.

I’m not using the script right now since I left the company where I wrote it, so I can’t easily test this.

Comment from Chris
Time January 24, 2012 at 10:00 pm

The HA Backup feature in FreePBX works very well, but you cannot use it when Asterisk is not running. If this script is put in place you cannot run an HA Backup from your Primary server and have it restored to your secondardy server automatically, as Asterisk is disabled on the hot spare. I am wondering if this script does the same things as the HA backup feature in FreePBX.

Comment from jotten
Time November 21, 2012 at 2:48 pm

Although I got this working it was too simplistic (different types of failures caused inoperative Asterisk to stay up, etc). I’m using HAAST (www.generationd.com) now. We have it running on 6 pairs of Ast servers. If you give up on the home brew approach too, try it out.

Comment from jbayer
Time November 21, 2012 at 4:06 pm

Not using this anymore, was for a previous job where it worked well.

Comment from Isuru
Time June 1, 2014 at 10:59 pm

We are operating a telco switch MSC which connects to two Asterisx server on two different IP domains. (connected via two routers in middle. We are planning on setting up High Availability between the two servers. In MSC we can give one server as active and the other server as the failover, where MSC switch between servers in case of any reachability issue. But still we want to replicate asterisk database and configuration files real time. Can you give us an idea how we could proceed.

Comment from jbayer
Time June 10, 2014 at 7:16 am

Real-time is a bit complicated, and I’m not an Asterisk expert. If you aren’t comfortable doing sysadmin work, I’d suggest finding a better expert.

That being said, I’d do the following:

Use the iNotify mechanisms to copy over any config files which get changed in the /etc/asterisk directory. You should also find any other directories where files get saved/changed and do the same thing.

Use Mysql as a database, and set up replication between the master and the slave. That will keep the databases in sync.

If you do failover, then you will have the problem of switching back. I’d suggest that you simply use the new master as the master and reconfigure everything to reverse the flow of data files. This a significantly complicated, and would be the subject of significant amount of work.

Comment from Nan Ling
Time November 14, 2015 at 6:59 pm

We have HAAst running with Asterisk realtime. HAAst synchronize all configuration, diaplan, etc. HAAst is from http://www.telium.ca (not generationd.com). It work perfect for over 2 years.

Comment from Jason Dupont
Time June 14, 2016 at 7:12 pm

I tried using iNotify to keep directories in sync, but when one server fails and corrupts files, they get copied to the other server. Not workable! (I also tried DRBD, NFS share, etc). 2 years later and no one has a working open source solution. For now i’m using the free edition of HAAst (www.telium.ca) and that works. It also solves the problem of only syncing database contents if the source peer for the copy is healthy (otherwise Asterisk realtime breaks)

Comment from jdawson
Time January 12, 2017 at 10:10 am

Way to complicated, and doesn’t handle so many use cases. I would suggest using the free version of HAAst. One program to install and it does all sensing/syncing/heartbeat/failover/etc. (www.telium.ca) This solution only works in the most simplistic failure scenarios

Write a comment

asd