HOWTO: Two Server Linux Cluster

This covers how to set up two linux servers to serve up identical content. This is ideal if the two servers are behind a loadbalancer or using Round Robin DNS.

At the end of this HOWTO you will end up with the following:

Apache 2
PHP 4
MySQL 4.1
Awstats
A secure box

This HOWTO however assumes that you are using another server as a DNS server or are using a third party DNS server such as zoneedit.

The two servers will be totally redundent - i.e. either server could die and then come back up and it would join the cluster again.

Please Note that this is for a "plain" redhat server, not one with a control panel on it.

STEP 1: Preparation

Firstly, prepare yourself for a struggle. Some things are bound to go wrong and you will have to either be prepared to pay and contact me to complete it or spend the time troubleshooting yourself. (if you do decide it is too much I would be delighted to set a cluster like this up for less than you think!)

Secondly, download a copy of putty (save the file to disk, I suggest your desktop). This allows you to connect to your servers.

Thirdly, go to whatever server you are using for your DNS (I recommend zoneedit.com) and add the following DNS entries (A records):

srv01.yourdomain.com	192.168.0.1
srv02.yourdomain.com	192.168.0.1
cluster.yourdomain.com	192.168.0.3

(You need to get these IPs from your hosting provider).

STEP 2: Initial Setup

Now you need to log into your server. Firstly, open the putty.exe file you downloaded above (I suggested you put it on your desktop so its easy to get to). Then,

Enter your first server's IP address in the Hostname/IP address box at the top
Select "SSH" from the box below that
Enter a name in the "Saved Sessions" Box (I suggest "srv01")
Click Save
Repeat for second server (must have a different name).

STEP 3: Setup both servers

We are now going to do the things that you have to do to BOTH servers. Complete this whole section for srv01, then go back and do it all over again on srv02.

Hostname

Firstly, we are going to set the hostname. Thanks to NightHawk for the script:

su -
wget http://ezsm.net/hostname.sh
sh hostname.sh

And answer the questions.

Apache Install

We are now going to install Apache 2 from source. If you already have an apache rpm, remove it (rpm -e httpd). You will need to remove all the things that depend on it as well. You can also just stop it.

wget http://www.mirror.ac.uk/mirror/ftp.apache.org/httpd/httpd-2.0.52.tar.gz
mv httpd-2.0.52 /usr/src
cd /usr/src
tar -zxvf httpd-2.0.52
rm -f httpd-2.0.52.tar.gz
cd httpd-2.0.52
./configure --prefix=/wwwroot --enable-so --enable-rewrite --enable-headers --enable-expires
	--enable-deflate --with-z --enable-speling --enable-ssl
make
make install

Now, install SSL for self-signed certificated:

cd /wwwroot/certs/
openssl genrsa -des3 -out hostname.key 1024
openssl req -new -key hostname.key -out hostname.csr
openssl x509 -req -days 30 -in hostname.csr -signkey hostname.key -out hostname.crt

Then in /wwwroot/conf/httpd.conf, REMOVE any Listen lines and add these lines:

Listen IP:80
Listen IP:443
# SSL

        SSLEngine on
        SSLCertificateFile /wwwroot/certs/hostname.crt
        SSLCertificateKeyFile /wwwroot/certs/hostname.key

Now, you need to add the apache bin to your $PATH variable:

cd
vi .bash_profile

Look for the line that starts with PATH= and add :/wwwroot/bin to the end, so it would look something like this:

PATH=$PATH:$HOME/bin:/wwwroot/bin

You should now exit and re-login so the new $PATH variable takes effect:

exit

Now, log in again (putty).

Install MySQL

In order to set up the mysql cluster you must get the mysql-max distribution. At time of writing latest version is 4.1.8. The best way to download it is to lynx the final download page (for various reasons wget does not work):

cd /usr/src/
wget http://dev.mysql.com/get/Downloads/MySQL-4.1/mysql-max-4.1.8-pc-linux-i686.tar.gz/
	from/http://mysql.binarycompass.org/
groupadd mysql
useradd -g mysql mysql
tar -zxvf mysql-max-4.1.8-pc-linux-i686.tar.gz
rm mysql-max-4.1.8-pc-linux-i686.tar.gz
ln -s mysql-max-4.1.8-pc-linux-i686 mysql
cd mysql
scripts/mysql_install_db --user=mysql
chown -R root  .
chown -R mysql data
chgrp -R mysql .
cp support-files/mysql.server /etc/rc.d/init.d/
chmod +x /etc/rc.d/init.d/mysql.server
chkconfig --add mysql.server

Do not start mysql yet.

Install PHP

Wget wget http://uk.php.net/get/php-4.3.10.tar.gz/from/us3.php.net/mirror
tar -zxvf php-4.3.10.tar.gz
rm -f php-4.3.10.tar.gz
cd php-4.3.10
./configure --prefix=/wwwroot/php --with-apxs2=/wwwroot/bin/apxs --with-config-file-path=/wwwroot/php --with-mysql --enable-gd --with-mysqli=/usr/local/mysql/bin/mysql_config
make
apachectl stop
make install
apachectl start

Install APF Firewall

This will install APF:

wget http://www.rfxnetworks.com/downloads/apf-current.tar.gz
tar -xvzf apf-current.tar.gz
cd apf*
./install.sh
vi /etc/apf/conf.apf

Change the following lines: (press i to insert, :wq then enter to save and exit after you have finished changing)

IG_TCP_CPORTS="22,80,1186,2200"

Then save and exit. Start APF like this:

service apf start

Press enter a few times. If you are not disconnected then it works, so re-edit it and change DEVM="1" to DEVM="0". Restart apf (service apf restart). Make apf start at boot:

chkconfig --level 2345 apf on

Install Chkrootkit

This installs a script that searches for rootkits (scripts hackers leave to allow them to get in):

cd /usr/src/
wget ftp://ftp.pangeia.com.br/pub/seg/pac/chkrootkit.tar.gz
tar xvzf chkrootkit.tar.gz
rm chkrootkit.tar.gz
mv chkrootkit* chkrootkit
cd chkrootkit
make sense
./chkrootkit

Everything should be 'not found' or 'not infected'. Now, make it run every night and email you the report:

crontab -e

Insert this into the crontab (press i, then :wq to save as usual. Remember to change the email to your email address.):

0 1 * * * (cd /usr/local/chkrootkit; ./chkrootkit 2>&1 | mail -s "chkrootkit output" youremail@yourdomain.tld)

Install Tripwire

Install this to detect hackers as they attempt to change files:

rpm -Uhv ftp://194.199.20.114/linux/redhat/9/en/os/i386/RedHat/RPMS/tripwire-2.3.1-17.i386.rpm

/etc/tripwire/twinstall.sh

Enter a passphrase (this should be different to your root password). You will have to enter this 4 times (I use the same one, you can use different site and local passwords). Now we need a script to clean out the policy file:

vi cleantw.pl

Insert this into the file:(Credit dafonso and siliconjunkie here)

# cat > ./cleantw.pl
#!/usr/bin/perl

while (<> ){
#look at the line, and check for a line that can be
    # Construed as a file name
  CASE:{
      ( m|(^\s*)(/[/\w\.\_\-]+)(\s+->.*)| ) and do {
          print $1;
          print "#" unless (-e $2);
          print "$2$3\n";
          last;
      };

      print $_;
  }
};
#

Insert the follwing in cleanedpol.txt just below "rulename = "Invariant Directories"", you will see some similar entries (I am going to stop giving every command necessary - by now you should have worked out how to use vi to edit files. If you still have not there is a tutorial here.

/var/run -> $(SEC_INVARIANT)
/var/log -> $(SEC_INVARIANT)
/var/tty[1-8] -> $(SEC_INVARIANT)

Now, start tripwire:

/usr/sbin/twadmin -m P cleanedpol.txt
/usr/sbin/tripwire --init

Mount /tmp with noexec

I see no nead to replicate This thread here. It is very good - just follow it.

Only allow root to use some binaries

A lot of recent exploits have used the following binaries. By only allowing root to execute them you make yourself less at risk. Make sure there are no more copies of these binaries by running a "Locate binaryname" but if they are all in /usr/bin run these commands:

chmod 750 /usr/bin/rcp
chmod 750 /usr/bin/wget
chmod 750 /usr/bin/lynx
chmod 750 /usr/bin/links
chmod 750 /usr/bin/scp

To be continued. . .

Looking for exceptionally high quality, low cost and satisfaction guaranteed systems administration?
Contact Me for details.

See more HOWTOs like this