How to Setup an Apache Cassandra Cluster

On By Tony ZampognaIn Devops, Programming, Technology

This article is a guide to setup an Apache Cassandra cluster. The cluster runs on local CentOS virtual machines using Virtualbox. I use this to have a local environment for development and testing.

Prerequisites

It assumes you are using the following “software” versions.

  • MacOS 10.11.3
  • Vagrant 1.8.5
  • Java 1.8.0
  • Apache Cassandra 1.2.2

Here are the steps I used:

  1. First, create a workspace.

    mkdir -p ~/vagrant_boxes/cassandra

    cd ~/vagrant_boxes/cassandra

  2. Next, create a new vagrant box. I’m using a minimal CentOS vagrant box.

    vagrant box add “CentOS 6.5 x86_64” https://github.com/2creatives/vagrant-centos/releases/download/v6.5.3/centos65-x86_64-20140116.box

  3. I need a basic VM to install the packages. This command creates one.

    vagrant init -m “CentOS 6.5 x86_64” cassandra_base

  4. Next, change the Vagrantfile to the following: 

      Vagrant.configure(2) do |config|
    config.vm.box = "CentOS 6.5 x86_64"
    config.vm.box_url = "cassandra_base"
    config.ssh.insert_key = false
  end

  5. Now, install Cassandra and it’s dependencies.

    vagrant up

    vagrant ssh

    sudo yum install java-1.8.0-openjdk-devel

    sudo yum install wget

    wget http://www-us.apache.org/dist/cassandra/3.9/apache-cassandra-3.9-bin.tar.gz ~

    gunzip -c *gz | tar xvf –

  6. Open up your ~/.bash_profile and append the following lines. 

      export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk.x86_64
  export PATH=$PATH:$JAVA_HOME/bin
  export CASSANDRA_HOME=~/apache-cassandra-3.9
  export PATH=$PATH:$CASSANDRA_HOME/bin
  export CASSANDRA_CONF_DIR=$CASSANDRA_HOME/conf

  7. Source the profile.

    source ~/.bash_profile

  8. Create a ~/.ssh/config file to avoid host key checking for SSH. Since these are DEV servers, this is ok. Note that the indentation here before StrictHostKeyChecking must be a tab. 

      Host *
        StrictHostKeyChecking no

  9. Now run these commands to finish the password-less authentication.

    chmod 600 ~/.ssh/config

    ssh-keygen -f ~/.ssh/id_rsa -t rsa -P “”

    cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

  10. In /etc/hosts, add the following lines. 

      192.168.50.41 cassandra1.example.com
  192.168.50.42 cassandra2.example.com
  192.168.50.43 cassandra3.example.com
  192.168.50.44 cassandra4.example.com
  192.168.50.45 cassandra5.example.com

  11. In ~/apache-cassandra-3.9/conf/cassandra.yaml, comment out the line below. This will tell Cassandra to bind to the IP Address and allow our host machine to connect to it.

    rpc_address: localhost

  12. It’s a good idea to install Python 2.7 here, since the CQL shell (cqlsh) command requires that version. The image of CentOS we are using has Python 2.6. Follow the instructions here to update Python.

  13. Exit the SSH session and copy the VM for the other Cassandra nodes.

    exit

    vagrant halt

    vagrant package

    vagrant box add cassandra ~/vagrant_boxes/cassandra/package.box

  14. Edit the Vagrantfile to look like the following below. This will create 5 Cassandra nodes. 

      Vagrant.configure("2") do |config|
    (1..5).each do |i|
      config.vm.define "cassandra#{i}" do |node|
        node.vm.box = "cassandra"
        node.vm.box_url = "cassandra#{i}.example.com"
        node.vm.hostname = "cassandra#{i}.example.com"
        node.vm.network :private_network, ip: "192.168.50.4#{i}"
        node.ssh.insert_key = false

        # Replace the "listen_address" line in the conf file.
        node.vm.provision "shell", inline: "sed -i 's/^cluster_name: .*/cluster_name: \"My Cluster\"/' ~/apache-cassandra-3.9/conf/cassandra.yaml", privileged: false
        node.vm.provision "shell", inline: "sed -i 's/- seeds: .*/- seeds: \"192.168.50.41\"/' ~/apache-cassandra-3.9/conf/cassandra.yaml", privileged: false
        node.vm.provision "shell", inline: "sed -i 's/listen_address: .*/listen_address: \"192.168.50.4#{i}\"/' ~/apache-cassandra-3.9/conf/cassandra.yaml", privileged: false
      end
    end
  end

  15. Bring the new Vagrant VMs up.

    vagrant up –no-provision

    vagrant provision

  16. Start Cassandra. It’s tricky because each node has to start up completely so it can join the cluster. I ssh into each VM individually and start Cassandra and wait about 60 seconds until it has completely started.

    vagrant ssh cassandra1

    ~/apache-cassandra-3.9/bin/cassandra

    # Wait 60 seconds

    exit

    vagrant ssh cassandra2

    ~/apache-cassandra-3.9/bin/cassandra

    # Wait 60 seconds

    exit

    vagrant ssh cassandra3

    ~/apache-cassandra-3.9/bin/cassandra

    # Wait 60 seconds

    exit

    vagrant ssh cassandra4

    ~/apache-cassandra-3.9/bin/cassandra

    # Wait 60 seconds

    exit

    vagrant ssh cassandra5

    ~/apache-cassandra-3.9/bin/cassandra

    # Wait 60 seconds

    exit

To verify Cassandra

  1. Verify that Cassandra is running correctly.

    vagrant ssh cassandra1

    ~/apache-cassandra-3.9/bin/nodetool status
    Datacenter: datacenter1
    =======================
    Status=Up/Down
    |/ State=Normal/Leaving/Joining/Moving
    — Address Load Tokens Owns (effective) Host ID Rack
    UN 192.168.50.41 108.49 KiB 256 38.0% 74b9bb75-e941-4e4c-9900-0b30c7b204b4 rack1
    UN 192.168.50.42 108.2 KiB 256 41.8% 8ee92d98-2a83-491d-b39f-fc3263217dc7 rack1
    UN 192.168.50.43 84.1 KiB 256 41.0% f30a37fd-8763-4f11-a4be-7f413e062bd9 rack1
    UN 192.168.50.44 115.62 KiB 256 38.8% 15356519-30e0-435e-a865-47abfbef5a81 rack1
    UN 192.168.50.45 96.62 KiB 256 40.4% 213e082a-92fd-4948-84be-fe941a0801b1 rack1

Leave a Reply