Friday, April 18, 2014

Riak NoSQL database on Mac OS-X 10.9

   
In world lot of databases available in nowadays, Riak is most power full opensource database from NoSQL database category.In web based application I have tried to implement Riak database. It is hard to install and configure  at first time in my linux system, because that time I am new to this DB. Riak is written in Erlang, known for its ability to distribute data across nodes using consistent hashing in a simple key/value scheme in namespaces called buckets.


Main Advantages of Riak


Fault-Tolerant Availability
    Riak replicates key/value stores across a cluster of nodes with a default n_val of three. In the case of node outages due to network partition or hardware failures, data can still be written to a neighboring node beyond the initial three, and read-back due to its "masterless" peer-to-peer architecture.

Queries

    Riak supports a REST API through HTTP and Protocol Buffers for basic PUT, GET, POST, and DELETE functions. Additional query choices are offered including secondary indices, Riak Search leveraging the Apache Solr Engine with Solr client query APIs and MapReduce. MapReduce has native support for both JavaScript (using the SpiderMonkey runtime) and Erlang.

Predictable Latency

    Riak evenly distributes data across nodes with consistent hashing and can provide an excellent latency profile, even in the case of multiple node failures. Key/Values can be stored in memory, disk, or a combination depending on which pluggable backend one chooses.

Multi-Datacenter Replication

    In multi-datacenter replication, one cluster acts as a "primary cluster". The primary cluster handles replication requests from one or more "secondary clusters" (generally located in other regions or countries). If the datacenter with the primary cluster goes down, a secondary cluster can take over as the primary cluster.
    There are two primary modes of operation: fullsync and realtime. In fullsync mode, a complete synchronization occurs between primary and secondary cluster(s), by default every 360 minutes. In realtime mode, continual, incremental synchronization occurs - replication is triggered by new updates.


Install Riak from Source


    Let's install Riak on Mac OS-X 10.9 Maverick and build a five-node cluster running on your local machine.
 Instructions for downloading and installing Riak from source.

Impotent thing is, You must have Xcode tools installed from Apple's Developer website


Riak requires Erlang R15B01. Installation of Erlang R15B01 from source..


First download and unpack the source:


curl -O http://erlang.org/download/otp_src_R15B01.tar.gz

tar zxvf otp_src_R15B01.tar.gz
cd otp_src_R15B01

Then configure Elang using LLVM (the default):


 CFLAGS=-O0 ./configure --disable-hipe --enable-smp-support --enable-threads \

--enable-kernel-poll --enable-darwin-64bit

Now build and install, you will prompted for your sudo password.


make && sudo make install


Download and unpack Riak-1.4.8


After installing Erlang, We are going to download and unpack the source distribution of Riak-1.4.8.


wget http://s3.amazonaws.com/downloads.basho.com/riak/1.4/1.4.8/riak-1.4.8.tar.gz

tar zxvf riak-1.4.8.tar.gz
cd riak-1.4.8 
make all

Setup and startup Five nodes


After successful built,  we are going to setup five self-contained nodes on Riak. 


First, make five nodes:


make devrel DEVNODES=5


You have just generated a dev directory. Let's go into that directory to check out its these directories dev1, dev2, dev3, dev4:


cd dev; ls


Each directory starting with dev is a complete, self-contained package containing a Riak node. We need to start all nodes,  you can use a for loop to iterate through and start the available nodes::


 for node in `ls`; do $node/bin/riak start; done



--ulimit warning--
At this point you may receive a warning message to increase the number of open file handles (ulimit). 

To check the current limits:


launchctl limit maxfiles


 To adjust the maximum open file limits,   edit(create) /etc/launchd.conf (using any editor with sudo previllage)and increase the limits for soft and hard as appropriate. add the lines as follows:


limit maxfiles 16384 32768


Save the file, and restart the system for the new limits to take effect. After restarting,

verify current limits:

launchctl limit


    cpu         unlimited      unlimited

    filesize    unlimited      unlimited
    data        unlimited      unlimited
    stack       8388608        67104768
    core        0              unlimited
    rss         unlimited      unlimited
    memlock     unlimited      unlimited
    maxproc     709            1064
    maxfiles    16384          32768

After that start the all five nodes again using above for loop command and check running nodes, to do this run : 


ps aux | grep beam


This should give you granular details on the five running Riak nodes.  If you'd like to simply check which nodes are running and which are not, you can run the riak ping command on for loop:


 for node in `ls`; do $node/bin/riak ping; done


If the response is PONG, then the node is up and running. Otherwise, the node is currently stopped


Cluster joining


The next step is to join these five nodes together to form a cluster. You can do this using the Riak Admin tool. Specifically, what we want to do is join dev2,dev3, dev4, and dev5 to dev1: , to do this run:


 for n in {2..5}; do dev$n/bin/riak-admin cluster join dev1@127.0.0.1; done


To make the above joins take effect, you first must review the plan:


 dev1/bin/riak-admin cluster plan


The plan will print out a synopsis of what it plans to do, and how the cluster will look after this is completed.


 =============================== Staged Changes ================================

Action         Nodes(s)
-------------------------------------------------------------------------------
join           'dev2@127.0.0.1'
join           'dev3@127.0.0.1'
join           'dev4@127.0.0.1'
join           'dev5@127.0.0.1'
-------------------------------------------------------------------------------


NOTE: Applying these changes will result in 1 cluster transition


###############################################################################

                         After cluster transition 1/1
###############################################################################

================================= Membership ==================================

Status     Ring    Pending    Node
-------------------------------------------------------------------------------
valid     100.0%     20.3%    'dev1@127.0.0.1'
valid       0.0%     20.3%    'dev2@127.0.0.1'
valid       0.0%     20.3%    'dev3@127.0.0.1'
valid       0.0%     20.3%    'dev4@127.0.0.1'
valid       0.0%     18.8%    'dev5@127.0.0.1'
-------------------------------------------------------------------------------
Valid:5 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

Transfers resulting from cluster changes: 51

  12 transfers from 'dev1@127.0.0.1' to 'dev5@127.0.0.1'
  13 transfers from 'dev1@127.0.0.1' to 'dev4@127.0.0.1'
  13 transfers from 'dev1@127.0.0.1' to 'dev3@127.0.0.1'
  13 transfers from 'dev1@127.0.0.1' to 'dev2@127.0.0.1'


Finally, you can commit the batch:


 dev2/bin/riak-admin cluster commit


Test the cluster 


Now we now a have a running five-node Riak cluster. Let's make sure it's working properly. For this we have a couple of options. A simple option is to run the member-status command on one of our nodes:


dev1/bin/riak-admin member-status


This will give us a high-level view of our cluster and tell us the percentage of the ring that each node manages:


 ================================= Membership ==================================

Status     Ring    Pending    Node
-------------------------------------------------------------------------------
valid      20.3%      --      'dev1@127.0.0.1'
valid      20.3%      --      'dev2@127.0.0.1'
valid      20.3%      --      'dev3@127.0.0.1'
valid      20.3%      --      'dev4@127.0.0.1'
valid      18.8%      --      'dev5@127.0.0.1'
-------------------------------------------------------------------------------
Valid:5 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

You should now have a five-node Riak cluster up and running. 




Shortcut For running Riak node


I have written two shell script to ease for running Riak nodes.It is very ease to use in every day.


First, create riak_run.sh using text editor and add following lines and save:


#!/bin/bash

#script for initiating riak database.

echo "Initiating Riak DB..."


cd riak-1.3.2/dev/


for node in `ls`; do $node/bin/riak start; done


echo "Riak DB initiated."


echo "-----------------"


echo "Checking Riak....."

ps aux | grep beam


echo "Riak checking completed."


Second, create riak_stop.sh file, add and save below lines:


#!/bin/bash

#script for initiating riak database.

echo "Stoping Riak DB..."


cd riak-1.3.2/dev/


for node in `ls`; do $node/bin/riak stop; done


echo "Riak DB stoped."


echo "-----------------"


echo "Checking Riak....."

ps aux | grep beam


echo "Riak checking completed."

must save this two files in home directory and run the below command in your Teminal:


make it as executable

chmod +x riak_run.sh
chmod +x riak_stop.sh
  For run and check the status of Riak nodes:

./riak_run.sh


For stop Riak nodes:


./riak_stop.sh


Congratulations!....Cheers!!!!


Reference: http://docs.basho.com/riak