ElasticSearch cluster setup on AWS

Elasticsearch is a distributed search server offering powerful search functionality over schema-free documents in (near) real time. All of this functionality is exposed via a RESTful JSON API. It’s built on top of Apache Lucene and like all great projects it’s open source.

I’ll assume you already have an Amazon AWS account, and can navigate yourself around the AWS console. If not – get started over at aws.amazon.com.

Fire up instances with your favorite AMI. I chose 3 instances of m3.xlarge running the latest Ubuntu, but whatever works for you.

Steps to setup Elasticsearch cluster :

1. Download a copy of Elasticsearch 1.5.2

wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.5.2.zip

2. Unzip the zipped folder

unzip elasticsearch-1.5.2.zip

3. Install the cloud AWS plugin so you can take advantage of features like automatic node discovery-

– Go to bin folder of ELASTICSEARCH_HOME and type –

. /plugin install elasticsearch/elasticsearch-cloud-aws/2.5.0

– You can check installed plugin through:

. /plugin –list

4. Configuring Elasticsearch by editing its elasticsearch.yml file –

cluster.name: test-cluster

node.name: "AWS-ES-1"

node.master: true

node.data: true

index.number_of_shards: 3

index.number_of_replicas: 0

plugin.mandatory: cloud-aws

network.host: _ec2_

discovery.type: ec2

discovery.zen.ping.unicast.hosts: ["52.71.255.37", "54.165.19.167", "52.91.6.18"]

cloud.aws.access_key: xxxxxxxxxxxxxxx

cloud.aws.secret_key: xxxxxxxxxxxxxxx

Note:

Pink colored configurations can be changed as per requirement.
name.node will be different for all the nodes and cluster.name should be same.
Blue colored configurations will remain same for all the nodes.

5. Run elasticsearch

./bin/elasticsearch

Your node is now running and is accessible (when using default configuration) on port 9200

curl -XGET ‘http://<EC2 instance ip>:9200/_cluster/health?pretty=true’
{
“cluster_name” : “test-cluster”,
“status” : “green”,
“timed_out” : false,
“number_of_nodes” : 3,
“number_of_data_nodes” : 3,
“active_primary_shards” : 3,
“active_shards” : 3,
“relocating_shards” : 0,
“initializing_shards” : 0,
“unassigned_shards” : 0,
“delayed_unassigned_shards”: 0,
“number_of_pending_tasks” : 0,
“number_of_in_flight_fetch”: 0,
“task_max_waiting_in_queue_millis”: 0,
“active_shards_percent_as_number”: 100
}