Deploying Java Web application with Docker and Mesos for fault-tolerance and scalability

In this post we well deploy simple java/scala microservice application to Mesos and learn how to scale it up and down. If you read my previous post, you know that Mesos fully support Dockers, most popular today container technology. We will take advantage of Dockers and deploy our application as a docker image. With Dockers we can deploy only bare minimum required for our application, have everything packaged in a container and we can even run many containers on the same host.

Here is the architecture of our deployment.
mesos_blog_2

I understand than not many have access to 3 or more node cluster, so we will do everything on a single node virtual machine. This setup allow us to learn how to deploy application to Mesos on a single laptop or server. We will be using great virtualization environment, Vagrant. So lets get started:

Requirements:

1. Getting Mesos, Marathon up and running.

We will be using pre-packaged Virtual environment, named playa-mesos. It come with Mesos, Marathon, Docker and git preinstalled. On your computer, open terminal then download and start playa-mesos virtual environment.


git clone https://github.com/mesosphere/playa-mesos.git
cd playa-mesos
vagrant up
vagrant ssh

This will prepare for us Mesos and Marathon. You can access Mesos UI at http://10.141.141.10:5050/ ans Marathoon UI at http://10.141.141.10:8080/. Now we have Mesos up and running on our single machine, let's prepare our java application inside docker container.  We will build our application based on repository I have prepared and our development environment will be this new vagrant box.

Prepare Java Application


#ssh to vagrant box
vagrant ssh
#install sbt - package manager for scala (for java projects it would be maven or gradle
echo "deb http://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list
sudo apt-get update
sudo apt-get install sbt
#let's download and build fat jar for our scala project
git clone https://github.com/edvorkin/simple-docker-scala-app.git /vagrant/simple-docker-scala-app
#create fat jar
cd /vagrant/simple-docker-scala-app 
sbt assembly


At this point we should have jar archive in target/scala-2.11 directory.

Create Docker image
Now lets build our docker image based on Dockerfile provided.

This file tells Docker to download Centos system, install java and then copy our fat jar into /tmp directory.


sudo docker build -t simple-docker-scala-app .


This will build our docker image with our little apps inside. Let test how it works. Execute following command:


sudo docker run -d -p 7070:8080  simple-docker-scala-app java -jar /tmp/simple-docker-scala-app-assembly-0.0.2.jar Boot


Now check out http://10.141.141.10:7070/
You should see application up and running.
simple-app
Lets stop our docker container.


sudo docker stop 

Deploy docker container to Marathon
So now we have working Docker container and Mesos up and running. Let's deploy our application into Marathon using Marathons REST API.
Marathon downloads docker images from either Docker Hub - a public repository or docker private repositories. We will be using public registry. The Docker Hub is a centralized resource for working with Docker and its components. Docker Hub helps you collaborate with colleagues and get the most out of Docker. To do this, it provides services such as:

Docker image hosting.
User authentication.
Automated image builds and work-flow tools such as build triggers and web hooks.
Integration with GitHub and BitBucket.

In order to use Docker Hub, you will first need to register and create an account. Don’t worry, creating an account is simple and free.
Docker Hub works like github, you have to create your account, and then you will be able to upload docker images. Please register and and execute line below (replace {username} with your name):


sudo docker login
sudo docker tag simple-docker-scala-app edvorkin/simple-docker-scala-app
sudo docker push {username}/simple-docker-scala-app


You should see following output:
Screen Shot 2015-07-18 at 11.40.25 PM

Now we ready to deploy our application to Mesos. We will be using Marathon framework, which will monitor our application, perform healthchecks, restart it if it crash and scale it. Marathon provide a nice user interface as well as rest API. We will be using Marathon Rest API to deploy our application. This require us to write application description file in json format. Here is the file for our apps. You can find this file app-slick.json in project source directory.
Most of it self explanatory, again, replace line edvorkin/simple-docker-scala-app with your account on Docker registry:

Let's deploy using Marathon Rest API:


curl -X POST http://10.141.141.10:8080/v2/apps -d @app-slick.json -H "Content-type: application/json"

Application get deployed as you can see in marathon UI.
marathounUI-deploy

Scale our application up and down
Now let scale the application, for example to 3 instances.
marathonUI-scale
As you can see, our application is running as three separate docker containers. You can use load balancers and DNS solution (HAProxy, Consul, Mesos-DNS) to distribute load across the instances.
marathon-ui-scaled

In this post I covered how to install simple Docker container into Mesos and use Marathon framework for fault tolerance and scaling. Please play with your new setup to learn more. Check log files, mesos UI, kill the application and so on. We were able to scale it easily because our application is stateless. We should always strive for developing stateless web services, or store state in scalable, in-memory storage like memcached or Redis.
Now imagine you have cluster of nodes in your datacenter and by using Mesos, dockers and Marathon your can scale almost as easily as on Amazon or Google cloud.

Submit a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>