June 29, 2015

Docker from Development to Deployment

Are we bracing ourselves for a world where we are going to see developers, testers, and other stakeholders in a development process working on their own sandbox containers?  The answer seems to be a resounding YES.  Let’s jump into the concept of container-driven development (CDD) - I think I just found a new acronym that’s going to be famous, in a world that’s begging for more acronyms :).   Or do you think DDD - Docker Driven Development, sounds better than CDD? (;-) Anyways, let's jump in.

In an application life-cycle process, let's take the 3 biggest stages - Development, Testing, and Production.  This post is going to focus on how Docker containers can play an active role in each of these life-cycle stages and benefits that can be derived from them?

Note, I am going to make assumptions (like just 3 life-cycle stages) to keep this post short and concise, but an enterprise is free to extrapolate and devise better ways of doing the same.

Develop in a container
Consider a scenario where every developer builds his application inside a container, which is provisioned exclusively for the project being developed.  These Docker-based containers are extremely lightweight.  They can be instantly provisioned in a developer laptop or an enterprise server infrastructure that can be accessed remotely through console utilities.  These development containers will be based on a Docker image that can contain all the relevant utilities for a developer.  Docker image itself can be stored in a Docker Registry.  These containers, based on the Docker image, can be provisioned in a matter of seconds. For example, if a new developer joins a team, he can provision a container instantly with all the utilities (like Eclipse IDE) pre-installed.  And finally, after the developer gets his app to work, he can push the image of the app build to the registry.  A docker container for a developer can look like how it is.

A container has 2 primary configurations to it.

  1. The hardware configuration is a factor of 3 parameters - CPU, memory, and disk.  A sample configuration of Dev, Test, and Production containers.
  2. The software stack is contained in the container.  In this case, from Fig-1, you see that the Dev-Container has Eclipse, Java, and Tomcat as a part of it.  There is also a dedicated DB container that is used by the development team that contains MySQL as a part of the container.

Also note that the utilities that need modifications to their configurations like the DB settings, utilities for monitoring, and log files need not be a part of the container necessarily.  They can be configured when the same container is provisioned in different release stages.

What are the benefits of using containers during development?

  • Instant provisioning of the requisite hardware resource as a container
  • Instant provisioning of the requisite software stack configuration of the container
  • Option to the de-provision container if it’s not used (Needs to be coded)
  • Integrated DevOps.  The development and operations are not disconnected anymore.

Test in a container
Once the app-build (See App v1.1 in Fig 1 & 3) image is pushed into the registry by the developer, it indicates that the build is ready to be tested.  In pre-container days, the tester would set up a new physical machine or a VM instance, install the requisite software, and then deploy the app.  It never worked the first time.  The developer just says “It worked for me”.

All these kinds of excuses are going to be a thing of the past, now that the containers have emerged.  In my previous post on Docker, one of the benefits I highlighted was “Guaranteed Reproduce-ability”. That means, by packaging your ap­pli­ca­tions into containers you can be sure that they will run as it did in a development environment, wherever they are deployed.

In summary, there is a guarantee of reproducing the same behavior wherever it is executed.  Hence when a Tester/QA provisions a container based on the app-build image, it's guaranteed to work as it did for the developer.

QA can provision individual containers for themselves to focus on various aspects of the app like performance etc or all individual QA members can work on just one provisioned test container and focus on a particular area/feature of the app.

QA can find bugs in the app that can be reported back to the development in no time and this process can speed up the overall delivery process, with a little automation. In fact, consumer-facing services/apps like Amazon do multiple releases in a very short span of time (an interval of one to few days) using continuous delivery processes and containers are going to make that even more faster.  The idea of having a 1:1 mapping between the app being developed and the container also speeds up the delivery process with a lot more clarity, since now the deployment of the app is nothing but simply provisioning a container.

At this stage of the app lifecycle, the testing containers are provisioned and the lifecycle is visually represented.

What are the benefits of using containers during testing?

  • No more writing huge complex scripts to deploy apps and make them work
  • Developers can heave a sigh of relief - no more excuses like “It used to work for me”
  • Automate everything.  No time wasted.  Execute continuous delivery and make frequent product releases

 Deploy in a container
Once the app build has passed the QA, it’s ready to be deployed into the production environment.  All that needs to be done is to provision a new production container (bigger size as given in Fig-2) and it's all set.  The guaranteed reproduce-ability will make sure that the app deploys and works as effectively as it did in the dev and testing environments.  There are few more steps like migrations of DB, services, etc but that looks like a topic of another blog post.

The instant provisioning and de-provisioning of containers offer tremendous flexibility to update to newer app versions with rolling upgrades or blue-green deployment.  The lightweight nature of the container also makes localized error detection and fixing extremely easy.  All that needs to be done is to image the production container and move it into a debugging environment for a quick inspection.  An app in a container has also added fuel to the new MicroServices-based app architecture, where independent modules are deployed in separate containers and are scaled independently.  The commoditization of hardware and container-based PaaS platforms offers tremendous benefits to instantly scale the application.  Check out my previous blog for more details on localized error detection, horizontal scalability, and lightweight containers.

The complete app life cycle after the app is deployed into production should something like this

What are the benefits of using containers for the production phase? 

  • Every app gets a container. targeted error detection.
  • Instant rollback of faulty builds
  • Rolling upgrades with zero downtime
  • Enabled for Microservice apps
  • Instant horizontal scalability

So, What next?
It all seems so easy, isn't it? Not really.  There are a lot of unresolved issues during this app delivery process that is being/yet to be addressed.  Some of them include...

  • Continuous deployment needs governance semantics around it
  • Migration from one life-cycle stage to another (say dev to testing) requires migrations of DB, services, logs, etc and that is not easy.
  • Do-It-Yourselves options for managing containers are not there.  Still, this area looks geeky with command-line options.

 How can you solve this? Are there any tools that can ease the task for you? I would suggest you take a look at WaveMaker's case study on "How WaveMaker Got Faster, Better, More Agile with Docker".  It also touches on the general benefits of Docker and issues of management complexity.[/vc_column_text][/vc_column][/vc_row]