Continued delivery series (1): use Docker, Mesos to achieve continuous delivery

[Editor's Note] This article focuses on how to build a continuous integration environment for Node.js applications. Including how to set up a Node.js application on the development machine and how to deploy the Jenkins and Docker registry for continuous integration. It is recommended that students who are interested in using Docker's CI work, CI is also one of Docker's application scenarios. This article is a good tutorial.

You first need to install Docker and Fig on your Linux machine. This article will introduce how to use Docker and Mesos to achieve continuous integration.

Testing, build the production environment available Mesos cluster and other complex work is not within the scope of this article, I will follow in the follow-up article in detail. Next, we will support the solution on more cloud platforms, such as GCE, AWS, and gradually integrate more third-party components, such as Flocker , Weave and Consul .

All of the code in this article can be found on GitHub .

Throughout the solution, we used the following tools:

  • Node.js in the HelloWorld project
  • Git
  • Jenkins
  • Docker
  • Docker Registry
  • Fig
  • Mesos
  • Mesosphere Marathon
  • Zookeeper

Development environment

Let's start with a simple HelloWorld first, which is written using Node.js and runs in the Docker container.

From the figure below, you can see that the developer uses the container to run the build, test, and other processes in the local environment, as shown in the light blue box below. Once tested, they will submit the code to the central Git repository, which will then be released to the production environment.
15.02_.11_diagram_.jpg
I assume that readers who read this article can understand and implement this step on their own.

First is Node.js application app.js:

  // Load the http module to create an http server. 
Var http = require ('http');

// Configure our HTTP server to respond with Hello World to all requests.
Var server = http.createServer (function (request, response) {
Response.writeHead (200, {"Content-Type": "text / plain"});
Response.end ("Hello World \ n");
});

// Listen on port 8000, IP defaults to "0.0.0.0"
Server.listen (8000);

// Put a friendly message on the terminal
Console.log ("Server running at http://127.0.0.1:8000/");

Corresponding configuration file package.json:

  { 
"Name": "hello-world",
"Description": "hello world",
"Version": "0.0.1",
"Private": true,
"Dependencies": {
"Express": "3.x"
},
"Scripts": {"start": "node app.js"}
}

After the above two steps, you can use Google's Node.js mirror to Docker the application, and add the following Dockerfile :

  FROM google / nodejs 

WORKDIR / app
ADD package.json / app /
RUN npm install
ADD. / App

EXPOSE 8000
CMD []
ENTRYPOINT ["/ nodejs / bin / npm", "start"]

The development environment is ready. Now we can build the container and confirm that the mirror is working properly.

  $ Docker build -t my_nodejs_image. 
$ Docker run -p 8000: 8000 my_nodejs_image

Now you can visit http://127.0.0.1:8000/ to see if you can see "Hello World"!

You can modify app.js to let it show other strings. Just need to rebuild the mirror, and then start a new container, refresh the browser, you can see this new string.
Screen-Shot-2015-02-28-at-14.17_.34_.png
The configuration here does not have any tests. Not because it does not need to, but to simplify the configuration, will focus on Docker and Mesos / Marathon. Excellent programmers should know that no test, then the development project can not directly on the production environment.

Now go to the next step.

Continuous integration / delivery

15.02_.11_diagram2_.jpg
We need to have a version control system and build a server to run the build and test after the code is submitted. To this end, we used Jenkins and published the results (artefact) to the repository. Because we use Docker deployment, the final artefact is a Docker image, push is also pushed to the local Docker Registry.

In order to simplify the complexity, I did not use Git warehouse management tools, such as GitLab or GitHub. Connecting Jenkins to the Git server is simple and will not go on here. If you want to deploy Git servers on Docker, you can use this image .

Now let's start writing fig.yml now. Fig is a choreographer, and we can use a command to deploy all the centrally located services. Fig is just a development tool that needs to be replaced with more sophisticated automation tools if you want to use it in a production environment.

A few days ago, Docker released the first version of Docker Compose, which could replace Fig. I have not tested it yet, but I do not need any change, Docker Compose can replace Fig.

The following is to build the local Docker Registry fig.yml :

  Registry: 
Image: registry
Environment:
- STORAGE_PATH = / registry
Volumes:
- registry-stuff: / registry
Ports:
- "5000: 5000"

Here is the standard Docker Registry image, and the container outside a persistent storage of the volume mounted to the top, because we need to restart the container can retain the built mirror.

Next run:

  $ Fig up 

Docker Registry runs on http://localhost:5000 .

The next step is to build the mirror and push it to the Registry.

  # Build an image 
$ Docker build -t localhost: 5000 / containersol / nodejs_app

# Push it to the registry
$ Docker push localhost: 5000 / containersol / nodejs_app

At this time Docker Daemon will report the following error:

  Forbidden. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add -insecure-registry 10.0.0.26:5000 to the daemon's arguments. 

To solve this problem, see StackOverflow
The quickest and easiest way is to add the following line to /etc/default/docker and then restart the Docker Daemon:

  DOCKER_OPTS = "- insecure-registry localhost: 5000" 

After the problem is solved, push the mirror onto the repo and access the URL to see if the mirror has been successfully pushed.
Screen-Shot-2015-02-28-at-14.21_.13_.png
Then add Jenkins configuration to fig.yml file, reboot the system:

  Jenkins 
Image: containersol / jenkins_with_docker
Volumes:
- jenkins-stuff: / var / jenkins_home
-.: / Var / jenkins_data
- /var/run/docker.sock:/var/run/docker.sock
- / usr / bin / docker: / usr / bin / docker
Ports:
- "8081: 8080"
Registry:
Image: registry
Environment:
- STORAGE_PATH = / registry
Volumes:
- registry-stuff: / registry
Ports:
- "5000: 5000"

Here is a detailed explanation of the next Jenkins configuration. First, we mount the Docker binaries in the Jenkins container and the socket used by the docker daemon. This is because you need to allow Jenkins to run the Docker command on the host from the container. The problem is that only users in the Docker group have the right to do so, and the Jenkins container is a Jenkins user, not a root user. So we used the following Dockerfile to build our own Jenkins mirror:

  FROM jenkins 

MAINTAINER ContainerSolutions

USER root
#TODO the group ID for docker group on my Ubuntu is 125, therefore I can only run docker commands if I have same group id inside.
# Otherwise the socket file is not accessible.
RUN groupadd -g 125 docker && usermod -a -G docker jenkins
USER jenkins

In the Dockerfile folder run the following command to build the mirror:

  $ Docker build -t containersol / jenkins_with_docker. 

This is not a very clever solution, if you have a better way, be sure to leave a message in this article tell me.

Fig will create a persistent store for the Jenkins container and mount the current directory in the container. This is useful after running build, push, and deploy scripts.

To complete the continuous presentation of the text, you also need to add some scripts to build and push the Docker image, configure Jenkins to run them.

build.sh contains additional mirror version information. Each Jenkins build creates a new tag for the Docker image. Need to pay attention, I am not saying that this is the correct version of the way. This area is very complex, may be written later in the article to discuss, but now skip it.

  #! / Bin / bash 

If [-z "$ {1}"]; then
Version = "latest"
Else
Version = "$ {1}"
Fi

Cd nodejs_app
Docker build -t localhost: 5000 / containersol / nodejs_app: $ {version}.
Cd

Note the name of the mirror that was built. It first contains the URL of the Registry, then the full name of the mirror, and then the label.
Also note that Jenkins runs in the container and uses the script to build the mirror, but the docker command executed in the Jenkins container will actually run on the host because we mounted the socket used by the Docker before.

push.sh will push the previous steps to build the mirror. Use the same version as the previous script.

  #! / Bin / bash 

If [-z "$ {1}"]; then
Version = "latest"
Else
Version = "$ {1}"
Fi

Docker push localhost: 5000 / containersol / nodejs_app: "$ {version}"

The last step is to restart the entire system with Fig, and configure Jenkins job to run build.sh and push.sh Jenkins runs on the URL http://localhost:8081 .
This is a standard build job, the implementation of two shell orders, build version of its input parameters:

  ./build.sh $ {BUILD_ID} 
./push.sh $ {BUILD_ID}

Screen-Shot-2015-02-28-at-17.28_.21_.png
Now we can perform the build, you can see the new image deployed to the Registry. Note that the URL contains a new label.
Screen-Shot-2015-02-28-at-17.31_.54_.png
This article, as the first part of the series, describes how Docker has applied a Node.js application on the development machine and how to deploy Jenkins and Docker Registry for Node.js applications.

The second part will introduce the Mesos and Marathon structures, and how to complete the continuous integration of workflow.

Source: Continuous delivery with Docker on Mesos in less than a minute – Part 1 (translation: Cui Jingwen proof: Guo Lei)

===========================
Translator introduction Cui Jingwen, now working at VMware, senior software engineer, responsible for desktop virtualization products, quality assurance work. Worked for years in the IBM WebSphere Business Process Management software. On the virtualization, middleware technology has a strong interest.

    Heads up! This alert needs your attention, but it's not super important.