29. September 2019
introduction
Jenkins is one of the best continuous integration (CI) tools because of its maturity, large number of plugins, support for distributed builds, etc.
One of the key features of Jenkins is Docker container support (via the Docker plugin), as it makes it easy to build CI pipelines based on specific tools without having to install the tools on the host itself.
In fact, Jenkins itself is distributed as a Docker container, which is convenient since you don't have to install Jenkins and its dependencies directly on the host machine.
However, it turns out that there are several problems when running Jenkins in a container and using the Jenkins Docker plugin in your pipelines.
This article describes these problems and shows an easy way to fix them with aSystemcontainer Nestybox.
If you plan to use Jenkins for your CI, this article will help you get set up in minutes.
contents
- Problems with Jenkins in a container
- Issues with Jenkins Docker agents
- Even more problems
- recap
- Simple solution with Nestybox system containers
- Why does this work?
- Try our free trial!
Problems with Jenkins in a container
When installing Jenkins on a host, the easiest way is to run a Docker container that contains the Jenkins server. This saves you the trouble of having to install Jenkins and all of its dependencies directly on the host machine.
Jenkins provides a Docker image calledJenkins/Jenkinsthat has the jenkins server in it.
However, if your Jenkins pipeline uses Docker as an agent (which is common and practical), theJenkins/Jenkinspicture doesn't work.
For example, if your pipeline has something like this:
Jenkinsfile:
Pipeline { Agent { Docker { Bild 'golang' } } stages { stage('to build') { steps { Sch 'go build main.go' } } }}then Jenkins asks Docker to run itgolangPicture. But since Docker is not present in theJenkins/Jenkinscontainer, the pipeline fails.
To solve this, you first need to build a custom Docker image that includes Jenkins and the Docker CLI. Here is a Dockerfile example for such an image:
OUTJenkins/Jenkins# Docker-InstallationUSERrootRUNapt-get update&&apt-getTo install -y \apt-transport-https\ca certificates\ruffle\gnupg2\Software-Features-CommonRUNruffle-fsSLhttps://download.docker.com/linux/debian/gpg | add apt key -RUNapt-key fingerprint 0EBFCD88RUNadd-apt-repository\ "deb [arch=amd64] https://download.docker.com/linux/debian\ $(lsb_release-cs) \stable"RUNapt-get update&&apt-getTo install -ydocker-ce-cliUSERjenkinsYou can build it with:
$ docker build -t jenkins-docker .When you run this image, you now need to mount the host machine's Docker socket into the Jenkins master container. That's because this image only includes Jenkins and the Docker CLI, butNotthe docker daemon. Therefore, you need to connect the Docker CLI in the Jenkins container to the Docker daemon on the host by mounting the daemon's socket into the container:
$ docker run --rm -d -v /var/run/docker.sock:/var/run/docker.sock -P jenkins-dockerBut even that is not enough. It turns out that theJenkins/Jenkinsimage (which uses the Dockerfile above as the base image) is a non-root user inside the container by default. This user does not have permission to access the docker daemon socket mounted in the container. So when you run your pipeline with Jenkins you get this error:
Permission was denied trying to connect to the docker daemon socket at unix:///var/run/docker.sock...The solution is to run the above image with the following command:
$ docker run --rm -d --group-add $(stat -c '%g' /var/run/docker.sock) -v /var/run/docker.sock:/var/run/docker.sock -P jenkins-dockerWith this command, the Jenkins master runs inside the container and can invoke the Docker daemon on the host to load the agent containers (e.g. thegolangcontainer for the pipeline example above).
The figure below shows the configuration.
While this configuration works, note that all containers created by Jenkins are created at the host level. This is fine in many cases, but causes problems if your pipeline contains Docker commands, as described in the next section.
Issues with Jenkins Docker agents
Sometimes it is useful for a pipeline stage to invoke Docker commands. For example, the following pipeline definition configures a Docker agent with theDockerimage and runs aDocker-Buildenter.
Jenkinsfile:
Pipeline { Agent { Docker { Bild 'Docker' } } stages { stage('to build') { steps { Sch 'docker build -t mein-container .' } } }}Ideally, this should run without problems. But unfortunately that is not the case. Running this pipeline results in a "permission denied" error reported by the Jenkins Docker agent:
Permission was denied trying to connect to the docker daemon socket at unix:///var/run/docker.sock...The problem with this is that when Jenkins uses Docker to create the agent container, the host's Docker socket is mounted on the agent, so the agent can run Docker commands (such asDocker-Buildcommand in the pipeline shown above).
The following figure shows the structure:
Mounting the host's docker socket into the agent is fine, except that Jenkins also sets up the default user in the agent container as a non-root user. This non-root user does not have permission to access the mounted docker socket.
As a result, if the pipeline step withDocker-Buildinside the agent container, we get the permissions error shown above.
To solve this, you now need to create a custom Docker agent image in which the non-root user is added to the Dockersocket group. It's painful since you just wanted to run a simple pipeline.
Even more problems
Even if you build a custom Docker agent image to work around the problem described in the previous section, you may still encounter other problems.
For example, assume image in the following pipeline configurationmy-docker-agentis a custom Docker agent image that solves the problem in the previous section.
The pipeline has a single step that requires running a Docker container namedmein-containerinside the docker agent container.
Jenkinsfile:
Pipeline { Agent { Docker { Bild 'my-docker-agent' } } stages { stage('to build') { steps { Sch 'docker run --name mein-container mein-container' } } }}That looks perfectly fine, but the pipeline stepdocker run --namemy-container mein-containerfails if the host already has a running container with the namemein-container.
This error occurs because the docker daemon in the host is the only entity that creates containers. The Jenkins agent container is simply connected to the docker daemon on the host using the host's docker socket. Hencemein-containeris created by the docker daemon on the host, and this creation will only succeed if there is no other container with the same name.
Now, to avoid these kinds of errors, you must either avoid naming containers in your pipeline steps (which is fine, but restrictive) or just use the machine for running Jenkins. By doing this, you eliminate the chance of name collisions between containers running Jenkins and other containers.
The problem is not specific to container names. It also occurs when naming other Docker resources such as networks or volumes.
All of this adds to the previously discussed pain points.
recap
Let's briefly recap the issues we've faced so far in our quest to simply run Jenkins in a container and use the JenkinsDocker plugin:
1) We needed to create a custom Docker image for the Jenkins master. pictureJenkins/Jenkinswasn't enough.
2) We needed to run this jenkins master container with a volume mount of the host's docker daemon and with the--group-add-flag specifying the correct permission to mount the volume. This reduces the isolation between the Jenkins master container and the host.
3) If we want to run Docker commands in our Jenkins pipeline, we had to build a custom Docker image for the Jenkins agent. pictureDockerwasn't enough.
4) In the pipeline steps, the docker commands we issued were run against the host's docker daemon. Therefore, these can fail when naming Docker resources (containers, networks, volumes) that collide with resources already used in the host.
While these may seem like trivial issues, they will likely take you several hours to figure them out (like they cost me when investigating Jenkins interactions with Docker), taking valuable time away from your ultimate goal of running your Jenkins pipeline while leveraging the power of Docker containers.
Simple solution with Nestybox system containers
It is possible to completely avoid the problems (2), (3) and (4) described in the previous section by running Jenkins inside aSystemcontainer Nestybox.
Problem (1) is unavoidable because whenever we want to run Jenkins in a container and configure it with its Docker plugin, we need a container image containing Jenkins and Docker.
The solution consists of using a system container image that acts as a dedicated Jenkins sandbox and contains the following software:
-
Jenkins-Meister
-
Docker (not just the CLI, but the Docker daemon too!)
-
supervisord (the process manager of the system container)
We have such an image in Nestybox's docker hub, along with its Dockerfile. It's callednestybox/jenkins-syscontand behere.
The Dockerfile and Supervisor configuration for this image can be found on the Nestybox GitHub sitehere.
Here's how things would work. We assume that Nestybox'sSystemboxContainer Runtime is already installed on your host (seeFree trial periodbelow for information on how to obtain it).
First, start the system container with this simple Docker command:
$ docker run --rm -d --runtime=sysbox-runc -P nestybox/jenkins-syscontThis command tells Docker to use the sysbox container runtime to start the system container.
When the system container starts, supervisord starts jenkins and the docker daemon inside the system container. You can think of the system container as a container-based virtual host: similar in many ways to a VM, but faster, easier to use, and much more efficient.
Notice how the simpler command avoids the problem (2) above (i.e. novolume mount or permission to the host's docker socket is required). In fact, the docker daemon in the system container is completely isolated from the docker daemon on the host.
Also note that while the system container runs the docker daemon it contains, it is not an insecure privileged container. This is a key feature of Nestybox system containers.
The figure below shows the structure.
Once the system container is running, simply access Jenkins from its web-based UI and configure it to run your pipeline. You can access the Jenkins UI by directing your browser to your host's IP and port mapped to the system container.
For example, on my host, Jenkins is listening on the port32789As shown below:
$ docker psCONTAINER ID IMAGE COMMAND CREATE STATUS PORTS NAMES3a28935e5c5e jenkins-syscont "/usr/bin/supervisord" 4 seconds ago Up 2 seconds 0.0.0.0:32789->8080/tcp, 0.0.0.0:32788->50000/tcp zealous_taussigSo when I point my browser to that port, I see Jenkins there:
I can then access Jenkins by pulling credentials from the system container:
$ docker exec -it zealous_taussig cat /var/jenkins_home/secrets/initialAdminPassword10762f55c17d42bfb7a1e98fb2ee278cYou can then configure and run your pipeline as usual. For example, the pipeline configuration for the golang image we showed earlier (repeated below for simplicity) works fine.
Jenkinsfile:
Pipeline { Agent { Docker { Bild 'golang' } } stages { stage('to build') { steps { Sch 'go build main.go' } } }}In this case, the docker container would be built with the golang image from the running docker daemonInsidethe system container.
Now what if your pipeline steps need to run Docker commands (e.g. to build or run a container inside your pipeline)? It's easy now, just specify the agent configuration of the pipeline like this:
Jenkinsfile:
Pipeline { Agent { Docker { Bild 'Docker' argument '-v /var/run/docker.sock:/var/run/docker.sock' } } stages { Sch 'docker run hallo-welt' }}As shown, the Docker agent uses theDockerimage (the official Docker Hub image containing the Docker CLI). No more custom image required. Problem (3) above is gone.
Also those of the Docker agentDockerImage runs with a volume mount of docker socket (Arguments '-v /var/run/docker.sock:/var/run/docker.sock').This is required since the Docker agentDockerimage only contains the Docker CLI, not the daemon.
But here's the key: This is a volume mount for the docker daemon running in the system container.Notthe Docker daemon on the host. In other words, the Docker daemon on the host is completely isolated from any Docker containers launched by Jenkins. This solves problem (4).
Why does this work?
Basically, this solution works because the system container image acts as a virtual host (or sandbox environment) in which Jenkins can create containers using a dedicated docker daemon completely isolated from the underlying host.
The permissions issues are gone as all work is done inside the system container under the "root" user (which is mapped by the Nestybox system container to a non-root user on the host for increased security).
This means you can use the host for other tasks as well, without ever worrying about collisions with Jenkins and its container operations.
This also means that you are not restricted when writing the pipeline steps, even if those pipeline steps require creating or launching Docker containers. Because the pipeline runs inside the system container, you can rest assured that you'll never collide with Docker resources on the host itself.
With this approach, you can launch Jenkins, configure it through its WebUI, and run your first pipeline in minutes.
Try ourFree trial period!
We have developed a prototype and are looking for early adopters.
You can access our free trialwebsite. We believe you will find it easy to use and useful. Your feedback is greatly appreciated!
FAQs
Can Jenkins create a Docker container? ›
Whenever a Jenkins build requires Docker, it will create a “Cloud Agent” via the plugin. The agent will be a Docker Container configured to talk to our Docker Daemon. The Jenkins build job will use this container to execute the build and create the image before being stopped.
What is a container in Jenkins? ›Docker is a platform for running applications in an isolated environment called a "container" (or Docker container). Applications like Jenkins can be downloaded as read-only "images" (or Docker images), each of which is run in Docker as a container.
Why run Jenkins in a container? ›All of your Jenkins configuration files live inside the container rather than the host machine. Knowing that all the files you need are inside the container, you can eliminate the issue of accidentally mixing your files with Jenkins configuration files.
Can I run any application in a container? ›Containers aren't a solution equivalent to desktop application virtualization. They support only server-side applications that don't require an interactive session. Because they run on specialized container images, they support only those applications that don't need a graphical front end.
Can Jenkins be containerized? ›Containerization is a great way to simplify migration of Jenkins instances to different machines, as well as simplify ongoing maintenance and upgrades. Starting with versions 2.5 and higher, Jenkins Pipeline has built-in support for interacting with Docker from within a Jenkinsfile.
Can Jenkins be used without Docker? ›I know that jenkins can be installed without a docker at all (“Installing Jenkins - Linux”). I also assume that I can use the jenkins/jenkins container by itself (implied by this section).
Which is better Jenkins or Docker? ›Docker Swarm is more comprehensive and highly customizable, whereas Jenkins is customizable with the help of plugins. Kubernetes and Jenkins provide low fault tolerance unlike Docker. Docker is a container engine that can make and handle containers, whereas Jenkins is a CI/CD model that can build and test applications.
Is a container the same as a node? ›Containers are packages of applications and execution environments. Pods are collections of closely-related or tightly coupled containers. Nodes are computing resources that house pods to execute workloads.
What is difference between node and container? ›To summarize, nodes represent physical or virtual machines that provide CPU and RAM resources for container-based applications. Nodes are grouped together into clusters. And finally, instead of managing containers individually, Kubernetes containers are housed into pods for scheduling and execution.
How do I run a Docker container in Jenkins pipeline? ›- Automatically grab an agent and a workspace (no extra node block is required).
- Pull the requested image to the Docker server (if not already cached).
- Start a container running that image.
- Mount the Jenkins workspace as a "volume" inside the container, using the same file path.
Can we use Docker container as a node in Jenkinsfile? ›
Many organizations use Docker to unify their build and test environments across machines, and to provide an efficient mechanism for deploying applications. Starting with Pipeline versions 2.5 and higher, Pipeline has built-in support for interacting with Docker from within a Jenkinsfile .
Can Jenkins run Docker commands? ›To set up Jenkins in Docker to be able to run Docker (specifically: build, run, and push an image), there are at least two options: Docker-in-Docker, and. Create a custom Jenkins Docker image and bind-mount the container to the host system daemon.
Why Jenkins is outdated? ›Jenkins again is an old tool and was not designed for the new container age technologies. Jenkins also does not get well with a microservices kind of architecture. In general Jenkins as a tool still holds value for following use-cases: You are using an on-premise solution.
What are the 3 types of pipelines in Jenkins? ›Different Types of Jenkins CI/CD Pipelines. Scripted Pipeline. Declarative Pipeline. The Concept of Stages in Jenkins Pipeline.
Why would a developer use a container? ›Containers allow applications to be more rapidly deployed, patched, or scaled. Containers support agile and DevOps efforts to accelerate development, test, and production cycles.
Can a container have a GUI? ›GUI applications running in the container would then appear on your existing desktop. Running a GUI program in Docker can be a useful technique when you're evaluating a new piece of software. You can install the software in a clean container, instead of having to pollute your host with new packages.
What is the difference between a system container and an application container? ›So in general when you want to package and distribute your application as components, application containers serve as a good resort. Whereas, if you just want an operating system in which you can install different libraries, languages, databases, etc., OS containers are better suited.
When should you not Containerize an application? ›So, one example of when not to use containers is if a high level of security is critical. They can require more work upfront: If you're using containers right, you will have decomposed your application into its various constituent services, which, while beneficial, isn't necessary if you are using VMs.
Can Jenkins run jar files? ›How do I run a JAR file in Jenkins? Download the latest Jenkins WAR file to an appropriate directory on your machine. Open up a terminal/command prompt window to the download directory. Run the command java -jar jenkins.
What are the disadvantages of Jenkins? ›Here are some disadvantages of Jenkins: Single server architecture—uses a single server architecture, which limits resources to resources on a single computer, virtual machine, or container. Jenkins doesn't allow server-to-server federation, which can cause performance issues in large-scale environments.
Should you run Jenkins in Kubernetes? ›
Hosting Jenkins on a Kubernetes Cluster is beneficial for Kubernetes-based deployments and dynamic container-based scalable Jenkins agents. Here, we see a step-by-step process for setting up Jenkins on a Kubernetes Cluster.
What will replace Jenkins? ›- Buddy.
- FinalBuilder.
- GoCD.
- IBM Urbancode.
- CircleCI.
- TeamCity.
- GitLab CI.
Jenkins is a platform for creating a Continuous Integration/Continuous Delivery (CI/CD) environment. The system offers many different tools, languages, and automation tasks to aid in pipeline creation when developing and deploying programs.
Can I run Jenkins without Tomcat? ›Install Java Version 8 – Jenkins is a Java based application, hence Java is a must. Install Apache Tomcat Version 9 – Tomcat is required to deploy Jenkins war file. Download Jenkins war File – This war is required to install Jenkins.
Is Docker still relevant 2022? ›Is Docker Still Relevant In 2022? Docker remains relevant to most container projects, applications, and developers today thanks to its modern tools, compatibility, large community, and ease of use. However, Docker Inc has undergone changes recently, among them changes to Docker Desktop licensing.
Can Kubernetes replace Jenkins? ›Jenkin vs.
Kubernetes and Jenkins do not compete. Instead, the two systems complement each other and work together to automate software releases using Agile methods. Outlining the critical differences between the two tools helps clarify how to apply each and maximize the benefits in a development environment.
Gitlab CI
Every GitLab user has access to the built-in CI/CD tool GitLab CI. GitLab CI offers an excellent UI which makes it easy to implement CI/CD without employing another third-party tool like Jenkins. This tool is cloud-based and highly scalable.
Containers share the same operating system kernel and isolate the application processes from the rest of the system.
What is Docker vs container? ›The key difference between a Docker image Vs a container is that a Docker image is a read-only immutable template that defines how a container will be realized. A Docker container is a runtime instance of a Docker image that gets created when the $ docker run command is implemented.
How many containers can run in a node? ›No more than 110 pods per node. No more than 5000 nodes. No more than 150000 total pods. No more than 300000 total containers.
What are the 3 main types of containers in Java? ›
Three of the most useful container types are JFrame , JPanel , and JApplet . A JFrame is a top-level window on your display. JFrame is derived from JWindow , which is pretty much the same but lacks a border. A JPanel is a generic container element used to group components inside of JFrame s and other JPanel s.
Is a JFrame a container? ›JFrame is a top-level container that provides a window on the screen. A frame is actually a base window on which other components rely, namely the menu bar, panels, labels, text fields, buttons, etc. Almost every other Swing application starts with the JFrame window.
What are the 3 types of Java Swing containers? ›- Swing Containers. Swing containers can be classified into three main categories:
- • Top-level containers: JFrame, JWindow, and JDialog.
- • General-purpose containers: JPanel, JScrollPane,JToolBar,JSplitPane, and JTabbedPane.
- • Special-purpose containers: ...
- Swing components can be broadly classified as:
Databases are standard stateful and durable, containers are stateless and ephemeral and, finally, databases have many - dynamic - configuration parameters. However, the latter is also a challenge with virtual machines.
What is difference between server and container? ›The main difference between the web containers and application server is that most web containers such as Apache Tomcat implements only basic JSR like Servlet, JSP, JSTL wheres Application servers implements the entire Java EE Specification. Every application server contains web container.
What is a container in API? ›Containers are packages of software that contain all of the necessary elements to run in any environment. In this way, containers virtualize the operating system and run anywhere, from a private data center to the public cloud or even on a developer's personal laptop.
Why do we need Docker in Jenkins? ›Docker is a container engine that can create and manage containers, whereas Jenkins is a CI engine that can run build/test on your app. Docker is used to build and run multiple portable environments of your software stack. Jenkins is an automated software testing tool for your app.
How do I make a CI CD pipeline in Jenkins and Docker? ›- Logging in to the Amazon Web Services Console.
- Connecting to an EC2 Instance Using Amazon EC2 Instance Connect.
- Launch Jenkins and Splunk Docker Containers.
- Log in to Splunk and Complete Default Installation.
- Create a working directory.
- In your favorite text editor, create a new file called “Dockerfile”
- Add the following to the file and save it: FROM jenkins/jenkins:2.112 LABEL maintainer=”yourcontact@somemail.com”
- Then at the command line enter:
The simple answer is, Agent is for declarative pipelines and node is for scripted pipelines. In declarative pipelines the agent directive is used for specifying which agent/slave the job/task is to be executed on.
What should you not use Docker for? ›
Docker is great for developing web applications, but if your end-product is a desktop application, then we would suggest you not to use Docker. As it doesn't provide the environment for running the software with a graphical interface, you would need to perform additional workarounds.
Can you run graphical applications in Docker? ›We can easily run the most common GUI applications without getting into trouble inside a Docker Container.
Is Docker discontinued? ›Containerd and CRI-O: Docker Alternatives
Before upgrading to a Kubernetes version removes support for Docker (which is currently estimated to release in late 2021), you will need to modify (or replace) existing Kubernetes nodes so that they use a supported container runtime other than Docker.
- Automatically grab an agent and a workspace (no extra node block is required).
- Pull the requested image to the Docker server (if not already cached).
- Start a container running that image.
- Mount the Jenkins workspace as a "volume" inside the container, using the same file path.
- To start your container, use the following command. Bash Copy. docker run -dp 3000:3000 getting-started. ...
- Add an item or two and see that it works as you expect. You can mark items as complete and remove items. Your frontend is successfully storing items in the backend.
- Use docker ps to get the name of the existing container.
- Use the command docker exec -it <container name> /bin/bash to get a bash shell in the container.
- Or directly use docker exec -it <container name> <command> to execute whatever command you specify in the container.
To run a command in a certain directory of your container, use the --workdir flag to specify the directory: docker exec --workdir /tmp container-name pwd.
Can you run a GUI in a Docker container? ›We can easily run the most common GUI applications without getting into trouble inside a Docker Container.
What can I run in a container? ›You can run both Linux and Windows programs and executables in Docker containers. The Docker platform runs natively on Linux (on x86-64, ARM and many other CPU architectures) and on Windows (x86-64). Docker Inc. builds products that let you build and run containers on Linux, Windows and macOS.
What technologies can be run inside a container? ›- Docker. The first and still most popular container technology, Docker's open-source containerization engine works with most of the products that follow, as well as many open-source tools.
- Docker Enterprise. ...
- CRI-O. ...
- rktlet. ...
- containerd. ...
- Microsoft Containers. ...
- Kubernetes. ...
- Istio and Envoy.
Can you run DB in a container? ›
If you're working on a small project, and are deploying to a single machine, it's completely okay to run your database in a Docker container. Be sure to mount a volume to make the data persistent, and have backup processes in place. Try to restore them every once in a while to make sure your backups are any good.
Can you interact with an application inside a container? ›Performing similar process, you can run almost any GUI application inside Docker Containers and display and interact with them on your screen. Running GUI Apps in a Docker Container is really an awesome experience which will never harm/use your host Filesystem.
How to run code in docker container? ›- Install Docker on your machine. For Ubuntu: ...
- Create your project. ...
- Edit the Python file. ...
- Edit the Docker file. ...
- Create the Docker image. ...
- Run the Docker image.
Docker exec is a command that allows the execution of any given command within a Docker container. This means it will interpret the arguments passed to it as commands to be run inside the container.