Securing GitLab + Docker CI Pipelines

Intro

TL;DR

Contents

Security Problems with GitLab + Docker

  • The Shell Executor
  • The Docker Executor

Security issues with the Shell Executor

build_image:
script:
- docker build -t my-docker-image .
- docker run my-docker-image /script/to/run/tests

Security issues with the Docker Executor

Binding the host Docker Socket into the Job Container

image: docker:19.03.12

build:
stage: build
script:
- docker build -t my-docker-image .
- docker run my-docker-image /script/to/run/tests
[[runners]]
url = "https://gitlab.com/"
token = REGISTRATION_TOKEN
executor = "docker"
[runners.docker]
tls_verify = false
image = "docker:19.03.12"
privileged = false
disable_cache = false
volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]

Using a Docker-in-Docker Service Container

image: docker:19.03.12

services:
- docker:19.03.12-dind

build:
stage: build
script:
- docker build -t my-docker-image .
- docker run my-docker-image /script/to/run/tests
[[runners]]
url = "https://gitlab.com/"
token = REGISTRATION_TOKEN
executor = "docker"
[runners.docker]
tls_verify = true
image = "docker:19.03.12"
privileged = true
disable_cache = false
volumes = ["/certs/client", "/cache"]

Solution: Using Docker + Sysbox

A Secure DinD Service Container

[[runners]]
url = "https://gitlab.com/"
token = REGISTRATION_TOKEN
executor = "docker"
[runners.docker]
tls_verify = true
image = "docker:19.03.12"
privileged = false
disable_cache = false
volumes = ["/certs/client", "/cache"]
runtime = "sysbox-runc"
{
"default-runtime": "sysbox-runc",
"runtimes": {
"sysbox-runc": {
"path": "/usr/local/sbin/sysbox-runc"
}
}
}
image: docker:19.03.12

services:
- docker:19.03.12-dind

build:
stage: build
script:
- docker build -t my-docker-image .
- docker run my-docker-image /script/to/run/tests

GitLab Runner & Docker in a Container

  • It allows the GitLab CI jobs to use the shell executor or Docker executor (either the DooD or DinD approaches) without compromising host security, because the system container provides a strong isolation boundary.
  • You can run many GitLab runners on the same host machine, in full isolation from one another. This way, you can easily deploy multiple customized GitLab runners on the same machine as you see fit, giving you more flexibility and improving machine utilization.
  • You can easily deploy this system container on bare-metal machines, VMs in the cloud, or any other machine where Linux, Docker, and Sysbox are running. It’s a self-contained and complete GitLab runner + Docker environment.
  • For CI jobs that interact with Docker, the isolation boundary is at the system container boundary rather than at the job level. That is, such a CI job could easily gain control of the system container and thus compromise the GitLab runner environment, but not the underlying host.
$ docker run --runtime=sysbox-runc -d --name gitlab-runner --restart always -v /srv/gitlab-runner/config:/etc/gitlab-runner nestybox/gitlab-runner-docker
$ docker run --rm -it -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register
[[runners]]
name = "syscont-runner-docker"
url = "https://gitlab.com/"
token = REGISTRATION_TOKEN
executor = "docker"
[runners.docker]
tls_verify = false
image = "docker:19.03.12"
privileged = false
disable_cache = false
volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
$ docker restart gitlab-runner

Inner Docker Image Caching

[[runners]]
url = "https://gitlab.com/"
token = REGISTRATION_TOKEN
executor = "docker"
[runners.docker]
tls_verify = true
image = "docker:19.03.12"
privileged = false
disable_cache = false
volumes = ["/certs/client", "/cache", "/var/lib/docker"]
runtime = "sysbox-runc"
$ docker run --runtime=sysbox-runc -d --name gitlab-runner --restart always -v /srv/gitlab-runner/config:/etc/gitlab-runner -v inner-docker-cache:/var/lib/docker nestybox/gitlab-runner-docker
  • By making the containerized Docker’s image cache persistent, you are not just persisting images downloaded by the containerized Docker daemon; you are persisting the entire state of that Docker daemon (i.e., stopped containers, volumes, networks, etc.) Keep this in mind to make sure you CI jobs persist the state that you wish to persist, and explicitly cleanup any state you wish to not persist across CI jobs.
  • A given host volume bind-mounted into a system container’s /var/lib/docker must only be mounted on a single system container at any given time. This is a restriction imposed by the inner Docker daemon, which does not allow its image cache to be shared concurrently among multiple daemon instances. Sysbox will check for violations of this rule and report an appropriate error during system container creation.

Conclusion

Founder and CEO of Nestybox, Inc.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

More project ideas to improve your coding skills

from fit6 http://ift.tt/2k5vKC8 via alanafalk.jimdo.com

Using Azure Key Vault with Power Automate Desktop

Screenshot of all actions in the flow

Decision Optimization Generally Available in Watson Studio Notebooks

Made with only the finest ingredients

Testing the commercial advantages to Beame’s XR telepresence running on Verizon 5G Edge with AWS…

Sticker pack

Difference between JDK, JRE, JVM | Java in Urdu

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Cesar Talledo

Cesar Talledo

Founder and CEO of Nestybox, Inc.

More from Medium

Accessing Docker tag as an environment variable inside a Docker container project

Keycloak Admin REST API

Deploy MongoDB on OpenShift using Helm

Deploying Maven package and Docker image to Github Registry using Github Action workflow