CI/CD in your local dev

Everyone should be able to run their CI/CD pipeline locally. Having to commit & push in order to test a change is a slow way to iterate on a pipeline. This guide shows you the Dagger way. Within 5 minutes, you will have a local CI/CD loop and run your first test & build pipeline.

macOS

We assume that you have Homebrew installed. If you do, you can install dagger with a single command:

brew install dagger/tap/dagger

Check that dagger installed correctly by verifying its execution path:

type dagger
dagger is /opt/homebrew/bin/dagger

Before we can build & test our example app with dagger, we need to have Docker running. You most likely already have Docker set up. If not, Docker Desktop makes this easy. With Docker running, we are ready to download our example app and run its CI/CD pipeline:

git clone https://github.com/dagger/dagger
cd dagger
git checkout v0.2.0

cd pkg/universe.dagger.io/examples/todoapp

With everything in place, we run the CI/CD pipeline locally:

dagger do build

With an empty cache, installing all dependencies, then testing & generating a build for this example app completes in just under 3 minutes:

[✔] client.filesystem.".".read                                    0.1s
[✔] actions.deps                                                118.8s
[✔] actions.test.script                                           0.1s
[✔] actions.test                                                  6.3s
[✔] actions.build.run.script                                      0.0s
[✔] actions.build.run                                            43.7s
[✔] actions.build.contents                                        0.4s
[✔] client.filesystem.build.write                                 0.1s

Since this is a static application, we can open the files which are generated in actions.build.contents in a browser. The last step copies the build result into the build directory on the host. On macOS, we run open build/index.html in our terminal and see the following app preview:

One of the big advantages to this approach is that we did not have to install any dependencies specific to this application. Dagger managed all the intermediary steps, and we ended up with the end-result on our host, without any of the transient dependencies.

Now that we have everything running locally, let us make a change and get a feel for our local CI/CD loop. The quicker we can close this loop, the quicker we can learn what actually works. With Dagger, we can close this loop locally, without committing and pushing our changes. And since every action is cached, subsequent runs will be quicker.

In the todoapp directory, edit line 25 of src/components/Form.js and save the file.

I change this line to What must be done today? and run the build locally again:

dagger do build

[✔] client.filesystem.".".read                                    0.0s
[✔] actions.deps                                                  7.5s
[✔] actions.test.script                                           0.0s
[✔] actions.test                                                  6.0s
[✔] actions.build.run.script                                      0.0s
[✔] actions.build.run                                            29.2s
[✔] actions.build.contents                                        0.0s
[✔] client.filesystem.build.write                                 0.1s

The total 42.8 time is macOS specific, since the Linux alternative is more than 8x quicker. Either way, this local test & build loop is likely to change our approach to iterating on changes. It becomes even more obvious when the change is not as straightforward as knowing exactly which line to edit.

Linux

Download the latest 64bit Linux binary release. We support both x86 & ARM architectures.

We install dagger in /usr/local/bin, but anywhere in your PATH works:

type dagger
dagger is /usr/local/bin/dagger

Before we can build, test & deploy our example app with dagger, we need to have Docker Engine running. You most likely already have Docker Engine set up. If not, Docker on Linux install makes this easy. With Docker Engine running, we are ready to download our example app and run its CI/CD pipeline:

git clone https://github.com/dagger/dagger
cd dagger
git checkout v0.2.0

cd pkg/universe.dagger.io/examples/todoapp

With everything in place, we run the CI/CD pipeline locally:

dagger do build

With an empty cache, installing all dependencies, then testing & generating a build for this example app completes in just under 1 minute:

[✔] client.filesystem.".".read                                    0.3s
[✔] actions.deps                                                 39.7s
[✔] actions.test.script                                           0.2s
[✔] actions.test                                                  1.9s
[✔] actions.build.run.script                                      0.1s
[✔] actions.build.run                                            10.0s
[✔] actions.build.contents                                        0.6s
[✔] client.filesystem.build.write                                 0.1s

Since this is a static application, we can open the files which are generated in actions.build.contents in a browser. The last step copies the build result into the build directory on the host. On Linux, we run xdg-open build/index.html in our terminal and see the following app preview:

One of the big advantages to this approach is that we did not have to install any dependencies specific to this application. Dagger managed all the intermediary steps, and we ended up with the end-result on our host, without any of the transient dependencies.

Now that we have everything running locally, let us make a change and get a feel for our local CI/CD loop. The quicker we can close this loop, the quicker we can learn what actually works. With Dagger, we can close this loop locally, without committing and pushing our changes. And since every action is cached, subsequent runs will be quicker.

In the todoapp directory, edit line 25 of src/components/Form.js and save the file.

I change this line to What must be done today? and run the build locally again:

dagger do build

[✔] client.filesystem.".".read                                    0.0s
[✔] actions.deps                                                  1.1s
[✔] actions.test.script                                           0.0s
[✔] actions.test                                                  0.0s
[✔] actions.build.run.script                                      0.8s
[✔] actions.build.run                                             2.9s
[✔] actions.build.contents                                        0.0s
[✔] client.filesystem.build.write                                 0.0s

Being able to re-run the test & build loop locally in 4.8s, at the same speed as running yarn scripts locally and without adding any extra dependencies to our host, is likely to change our approach to iterating on changes. It becomes even more obvious when the change is not as straightforward as knowing exactly which line to edit.

Windows

We assume that you have curl installed. If you do, you can install dagger with a single command. From a powershell terminal, run the following command:

curl https://releases.dagger.io/dagger/install.ps1 -OutFile install.ps1 ; ./install.ps1; rm install.ps1

We try to move the dagger binary under C:\Windows\System32 but in case we miss the necessary permissions, we'll save everything under <your home folder>/dagger

Check that dagger is installed correctly by opening a Command Prompt terminal and run:

where dagger
C:\<your home folder>\dagger.exe

Before we can build & test our example app with dagger, we need to have Docker running. You most likely already have Docker set up. If not, Docker Desktop makes this easy. With Docker running, we are ready to download our example app and run its CI/CD pipeline. Still in your Command Prompt terminal:

git clone https://github.com/dagger/dagger
cd dagger
git checkout v0.2.0

cd pkg/universe.dagger.io/examples/todoapp

With everything in place, we run the CI/CD pipeline locally:

dagger do build

With an empty cache, installing all dependencies, then testing & generating a build for this example app completes in just under a minute:

[✔] actions.deps                                                 62.1s
[✔] actions.build.run.script                                      0.4s
[✔] actions.test.script                                           0.5s
[✔] client.filesystem.".".read                                    0.6s
[✔] actions.test                                                  2.0s
[✔] actions.build.run                                            12.4s
[✔] actions.build.contents                                        0.1s
[✔] client.filesystem.build.write                                 0.2s
[✔] client.filesystem.".".read                                    0.2s

Since this is a static application, we can open the files which are generated in actions.build.contents in a browser. The last step copies the build result into the build directory on the host. On Windows, we run start build/index.html in our Command Prompt terminal and see the following app preview:

One of the big advantages to this approach is that we did not have to install any dependencies specific to this application. Dagger managed all the intermediary steps, and we ended up with the end-result on our host, without any of the transient dependencies.

Now that we have everything running locally, let us make a change and get a feel for our local CI/CD loop. The quicker we can close this loop, the quicker we can learn what actually works. With Dagger, we can close this loop locally, without committing and pushing our changes. And since every action is cached, subsequent runs will be quicker.

In the todoapp directory, edit line 25 of src/components/Form.js and save the file.

I change this line to What must be done today? and run the build locally again:

dagger do build
INF upgrading buildkit    have host network=true version=v0.10.0
[✔] actions.build.run.script                                     0.0s
[✔] actions.deps                                                 3.4s
[✔] client.filesystem.".".read                                   0.2s
[✔] actions.test.script                                          0.0s
[✔] actions.test                                                 1.8s
[✔] actions.build.run                                            7.7s
[✔] actions.build.contents                                       0.2s
[✔] client.filesystem.build.write                                0.2s
[✔] client.filesystem.".".read                                   0.1s

Being able to re-run the test & build loop locally in 13.6s, without adding any extra dependencies to our host, is likely to change our approach to iterating on changes. It becomes even more obvious when the change is not as straightforward as knowing exactly which line to edit.

tip

Now that we are comfortable with our local CI/CD loop, let us configure a remote CI environment in the second part. The difference is that we will also deploy the build output to Netlify. Dagger makes this easy.