Skip to content

Continuous delivery for free using Docker, CircleCI and Heroku

Posted in .NET Core, Building future-proof software, and Tutorials

Continuous what?

Continuous delivery. You may recall that in my previous post I announced that today’s entry would be revolving around continuous integration. And technically it can count as such since we will cover continuous integration along the next step. That next step is continuous delivery. If you are not familiar with these terms and the concepts behind them I will sum them up briefly.

Basically, continuous integration allows verifying that your codebase still builds and passes tests passing whenever you push changes. Add a trigger to deploy your code to production upon success and you pretty much have the idea around continuous delivery.

These practices help mainly to make sure that you don’t break your codebase when pushing changes. This is good when you work alone but a lifesaver when working in a team. You cannot imagine how many hours I wasted mostly during my studying years because of coding breaking without us realizing before days. Using source control was already a miracle in itself at a time when there limited options for continuous integration, especially for students. If you want more details about source control workflows the GitHub Flow is a great place to start.

Let’s just jump into it!

Back in today’s topic, continuous delivery. Before I start inundating you with scripts and screen captures you need to be familiar with a few things:

Retrieving the code

Since you read my previous tutorial, you should know more or less what the code does. It is the classic Values API sample returning an array with two values “value1” and “value2”. From there, the easiest step is to fork the repository created from that previous post which you can reach by clicking here.

fork that repo
Fork that repo, yep that one. Just do it!

Once the fork completed you will have an exact copy of my repository where you can push changes for the rest of the tutorial. If you have not yet, you need to clone your fork to your machine for the next stage.

Getting acquainted with Docker

Docker is going to be key for today’s tutorial. Why? I hear you ask. Because CircleCI does not support C# for continuous integration. Neither does Heroku for deployment, at least not officially but we’ll get back to that later. But do you know what is supported by both that we can use? Docker container images.

Basically consider a Docker container image as a box in where you put everything your software needs to run properly, from code to settings to system tools and libraries. A containerized software will always run the same way regardless of the environment. It will be completely isolated from its surroundings. The cool thing about this? Well it works on any environment, whether you run it on Windows, Mac and Linux. It is true as long as your computer supports VT-d virtualization. Then, you can make sure your container behaves as you expect locally before deploying. This should be the case if your device is no more than a couple years old. Also note that if you cannot run Docker on your local machine, you can still commit the docker files and it will work on CircleCI.

First things first, you will need to install Docker Community Edition which is free and available at this link. The installation is pretty straightforward so nothing special to mention here. If Docker is not supported on your machine you will get a message when trying to install it on Windows. The same should happen if you try to run it on Mac. If it is the case, don’t worry, you can still go through the tutorial and won’t be missing that much.

Making our tests ready to work on CircleCI

As mentioned previously, if we try to build our API straight away on CircleCI it will fail. Not because the code does not work but because it is not supported. In order to get our tests running, we will have them run in a containerized way. We don’t need to create a container image yet, only to get an existing container image that will support running them.

The first thing to do is to create docker compose file that will allow getting an image that supports running .NET Core 2 applications and run our tests inside of that image. Now you will copy a file definition that will do exactly that upon using the docker-compose command. You need to create a file named docker-compose.unittests.yml at the root of your repository.  Once it’s done, copy into it the contents of the code below:

version: '3'
services:
  unittests:
    image: microsoft/dotnet:2.1-sdk
    volumes:
      - .:/code
    command:
      - /bin/bash
      - /code/docker-run-unittests.sh

Now we need to write the script that will allow our continuous integration tool to restore the solution within the container image. After what the tests will be run. Here is the script to copy inside a file named `docker-run-unittests.sh` still at the root of your solution:

You may notice a line that is unusual to most people. The command `set -eu -o pipefail`. A short and stupid explanation is to say that it halts and makes the build process fail if an error occurs. If your build does not compile or that tests fail, that command will allow the `docker-compose` command to fail which will trigger an error and allow your CI system to know it failed.

Now that we have our tests ready to run within a container we will run them locally to make sure we’re all set. In order to do so, you will need to run the following command with your favourite terminal. This assumes that you are in your solution folder and that you can run Docker commands on your machine.

docker-compose -f docker-compose.unittests.yml run --rm unittests

Running that command will give you an output similar to this:

Docker test run result

We are now able to run tests on any environment supporting Docker. Let’s now setup our continuous integration tool.

Continuous integration

Configuring for CircleCI

Now that we have all the Docker configuration ready to run tests, we can configure our project to have our continuous integration on CircleCI. The first thing to do here is to create a .circleci folder in your solution folder. Then, you will create a config.yml file inside of it so that its relative path to your solution is .circleci/config.yml. Into that file you will copy these contents:

version: 2
jobs:
 build:
   machine: true
   steps:
     - checkout

     # test
     - run: |
         docker-compose -f docker-compose.unittests.yml run --rm unittests

Commit and push your changes, then move onto the next section.

Setting up CircleCI

CircleCI is a platform used for continuous integration and continuous delivery. I picked it for today’s post because it’s free and can be good if you just want to play around. Also, it can be great if you are creating a new business and want to keep the costs low before scaling up.

The first step here is to create an account. You can reach their signup page by clicking here. Once there you should see this screen:

Now you need to press “Sign Up with GitHub” to create your CircleCI account. This will land you on a page where GitHub will ask you if you want to grant CircleCI various permissions. As you will see below it will require your email address(es) and repository access rights.

Press “Authorize cicleci” to move onto the next step. Now you will see a welcome screen as below.

If you noticed the arrow and the red not circle you know where to click next. If not, press “Add projects”. You will see the forked repository name appear. Next to it, you will notice a “Setup project” link, press it.

No red mark to show where to click this time

Now you have pressed the right link you should see the project setup screen. You can leave the operating system as Linux and select “Other” as language.

Once you’ve done that a feedback box will appear asking what language you intend to use. I suppose it is to prioritise what they should add next to their roadmap. Don’t feel obliged to put C# as it might make the unit testing part of this post obsolete. Which I wouldn’t mind much because then I can update this post to avoid the build & test magic you were introduced to previously.

Next, scrolling down you should see a set of instructions to get the build to run but we already took care of that.

Now you can press “Start building” which will send you to your first build screen. Your build might be queuing for a few seconds before starting as below:

Your first Docker powered CircleCI build

In our case, there is not much going on apart from the test run so after up to a couple minutes you should get your successful build.

Successful build circleci
Build passing CI on the first try? A win in my book

Now that our CI tool is ready to build and validate our software, it’s time to prepare for deployment.

Time to deploy that API

Deployment over 9000 with Heroku

Heroku is a platform allowing developers to deploy, manage and scale web apps. They support most of the modern technologies and languages such as Node.js, Java, Go and many more. However, they do not officially support .NET Core even though they allow for extensions from Github (or buildpacks) to have some sort of support. But today we are not going to do that.

The first thing you will need to do now is creating an account. You can do so by clicking here. Once your account created, you will see a screen prompting you to create a new app.

heroku create app screen
Easy

Now you can press “Create New App”, and you will be asked to pick a name and region. For this tutorial, the region does not matter and you can pick any name you like.

heroku create app

From here, press “Create app” to create your app and access its dashboard.

heroku new app dashboard

Now that the app is ready to receive our API deployment, you need to get your Heroku API key so that we can deploy our code to Heroku from CircleCI. In order to do so, you will have to access your Heroku settings. To get there, click on your profile icon (top right of the screen), you should see this menu pop up.

Heroku profile menu

Next, click “Account settings”. Once on the settings page scroll down until you see this:

heroku api key

Finally, press “Reveal” to display your API key and save it somewhere close, for that we will use it soon.

Creating our own docker image to run the API

Here we are, the time where we create our own (maybe your first) Docker image. The first step is to create our Dockerfile in the project folder.

FROM microsoft/dotnet:2.1-sdk AS build-env
WORKDIR /app

# Copy csproj and restore as distinct layers
COPY *.csproj ./
RUN dotnet restore

# Copy everything else and build
COPY . ./
RUN dotnet publish -c Release -o out

# Build runtime image
FROM microsoft/dotnet:2.1-sdk
WORKDIR /app
COPY --from=build-env /app/out .
CMD ASPNETCORE_URLS=http://*:$PORT dotnet DotNetCoreSampleApi.dll

Our Dockerfile is pretty standard here, it generates an environment allowing to compile build and run .NET Core apps. Then, it restores our project and publishes it locally to eventually run it using port number passed by Docker.

Now that our Dockerfile is ready to go, we will add a .dockerignore file that is a list of files/folders we want Docker to ignore. In our case, we want to make our build context as small as possible so we will ignore binaries as you can see below:

bin\
obj\

Once the file created, if you can run Docker locally, you may run the following commands to make sure your setup is valid:

docker build -t aspnetapp DotNetCoreSampleApi

Yet again, if you cannot run Docker locally, you will see the results on CircleCI later.

Updating the CircleCI config to set up our continuous delivery

We are almost there! It is time to put the delivery in continuous delivery. Now that we have our Docker image configuration ready, we can finalize our CircleCI configuration. Before editing our configuration file will need to add our Heroku credentials to the project environment variables. In order to do so, go back to your dashboard. From there, press your build’s settings button, it should look like this:

Then, click “Environment Variables” and add the email address you registered with on Heroku as HEROKU_USERNAME. Afterwards, add your Heroku API key as HEROKU_API_KEY. Finally, add your Heroku app name as HEROKU_APP_NAME.

After adding the variables, we can now update our CircleCI configuration file with the deployment steps. This is what it should look like now.

version: 2
jobs:
 build:
   machine: true
   steps:
     - checkout

     # test
     - run: |
         docker-compose -f docker-compose.unittests.yml run --rm unittests
         
     # build image
     - run: |
         docker info
         docker build -t aspnetapp DotNetCoreSampleApi

     # deploy the image
     - run: |
         set -eu -o pipefail
         echo $HEROKU_API_KEY | docker login --username=$HEROKU_USERNAME --password-stdin registry.heroku.com
         docker tag aspnetapp registry.heroku.com/$HEROKU_APP_NAME/web
         docker push registry.heroku.com/$HEROKU_APP_NAME/web
         docker inspect registry.heroku.com/$HEROKU_APP_NAME/web --format={{.Id}} > WEB_DOCKER_IMAGE_ID_FILE
         export WEB_DOCKER_IMAGE_ID=$(cat WEB_DOCKER_IMAGE_ID_FILE)
         curl -n -X PATCH https://api.heroku.com/apps/$HEROKU_APP_NAME/formation \
            -d '{ "updates": [ { "type": "web", "docker_image": "'"$WEB_DOCKER_IMAGE_ID"'" }]}' \
          -H "Content-Type: application/json" \
          -H "Accept: application/vnd.heroku+json; version=3.docker-releases" \
          -H "Authorization: Bearer $HEROKU_API_KEY"

Basically, what we do in that file is building our Docker image then authenticating to Heroku to eventually push our image to Heroku’s container registry. Now it is time to commit and push our changes for the last time. If you go back to CircleCI, you should see your build was successful.

continuous delivery all green
Continuous delivery in action, simply beautiful

Now, if you go to your Heroku app using https://<your-app-name>.herokuapp.com/api/values, you will see the following result.

Continuous delivery ✓✓✓

Congratulations! You are now smarter than 30 minutes ago! Not only you know how to setup continuous delivery using CircleCI and Heroku but you can build a Docker container image. If you missed anything, don’t hesitate to check the source code there.

What your solution folder structure should look like now.

Note that the sake of brevity, I chose to put all the commands in the CircleCI build job. Also I did not put any condition on which branch gets deployed, which is a check that you should always have to avoid publishing a test build to production. In the case of continuous delivery, pushing code to the dev branch should trigger a deployment to the development environment. Pushing code to master should trigger a deployment to production and so on. You can figure how to do this using condition-based instructions and the deployment job here.

Based on your feedback I may write a quick guide on setting up CI for multiple environments using this post as a basis. Since I have a few other things in the pipeline for the next few months it might not happen before a while.

Thanks again for reading, if it was any use to you don’t hesitate to share and subscribe to get more of these. The next future-proof entry should be about what you can do to avoid your continuous delivery to turn into this:

2018-11-07 UPDATE

On May 15th this year, pushing a docker image to Heroku registry no longer creates a release. I updated the post to include a curl PATCH call to use the pushed image a release via the Heroku Platform API as advised.

Also on May 20th, I really should have paid attention during these days, .NET 2.1 came out. I updated the various docker files to use that version instead due to some vulnerabilities on .NET Core dependencies.

Last not the least, some chunks of code just went missing from the post. Not sure for how long nor how but they were just gone. I put them back with the aforementioned updates. I really need to be a little more alert on these things.

Enjoy your now restored continuous delivery post!

Be First to Comment

    Leave a Reply

    This site uses Akismet to reduce spam. Learn how your comment data is processed.

    %d bloggers like this: