In a recent blog post I talked about automating the testing of an advanced GopherJS library using a combination of QUnit, Ginkgo and Agouti. That allowed me to run a complete integration test suite against my library by automatically spinning up browsers and pointing them at my QUnit GopherJS page. This was a great start, but after running it a couple of times we find that there are several problems:
- It requires all browsers tested to be installed on the machine running the tests. This limits the scale of testing as no-one is going to be running around with Chrome, Firefox, Safari, Edge and whatever other browser we may want to test with. Especially not different versions of said browsers!
- It’s still a manual step. Wouldn’t it be great if these tests could be run on every pull request and every push to the master branch?
To solve these problems I spent some time figuring out how to use the CircleCI platform to automate my setup. I knew it wouldn’t be trivial as there are many different dependencies in the different parts of the protobuf repository. Building the GopherJS protoc plugin requires Go. Generating the well known types requires protoc and the GopherJS protoc plugin to both be installed. The integration tests require Firefox and Chrome to be installed on the machine running the tests.
Fortunately CircleCI 2.0 embraces Docker as a testing sandbox. At work we use GitLab whose CI solution is similarly centered around docker (though not exclusively!), so I was reasonable well versed in the quirks around using Docker in CI. The layout of the CircleCI configuration is quite different from the one used by GitLab, so it took me a while to get up to speed.
The integration test
In order to fix the issues I mentioned at the start of this post, I decided to rewrite
my tests to be runnable in docker containers, with the browsers running in separate
containers. As such, I wrote a
docker-compose file to define the setup required.
Here’s the docker-compose file I used:
version: '2' networks: internal: driver: bridge services: chromedriver: environment: CHROMEDRIVER_WHITELISTED_IPS: "" expose: - "4444" image: robcherry/docker-chromedriver:latest networks: - internal privileged: true selenium: expose: - "4444" image: selenium/standalone-firefox:latest networks: - internal testrunner: # This just sleeps, we execute the test command using docker-compose exec. container_name: testrunner command: sleep infinity depends_on: - chromedriver - selenium environment: CHROMEDRIVER_ADDR: chromedriver:4444 SELENIUM_ADDR: selenium:4444 GOPHERJS_SERVER_ADDR: testrunner:8080 expose: - "8080" - "9090" - "9095" - "9100" - "9105" image: golang:latest networks: - internal
It starts a ChromeDriver, a Selenium and a standard Go container.
This is easy enough to run on your own machine, with
docker-compose exec. I wish I could use
docker-compose run here, but because of
an old bug that seems to have been forgotten,
we’re forced to use
exec. This is how we run this:
bash -c "\ set -x \ trap '\ docker-compose logs selenium && \ docker-compose logs chromedriver && \ docker-compose down' EXIT; \ docker-compose up -d && \ docker-compose exec -T testrunner bash -c '\ mkdir -p /go/src/github.com/johanbrandhorst/protobuf/' && \ docker cp ./ testrunner:/go/src/github.com/johanbrandhorst/protobuf/ && \ docker-compose exec -T testrunner bash -c '\ cd /go/src/github.com/johanbrandhorst/protobuf && \ go install ./vendor/github.com/onsi/ginkgo/ginkgo && \ cd test && make test' \ "
Let me preface this by saying that I know this looks absolutely rubbish, but I really wanted the trap and I also wanted this in a Makefile so I could run it easily from my terminal, and I also did not want to have to write a separate shell script for it. This works, and if you squint and ignore the backslashes and ampersands you can kind of see what I was going for.
Something worth noting here is the use of
docker cp to get the contents of the repository
into the testrunner container. This is a workaround to a well known shortcoming of
running docker containers from other docker containers, namely that
you can’t volume mount from one docker container to another one.
So now that we’ve got a docker-compose file and a hacky bash line to run it, what is required to get it running on CircleCI? Look no further:
tests: docker: - image: ypereirareis/docker-compose working_directory: /go/src/github.com/johanbrandhorst/protobuf steps: - checkout - setup_remote_docker - run: name: Browser Integration Tests command: make integration
ypereirareis/docker-compose which just comes with
The magic here is using
setup_remote_docker to do just that. This is required to get
access to docker powers from within your CircleCI containers.
checkout job tells CircleCI that we want the git repo cloned into the
Bam! Just like that we’ve got our GopherJS integration tests up and running on CircleCI! The coolest thing here is that now in the future if we want to test other versions of browsers, or other browsers altogether, we need only add another docker container to the fray.
We’ve shown that you can run complex testing scenarios involving several interdependent docker containers on CircleCI. These tests run automatically on PRs and commits made to the repository to preserve confidence in the functionality of the code. You can check out the CI in action on my github repo.
If you enjoyed this blog post, have any questions or input,
don’t hesitate to contact me on
jbrandhorst on the Gophers Slack. I’d love to hear