Case: Containerized Buildbot CI/CD system
The client used Buildbot to ensure that their software builds cleanly on more than dozen operating systems, including all major Linux and *BSD distributions and Windows. The old Buildbot environment was getting very outdated, so we rebuilt it from scratch. The new setup was designed around Docker containers that Buildbot spun up on demand at build time.
Main technologies
Buildbot
Docker
Docker
Benefits
Portability
Scalability
Expanded capabilities
Scalability
Expanded capabilities
Numbers
21 operating systems supported
5 target applications
481 build permutations
5 target applications
481 build permutations
1 Starting point
The client had an old virtual machine-based Buildbot CI/CD system. It worked well but did not track all the applications and libraries the client was developing. Moreover, it was hard to extend beyond its initial, limited design.
2 Project
Wide operating system coverage was very important for the client, so Buildbot continued to be a very good choice. So, we redesigned the Buildbot environment on several levels. First, data ("what to build") and code ("how to execute builds") was separated. This easily allowed building of all the applications and libraries the client required and defining build permutations flexibly. The virtual machine-based workers were also converted to Docker workers launched on demand.
3 End result
At the end of the project there were two main Buildbot instances - one for production and one for staging. Developers could create private Buildbot instances for testing and development purposes on demand. Each Buildbot could be configured differently to support multiple use cases. Every commit to a tracked application launched 20-400 builds across a wide range of containerized operating systems. Some of those builds triggered a full integration test suite, whereas some built Debian, Ubuntu or MSI packages.
"Puppeteers helped us resolve our Red Hat Enterprise Linux issue. I'm looking forward to upgrading and improving our clients' production environments and our development setups with their help."
Aarre Pohjola
A&A Consulting Oy, Finland
