Case: Event-driven automation with Kafka and Ansible
The client had an application that needed to create and modify infrastructure dynamically based on Kafka messages while retaining change history. We implemented the infrastructure management part with Git and Ansible. The system created and managed resources in AWS, Keycloak and on virtual machines.
Main technologies
Ansible
Kafka
Keycloak
Kafka
Keycloak
Benefits
User-facing applications could trigger controlled infrastructure modifications
Removal of manual steps
Removal of manual steps
Numbers
82 Ansible tasks
5 new Ansible plugins developed
3 deployments
5 new Ansible plugins developed
3 deployments
1 Starting point
The client wanted to provide a new self-service capability for their own clients. Under the hood this new feature required creating, modifying and deleting infrastructure on demand. So far such changes had been done in a static way with infrastructure as code tools which was not dynamic enough.
2 Project
We started the project by creating a Vagrant-based development environment for the Ansible code that was required in the project. Once the code was mostly functional we spun up a staging environment in AWS EC2 for testing the automation code in a production-like environment where we could process messages from Kafka and do integrations with AWS and Keycloak. As the final step infrastructure automation code was deployed to production.
3 End result
At the end of the project the steps that were done manually earlier were automatically handled by infrastructure automation. The tooling created a Git commit for each change so that changes could not be introduced without leaving a trace. Ansible did not manage just the individual changes but all of the affected infrastructure. This ensured that configuration divergences got fixed automatically.
"Puppeteers helped us resolve our Red Hat Enterprise Linux issue. I'm looking forward to upgrading and improving our clients' production environments and our development setups with their help."
Aarre Pohjola
A&A Consulting Oy, Finland
