My Terraform AWS journey — HashiTimes Interview
I’d like to share an extended interview which I gave to HashiTimes (newsletter curated by the community and not affiliated with HashiCorp) in June 2019. I added some more additional links and information.
In this week’s HashiTimes Interview, we sat down with Anton Babenko, a Developer, Public Speaker, and Evangelist based in Oslo, Norway.
Tell us a little about yourself, your background around HashiCorp, and what do you do professionally?
I started working with Terraform and Packer in 2015. In December 2015, I gave a talk about how to use Terraform with AWS at a local AWS User Group meetup in Oslo because I wanted to share how awesome Terraform was (and still is). You can see a video recording here.
Around the same time, I requested access to push my first Terraform module and got access to manage all other repositories under terraform-community-modules organization. This was the starting point for my open-source participation within Terraform. I got full access, and soon I started answering questions about other modules even without using them myself. I was a little uncertain if I was doing it right by assuming positive intent, and soon, I realized that other contributors liked the way I act and joined the organization and submitted their modules.
Fast-forward to 2017, I cleaned up existing modules, added many missing features and rewrote the most popular AWS modules including VPC, RDS, Autoscaling, ELB, EC2 instance, and EC2 security group. These modules went live as verified Terraform AWS modules during HashiConf 2017 in Austin together with the announcement of the Terraform Registry. Right after the announcement at HashiConf during a break, some people were joking “why the VPC module is such a monster and creates so much…”. Well, because this is a powerful infrastructure module…
Since then I maintain terraform-aws-modules used by hundreds of companies worldwide, reviewing code, answering questions, adding new features, collecting use-cases from users, giving talks and workshops. There were more than 1K of pull-requests and issues open & closed in all modules I maintain, and more than 4 million downloads since the release of the Terraform Registry.Terraform AWS modules are now 0.12 ready!
In June 2019 Amazon recognized my work in the community and called me an AWS Community Hero. It took me several years of doing the things I enjoy.
Today, for the living I provide consulting services and workshops for companies who want to use Terraform and AWS. Most of my customers are using open-source modules and tools, as well as contribute back to open-source (few Terraform modules and projects were released like that, few more — in progress).
Anton, you have been a very active member in the HashiCorp Dev Community with some of your open source projects, Terraform Modules, and other tools available today. Can you share with the readers what modules.tf does, terraform docs as pdf and how these tools can enable them as Developers?
Terraform AWS modules were the starting point. Almost immediately after I started maintaining a couple of them (back in 2016) I have been looking into tools which could simplify the review, update, test, release process, and allow to scale up (in quality and in complexity). My ultimate goal is to make a system which will handle hundreds of high-quality modules with fewer efforts on the developer’s side. The first prototype called “terrapin” (see source code) I made in 2017. It was good and working as I wanted, but a bit too over-engineered according to myself after I came back from vacation and looked at it again soberly.
I like to make things which can impact developers or just fun to work on, learn from it, and go on to the next.
At some point, when I already realized that modules in Terraform are the building blocks of infrastructure, I wanted to figure out how to simplify infrastructure as code more and how to make it available for a wider group of people. I bought a start-up ticket to Web Summit 2017 in Lisbon, made a website to illustrate that infrastructure code can be generated by completing easy forms and stayed there for one day pitching my ideas and asking potential users. It turned out that many people didn’t understand the point, they didn’t have the problem I anticipated. They liked nice diagrams created using cloudcraft.co for AWS infrastructure. They liked visual, and I wanted to push Terraform. This was the beginning of modules.tf in the way it is now. Modules.tf — is an open-source project which allows converting visual diagrams created with cloudcraft.co into infrastructure configurations using Terraform.
As a developer myself I wanted users of modules.tf to think about it as an opinionated (yet, powerful) starting point for AWS projects. The project is still early in the development phase. The source code is available here. It is using Serverless Framework with Python 3 and running on AWS Lambda.
While working on modules.tf I needed to connect values in tfvars defined in different directories/stacks/layers dynamically so that outputs of one terraform module can be passed to another terraform module transparently. “Update values in terraform.tfvars using annotations” — this is tfvars-annotations is about. It is currently not complete but already usable for my cases in modules.tf. I also wanted to learn Go language better by doing something small and real.
In 2016, I released pre-commit-terraform — a collection of pre-commit git hooks to take care of Terraform configurations. Those hooks can automatically format, validate, generate documentation for Terraform code. I am glad to see more and more developers are using it in their work.
During my work, I develop various solutions and tools which I prefer to share with the hope that someone will find it useful and begin to cooperate (open issue, give feedback, write code, update documentation, etc). Often people find them useful because it saves them time and makes them more productive. I think this is the reason why some people appreciate “Terraform docs as pdf” I made after I didn’t have Terraform documentation during the cross-Atlantic flight.
After being in the community for some time I’ve gathered enough do and don’ts from real projects and use-cases, so I published them as Terraform Best Practices (source code). Some things are incomplete there, and people still ask me “how to structure my infrastructure as code?” even after I pointed them to this website. For me, this means that I need to find time to update it significantly.
Another piece of educational content I have created recently is — Terraform best practices workshop materials. I run various Terraform AWS workshops on-site for companies and at community events like DevOpsDays and AWS Community Summits.
In May 2019, I decided to share my knowledge regarding Terragrunt and published real code with some of the important points highlighted as Terragrunt Reference Architecture. I believe this will be relevant to people who want to write clean code and use Terraform modules in the most efficient ways.
You were instrumental in launching the HashiCorp User Groups program along with Kate Reese @ HashiCorp. For those in the community who have not attended an EMEA region HUG before, could you share the success you have seen in the EMEA region growing the HUG community?
There are more than 30 HashiCorp user groups in EMEA with several thousands of active members. The number of meetups and quality of talks is increasing. Few organizers are trying out different formats for their events and aiming meetups for different audiences.
The HUG in Hamburg, for example, there was a workshop with a dedicated group of people per product. Some HUGs are mixing advanced and beginner talks during one meetup to satisfy a wider audience. HUG in Zürich is doing a great job by having at least 2–4 talks during a meetup, while most meetups have one talk because it is hard to find good speakers.
When I started the HUG in Oslo, I wanted to have meetups about HashiCorp products and have speakers share their real stories and challenges. I’ve been co-organizing AWS User Group in Oslo since 2015 and already knew that it was very helpful and interesting to learn & share experiences with like-minded individuals. To make it more interesting, I decided to have meetups where attendees can learn about real and advanced use-cases. I don’t want us to repeat official documentation and well-known information which can be found easily online. In general, I want to avoid impractical talks.
Being a smaller community is actually good. The amount of generic DevOps meetups is decreasing, and attendees prefer to have specialization by product and by technologies (CNCF, K8S, Serverless) where they can dive deeper. Additionally, HUG meetups are often relevant to AWS or Google Cloud user groups as long as they touch those services. Win-win, and cross-links :)
In 2018, I proposed to HashiCorp a new format for the event where anyone in the HashiCorp community around the globe can participate and share whatever is interesting for the HUG community worldwide. Each speaker had 30 minutes; the whole event was virtual and lasted 24 hours to cover all time zones. This way HashiCorp All-Day HashiTalks was organized in February 2019.
Finally, If you are still reading this and you want to organize HashiCorp User Group in your area, please contact Katie Reese to help you get started.
Finally, how do you believe the HashiCorp Community plays a part in the overall success of HashiCorp as a Company?
The HashiCorp community is definitely playing a significant role in company success. Open-source versions of their products are driving most of the roadmaps of the individual projects. This is why I always recommend people to open issues if they found bugs or want some feature requests to be considered.
The community is also growing pretty nicely with lots of tools being created to integrate and extend the functionality provided by HashiCorp tools, as well as lots of Slack groups (eg, SweetOps and hangops) where a couple of thousand users are online daily. Some of the tools are being updated to support Terraform 0.12 and HCL2 syntax, while some people in the community discover short-term hacks to adopt Terraform 0.12 and its pretty long list of features as soon as possible. The community is certainly busier now more a month ago :)
PS: I’ve recently studied Amazon Leadership Principles and realized that I apply these principles on a daily basis: “Invent And Simplify”, “Customer Obsession”, “Ownership”, “Are Right, A Lot”, “Learn and Be Curious”, “Insist on the Highest Standards”, “Think Big”, “Bias for Action”, “Frugality”, “Earn Trust”, “Dive Deep”, “Have Backbone; Disagree and Commit”, “Deliver Results”. It didn’t give me a job at AWS if you wonder. :)
Please share this article on your social networks and feel free to connect with me: