HashiCorp Terraform enables you to safely and predictably create, change, and improve infrastructure. It is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.

Aiven has an officially supported Terraform provider, which allows you to manage all Aiven resources. The provider's Github pages have more detailed documentation for the various resources and resource attributes that are supported. Please refer to the README.md file there for additional details after reading through this help article.

Getting started

To get started with Terraform you need to do the following steps:

  1. Install Go: https://golang.org/doc/install (Note: instead of installing Go and building Terraform and Aiven from source code you can also get Terraform binaries from their web site and Aiven binaries for supported operating systems from Aiven Terraform provider releases page [make sure to remove the -{os}_{arch} part of the file names])
  2. Rest of the commands are to be executed in shell so open your shell
  3. Install Terraform: go get -u github.com/hashicorp/terraform 
  4. Install Aiven Terraform provider: go get -u github.com/aiven/terraform-provider-aiven 
  5. Make sure ~/go/bin  is in PATH : export PATH=$PATH:~/go/bin 
  6. Navigate to the directory with your Terraform scripts or create that directory as well as the scripts. See below for more info.
  7. Define API token, project name, card ID and other important properties. See below for more details.
  8. Run terraform init in the directory containing your Terraform scripts to initialize Terraform for your project.
  9. Run terraform apply in the directory containing your Terraform scripts to apply the configuration defined in your scripts. If you're importing existing resources you should run terraform import ... for those resources instead. See below for more details.

To help you get started with defining your Aiven infrastructure, the Aiven Terraform Github project comes with a sample script, which defines several resources to show you how that's done. There's also a variable definition file, which can be copied to the same directory with sample.tf and renamed to terraform.tfvars . Once you fill in values for the variables defined in terraform.tfvars you can apply the configuration to create your first Terraform managed Aiven project and you can tweak the various settings to see how it works. Note that as usual, running Aiven services does incur some costs. The services in the sample project cost bit over 1 dollar per hour altogether. Once you're done testing you can simply run terraform destroy to tear down all the services defined in the Terraform script.

Setting up user account and access token

In order for the Aiven Terraform provider to talk to Aiven API, you need to set the api_token value for the provider. In Aiven tokens are currently associated with individual Aiven users and if you don't already have an appropriate user you first need to create that by signing up from https://console.aiven.io or by inviting the user to join an existing project. Signing up to Aiven without an invitation always creates a new project but you don't need to use that project in your Terraform script. Regardless of whether you're creating a new project or using an existing one, the user account used for Terraform needs to be the one that owns the credit card used for billing unless invoicing is used (switching to invoicing requires contacting Aiven separately and cannot be done over API).

Once you have created the Aiven user account to use for Terraform integration and have added a credit card for that user (or set up invoicing billing), you can proceed to creating the API token used for Terraform. To do this, click the user name / gear icon at the top right hand side corner in Aiven web console, go to Authentication tokens tab and click the Generate Token button. Name the token some like "Terraform integration" and leave Max Age empty to ensure the token never expires. Click Generate Token to see the token and copy it to your Terraform variables file. The credit card ID to put in the configuration file is the last 4 digits of the card, as shown in the Aiven web console.

Handling existing projects and services

If you have an existing project and associated services that you want to bring under Terraform management, the Aiven Terraform provider supports import command for all resources. The Github project has additional details but you need to create a Terraform script defining your project and resources as usual but instead of running terraform apply once done with the scripts, you need to do

terraform import <terraform_resource_type>.<terraform_resource_id> <aiven_resource_id>

for all of the resources that already exist. Once you're done with the import phase you can run terraform plan to see how the existing resources differ from the definition in the Terraform scripts and keep on tweaking the parameters for the resources in the scripts until terraform plan no longer reports differences between the scripts and the current state of the system.

If you have existing projects that are set to use invoicing and you want to create new projects using Terraform that also use invoicing, you can define the existing project name in copy_from_project attribute for your new project to make the invoicing billing type get inherited for the newly created project.

Important considerations

Terraform resolves updates that are not supported by doing delete+create. While that may be acceptable for some resources, you will never want to delete a stateful service that has relevant data in it. Terraform always lists the changes it is about to perform so be mindful of the changes it is about to perform. If it ever shows that it is about to delete something, be extra sure that the resource being deleted isn't a service, database, topic, index or other such resource unless you really are trying to get rid of such a resource. Notable operations that cause unexpected service deletion are renaming a service, renaming a project, or trying to move a service from one project to another.

When choosing names for your projects, please keep in mind that project names need to be globally unique, i.e. unique among all Aiven customers.

If you encounter any problems with the Terraform integration or have improvement suggestions, please let us know. You can either contact our support directly or use Github issues for submitting your issue / improvement.

Did this answer your question?