The most practical way to migrate data from a self hosted InfluxDB 1.x service to an Aiven InfluxDB can be to export the data from the source service in line protocol using the influx_export -command.

The export file, which is line protocol, can then be pushed over to the Aiven destination service with the influx -client. This should be fast and work across different minor versions.

Creating the export

First, on the source service we will export the data to a dump file.

In our example we will use common defaults for the datadir and waldir, directories - those may differ on your system.

Our example database name will be "weather" and the export here would be a gzip compressed dump created to the /scratch directory.

$ influx_inspect export -datadir "/var/lib/influxdb/data" -waldir "/var/lib/influxdb/wal" -out "/scratch/weather.influx.gz" -database weather -compress

If you have a large database and only need a part of the data, you may optionally want to define a time span with the -start and -end -switches to reduce the size of the dump.

Pushing the export to the destination Aiven service

Now that you have successfully created the export file you can push it to the Aiven service. It is a good idea to do it from a host with a fast and stable network connection, e.g. through VPC peering, if possible.

One key point here is that the Influx database you will be migrating will need to be pre-created by you before pushing the export to the destination.

You can do it through the web console from under the "Databases" -tab for your Influx service. It can also be done via Aiven CLI and API. This is because the 'avnadmin' -admin user does not have full superuser access. Retention policies can be changed normally though - more detailed information about the available commands can be found here.

Once the database exists on the destination side you can push the export data to the destination. Before inserting the data, check that your retention policies are what you want them to be. More information about InfluxDB retention policies on Aiven here.

$ influx -import -host mkos-influx-dest-dev-sandbox.aivencloud.com -port 12691 -username 'avnadmin' -password 'secret' -ssl -precision rfc3339 -compressed -path ./weather.influx.gz
2021/01/12 03:39:37 error: error authorizing query: avnadmin not authorized to execute statement 'CREATE DATABASE weather WITH NAME autogen', requires admin privilege
2021/01/12 03:39:38 Processed 100000 lines. Time elapsed: 565.826523ms. Points per second (PPS): 176732
2021/01/12 03:39:38 Processed 200000 lines. Time elapsed: 1.136294768s. Points per second (PPS): 176010
2021/01/12 03:39:39 Processed 300000 lines. Time elapsed: 1.671407388s. Points per second (PPS): 179489
[...]
2021/01/12 03:39:53 Processed 1 commands
2021/01/12 03:39:53 Processed 2752902 inserts
2021/01/12 03:39:53 Failed 0 inserts

You will notice that there will be an initial error line about not being able to create the database. That is fine as we created it in the earlier step.

Now the migration should be done. You can verify your data and start using

your new Aiven InfluxDB service.

Did this answer your question?