Note: Aiven for OpenSearch replaces Aiven for Elasticsearch after Elasticsearch version 7.10.2 due to licensing changes. In addition, OpenSearch Dashboards replaces Kibana. Starting from September 24th, 2021 until March 23rd, 2022, Aiven supports services running both OpenSearch and Elasticsearch 7.10.2. For more information on Aiven for OpenSearch, see our developer documentation.

We recommend migrating to Aiven for Elasticsearch by reindexing from your remote cluster. The same process works for migrating from Aiven for Elasticsearch to a self-hosted Elasticsearch service.

For a larger number of indexes, we recommend that you use a script to execute these steps automatically.

As Aiven for Elasticsearch does not support joining external Elasticsearch servers to the same cluster, online migration is not currently possible.

To migrate or copy data:

  1. Create a hosted Aiven for Elasticsearch service.

  2. Use the Aiven CLI client to set the reindex.remote.whitelist parameter to point to your source Elasticsearch service:

    avn service update your-service-name -c elasticsearch.reindex_remote_whitelist=your.non-aiven-service.example.com:9200 

    Replace the port number with the one where your source Elasticsearch service is listening.

  3. Wait for the cluster to restart.
    This may take several minutes, as the service tries to do a rolling restart to minimize downtime. You do not need to power off the service.

  4. Start migrating the indexes.
    For each index:

    1. Stop writes to the index.
      This step is not necessary if you are testing the process.

    2. Create the empty index on your destination Elasticsearch service.
      For example, using curl:

      curl -XPUT https://avnadmin:yourpassword@es-123-demoprj.aivencloud.com:23125/logs-2024-09-21

    3. Submit the reindexing request.
      For example, using curl:

      curl -XPOST https://avnadmin:yourpassword@es-123-demoprj.aivencloud.com:23125/_reindex \
      -H 'Content-type: application/json' \
      -d '{"source":
      {"index": "logs-2024-09-21",
      "remote":
      {"username": "your-remote-username",
      "password": "your-remote-password",
      "host": "https://your.non-aiven-service.example.com:9200"
      }
      },
      "dest":
      {"index": "logs-2024-09-21"}
      }'

    4. Wait for the reindexing process to finish.
      If you see a message like the following in the response, check that the host name and port match the ones that you set earlier:

      [your.non-aiven-service.example.com:9200] not whitelisted in reindex.remote.whitelist

      Depending on the amount of data that you have, reindexing may take a significant amount of time.

    5. Point clients to use that index from Aiven for Elasticsearch for both read and write operations and resume any write activity.

    6. Delete the source index if necessary.

If you have any questions about this process, contact our support at support@aiven.io.

Learn how Aiven simplifies working with Elasticsearch:

Did this answer your question?