Note: Azure PrivateLink support is currently a beta feature. Please get in touch with support if you are interested in trying out this feature!

Azure Privatelink brings Aiven services to the selected virtual network in your subscription. In a traditional setup using VNet peering traffic is routed through an Azure virtual network peering to and from your Aiven services. With Privatelink it is possible to create a Private Endpoint to your own virtual network and access an Aiven service through it. The endpoint creates network interface (NIC) into one of the subnets of the VNet and receives a private IP address from that subnet's IP range. The private endpoint is routed to your Aiven service located in one of Aiven's Azure subscriptions.

Prerequisites

  • The Aiven service must be located in a project VPC. This ensures the service is not accessible from the public internet. If you do not plan adding regular VNet peerings in addition to Privatelink, any private IP range can be used for the VPC. There is no network routing between your subscription and the Aiven VPC and subsequently overlapping IP ranges are not an issue as with VNet peering.

  • The service must be using static IP addresses. Even though the service is in a VPC and only communicates using private addresses, Azure load balancers require using standard SKU IP addresses for target virtual machines instead of basic SKU addresses. Even though the Aiven service nodes have a public IP address, the only "public" access is TCP health probes that Azure sends to load balancer target ports from a public IP address. Aiven only allows opening and closing a TCP connection from this address without transmitting any data.

Setting it up

Create a Privatelink Service

Once the Aiven service in a VPC has been created and is using static IP addresses, use the Aiven CLI to create a privatelink resource on the Aiven service. This creates an Azure Standard Internal Load Balancer dedicated to your Aiven service and attaches it to an Azure Privatelink Service which we'll later use to connect your subscription's private endpoint to.

# avn service privatelink azure create --user-subscription-id 8eefec94-5d63-40c9-983c-03ab083b411d my-privatelink-service

The commands creates a privatelink resource for the my-privatelink-service Aiven service. The --user-subscription-id argument adds that azure subscription ID to the list of subscriptions that are allowed to connect endpoints to the subscription. Connections from other subscriptions will be automatically rejected. You can later update the subscription list with service privatelink azure update. The privatelink will be in state creating until Azure has provisioned a load balancer and privatelink service:

# avn service privatelink azure get my-privatelink-service
AZURE_SERVICE_ALIAS AZURE_SERVICE_ID STATE USER_SUBSCRIPTION_IDS
=================== ================ ======== ====================================
null null creating 8eefec94-5d63-40c9-983c-03ab083b411d

After the load balancer and privatelink service have been provisioned state changes to active and azure_service_alias and azure_service_id are set:

# avn service privatelink azure get my-privatelink-service
aivenprod-ss35838b0dc89.ae1a3fec-c06b-40b5-9692-f4fff3acfbcd.westeurope.azure.privatelinkservice /subscriptions/2280b6de-6fd6-406e-84d5-2fbc7758d1fc/resourceGroups/aivenprod-1af4cfbb-3259-4c12-95de-a92641822504/providers/Microsoft.Network/privateLinkServices/aivenprod-ss35838b0dc89 active 8eefec94-5d63-40c9-983c-03ab083b411d

Create a Private Endpoint

At this point Azure resources in the Aiven subscription have been set up and are ready to be connected to from your subscription and virtual network. Create a private endpoint using the web console, Azure CLI or any other method. If using the web console, select "Connect to an Azure resource by resource ID or alias" and fill in azure_service_alias or azure_service_id referred to earlier.
When the endpoint has been created refresh the Aiven privatelink service. Azure does not provide notifications about incoming endpoint connections, and the Aiven API is oblivious to new endpoints until refreshed:

# avn service privatelink azure refresh my-privatelink-service

After a moment the endpoint should be listed as connected to the service:

# avn service privatelink azure connection list my-privatelink-service
PRIVATELINK_CONNECTION_ID PRIVATE_ENDPOINT_ID STATE USER_IP_ADDRESS
========================= ========================================================================================================================================================== ===================== ===============
plc35843e8054b /subscriptions/8eefec94-5d63-40c9-983c-03ab083b411d/resourceGroups/test-privatelink/providers/Microsoft.Network/privateEndpoints/my-endpoint pending-user-approval null

Validate the endpoint ID matches the one created in your subscription and approve it:

# avn service privatelink azure connection approve my-privatelink-service plc35843e8054b
PRIVATE_ENDPOINT_ID PRIVATELINK_CONNECTION_ID STATE USER_IP_ADDRESS
========================================================================================================================================================== ========================= ============= ===============
/subscriptions/8eefec94-5d63-40c9-983c-03ab083b411d/resourceGroups/test-privatelink/providers/Microsoft.Network/privateEndpoints/my-endpoint plc35843e8054b user-approved null

The endpoint in your subscription is now connected to the privatelink service in the Aiven subscription. The connection state as reported by Azure turn into "Pending" after a small delay. This is the state endpoints are in after being connected.

The endpoint has a private IP address in the subnet you created the endpoint to. Because this IP address is not visible to the Aiven subscription, you'll need to provide the address to the Aiven API. Go to the private endpoint -> Network interface and copy "Private IP address". In our example the value is 10.0.0.4, but yours will very, very likely be a different one. Update the connection with the address:

# avn service privatelink azure connection update --endpoint-ip-address 10.0.0.4 my-privatelink-service plc35843e8054b
PRIVATE_ENDPOINT_ID PRIVATELINK_CONNECTION_ID STATE USER_IP_ADDRESS
========================================================================================================================================================== ========================= ====== ===============
/subscriptions/8eefec94-5d63-40c9-983c-03ab083b411d/resourceGroups/test-privatelink/providers/Microsoft.Network/privateEndpoints/my-endpoint plc35843e8054b active 10.0.0.4

Once the endpoint IP address has been added, the connection's state turns to active. A DNS name for the service is registered pointing to the IP address provided.

Enable PrivateLink access for Aiven service components

Lastly, PrivateLink access needs to be enabled on the Aiven service. Each service component can be controlled separately - for example you can enable PrivateLink access for Kafka, while allowing Kafka Connect to only be connected via VNet peering. Set user_config.privatelink_access.<service component> to true for the components you wish to enable. With Aiven CLI:

# avn service update -c privatelink_access.pg=true my-privatelink-service
# avn service update -c privatelink_access.pgbouncer=true my-privatelink-service

Access can also be set up in the Aiven web console from Service -> Overview tab -> Advanced configuration. Add the options from the list and toggle the switch to set the value to true.

After toggling the values your privatelink resource will be rebuilt with load balancer rules added for the service component's ports. In the Aiven web console, see the connection information box on your Aiven service's overview page to find the URI or hostname and port to access the service through the private endpoint.

Delete a PrivateLink

Use the Aiven CLI to delete the Azure Load Balancer and Privatelink Service:

# avn service privatelink azure delete my-privatelink-service
Did this answer your question?