Egress Private Service Connect Endpoint Setup: ClickHouse on Google Cloud for Confluent Cloud

This topic presents the steps for configuring the ClickHouse Sink connector in Confluent Cloud using Egress Private Service Connect on Google Cloud.

For details, see Google Cloud Private Service Connect and ClickHouse.

Prerequisites

The following is a list of prerequisites for configuring the ClickHouse Sink connector with an Egress PrivateLink Endpoint:

• In Confluent Cloud, one of the following cluster types is set up with the specified network resource:

  • A Dedicated cluster with a Confluent Cloud network.

    For the steps to create a Confluent Cloud network, see Create a Confluent Cloud network. The Connection type of the network needs to be “PrivateLink Access”.

  • A Enterprise cluster with a network gateway

    For the steps to create a gateway, see Create a gateway for outbound connectivity in Confluent Cloud.

• A source topic is created to sink data into the ClickHouse database.

• ClickHouse instance is created to sink data into and is running within the same region and cloud as the Confluent Cloud cluster.

• A database and a schema are created in ClickHouse to sink data into.

Step 1. Obtain Endpoint Service name and DNS Name from ClickHouse

1. In the ClickHouse Cloud console, open the service you want to connect via PrivateLink, then navigate to the Settings menu.

2. Click Setup PrivateLink Endpoint in Private Endpoint section, and make note of ClickHouse Service Name and DNS Name.

   

Step 2. Create an Egress PrivateLink Endpoint

Dedicated clusterEnterprise cluster

1. In the Network management page or tab of the desired Confluent Cloud environment, click the Confluent Cloud network you want to add the PrivateLink Endpoint to. The Connection Type of the network should be “PrivateLink Access”.

2. Click Create endpoint in the Egress connections tab.

3. Click the service you want to connect to, specifically, ClickHouse. Select Other if you do not see the specific service.

4. Follow the steps below to specify the following field values:

   • SERVICE: Name of service connecting to - ClickHouse.

   • Endpoint Name: Name of the PrivateLink Endpoint.

   • PrivateLink service name: The name of the PrivateLink service you retrieved in Step 1. Obtain Endpoint Service name and DNS Name from ClickHouse.

   • Create an endpoint with high availability: Check the box if you wish to deploy an endpoint with High Availability.

     Endpoints deployed with high availability have network interfaces deployed in multiple availability zones.

5. Click Create to create the PrivateLink Endpoint.

6. If there are additional steps for the specific target service, follow the prompt to complete the tasks, and then click Finish.

Step 3. Add Endpoint ID to ClickHouse Cloud organization

1. Open the ClickHouse Cloud Console, and Copy the VPC endpoint ID from the Egress connection tab.

   

2. Go to the ClickHouse Console → Settings → Set up Private Endpoint. Add the VPC endpoint ID retrieved in the previous step to the Endpoint ID field. This will allow requests from the Confluent cluster endpoint to access the ClickHouse service.

   

Step 4. Create a DNS record

Dedicated clusterEnterprise cluster

1. When the PrivateLink Endpoint status transitions to “Ready”, in the DNS tab, click Create record on the associated PrivateLink Endpoint.

   

2. Specify the following, and click Save.

   • Access point: Select the PrivateLink Endpoint you created above in Step 2. Create an Egress PrivateLink Endpoint.
   • Domain: Enter the domain name as DNS name we got from ClickHouse in Step 1. Obtain Endpoint Service name and DNS Name from ClickHouse.
   

Step 5. Create the ClickHouse Sink connector

1. Specify the ClickHouse hostname as the DNS name fetched from ClickHouse in Step 1. Obtain Endpoint Service name and DNS Name from ClickHouse.

   

2. See the ClickHouse Sink connector documentation for the steps to create the sink connector Confluent Cloud.