Use Pre-BYOK-API-V1 Self-managed Encryption Keys in Confluent Cloud¶
Prerequisites¶
To manage your Pre-BYOK-API-V1 Confluent Cloud cluster on AWS, you need the following:
- An AWS account with access to AWS KMS.
- A Confluent Cloud cluster provisioned on AWS before August 11, 2023.
- The Confluent CLI installed and configured.
- Appropriate AWS IAM permissions to manage KMS keys.
If you are using Confluent Cloud clusters provisioned on AWS before August 11, 2023, follow the information below to manage your Pre-BYOK-API-V1 self-managed encryption keys to encrypt data at rest for your Kafka clusters.
Important
If you are using Confluent Cloud clusters provisioned on AWS after August 11, 2023, do not use the BYOK V1 API to manage your self-managed encryption keys. keys.
Identify a Pre-BYOK-API-V1 Confluent Cloud cluster¶
If your Confluent Cloud cluster was created and provisioned on AWS before August 11, 2023, then it is likely a Pre-BYOK-API-V1 cluster. To verify that your cluster is a Pre-BYOK-API-V1 cluster, run the following Confluent CLI command to inspect the cluster and verify that the cluster cannot use the BYOK V1 API:
confluent kafka cluster describe <cluster-id> --output json
If you don’t know the cluster ID, you can use the following Confluent CLI command to list all your clusters and get the ID for the cluster you want to inspect:
confluent kafka cluster list
For a Pre-BYOK-API-V1 cluster, the output includes a key_id field within the
encryption_key_id section, like this:
{
"is_current": true,
"id": "lkc-abc123",
"name": "test-byok-cluster",
"type": "DEDICATED",
"cluster_size": 1,
"ingress_limit": 50,
"egress_limit": 150,
"storage": "Infinite",
"cloud": "aws",
"region": "us-west-2",
"availability": "single-zone",
"status": "UP",
"endpoint": "SASL_SSL://pkc-19m9ov.us-west-2.aws.confluent.cloud:9092",
"encryption_key_id": "arn:aws:kms:us-west-2:111122223333:key/mrk-1234abcd-12ab-34cd-56ef-1234567890ab",
"rest_endpoint": "https://pkc-19m9ov.us-west-2.aws.confluent.cloud:443"
}
If you see this structure in your cluster’s output, it confirms that you have a
Pre-BYOK-API-V1 cluster. Newer Confluent Cloud clusters will not include the key_id
field in this format.
Manage Pre-BYOK-API-V1 self-managed encryption keys¶
Pre-BYOK-API-V1 clusters differ from newer Confluent Cloud clusters in the following ways:
- A distinct CCK ID (Confluent Cloud key ID) is not associated with the cluster. This means you cannot use the BYOK V1 API to manage your keys through the Confluent Cloud Console or CLI.
- You cannot retrieve your KMS key policy directly using the Confluent Cloud APIs or the Confluent CLI for Pre-BYOK-API-V1 clusters. You must manage your KMS key policies directly in the AWS Console, but you must include the policy statement shown in the Replace a lost IAM policy section in your policy.
To manage your Pre-BYOK-API-V1 self-managed encryption keys:
- Use the AWS Console or AWS CLI to manage your KMS keys.
- Keep track of your key ARNs and policies manually.
- Ensure your IAM policies are properly configured as shown in the Replace a lost IAM policy section.
Replace a lost IAM policy¶
Warning
Do not reuse or re-onboard encryption keys that were used for a Pre-BYOK-API-V1 cluster into a new Confluent Cloud cluster. Unexpected results might occur.
If you lose your IAM policy for a Pre-BYOK-API-V1 cluster, append the following JSON statement to your existing policy to ensure that you can continue to manage your Kafka cluster:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Allow Confluent account(s) to use the key",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::050451403612:root",
"arn:aws:iam::050879227952:root",
"arn:aws:iam::121194963621:root",
"arn:aws:iam::259082970751:root",
"arn:aws:iam::266735807085:root",
"arn:aws:iam::390327825978:root",
"arn:aws:iam::417601102659:root",
"arn:aws:iam::457492987184:root",
"arn:aws:iam::479111403931:root",
"arn:aws:iam::563559260525:root",
"arn:aws:iam::727646498665:root",
"arn:aws:iam::891612556359:root",
"arn:aws:iam::897722682106:root",
"arn:aws:iam::908027390566:root",
"arn:aws:iam::980921717494:root",
"arn:aws:iam::982081071159:root",
"arn:aws:iam::649317483566:root",
"arn:aws:iam::439989760662:root",
"arn:aws:iam::819787557123:root",
"arn:aws:iam::571452958320:root",
"arn:aws:iam::702463834586:root",
"arn:aws:iam::350927345273:root",
"arn:aws:iam::152535741197:root",
"arn:aws:iam::161406349951:root",
"arn:aws:iam::219109013385:root",
"arn:aws:iam::777099487581:root",
"arn:aws:iam::590184035729:root",
"arn:aws:iam::637423370172:root",
"arn:aws:iam::058264492257:root",
"arn:aws:iam::851725567684:root"
]
},
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": "*"
},
{
"Sid": "Allow Confluent account(s) to attach persistent resources",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::050451403612:root",
"arn:aws:iam::050879227952:root",
"arn:aws:iam::121194963621:root",
"arn:aws:iam::259082970751:root",
"arn:aws:iam::266735807085:root",
"arn:aws:iam::390327825978:root",
"arn:aws:iam::417601102659:root",
"arn:aws:iam::457492987184:root",
"arn:aws:iam::479111403931:root",
"arn:aws:iam::563559260525:root",
"arn:aws:iam::727646498665:root",
"arn:aws:iam::891612556359:root",
"arn:aws:iam::897722682106:root",
"arn:aws:iam::908027390566:root",
"arn:aws:iam::980921717494:root",
"arn:aws:iam::982081071159:root",
"arn:aws:iam::649317483566:root",
"arn:aws:iam::439989760662:root",
"arn:aws:iam::819787557123:root",
"arn:aws:iam::571452958320:root",
"arn:aws:iam::702463834586:root",
"arn:aws:iam::350927345273:root",
"arn:aws:iam::152535741197:root",
"arn:aws:iam::161406349951:root",
"arn:aws:iam::219109013385:root",
"arn:aws:iam::777099487581:root",
"arn:aws:iam::590184035729:root",
"arn:aws:iam::637423370172:root",
"arn:aws:iam::058264492257:root",
"arn:aws:iam::851725567684:root"
]
},
"Action": [
"kms:CreateGrant",
"kms:ListGrants",
"kms:RevokeGrant"
],
"Resource": "*"
}
]
}