Skip to content

Commit

Permalink
Merge branch 'main' into DOC-11287
Browse files Browse the repository at this point in the history
  • Loading branch information
mdlinville authored Oct 23, 2024
2 parents dbac59b + ef38e87 commit c5bf1c6
Show file tree
Hide file tree
Showing 17 changed files with 1,031 additions and 307 deletions.
3 changes: 3 additions & 0 deletions src/current/_data/redirects.yml
Original file line number Diff line number Diff line change
Expand Up @@ -666,6 +666,9 @@
- destination: cockroachcloud/cmek.md#faq
sources: ['cockroachcloud/cmek-faq.md']

- destination: cockroachcloud/managing-cmek.md
sources: ['cockroachcloud/cmek-ops-*.md']

- destination: cockroachcloud/connect-to-your-cluster.md
sources: ['cockroachcloud/stable/cockroachcloud-connect-to-your-cluster.md']

Expand Down
43 changes: 43 additions & 0 deletions src/current/_includes/v24.2/cdc/cluster-iam-role-step.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
1. Navigate to the [IAM console](https://console.aws.amazon.com/iam/), select **Roles** from the navigation, and then **Create role**.
1. Select **AWS service** for the **Trusted entity type**. For **Use case**, select **EC2** from the dropdown. Click **Next**.
1. On the **Add permissions** page, click **Next**.
1. Name the role (for example, `ec2-role`) and click **Create role**.
1. Once the role has finished creating, copy the ARN in the **Summary** section. Click on the **Trust relationships** tab. You'll find a **Trusted entities** policy:

{% include_cached copy-clipboard.html %}
~~~json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
~~~

1. Navigate to the [IAM console](https://console.aws.amazon.com/iam/) and search for the role (`msk-role`) you created in Step 2 that contains the MSK policy. Select the role, which will take you to its summary page.
1. Click on the **Trust relationships** tab, and click **Edit trust policy**. Add the ARN of the EC2 IAM role (`ec2-role`) to the JSON policy:

{% include_cached copy-clipboard.html %}
~~~json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com",
"AWS": "arn:aws:iam::{account ID}:role/{ec2-role}"
},
"Action": "sts:AssumeRole"
}
]
}
~~~

Once you've updated the policy, click **Update policy**.
1 change: 1 addition & 0 deletions src/current/_includes/v24.2/cdc/msk-dedicated-support.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
If you would like to connect a changefeed running on a CockroachDB Dedicated cluster to an Amazon MSK Serverless cluster, contact your Cockroach Labs account team.
51 changes: 51 additions & 0 deletions src/current/_includes/v24.2/cdc/msk-iam-policy-role-step.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
1. In the AWS Management Console, go to the [IAM console](https://console.aws.amazon.com/iam/), select **Policies** from the navigation, and then **Create Policy**.
1. Using the **JSON** tab option, update the policy with the following JSON. These permissions will allow you to connect to the cluster, manage topics, and consume messages. You may want to adjust the permissions to suit your permission model. For more details on the available permissions, refer to the AWS documentation on [IAM Access Control](https://docs.aws.amazon.com/msk/latest/developerguide/iam-access-control.html#kafka-actions) for MSK.

Replace the instances of `arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}` with the MSK ARN from your cluster's summary page and add `/*` to the end, like the following:

{% include_cached copy-clipboard.html %}
~~~json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"kafka-cluster:Connect",
"kafka-cluster:AlterCluster",
"kafka-cluster:DescribeCluster"
],
"Resource": [
"arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}/*"
]
},
{
"Effect": "Allow",
"Action": [
"kafka-cluster:*Topic",
"kafka-cluster:WriteData",
"kafka-cluster:ReadData"
],
"Resource": [
"arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}/*"
]
},
{
"Effect": "Allow",
"Action": [
"kafka-cluster:AlterGroup",
"kafka-cluster:DescribeGroup"
],
"Resource": [
"arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}/*"
]
}
]
}
~~~

1. Once you have added your policy, add a policy name (for example, `msk-policy`), click **Next**, and **Create policy**.
1. Return to the [IAM console](https://console.aws.amazon.com/iam/), select **Roles** from the navigation, and then **Create role**.
1. Select **AWS service** for the **Trusted entity type**. For **Use case**, select **EC2** from the dropdown. Click **Next**.
1. On the **Add permissions** page, search for the IAM policy (`msk-policy`) you just created. Click **Next**.
1. Name the role (for example, `msk-role`) and click **Create role**.
33 changes: 33 additions & 0 deletions src/current/_includes/v24.2/cdc/msk-tutorial-crdb-setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
1. (Optional) On the EC2 instance running CockroachDB, run the [Movr]({% link {{ page.version.version }}/movr.md %}) application workload to set up some data for your changefeed.

Create the schema for the workload:

{% include_cached copy-clipboard.html %}
~~~shell
cockroach workload init movr
~~~

Then run the workload:

{% include_cached copy-clipboard.html %}
~~~shell
cockroach workload run movr --duration=1m
~~~

1. Start a SQL session. For details on the available flags, refer to the [`cockroach sql`]({% link {{ page.version.version }}/cockroach-sql.md %}) page.

{% include_cached copy-clipboard.html %}
~~~ shell
cockroach sql --insecure
~~~

{{site.data.alerts.callout_info}}
To set your {{ site.data.products.enterprise }} license, refer to the [Licensing FAQs]({% link {{ page.version.version }}/licensing-faqs.md %}#set-a-license) page.
{{site.data.alerts.end}}

1. Enable the `kv.rangefeed.enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}):

{% include_cached copy-clipboard.html %}
~~~ sql
SET CLUSTER SETTING kv.rangefeed.enabled = true;
~~~
21 changes: 16 additions & 5 deletions src/current/_includes/v24.2/sidebar-data/stream-data.json
Original file line number Diff line number Diff line change
Expand Up @@ -87,15 +87,26 @@
"title": "Changefeed Tutorials",
"items": [
{
"title": "Connect to a Changefeed Kafka Sink with OAuth Using Okta",
"urls": [
"/${VERSION}/connect-to-a-changefeed-kafka-sink-with-oauth-using-okta.html"
"title": "Stream a Changefeed to an Amazon MSK Cluster",
"items": [
{
"title": "Amazon MSK",
"urls": [
"/${VERSION}/stream-a-changefeed-to-amazon-msk.html"
]
},
{
"title": "Amazon MSK Serverless",
"urls": [
"/${VERSION}/stream-a-changefeed-to-amazon-msk-serverless.html"
]
}
]
},
{
"title": "Stream a Changefeed to Amazon MSK Serverless",
"title": "Connect to a Changefeed Kafka Sink with OAuth Using Okta",
"urls": [
"/${VERSION}/stream-a-changefeed-to-amazon-msk-serverless.html"
"/${VERSION}/connect-to-a-changefeed-kafka-sink-with-oauth-using-okta.html"
]
},
{
Expand Down
41 changes: 41 additions & 0 deletions src/current/_includes/v24.3/cdc/cluster-iam-role-step.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
1. Navigate to the [IAM console](https://console.aws.amazon.com/iam/), select **Roles** from the navigation, and then **Create role**.
1. Select **AWS service** for the **Trusted entity type**. For **Use case**, select **EC2** from the dropdown. Click **Next**.
1. On the **Add permissions** page, click **Next**.
1. Name the role (for example, `ec2-role`) and click **Create role**.
1. Once the role has finished creating, copy the ARN in the **Summary** section. Click on the **Trust relationships** tab. You'll find a **Trusted entities** policy:

~~~json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
~~~

1. Navigate to the [IAM console](https://console.aws.amazon.com/iam/) and search for the role (`msk-role`) you created in Step 2 that contains the MSK policy. Select the role, which will take you to its summary page.
1. Click on the **Trust relationships** tab, and click **Edit trust policy**. Add the ARN of the EC2 IAM role (`ec2-role`) to the JSON policy:

~~~json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com",
"AWS": "arn:aws:iam::{account ID}:role/{ec2-role}"
},
"Action": "sts:AssumeRole"
}
]
}
~~~

Once you've updated the policy, click **Update policy**.
1 change: 1 addition & 0 deletions src/current/_includes/v24.3/cdc/msk-dedicated-support.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
If you would like to connect a changefeed running on a CockroachDB Dedicated cluster to an Amazon MSK Serverless cluster, contact your Cockroach Labs account team.
51 changes: 51 additions & 0 deletions src/current/_includes/v24.3/cdc/msk-iam-policy-role-step.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
1. In the AWS Management Console, go to the [IAM console](https://console.aws.amazon.com/iam/), select **Policies** from the navigation, and then **Create Policy**.
1. Using the **JSON** tab option, update the policy with the following JSON. These permissions will allow you to connect to the cluster, manage topics, and consume messages. You may want to adjust the permissions to suit your permission model. For more details on the available permissions, refer to the AWS documentation on [IAM Access Control](https://docs.aws.amazon.com/msk/latest/developerguide/iam-access-control.html#kafka-actions) for MSK.

Replace the instances of `arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}` with the MSK ARN from your cluster's summary page and add `/*` to the end, like the following:

{% include_cached copy-clipboard.html %}
~~~json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"kafka-cluster:Connect",
"kafka-cluster:AlterCluster",
"kafka-cluster:DescribeCluster"
],
"Resource": [
"arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}/*"
]
},
{
"Effect": "Allow",
"Action": [
"kafka-cluster:*Topic",
"kafka-cluster:WriteData",
"kafka-cluster:ReadData"
],
"Resource": [
"arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}/*"
]
},
{
"Effect": "Allow",
"Action": [
"kafka-cluster:AlterGroup",
"kafka-cluster:DescribeGroup"
],
"Resource": [
"arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}/*"
]
}
]
}
~~~

1. Once you have added your policy, add a policy name (for example, `msk-policy`), click **Next**, and **Create policy**.
1. Return to the [IAM console](https://console.aws.amazon.com/iam/), select **Roles** from the navigation, and then **Create role**.
1. Select **AWS service** for the **Trusted entity type**. For **Use case**, select **EC2** from the dropdown. Click **Next**.
1. On the **Add permissions** page, search for the IAM policy (`msk-policy`) you just created. Click **Next**.
1. Name the role (for example, `msk-role`) and click **Create role**.
33 changes: 33 additions & 0 deletions src/current/_includes/v24.3/cdc/msk-tutorial-crdb-setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
1. (Optional) On the EC2 instance running CockroachDB, run the [Movr]({% link {{ page.version.version }}/movr.md %}) application workload to set up some data for your changefeed.

Create the schema for the workload:

{% include_cached copy-clipboard.html %}
~~~shell
cockroach workload init movr
~~~

Then run the workload:

{% include_cached copy-clipboard.html %}
~~~shell
cockroach workload run movr --duration=1m
~~~

1. Start a SQL session. For details on the available flags, refer to the [`cockroach sql`]({% link {{ page.version.version }}/cockroach-sql.md %}) page.

{% include_cached copy-clipboard.html %}
~~~ shell
cockroach sql --insecure
~~~

{{site.data.alerts.callout_info}}
To set your {{ site.data.products.enterprise }} license, refer to the [Licensing FAQs]({% link {{ page.version.version }}/licensing-faqs.md %}#set-a-license) page.
{{site.data.alerts.end}}

1. Enable the `kv.rangefeed.enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}):

{% include_cached copy-clipboard.html %}
~~~ sql
SET CLUSTER SETTING kv.rangefeed.enabled = true;
~~~
21 changes: 16 additions & 5 deletions src/current/_includes/v24.3/sidebar-data/stream-data.json
Original file line number Diff line number Diff line change
Expand Up @@ -87,15 +87,26 @@
"title": "Changefeed Tutorials",
"items": [
{
"title": "Connect to a Changefeed Kafka Sink with OAuth Using Okta",
"urls": [
"/${VERSION}/connect-to-a-changefeed-kafka-sink-with-oauth-using-okta.html"
"title": "Stream a Changefeed to an Amazon MSK Cluster",
"items": [
{
"title": "Amazon MSK",
"urls": [
"/${VERSION}/stream-a-changefeed-to-amazon-msk.html"
]
},
{
"title": "Amazon MSK Serverless",
"urls": [
"/${VERSION}/stream-a-changefeed-to-amazon-msk-serverless.html"
]
}
]
},
{
"title": "Stream a Changefeed to Amazon MSK Serverless",
"title": "Connect to a Changefeed Kafka Sink with OAuth Using Okta",
"urls": [
"/${VERSION}/stream-a-changefeed-to-amazon-msk-serverless.html"
"/${VERSION}/connect-to-a-changefeed-kafka-sink-with-oauth-using-okta.html"
]
},
{
Expand Down
Loading

0 comments on commit c5bf1c6

Please sign in to comment.