-
Notifications
You must be signed in to change notification settings - Fork 458
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
17 changed files
with
1,031 additions
and
307 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
1. Navigate to the [IAM console](https://console.aws.amazon.com/iam/), select **Roles** from the navigation, and then **Create role**. | ||
1. Select **AWS service** for the **Trusted entity type**. For **Use case**, select **EC2** from the dropdown. Click **Next**. | ||
1. On the **Add permissions** page, click **Next**. | ||
1. Name the role (for example, `ec2-role`) and click **Create role**. | ||
1. Once the role has finished creating, copy the ARN in the **Summary** section. Click on the **Trust relationships** tab. You'll find a **Trusted entities** policy: | ||
|
||
{% include_cached copy-clipboard.html %} | ||
~~~json | ||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Effect": "Allow", | ||
"Principal": { | ||
"Service": "ec2.amazonaws.com" | ||
}, | ||
"Action": "sts:AssumeRole" | ||
} | ||
] | ||
} | ||
~~~ | ||
|
||
1. Navigate to the [IAM console](https://console.aws.amazon.com/iam/) and search for the role (`msk-role`) you created in Step 2 that contains the MSK policy. Select the role, which will take you to its summary page. | ||
1. Click on the **Trust relationships** tab, and click **Edit trust policy**. Add the ARN of the EC2 IAM role (`ec2-role`) to the JSON policy: | ||
|
||
{% include_cached copy-clipboard.html %} | ||
~~~json | ||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Effect": "Allow", | ||
"Principal": { | ||
"Service": "ec2.amazonaws.com", | ||
"AWS": "arn:aws:iam::{account ID}:role/{ec2-role}" | ||
}, | ||
"Action": "sts:AssumeRole" | ||
} | ||
] | ||
} | ||
~~~ | ||
|
||
Once you've updated the policy, click **Update policy**. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
If you would like to connect a changefeed running on a CockroachDB Dedicated cluster to an Amazon MSK Serverless cluster, contact your Cockroach Labs account team. |
51 changes: 51 additions & 0 deletions
51
src/current/_includes/v24.2/cdc/msk-iam-policy-role-step.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
1. In the AWS Management Console, go to the [IAM console](https://console.aws.amazon.com/iam/), select **Policies** from the navigation, and then **Create Policy**. | ||
1. Using the **JSON** tab option, update the policy with the following JSON. These permissions will allow you to connect to the cluster, manage topics, and consume messages. You may want to adjust the permissions to suit your permission model. For more details on the available permissions, refer to the AWS documentation on [IAM Access Control](https://docs.aws.amazon.com/msk/latest/developerguide/iam-access-control.html#kafka-actions) for MSK. | ||
|
||
Replace the instances of `arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}` with the MSK ARN from your cluster's summary page and add `/*` to the end, like the following: | ||
|
||
{% include_cached copy-clipboard.html %} | ||
~~~json | ||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Effect": "Allow", | ||
"Action": [ | ||
"kafka-cluster:Connect", | ||
"kafka-cluster:AlterCluster", | ||
"kafka-cluster:DescribeCluster" | ||
], | ||
"Resource": [ | ||
"arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}/*" | ||
] | ||
}, | ||
{ | ||
"Effect": "Allow", | ||
"Action": [ | ||
"kafka-cluster:*Topic", | ||
"kafka-cluster:WriteData", | ||
"kafka-cluster:ReadData" | ||
], | ||
"Resource": [ | ||
"arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}/*" | ||
] | ||
}, | ||
{ | ||
"Effect": "Allow", | ||
"Action": [ | ||
"kafka-cluster:AlterGroup", | ||
"kafka-cluster:DescribeGroup" | ||
], | ||
"Resource": [ | ||
"arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}/*" | ||
] | ||
} | ||
] | ||
} | ||
~~~ | ||
|
||
1. Once you have added your policy, add a policy name (for example, `msk-policy`), click **Next**, and **Create policy**. | ||
1. Return to the [IAM console](https://console.aws.amazon.com/iam/), select **Roles** from the navigation, and then **Create role**. | ||
1. Select **AWS service** for the **Trusted entity type**. For **Use case**, select **EC2** from the dropdown. Click **Next**. | ||
1. On the **Add permissions** page, search for the IAM policy (`msk-policy`) you just created. Click **Next**. | ||
1. Name the role (for example, `msk-role`) and click **Create role**. |
33 changes: 33 additions & 0 deletions
33
src/current/_includes/v24.2/cdc/msk-tutorial-crdb-setup.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
1. (Optional) On the EC2 instance running CockroachDB, run the [Movr]({% link {{ page.version.version }}/movr.md %}) application workload to set up some data for your changefeed. | ||
|
||
Create the schema for the workload: | ||
|
||
{% include_cached copy-clipboard.html %} | ||
~~~shell | ||
cockroach workload init movr | ||
~~~ | ||
|
||
Then run the workload: | ||
|
||
{% include_cached copy-clipboard.html %} | ||
~~~shell | ||
cockroach workload run movr --duration=1m | ||
~~~ | ||
|
||
1. Start a SQL session. For details on the available flags, refer to the [`cockroach sql`]({% link {{ page.version.version }}/cockroach-sql.md %}) page. | ||
|
||
{% include_cached copy-clipboard.html %} | ||
~~~ shell | ||
cockroach sql --insecure | ||
~~~ | ||
|
||
{{site.data.alerts.callout_info}} | ||
To set your {{ site.data.products.enterprise }} license, refer to the [Licensing FAQs]({% link {{ page.version.version }}/licensing-faqs.md %}#set-a-license) page. | ||
{{site.data.alerts.end}} | ||
|
||
1. Enable the `kv.rangefeed.enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}): | ||
|
||
{% include_cached copy-clipboard.html %} | ||
~~~ sql | ||
SET CLUSTER SETTING kv.rangefeed.enabled = true; | ||
~~~ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
1. Navigate to the [IAM console](https://console.aws.amazon.com/iam/), select **Roles** from the navigation, and then **Create role**. | ||
1. Select **AWS service** for the **Trusted entity type**. For **Use case**, select **EC2** from the dropdown. Click **Next**. | ||
1. On the **Add permissions** page, click **Next**. | ||
1. Name the role (for example, `ec2-role`) and click **Create role**. | ||
1. Once the role has finished creating, copy the ARN in the **Summary** section. Click on the **Trust relationships** tab. You'll find a **Trusted entities** policy: | ||
|
||
~~~json | ||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Effect": "Allow", | ||
"Principal": { | ||
"Service": "ec2.amazonaws.com" | ||
}, | ||
"Action": "sts:AssumeRole" | ||
} | ||
] | ||
} | ||
~~~ | ||
|
||
1. Navigate to the [IAM console](https://console.aws.amazon.com/iam/) and search for the role (`msk-role`) you created in Step 2 that contains the MSK policy. Select the role, which will take you to its summary page. | ||
1. Click on the **Trust relationships** tab, and click **Edit trust policy**. Add the ARN of the EC2 IAM role (`ec2-role`) to the JSON policy: | ||
|
||
~~~json | ||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Effect": "Allow", | ||
"Principal": { | ||
"Service": "ec2.amazonaws.com", | ||
"AWS": "arn:aws:iam::{account ID}:role/{ec2-role}" | ||
}, | ||
"Action": "sts:AssumeRole" | ||
} | ||
] | ||
} | ||
~~~ | ||
|
||
Once you've updated the policy, click **Update policy**. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
If you would like to connect a changefeed running on a CockroachDB Dedicated cluster to an Amazon MSK Serverless cluster, contact your Cockroach Labs account team. |
51 changes: 51 additions & 0 deletions
51
src/current/_includes/v24.3/cdc/msk-iam-policy-role-step.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
1. In the AWS Management Console, go to the [IAM console](https://console.aws.amazon.com/iam/), select **Policies** from the navigation, and then **Create Policy**. | ||
1. Using the **JSON** tab option, update the policy with the following JSON. These permissions will allow you to connect to the cluster, manage topics, and consume messages. You may want to adjust the permissions to suit your permission model. For more details on the available permissions, refer to the AWS documentation on [IAM Access Control](https://docs.aws.amazon.com/msk/latest/developerguide/iam-access-control.html#kafka-actions) for MSK. | ||
|
||
Replace the instances of `arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}` with the MSK ARN from your cluster's summary page and add `/*` to the end, like the following: | ||
|
||
{% include_cached copy-clipboard.html %} | ||
~~~json | ||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Effect": "Allow", | ||
"Action": [ | ||
"kafka-cluster:Connect", | ||
"kafka-cluster:AlterCluster", | ||
"kafka-cluster:DescribeCluster" | ||
], | ||
"Resource": [ | ||
"arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}/*" | ||
] | ||
}, | ||
{ | ||
"Effect": "Allow", | ||
"Action": [ | ||
"kafka-cluster:*Topic", | ||
"kafka-cluster:WriteData", | ||
"kafka-cluster:ReadData" | ||
], | ||
"Resource": [ | ||
"arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}/*" | ||
] | ||
}, | ||
{ | ||
"Effect": "Allow", | ||
"Action": [ | ||
"kafka-cluster:AlterGroup", | ||
"kafka-cluster:DescribeGroup" | ||
], | ||
"Resource": [ | ||
"arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}/*" | ||
] | ||
} | ||
] | ||
} | ||
~~~ | ||
|
||
1. Once you have added your policy, add a policy name (for example, `msk-policy`), click **Next**, and **Create policy**. | ||
1. Return to the [IAM console](https://console.aws.amazon.com/iam/), select **Roles** from the navigation, and then **Create role**. | ||
1. Select **AWS service** for the **Trusted entity type**. For **Use case**, select **EC2** from the dropdown. Click **Next**. | ||
1. On the **Add permissions** page, search for the IAM policy (`msk-policy`) you just created. Click **Next**. | ||
1. Name the role (for example, `msk-role`) and click **Create role**. |
33 changes: 33 additions & 0 deletions
33
src/current/_includes/v24.3/cdc/msk-tutorial-crdb-setup.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
1. (Optional) On the EC2 instance running CockroachDB, run the [Movr]({% link {{ page.version.version }}/movr.md %}) application workload to set up some data for your changefeed. | ||
|
||
Create the schema for the workload: | ||
|
||
{% include_cached copy-clipboard.html %} | ||
~~~shell | ||
cockroach workload init movr | ||
~~~ | ||
|
||
Then run the workload: | ||
|
||
{% include_cached copy-clipboard.html %} | ||
~~~shell | ||
cockroach workload run movr --duration=1m | ||
~~~ | ||
|
||
1. Start a SQL session. For details on the available flags, refer to the [`cockroach sql`]({% link {{ page.version.version }}/cockroach-sql.md %}) page. | ||
|
||
{% include_cached copy-clipboard.html %} | ||
~~~ shell | ||
cockroach sql --insecure | ||
~~~ | ||
|
||
{{site.data.alerts.callout_info}} | ||
To set your {{ site.data.products.enterprise }} license, refer to the [Licensing FAQs]({% link {{ page.version.version }}/licensing-faqs.md %}#set-a-license) page. | ||
{{site.data.alerts.end}} | ||
|
||
1. Enable the `kv.rangefeed.enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}): | ||
|
||
{% include_cached copy-clipboard.html %} | ||
~~~ sql | ||
SET CLUSTER SETTING kv.rangefeed.enabled = true; | ||
~~~ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.