[occm] The pods cannot access each other when across nodes

[jeffyjf]

Is this a BUG REPORT or FEATURE REQUEST?:

/kind bug

What happened:

I useed CAPO deployed a two nodes cluser. And start OCCM with below configurations:

[Global]
auth-url=http://192.168.1.29:5000
username=admin
password=123456
region=RegionOne
tenant-id=666853c5323d486cbd81d372359f8942
domain-id=default

[Networking]
public-network-name=ext-net

[LoadBalancer]
enabled=true
floating-network-id=a03550ce-f3ff-4b71-bd37-00c12bcaa3f0
floating-subnet-id=f0db7750-6d43-4259-a9f5-4b171469b2c7
lb-provider=amphora
subnet-id=6cf6198d-8e89-4bf3-9c6e-5dc79509118f
network-id=c2d42f5b-1b1e-4d25-94eb-ea1a7449a8da

[Metadata]
search-order=metadataService,configDrive

[Route]
router-id=89aac4b9-9004-464d-8e01-409149144e28  # Note: router controller be enabled here. And OCCM start with --allocate-node-cidrs=true and --cluster-cidr=10.255.0.0/16

And the CNI configurations like below:

{
    "cniVersion": "0.3.1",
    "name": "mynet",
    "type": "bridge",
    "bridge": "mynet0",
    "isDefaultGateway": true,
    "forceAddress": false,
    "ipMasq": true,
    "hairpinMode": true,
    "ipam": {
        "type": "host-local",
        "subnet": "10.255.0.0/24"     # Node 1
        "subnet": "10.255.1.0/24"     # Node 2
    }
}

But, the pods of node1 cannot access the pods of node 2 and vice versa.

What you expected to happen:

All of the pods can access each other.

How to reproduce it:

Deploy a multiple nodes cluser and config OCCM and CNI plugin as above.

Anything else we need to know?:

IMO, This due to the node's security group has no ingress rule to permit the network packet of other node's pods through directly

Environment:

• openstack-cloud-controller-manager(or other related binary) version: xxx
• OpenStack version: xxx
• Others: xxx

[dulek]

I think CPO expects CAPO or other deployment tool to manage this. Why can't it be done there?

[jeffyjf]

I think CPO expects CAPO or other deployment tool to manage this. Why can't it be done there?

As I mentioned here, This is route controller's duty to ensure the containers on different nodes in one Kubernetes cluster can communicate with each other.

[kayrus]

@jeffyjf are you sure that the problem lies in node's security groups? could it be related to #2491?

[jeffyjf]

@jeffyjf are you sure that the problem lies in node's security groups?

Yep, I'm sure. I've already done test, add extra node's security group can deal with this issue.

could it be related to #2491?

They are different issues. For ingress network traffic, the AllowedAddressPairs just used to check destination address, the SecurityGroupRule used to check source address. They must all be set for a new node.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.