Add support for kubernetes device-ownership-from-security-context

[vigh-m]

Description of changes:

• Add default support for the new setting being introduced in
  • feat: Add kubernetes device ownership bottlerocket-settings-sdk#69
  • Use kubernetes device-ownership setting bottlerocket-core-kit#329
• Add migration for the new setting

Testing done:

• The setting shows up as expected in an AMI built using this settings-sdk

  # apiclient get settings.kubernetes.device-ownership-from-security-context
  {
    "settings": {
      "kubernetes": {
        "device-ownership-from-security-context": false
      }
    }
  }
  

• Changing the setting works as defined:

  [root@admin]# apiclient set settings.kubernetes.device-ownership-from-security-context="true"
  [root@admin]# apiclient get settings.kubernetes.device-ownership-from-security-context
  {
    "settings": {
      "kubernetes": {
        "device-ownership-from-security-context": true
      }
    }
  }
  [root@admin]# sheltie cat /etc/containerd/config.toml | grep device
  device_ownership_from_security_context = true
  

• The setting has the desired effect on device ownership

  • When true:

    # kubectl exec -it single-node-test -- bash
    ubuntu@single-node-test:/$ ls -lah /dev/ | grep neuron
    crw-------. 1 ubuntu 2000 244, 0 Dec 20 00:25 neuron0
    

  • When false:

    # kubectl exec -it single-node-test -- bash
    ubuntu@single-node-test:/$ ls -lah /dev/ | grep neuron
    crw-------. 1 root root 244, 0 Dec 19 20:03 neuron0
    

• migration testing
  Starting AMI -> bottlerocket-aws-k8s-1.30-x86_64-v1.29.0-c55d099c

  # updog check-update -a --json
  [
    {
      "variant": "aws-k8s-1.30",
      "arch": "x86_64",
      "version": "1.29.1",
      "max_version": "1.29.1",
      "waves": {
        "0": "2024-12-24T19:26:27.976238949Z",
        "20": "2024-12-24T22:26:27.976238949Z",
        "102": "2024-12-25T18:26:27.976238949Z",
        "307": "2024-12-26T18:26:27.976238949Z",
        "819": "2024-12-28T18:26:27.976238949Z",
        "1228": "2024-12-29T18:26:27.976238949Z",
        "1843": "2024-12-30T18:26:27.976238949Z"
      },
      "images": {
        "boot": "bottlerocket-aws-k8s-1.30-x86_64-1.29.1-054c931f-dirty-boot.ext4.lz4",
        "root": "bottlerocket-aws-k8s-1.30-x86_64-1.29.1-054c931f-dirty-root.ext4.lz4",
        "hash": "bottlerocket-aws-k8s-1.30-x86_64-1.29.1-054c931f-dirty-root.verity.lz4"
      }
    }
  ]
  

  On Upgrade, the settings show up as expected

  # apiclient get settings.kubernetes.device-ownership-from-security-context
  {
    "settings": {
      "kubernetes": {
        "device-ownership-from-security-context": false
      }
    }
  }
  

  # cat /var/lib/bottlerocket/datastore/current/live/settings/kubernetes/device-ownership-from-security-context
  false
  

  bash-5.1# cat /etc/containerd/config.toml | grep device_ownership
  device_ownership_from_security_context = false
  

  On downgrade, the settings are removed

  # apiclient get settings.kubernetes.device-ownership-from-security-context
  {}
  

  # cat /var/lib/bottlerocket/datastore/current/live/settings/kubernetes/device-ownership-from-security-context
  cat: /var/lib/bottlerocket/datastore/current/live/settings/kubernetes/device-ownership-from-security-context: No such file or directory
  

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

[ytsssun]

You will also want to add migration for the affected services so that the containerd service gets restarted upon migration.

Reference this code - https://github.com/bottlerocket-os/bottlerocket/blob/develop/sources/settings-migrations/v1.23.0/nvidia-container-runtime-metadata/src/main.rs#L7-L10

[vigh-m]

That makes sense.

Should the Metadata Migration be in addition to the existing SettingsMigration or in place of it?

From my read of the migration logic for AddMetadataMigration it looks like it only removes the affected-services and not the actual setting from the datastore.

[vigh-m]

Discussed offline. Reference PR #4182 where both a settings and a metadata migrations were added to support a new setting