From e91dc8787e94ec5adeb0b158871f7477dc14d2ef Mon Sep 17 00:00:00 2001 From: David Cassany Date: Tue, 22 Oct 2024 13:48:09 +0200 Subject: [PATCH] Update custom installation docs Signed-off-by: David Cassany --- docs/custom-install.md | 305 ++++++++++++++----------------------- docs/extra-rpms.md | 100 ++++++++++++ docs/lvm-drives-example.md | 102 +++++++++++++ sidebars.js | 12 +- 4 files changed, 324 insertions(+), 195 deletions(-) create mode 100644 docs/extra-rpms.md create mode 100644 docs/lvm-drives-example.md diff --git a/docs/custom-install.md b/docs/custom-install.md index df7a6b7c9..dce220a3f 100644 --- a/docs/custom-install.md +++ b/docs/custom-install.md @@ -12,71 +12,70 @@ title: '' Elemental installed OS images can be customized in different ways. One option is to remaster container OS images by simply using a docker build. -SLE Micro images are regular container images, so it is absolutely possible to create +SL Micro images are regular container images, so it is absolutely possible to create a new image using a Dockerfile based on SLE Micro. See [Build Custom OS Images](/custom-images.md) -section for further details on that possibility. +section for further details, this is the preferred option. -Alternatively, it is also possible to provide additional resources within the installation -media so that during installation, or eventually at boot time, additional binaries such as -drivers or extra configuration files can be included. +Alternatively, it is also possible to provide additional resources and configuration +within the installation media so that during installation, or eventually at boot time, +additional binaries such as drivers or extra configuration files can be included. This section focuses on how to customize the installation process from a given OS image. -## Customize installation ISO and installation process +## Customization options -In order to adapt the installation ISO a simple approach is to append extra configuration -files into the ISO root in an analog way the registration yaml configuration file -is added. +Elemental installation can be customized in three different non-exclusive ways. First, +including additional cloud-config files into the installed system, second, by including +additional cloud-config file into the installation media, and finally, by adding a custom +Elemental client configuration file (`/etc/elemental/config.yaml`). -### Common customization pattern +1. The additional cloud-config files included into the installed system are useful to run + custom operations at boot time. See the [Cloud Config Reference](cloud-config-reference.md). -Elemental installation can be customized in three different non-exclusive ways. First, including -some custom Elemental client configuration file, second, by including additional cloud-init files to execute at -boot time, and finally, by including `cloud-init` files such as installation hooks or boot stages evaluated during -the live system boot itself. +2. The additional cloud-config files into the installation media are useful to run custom operations + at install time or to customize the installation environment to match certain specific needs. -A common pattern is to combine the three ways described above. This pattern will allow you to add custom steps during the installation and add `cloud-init` files to be evaluated at boot time. +3. A custom Elemental client configuration file is, by default, located at `/etc/elemental/config.yaml` + and it can also be splitted into multiple yaml files under `/etc/elemental/config.d` directory. + See the configuration file + [reference](https://rancher.github.io/elemental-toolkit/docs/customizing/general_configuration/). -Additional config files can be added dynamically boot time by generating the ISO via a [SeedImage](/seedimage-reference.md) including custom cloud-config data. +A common pattern is to combine the three ways described above depending on the specific needs. -To apply this pattern, the following files need to be included in the ISO or generated at boot time: +### Adding additional cloud-config files within the installed OS -1. A [configuration file](https://rancher.github.io/elemental-toolkit/docs/customizing/general_configuration/) - for the elemental client. The file must be named `config.yaml` and located by default in `/etc/elemental`. - This path can be configured as part of the installation parameters of a [MachineRegistration](/machineregistration-reference) resource. +In order to include additional cloud-config files during the installation they need +to be added to the installation data into the MachineRegistration resource. The most simple +way to achieve it is by adding the desired cloud-config directly as part of the MachineRegistration [devoted +section](machineregistration-reference.md#configcloud-config). See the example below -2. The additional `cloud-init` files to be included into the installed system. They - allow to perform custom operations at boot time. -3. The installation hooks are evalutated at install time. They allow to perform custom operations - during the installation process (include extra software, set additional disks...). The same - way `cloud-init` files can be included to perform custom operations during the live installation - media boot time. - -#### Custom Elemental client configuration file - -[Elemental client](https://github.com/rancher/elemental-toolkit/blob/main/docs/elemental.md) `install`, `upgrade` and `reset` commands can be configured with a [custom configuration file](https://rancher.github.io/elemental-toolkit/docs/customizing/general_configuration/) located by default in `/etc/elemental/config.yaml`. -If you have multiple yaml files, you need to add them in the `/etc/elemental/config.d` directory. - -A simple example to set hooks location could be: - -```yaml -extra-partitions: - - size: 10240 - fs: ext4 - label: EXTRA_PARTITION +```yaml showLineNumbers +apiVersion: elemental.cattle.io/v1beta1 +kind: MachineRegistration +metadata: + name: my-nodes + namespace: fleet-default +spec: + ... + config: + ... + cloud-config: + stages: + boot: + - name: "Setting root user password" + users: + root: + passwd: root ``` -The above example sets an additional extra partition during the installation. - -#### Adding additional cloud-init files within the installed OS - -In order to include additional cloud-init files during the installation they need -to be added to the installation data into the MachineRegistration resource. More specific -the `config-urls` field is used for this exact purpose. See [MachineRegistration reference](/machineregistration-reference) page. +Alternatively, cloud-config files path can also be referenced explicitly so the configuration is not +extrictly needed to live within the MachineRegistration resource itself. The `config-urls` section of the +MachineRegistration is used for this exact purpose. See +[MachineRegistration reference](/machineregistration-reference) page. `config-urls` is a list of string literals where each item is an HTTP URL or a local path pointing to a -cloud-init file. The local path is evaluated during +cloud-config file. The local path is evaluated during the installation, hence it must exists within the installation media, commonly an ISO image. By default, Elemental live systems mount the ISO root at `/run/initramfs/live` which is also the default path set for `config-url` in `MachineRegistrations`: @@ -106,165 +105,83 @@ According to that, the ISO for the example above is expected to include the `/oe `/run/initramfs/live` is a readonly mountpoint and it's not an appropriate path for dynamically generated content at ISO boot. ::: -#### Adding installation hooks or boot stages for the live system - -[Elemental client](https://github.com/rancher/elemental-cli) `install`, `upgrade` and `reset` procedures include four different hooks: - -* `before-install`: executed after all partition mountpoints are set. -* `after-install-chroot`: executed after deploying the OS image and before unmounting the associated loop filesystem image. Runs chrooted to the OS image. -* `after-install`: executed just after the after-install-chroot hook. It is not chrooted. -* `post-install`: executed as the very last step before ending the installation, partitions are still mounted, the loop devices for the image is not. - -Hooks are provided as `cloud-init` stages. Equivalent hooks exist for `reset` and `upgrade` procedures. -They are loaded from the `/iso-config` folder in ISO filesystem root. In fact, hooks are regular `cloud-init` stages with the -only difference that Elemental client parses them during `install`, `upgrade` or `reset` actions, rather than boot time. -Note any boot stage included this way will be executed during the live installation media boot. - -Hooks are evaluated during `install`,`reset` and `upgrade` action from `/oem`, `/system/oem` and `/usr/local/cloud-config` paths, -however for the live ISOs there is an additional the path `/run/initramfs/live/iso-config` included. Note the `/run/initramfs/live` -prefix is the mount point of the ISO filesystem of the Elemental Live ISO once booted. - -### Adding extra driver binaries into the ISO example - -This example is covering the case in which extra driver binaries are included into the ISO -and during the installation they are installed over the OS image. +### Adding additional cloud-config files within the installation media -For that use case the following files are required: +Adding additional cloud-config files within he installtion media might be required to configure the +installation environment (e.g. setting the network connectivity to register) or to provide some +[installation hooks](cloud-config-reference.md#elemental-client-cloud-config-hooks) to run custom +logic during the installation itself. -* additional binaries to install (they could be in the form of RPMs) -* additional hooks file to copy binaries into the persistent storage and to install them - -Let's create an `overlay` directory to create the directory tree that needs to be -added into the ISO root. In that case the `overlay` directory could contain: - -```yaml showLineNumbers -overlay/ - data/ - extra_drivers/ - some_driver.rpm - iso-config/ - install_hooks.yaml -``` - -The `overlay/iso-config/install_hooks.yaml` could be as: +In Elemental the [SeedImage](seedimage-reference.md) resources are the responsibles of handling the installation +media. So the simplest place to include additional cloud-config data is within the installation media is by +adding it to the [`cloud-config` section](seedimage-reference.md#seedimagespec-reference). By doing so the +given cloud-config will be already evaluated form the very first boot of the installation media and also +honored during the installation phase in case some install hook is provided. Consider the following example: ```yaml showLineNumbers -name: "Install extra drivers" -stages: - before-install: - # Preload data to the persistent storage - # During installation persistent partition is mounted at /run/cos/persistent - - commands: - - rsync -a /run/initramfs/live/data/ /run/cos/persistent - after-install-chroot: - # extra_drivers folder is at `/usr/local/extra_drivers` from the OS image chroot - - commands: - - rpm -iv /usr/local/extra_drivers/some_driver.rpm +apiVersion: elemental.cattle.io/v1beta1 +kind: SeedImage +metadata: + name: custom-seed + namespace: fleet-default +spec: + ... + cloud-config: + stages: + post-install: + - name: "Run custom script after installation" + commands: + - | + echo "This is a custom script" + echo "For instance, this could be used to handle extra drives for an LVM group" + boot: + - name: "Add proxy setup for the installation media" + files: + - path: /etc/sysconfig/proxy + permissions: 0664 + content: | + PROXY_ENABLED="yes" + HTTP_PROXY=http://: + HTTPS_PROXY=https://: + NO_PROXY="localhost, 127.0.0.1" ``` -Note the installation hooks only cover installation procedures, for upgrades equivalent -`before-upgrade` and/or `after-upgrade-chroot` should be defined. - -### Adding extra LVM volume group disks during the installation - -This example is covering the setup of an host with multiple disks and some of them used -as part of an LVM setup. - -As an example, we have an host with three disks (`/dev/sda`, `/dev/sdb` -and `/dev/sdc`). - -The first disk is used for a regular Elemental installation -and the other remaining two are used as part of a LVM group where arbitrary logical volumes -are created, formatted and mounted at boot time via an extended `fstab` file. - -For this example, the following files are required: - -* additional `clout-init` files included in the installed system -* additional installation hooks to prepare the LVM volumes during the installation - -Let's create an `overlay` directory to create the directory tree that needs to be -added into the ISO root. In that case the `overlay` directory could contain: +### Custom Elemental client configuration file -```yaml showLineNumbers -overlay/ - oem/ - lvm_volumes_in_fstab.yaml - iso-config/ - lvm_volumes_hook.yaml -``` +[Elemental client](https://github.com/rancher/elemental-toolkit/blob/main/docs/elemental.md) `install`, `upgrade` and `reset` commands can be configured with a [custom configuration file](https://rancher.github.io/elemental-toolkit/docs/customizing/general_configuration/) located by default in `/etc/elemental/config.yaml`. +If you have multiple yaml files, you need to add them in the `/etc/elemental/config.d` directory. -The installation hook `overlay/iso-config/lvm_volumes_hook.yaml`: +A simple example to set hooks location could be: ```yaml showLineNumbers -name: "Create LVM logic volumes over some physical disks" -stages: - post-install: - - name: "Create physical volume, volume group and logical volumes" - if: '[ -e "/dev/sdb" ] && [ -e "/dev/sdc" ]' - commands: - - | - # Create the physical volume, volume group and logical volumes - pvcreate /dev/sdb /dev/sdc - vgcreate elementalLVM /dev/sdb /dev/sdc - lvcreate -L 8G -n elementalVol1 elementalLVM - lvcreate -l 100%FREE -n elementalVol2 elementalLVM - - # Trigger udev detection - if [ ! -e "/dev/elementalLVM/elementalVol1" ] || [ ! -e "/dev/elementalLVM/elementalVol2" ]; then - sleep 10 - udevadm settle - fi - - # Ensure devices are already available - [ -e "/dev/elementalLVM/elementalVol1" ] || exit 1 - [ -e "/dev/elementalLVM/elementalVol2" ] || exit 1 - - # Format logical volumes with a known label for later use in fstab - mkfs.xfs -L eVol1 /dev/elementalLVM/elementalVol1 - mkfs.xfs -L eVol2 /dev/elementalLVM/elementalVol2 +extra-partitions: +- size: 10240 + fs: ext4 + label: EXTRA_PARTITION ``` -The LVM devices are created and formatted as desired. This is a good -example of an installation hook, as this setup is only needed once, at installation -time. As an alternative, the same action could be done on first boot, however it would -require a more sophisticated logic to ensure it's only applied once at first boot. - -Finally, the boot time `cloud-init` files contain the mount points settings and trigger the -action of mounting those mountpoints. The Elemental OS `fstab` file is ephemeral and it's -dynamically created at boot time. That's why it doesn't exist during the installation and -can't be used in an installation hook. - -Here's an example of `overlay/oem/lvm_volumes_in_fstab.yaml`: +The above example sets an additional extra partition during the installation. In order to make it available at +at installation time this could be done my adding the extra file as part of the SeedImage resource cloud-config +as it is described in the previous section of this page. Consider the example: ```yaml showLineNumbers -name: "Mount LVM volumes" -stages: - initramfs: - - name: "Extend fstab to mount LVM logical volumes at boot" - commands: - - | - echo "LABEL=eVol1 /usr/local/eVol1 xfs defaults 0 0" >> /etc/fstab - echo "LABEL=eVol2 /usr/local/eVol2 xfs defaults 0 0" >> /etc/fstab -``` - -:::note -The `initramfs` stage is the last stage before switching to the actual root tree. -At this stage, the `/etc/fstab` file already exists and can be adapted before -switching root. Once running in the final root tree, SystemD will handle the rest of the initialization and apply it. -::: - -This cloud-init file should be included into the `/oem` directory on the installed -system. `/oem` is a mount point for the OEM partition. In order to include extra files, -they should be listed as `config-urls` within the Registration CRD at the -management cluster. - -### Repacking the ISO image with extra files - -Assuming an `overlay` folder was created in the current directory containing all -additional files to be appended, the following `xorriso` command adds the extra files: - -```bash showLineNumbers -xorriso -indev elemental.x86_64.iso -outdev elemental.custom.x86_64.iso -map overlay / -boot_image any replay +apiVersion: elemental.cattle.io/v1beta1 +kind: SeedImage +metadata: + name: custom-seed + namespace: fleet-default +spec: + ... + cloud-config: + stages: + boot: + - name: "Add Elemental client configuration file" + files: + - path: /etc/elemental/config.d/extra-partition.yaml + permissions: 0664 + content: | + extra-partitions: + - size: 10240 + fs: ext4 + label: EXTRA_PARTITION ``` - -For that a `xorriso` equal or higher than version 1.5 is required. diff --git a/docs/extra-rpms.md b/docs/extra-rpms.md new file mode 100644 index 000000000..33a6ce7d8 --- /dev/null +++ b/docs/extra-rpms.md @@ -0,0 +1,100 @@ +--- +sidebar_label: Add third party RPMs +title: '' +--- + + + + + +## How to add third party RPMs at install time + +This example is covering the case in which extra RPMS (e.g. specific hardware drivers) are included +into the ISO and during the installation they are installed over the OS image. + +For that use case the following files are required: + +* additional RPMs to install +* additional cloud-config file defining hooks to copy and install binaries as installation hooks + +We can handle it all into a single `SeedImage.spec.cloud-config` section assuming the additional +RPM is available to download it from a remote server at installation time. + +Consider the following cloud-config data which could be used as the content of the `cloud-config` +section in a [SeedImage resource](seedimage-reference.md#seedimagespec-reference). + +```yaml showLineNumbers +name: "Install extra drivers" +stages: + before-install: + # Preload data to the persistent storage + # During installation persistent partition is mounted at /run/elemental/persistent + - downloads: + - path: /tmp/some_package.rpm + url: "" + permissions: 0777 + - commands: + - mkdir -o /run/elemental/persistent/extra-pkgs + - cp -p /tmp/some_package.rpm /run/elemental/persistent/extra-pkgs + + after-install-chroot: + # Install the package at install time + - commands: + - rpm -iv /run/elemental/extra-pkgs/some_package.rpm + + # Include to the install system analog upgrade and reset hooks + - files: + - path: /oem/extra-pkg.yaml + permissions: 0664 + content: | + name: "Install extra drivers" + stages: + after-upgrade-chroot: + # Install the package after upgrading to a new image + - commands: + - rpm -iv /run/elemental/extra-pkgs/some_package.rpm + after-reset-chroot: + # Install the package on reset + - commands: + - rpm -iv /run/elemental/extra-pkgs/some_package.rpm +``` + +Note the installation hooks only cover installation procedures, so that additional cloud-config data should +be also part of the installed system in order to keep installing the package as part of the upgrade or reset +processes. + +### Repacking a generated ISO image with extra files + +Alternatively, if having the dynamic download of content at install time is not a desired behavior +and an ISO already including all the extra binaries is the actual goal this can is also possible +but requires manually repacking the ISO. + +Using `xorriso`, the linux utility to create ISOs, this turns to be a relatively easy process. + +Let's create an `overlay` directory to create the directory tree that needs to be +added into the ISO root. In that case the `overlay` directory could contain: + +```yaml showLineNumbers +overlay/ + data/ + extra-pkgs/ + some_package.rpm + iso-config/ + install_hooks.yaml +``` + +We are assuming the `install_hooks.yaml` is the content the actual cloud-config exposed in the previous +section which is manually included into image instead of being embedded in a SeedImage resource. The `data` folder is eventually including the binaries we want to append into the ISO. + +Assuming we already downloaded an Elemental ISO tied to a specific MachineRegistration with the following +`xorriso` call all the `overlay` folder contents would be included into a new ISO: + +```bash +xorriso -indev elemental.x86_64.iso -outdev elemental.custom.x86_64.iso -map overlay / -boot_image any replay +``` + +Requires `xorriso` equal or higher than version 1.5. + +The contents of `install_hooks.yaml` could eventually be the same as the previous section but omiting the RPM +package download and also adapting the RPM path to `/run/initramfs/live/data/extra-pkgs/some_package.rpm` as +the ISO root folder is mounted at `/run/initramfs/live`. diff --git a/docs/lvm-drives-example.md b/docs/lvm-drives-example.md new file mode 100644 index 000000000..1375bc478 --- /dev/null +++ b/docs/lvm-drives-example.md @@ -0,0 +1,102 @@ +--- +sidebar_label: Extra LVM volume group +title: '' +--- + + + + + +## How to add extra LVM volume group disks during the installation + +This example is covering the setup of a host with multiple disks and some of them used +as part of a LVM setup. + +As an example, we have an host with three disks (`/dev/sda`, `/dev/sdb` +and `/dev/sdc`). + +The first disk is used for a regular Elemental installation +and the other remaining two are used as part of a LVM group where arbitrary logical volumes +are created, formatted and mounted at boot time via an extended `fstab` file. + +For this example cloud-config steps are required in two different stages. First, some +[installation hooks](cloud-config-reference.md#elemental-client-cloud-config-hooks) are +needed to prepare and handle LVM volumes during the installation. Second, a cloud-config +is required at boot time to ensure the created LVM volumes are included in `/etc/fstab` +and consequently mounted. + +Installation hooks can be included to the `SeedImage.spec.cloud-config` section with something +like: + +```yaml showLineNumbers +apiVersion: elemental.cattle.io/v1beta1 +kind: SeedImage +metadata: + name: custom-seed + namespace: fleet-default +spec: + ... + cloud-config: + name: "Create LVM logic volumes over some physical disks" + stages: + post-install: + - name: "Create physical volume, volume group and logical volumes" + if: '[ -e "/dev/sdb" ] && [ -e "/dev/sdc" ]' + commands: + - | + # Create the physical volume, volume group and logical volumes + pvcreate /dev/sdb /dev/sdc + vgcreate elementalLVM /dev/sdb /dev/sdc + lvcreate -L 8G -n elementalVol1 elementalLVM + lvcreate -l 100%FREE -n elementalVol2 elementalLVM + # Trigger udev detection + if [ ! -e "/dev/elementalLVM/elementalVol1" ] || [ ! -e "/dev/elementalLVM/elementalVol2" ]; then + sleep 10 + udevadm settle + fi + # Ensure devices are already available + [ -e "/dev/elementalLVM/elementalVol1" ] || exit 1 + [ -e "/dev/elementalLVM/elementalVol2" ] || exit 1 + # Format logical volumes with a known label for later use in fstab + mkfs.xfs -L eVol1 /dev/elementalLVM/elementalVol1 + mkfs.xfs -L eVol2 /dev/elementalLVM/elementalVol2 +``` + +The LVM devices are created and formatted as desired. This is a good +example of an installation hook, as this setup is only needed once, at installation +time. As an alternative, the same action could be done on first boot, however it would +require a more sophisticated logic to ensure it's only applied once at first boot. + +Finally, the boot time cloud-config data contains the mount point settings to trigger the +mounts. The Elemental OS `fstab` file is ephemeral and it's dynamically created at boot time. +That's why it doesn't exist during the installation and can't be used in an installation hook. + +Consider the following example to customize the `/etc/fstab` file using the `cloud-config` section +of a MachineRegistration resource: + +```yaml showLineNumbers +apiVersion: elemental.cattle.io/v1beta1 +kind: MachineRegistration +metadata: + name: my-nodes + namespace: fleet-default +spec: + ... + config: + ... + cloud-config: + stages: + initramfs: + - name: "Extend fstab to mount LVM logical volumes at boot" + commands: + - | + echo "LABEL=eVol1 /usr/local/eVol1 xfs defaults 0 0" >> /etc/fstab + echo "LABEL=eVol2 /usr/local/eVol2 xfs defaults 0 0" >> /etc/fstab +``` + +:::note +The `initramfs` [stage](cloud-config-reference.md) is the last stage before switching to the actual +root tree. At this stage, the `/etc/fstab` file already exists and can be adapted before +switching root. Once running in the final root tree, SystemD will handle the rest of the +initialization and apply it. +::: diff --git a/sidebars.js b/sidebars.js index a91de2ef9..b420dd132 100644 --- a/sidebars.js +++ b/sidebars.js @@ -57,7 +57,6 @@ const sidebars = { "installation", "upgrade", "upgrade-lifecycle", - "custom-install", "elemental-plans", ] }, @@ -142,6 +141,17 @@ const sidebars = { "label": "How to", "items": [ "custom-images", + { + "type": "category", + "collapsible": true, + "collapsed": true, + "label": "Customize Elemental Installation", + "link": {"type": "doc", "id": "custom-install"}, + "items": [ + "extra-rpms", + "lvm-drives-example" + ] + }, "elemental_behind_proxy", "hostname", "ntp",