diff --git a/content/docs/latest/_index.md b/content/docs/latest/_index.md index 8f84ccc1..a183edfd 100644 --- a/content/docs/latest/_index.md +++ b/content/docs/latest/_index.md @@ -109,6 +109,7 @@ more. * [Configuring the update strategy][update-strategies] * [Flatcar update configuration specification][update-conf] * [Verifying Flatcar Images with GPG][verify-container-linux] + * [Nebraska][nebraska] #### Creating Clusters * [Cluster architectures][cluster-architectures] @@ -263,3 +264,4 @@ Flatcar tutorial to deep dive into some Flatcar fundamental concepts. [scaleway]: installing/community-platforms/scaleway [ovhcloud]: installing/community-platforms/ovhcloud [akamai]: installing/community-platforms/akamai +[nebraska]: nebraska diff --git a/content/docs/latest/nebraska/_index.md b/content/docs/latest/nebraska/_index.md new file mode 100644 index 00000000..cf6fd7b7 --- /dev/null +++ b/content/docs/latest/nebraska/_index.md @@ -0,0 +1,13 @@ +--- +content_type: nebraska +title: Nebraska +main_menu: true +weight: 40 +cascade: + github_edit_url: https://github.com/flatcar/flatcar-website/tree/master/content/docs/latest/nebraska + issues_url: https://github.com/flatcar/nebraska/issues/new +--- + +Nebraska is an update manager for [Flatcar Container Linux](https://flatcar.org/). + +It offers an easy way to monitor and manage the rollout of updates to applications that use the Omaha protocol, with special functionality for Flatcar Container Linux updates. diff --git a/content/docs/latest/nebraska/authorization/_index.md b/content/docs/latest/nebraska/authorization/_index.md new file mode 100644 index 00000000..b2470d59 --- /dev/null +++ b/content/docs/latest/nebraska/authorization/_index.md @@ -0,0 +1,348 @@ +--- +title: Deployment & Authorization +weight: 10 +--- + +Nebraska uses either a noop authentication or OIDC to authenticate and authorize users. + +# Preparing the database for Nebraska + +Nebraska uses the `PostgreSQL` database, and expects the used database to +be set to the UTC timezone. + +For a quick setup of `PostgreSQL` for Nebraska's development, you can use +the `postgres` container as follows: + +- Start `Postgres`: + - `docker run --rm -d --name nebraska-postgres-dev -p 5432:5432 -e POSTGRES_PASSWORD=nebraska postgres` + +- Create the database for Nebraska (by default it is `nebraska`): + - `psql postgres://postgres:nebraska@localhost:5432/postgres -c 'create database nebraska;'` + +- Set the timezone to Nebraska's database: + - `psql postgres://postgres:nebraska@localhost:5432/nebraska -c 'set timezone = "utc";'` + +## Tuning PostgreSQL auto-vacuum + +Autovacuum and autoanalyze in PostgreSQL are effectively disabled when tables +are very large. This is because the default is 20% of a table (and 10% +of a table for analyze). + +We advise to change the mentioned configuration in order to have autovacuum +and autoanalyse run when about 5,000 rows change. This value was chosen +based on getting the autovacuum to run every day, as it's large enough +to not cause the autovac to run all the time, but about the right size +to make a difference for query statistics and reducing table bloat. + +You can verify (and eventually use) this [SQL file](./autovacuum-tune.sql) +where we have set up these changes. + +The analyze threshold was chosen at half the autovacuum threshold +because the defaults are set at half. + +# Deploying Nebraska for testing on local computer (noop authentication) + +- Go to the nebraska project directory and run `make` + +- Start the database (see the section above if you need a quick setup). + +- Start the Nebraska backend: + + - `nebraska -auth-mode noop -http-static-dir $PWD/frontend/build + -http-log` + +- In the browser, access `http://localhost:8000` + +# Deploying Nebraska with OIDC authentication mode + +- Go to the nebraska project directory and run `make` + +- Start the database (see the section above if you need a quick setup). + +- Setup OIDC provider (Keycloak Recommended). + +- Start the Nebraska backend: + + - `nebraska --auth-mode oidc --oidc-admin-roles nebraska_admin --oidc-viewer-roles nebraska_member --oidc-client-id nebraska --oidc-issuer-url http://localhost:8080/auth/realms/master --oidc-client-secret ` + + Note: If roles array in the token is not in `roles` key, one can specify a custom JSON path using the `oidc-roles-path` flag. + +- In the browser, access `http://localhost:8000` + +# Preparing Keycloak as an OIDC provider for Nebraska + +- Run `Keycloak` using docker: + - `docker run -p 8080:8080 -e KEYCLOAK_USER=admin -e KEYCLOAK_PASSWORD=admin -d quay.io/keycloak/keycloak:13.0.1` + +- Open http://localhost:8080 in your browser to access keycloak UI and login with the username admin and password as admin. + +## Creating Roles + +### Member Role + +1. Click on `Roles` menu option and select `Add Role`. +2. Provide a name for the member role, here we will use `nebraska_member`. +3. Click `Save`. + +### Admin Role + +1. Click on `Roles` menu option and select `Add Role`. +2. Provide a name for the admin role, here we will use `nebraska_admin`. +3. Click `Save`. +4. After the admin role is created enable composite role to ON. In the Composite Roles section select the member role, In our case it is nebraska_member and click Add Selected. + +Now the member and admin roles are created, the admin role is a composite role which comprises of member role. + +

+ +

+ +## Creating a client + +1. Click on `Clients` menu option and click `Create`. +2. Set the client name as `nebraska` and click `Save`. +3. Change the `Access Type` to `Confidential` +4. Set `Valid Redirect URIs` to `http://localhost:8000/login/cb`. + +

+ +

+ +## Adding roles scope to token + +1. Click on `Mappers` tab in Client Edit View. Click on `Create`. +2. Set the name as `roles`, Select the `Mapper Type` as `User Realm Role`, `Token Claim Name` as `roles` and Select `Claim JSON Type` as String. +3. Click `Save` + +

+ +

+ +## Attaching Roles to User + +1. Click on `Users` menu option and click `View all users`. +2. Once the user list appears select the user and click on `Edit`. +3. Go to `Role Mapping` tab and select `nebraska_admin` role and click on add selected to attach role to user. If you want to provide only member access access select the member role. + +

+ +

+ +# Preparing Auth0 as an OIDC provider for Nebraska +## Create and configure new application + +1. Click on `Create Application`. +2. Provide the name as `nebraska`, select `Regular Web Application`. +3. Click `Create` +4. Click on the `settings` tab. +5. Under `Application URIs` section provide the `Allowed Callback URLs` as `http://localhost:8000/login/cb`. +6. Click on `Save Changes` + +

+ +

+ + +## Adding roles scope to token + +1. Click on `Rules` sub-menu from `Auth Pipeline` menu option. +2. Click on `Empty Rule` option. +3. Provide the name as `roles`. +4. Paste the following snippet in `Script` text box. +```js +function (user, context, callback) { + const namespace = 'http://kinvolk.io'; + const assignedRoles = (context.authorization || {}).roles; + + let idTokenClaims = context.idToken || {}; + let accessTokenClaims = context.accessToken || {}; + + idTokenClaims[`${namespace}/roles`] = assignedRoles; + accessTokenClaims[`${namespace}/roles`] = assignedRoles; + + context.idToken = idTokenClaims; + context.accessToken = accessTokenClaims; + callback(null, user, context); +} +``` +Now the rule to add the roles to the token is setup, the roles will be available in the key `http://kinvolk.io/roles`. + +Note: The `oidc-roles-path` argument accepts a JSONPath to fetch roles from the token, in this case set the value to `http://kinvolk\.io/roles`. + +

+ +

+ +# Preparing Dex with github connector as an OIDC provider for Nebraska + +## Setting up a Github App to be used as a connector for Dex + +- Create a new `organization` in Github. + +- Now you need a Github app, go to `https://github.com/organizations//settings/apps/new` and fill + the following fields: + + - `GitHub App name` - just put some fancy name. + + - `Homepage URL` - `http://localhost:8000` + + - `User authorization callback URL` - `http://0.0.0.0:5556/dex/callback` + + - `Permissions` - `Access: Read-only` to `Organization members` + + - `User permissions` - none needed + + - `Subscribe to events` - tick `Membership`, `Organization` and `Team` + + - `Where can this GitHub App be installed?` - `Only on this account` + +- Press `Create GitHub App` button + +- Next thing you'll get is `OAuth credentials` at the bottom of the + page of the app you just created, we will need both `Client ID` and + `Client secret` + +- You also need to install the app you just created + + - Go to `https://github.com/organizations//settings/apps` + + - Click `Edit` button for your new app + + - Choose `Install App` on the left of the page and perform the + installation + +## Creating Github Teams + +- Create two teams in your organization with the following names + `admin` and `viewer`. + +- Add the admin users to both `admin` and `viewer` team. Add the non-admin users to +`viewer` team. + +## Configuring and Running Dex IDP + +- Create a configuration for Dex based on the example. + +> example.yaml + +```yaml +issuer: http://0.0.0.0:5556/dex + +storage: + type: sqlite3 + config: + file: /var/dex/dex.db + +web: + http: 0.0.0.0:5556 + +staticClients: + - id: nebraska + redirectURIs: + - 'http://localhost:8000/login/cb' + name: 'nebraska' + secret: // Random Hash + +connectors: +- type: github + id: github + name: GitHub + config: + clientID: + clientSecret: + redirectURI: http://0.0.0.0:5556/dex/callback + loadAllGroups: true + teamNameField: slug + useLoginAsID: true + +enablePasswordDB: true +``` + +- Run Dex using docker with the example configuration. + +> docker run -p 5556:5556 -v ${PWD}/example.yaml:/etc/dex/example.yaml -v ${PWD}/dex.db:/var/dex/dex.db ghcr.io/dexidp/dex:v2.28.1 dex serve /etc/dex/example.yaml + + +## Running nebraska + +> go run ./cmd/nebraska --auth-mode oidc --oidc-admin-roles :admin --oidc-viewer-roles :viewer --oidc-client-id nebraska --oidc-issuer-url http://127.0.0.1:5556/dex --oidc-client-secret --oidc-roles-path groups --oidc-scopes groups,openid,profile + +# Deploying on Kubernetes using the Helm Chart + +We maintain a Helm Chart for deploying a Nebraska instance to Kubernetes. The +Helm Chart offers flexible configuration options such as: + +- Deploy a single-replica `PostgreSQL` database together with Nebraska. We use + the container image and also the Helm Chart (as a subchart) from + [Bitnami](https://github.com/bitnami/charts/tree/master/bitnami/postgresql) + +- Enabling / disabling, and configuring persistence for both Nebraska and PostgreSQL + (persistence is disabled by default) + +- Common deployment parameters (exposing through `Ingress`, replica count, etc.) + +- All [Nebraska application configuration](https://github.com/flatcar/nebraska/tree/main/charts/nebraska#nebraska-configuration) + +For the complete list of all available customization options, please read the +[Helm Chart README](https://github.com/flatcar/nebraska/blob/main/charts/nebraska/README.md). + +To install the Helm Chart using the default configuration (noop authentication), +you can execute: + +```console +$ helm repo add nebraska https://flatcar.github.io/nebraska +$ helm install my-nebraska nebraska/nebraska +``` + +You probably need to customize the installation, then use a Helm values file. +Eg.: + +```yaml +# nebraska-values.yaml +config: + app: + title: Nebraska + + auth: + mode: github + github: + clientID: + clientSecret: + sessionAuthKey: <64 random hexadecimal characters> + sessionCryptKey: <32 random hexadecimal characters> + webhookSecret: + +ingress: + annotations: + kubernetes.io/ingress.class: + hosts: + - nebraska.example.com + +postgresql: + postgresqlPassword: + persistence: + enabled: true + storageClass: + accessModes: + - ReadWriteOnce + size: 1Gi +``` + +Then execute: + +```console +$ helm install my-nebraska nebraska/nebraska --values nebraska-values.yaml +``` + +# Troubleshooting: + +- I'm getting a blank page! + + - You likely visited nebraska frontend website before, so browser + likely has cached the `index.html` page, so it won't get it from + Nebraska, but instead start asking for some CSS and javascript + stuff outright, which it won't get. That results in a blank + page. Force the browser to get `index.html` from Nebraska by + either doing a force refresh (ctrl+f5 on firefox), or by cleaning + the cache for localhost (or the server where the Nebraska instance + is deployed). We will try to improve this in the future. diff --git a/content/docs/latest/nebraska/authorization/autovacuum-tune.sql b/content/docs/latest/nebraska/authorization/autovacuum-tune.sql new file mode 100644 index 00000000..b2d85a92 --- /dev/null +++ b/content/docs/latest/nebraska/authorization/autovacuum-tune.sql @@ -0,0 +1,50 @@ +ALTER TABLE + instance +SET + ( + autovacuum_analyze_scale_factor = 0, + autovacuum_analyze_threshold = 3666, + autovacuum_vacuum_scale_factor = 0, + autovacuum_vacuum_threshold = 7333 + ); + + +ALTER TABLE + instance_application +SET + ( + autovacuum_analyze_scale_factor = 0, + autovacuum_analyze_threshold = 3666, + autovacuum_vacuum_scale_factor = 0, + autovacuum_vacuum_threshold = 7333 + ); + +ALTER TABLE + activity +SET + ( + autovacuum_analyze_scale_factor = 0, + autovacuum_analyze_threshold = 10000, + autovacuum_vacuum_scale_factor = 0, + autovacuum_vacuum_threshold = 20000 + ); + +ALTER TABLE + event +SET + ( + autovacuum_analyze_scale_factor = 0, + autovacuum_analyze_threshold = 950, + autovacuum_vacuum_scale_factor = 0, + autovacuum_vacuum_threshold = 1900 + ); + +ALTER TABLE + instance_status_history +SET + ( + autovacuum_analyze_scale_factor = 0, + autovacuum_analyze_threshold = 12500, + autovacuum_vacuum_scale_factor = 0, + autovacuum_vacuum_threshold = 25000 + ); diff --git a/content/docs/latest/nebraska/development.md b/content/docs/latest/nebraska/development.md new file mode 100644 index 00000000..948ffa52 --- /dev/null +++ b/content/docs/latest/nebraska/development.md @@ -0,0 +1,152 @@ +--- +title: Development +weight: 20 +--- + +This section shows how to set up a development environment for Nebraska. +Be also sure to read the [contributing](./contributing) section for the +general guidelines when contributing to Nebraska. + +Nebraska needs a running `PostgresSQL` database. Apart from that, the +recommended way to work on it, is to run the backend with the `noop` +authentication backend, and the frontend in a "watch" mode (where it +auto-refreshes when there are changes to the frontend). + +# Preparing the Database + +Nebraska uses the `PostgreSQL` database, and expects the used +database to be set to the UTC timezone. + +By default Nebraska uses a database with the name `nebraska`, and +`nebraska_tests` for the tests, respectively. For the main database, the full +URL (with a different database name if desired) can be overridden by the +environment variable `NEBRASKA_DB_URL`. + +For a quick setup of `PostgreSQL` for Nebraska's development, you can use +the `postgres` container as follows: + +- Start `Postgres`: + - `docker run --rm -d --name nebraska-postgres-dev -p 5432:5432 -e POSTGRES_PASSWORD=nebraska postgres` + +- Create the database for Nebraska (by default it is `nebraska`): + - `psql postgres://postgres:nebraska@localhost:5432/postgres -c 'create database nebraska;'` + +- Set the timezone to Nebraska's database: + - `psql postgres://postgres:nebraska@localhost:5432/nebraska -c 'set timezone = "utc";'` + +- Set up the nebraska_tests database for running unit tests + +```bash +psql postgres://postgres:nebraska@localhost:5432/postgres -c 'create database nebraska_tests;' +psql postgres://postgres:nebraska@localhost:5432/nebraska_tests -c 'set timezone = "utc";' +``` + +# Development Quickstart + +- Go to the Nebraska project directory and run `make` + +- Run the backend (with the `noop` authentication): `make run-backend` + +- Then, in a different terminal tab/window, run the frontend: `make run-frontend` + +Any changes to the backend means that the `make run-backend` command should be +run again. Changes to the frontend should be automatically re-built and the +opened browser page should automatically refresh. + +## Docker compose V2 + +You may be missing `docker-compose`, which is required to run some tests. + +```bash +docker-compose version + +cd backend +make check-backend-with-container +``` + +If so, see https://docs.docker.com/compose/install/other/ or https://github.com/containers/podman-compose (with setting up a `docker-compose` symlink) + +# Development Concepts + +## Frontend + +The [frontend](https://github.com/flatcar/nebraska/tree/main/frontend) side of Nebraska is a web application built using `React` and `Material-UI`. + +## Backend + +The Nebraska backend is written in Go. The backend source code is structured as follows: + +- **`pkg/api`**: provides functionality to do CRUD operations on all elements found in Nebraska (applications, groups, channels, packages, etc), abstracting the rest of the components from the underlying datastore (PostgreSQL). It also controls the groups' roll-out policy logic and the instances/events registration. + +- **`pkg/omaha`**: provides functionality to validate, handle, process and reply to Omaha updates and events requests received from the Omaha clients. It relies on the `api` package to get update packages, store events, or register instances when needed. + +- **`pkg/syncer`**: provides some functionality to synchronize packages available in the official Flatcar Container Linux channels, storing the references to them in your Nebraska datastore and even downloading packages payloads when configured to do so. It's basically in charge of keeping up to date your the Flatcar Container Linux application in your Nebraska installation. + +- **`cmd/nebraska`**: is the main backend process, exposing the functionality described above in the different packages through its http server. It provides several http endpoints used to drive most of the functionality of the dashboard as well as handling the Omaha updates and events requests received from your servers and applications. + +- **`cmd/initdb`**: is just a helper to reset your database, and causing the migrations to be re-run. `nebraska` will apply all database migrations automatically, so this process should only be used to wipe out all your data and start from a clean state (you should probably never need it). + + +### Backend Testing + +Most unit tests like beside the code for example inside `backend/pkg/`. +Tests which depend on a database mostly live in `backend/pkg/api`. +Some "server" binary integration tests are separate, and live in `backend/test/api/`. + +#### Environment variables + +Here are some test related environment variables. + +- NEBRASKA_SKIP_TESTS, if set do not run slow tests like DB using tests. +- NEBRASKA_RUN_SERVER_TESTS, if set run the integration tests. +- NEBRASKA_DB_URL, database connection URL. +- NEBRASKA_TEST_SERVER_URL, where the test server is running "http://localhost:8000" + +#### Test make targets. + +There are a number of make targets setup to run different tests. + +##### make ci + +Run all the tests that are run on CI with github actions. + +##### make code-checks + +Just build it, run quick tests, and lint it. + +Does not run tests that require a testing db, or a test server. + +##### make check + +Run tests except for the integration tests by default. Requires a test database to be running. + +You can use `NEBRASKA_RUN_SERVER_TESTS=1 make check` to also test the server integration tests, +but you need to be running a server (with `make run-backend`). + +##### make check-backend-with-container + +Run tests inside a container, including the integration tests. + +It starts it's own test server and test database. + +##### make check-code-coverage + +Like make check, but it outputs test coverage information. + +# Releasing + +To release a new version of Nebraska, first create a new `staging` git tag and push it: + +``` +git tag -d staging +git tag -as staging +git push origin --force staging +``` + +This will publish a new `ghcr.io/flatcar/nebraska:staging` image that can be tested on the [Flatcar staging instance](https://staging.updateservice.flatcar-linux.net) by redeploying (or pulling the new image and restarting the service). +When things look good, create the release tag in the format `MAJOR.MINOR.PATCH`. +This will publish a new `ghcr.io/flatcar/nebraska:MAJOR.MINOR.PATCH` and `:latest` image. +When done, continue by creating a GitHub Release for the git tag. + +Afterwards, bump the Nebraska version (`appVersion`) and the Helm chart version itself (`version`) in `charts/nebraska/Chart.yaml` in one commit. +When merged this will automatically create a new GitHub release for the Helm charts. diff --git a/content/docs/latest/nebraska/i18n/_index.md b/content/docs/latest/nebraska/i18n/_index.md new file mode 100644 index 00000000..e45ada70 --- /dev/null +++ b/content/docs/latest/nebraska/i18n/_index.md @@ -0,0 +1,31 @@ +--- +title: i18n Internationalization / Localization +linkTitle: Internationalization +--- + +Headlamp's internationalization uses the i18next, i18next-parser, and +react-i18next libraries. + +## Default language, and locales + +We started with an international English, and that will be the fallback language. + +Now we're starting with locales familiar, and will accept +translations through github. + +## Browser based language detection + +We use +[i18next-browser-languagedetector](https://github.com/i18next/i18next-browser-languageDetector#readme) + +This can select the browser language through various means like through +cookies, the html language tag and other ways. + +One way to change the locale is to use `?lng=en` in the URL. + +## Lazy load locale files + +Dynamic imports and the webpack code splitting feature we +load locale files from `src/js/i18n/locales/{{lng}}/{{ns}}.json` +where {{lng}} is the language code, and {{ns}} is the namespace. + diff --git a/content/docs/latest/nebraska/i18n/contributing.md b/content/docs/latest/nebraska/i18n/contributing.md new file mode 100644 index 00000000..dd4e64c5 --- /dev/null +++ b/content/docs/latest/nebraska/i18n/contributing.md @@ -0,0 +1,99 @@ +--- +title: Contributing to Internationalization +linkTitle: Contributing +--- + +This section introduces some concepts for contributing translations, and is +especially important when submitting a new language. + +**Important:** If you are adding a new language, keep in mind that while all +the specific Kubernetes components' names are translatable, it doesn't mean +that all of them should have a corresponding name in your language. Please, +refer to the [Kubernetes localized docs](https://kubernetes.io/docs/home/) in +your language (if they exist) in order to understand which components should +be translated and which should be left in the original form. + +## Namespaces + +[i18next namespaces](https://www.i18next.com/principles/namespaces) +are useful to keep things modular. + +We have a namespace for each app section, and also some frequently used global parts. +Namespaces are separated from the actual text by a `|` character. +E.g. `t('mynamescapce|This will be the translated text')`. + +### Frequent, and Glossary namespaces + +Additionally we have some global namespaces for frequently used and +jargony technical words. + +- frequent.json, Phrases reused many times, eg. 'save', 'cancel' +- glossary.json, Reusing these consistently inside texts like jargon words (Pods) + +#### Technical Jargon words + +For some technical/jargon terms there often isn't a good translation for +them. To find these ones, it can be good to look at existing documentation +like the k8s docs. + +The word "Pods" is a good example of a technical word that is used in Spanish. +Maybe it could directly translate to "Vainas", but "Pods" is used instead. + +- https://kubernetes.io/es/docs/concepts/workloads/pods/pod/ +- https://kubernetes.io/docs/concepts/workloads/pods/pod/ + +## Number formatting + +Numbers are formatted in a locale specific way. For example in 'en' +they are formatted like `1,000,000` but with 'de' they are formatted +like `1.000.000`. + +Here is an example which can use number formatting: + + +```JavaScript + return t('cluster|{{numReady, number}} / {{numItems, number}} Requested', { + numReady: podsReady.length, + numItems: items.length, + }); +``` + +Number formatting is being done with [Intl.NumberFormat](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/NumberFormat). + +## Date formatting + +Here's an example of using date formatting: + +```Javascript + return t('appsection|When {{ date, date }}', { + date: new Date(), + }); +``` + +## Adding a new component. + +See the `frontend/src/js/i18n/locales/en/` folder for a complete list of namespaces. +If you need a new namespace (e.g. when you're using a sentence that's very specific to +a single/new component), use that namespace as you would if it already existed. + +Then run `make i18n` and a new translation file for that namespace will show up in +all locale folders. + +## Adding a new language. + +Create a folder using the locale code in: `frontend/src/js/i18n/locales/`. + +Then run `make i18n`. This command parses the translatable strings in +the project and creates the corresponding catalog files. + +Integrated components may need to be adjusted (MaterialUI/Monaco etc). + +## Material UI + +Some Material UI components are localized, and are configured +via a theme provider. + +See the Material UI +[Localization Guide](https://material-ui.com/guides/localization/), +and also `frontend/src/js/i18n/ThemeProviderNexti18n.tsx` where integration is done. + diff --git a/content/docs/latest/nebraska/images/auth0-application.gif b/content/docs/latest/nebraska/images/auth0-application.gif new file mode 100644 index 00000000..7a324d87 Binary files /dev/null and b/content/docs/latest/nebraska/images/auth0-application.gif differ diff --git a/content/docs/latest/nebraska/images/auth0-token.gif b/content/docs/latest/nebraska/images/auth0-token.gif new file mode 100644 index 00000000..a0eb27b4 Binary files /dev/null and b/content/docs/latest/nebraska/images/auth0-token.gif differ diff --git a/content/docs/latest/nebraska/images/keycloak-client.gif b/content/docs/latest/nebraska/images/keycloak-client.gif new file mode 100644 index 00000000..130eb6d0 Binary files /dev/null and b/content/docs/latest/nebraska/images/keycloak-client.gif differ diff --git a/content/docs/latest/nebraska/images/keycloak-roles.gif b/content/docs/latest/nebraska/images/keycloak-roles.gif new file mode 100644 index 00000000..b7c5cf9a Binary files /dev/null and b/content/docs/latest/nebraska/images/keycloak-roles.gif differ diff --git a/content/docs/latest/nebraska/images/keycloak-token.gif b/content/docs/latest/nebraska/images/keycloak-token.gif new file mode 100644 index 00000000..93fd5dbf Binary files /dev/null and b/content/docs/latest/nebraska/images/keycloak-token.gif differ diff --git a/content/docs/latest/nebraska/images/keycloak-user.gif b/content/docs/latest/nebraska/images/keycloak-user.gif new file mode 100644 index 00000000..6d48f7ec Binary files /dev/null and b/content/docs/latest/nebraska/images/keycloak-user.gif differ diff --git a/content/docs/latest/nebraska/managing-updates.md b/content/docs/latest/nebraska/managing-updates.md new file mode 100644 index 00000000..59ad41af --- /dev/null +++ b/content/docs/latest/nebraska/managing-updates.md @@ -0,0 +1,128 @@ +--- +title: Managing Flatcar updates +weight: 10 +--- + +Once you have Nebraska up and running, a common use-case is to manage Flatcar Container Linux updates. + +By default, your Flatcar Container Linux instances use the public servers to get updates, so you have to point them to your Nebraska deployment for it to +manage their updates. The process for doing so is slightly different depending on whether you have existing machines or new ones. + +## New machines + +For new machines, you can set up the updates server in their Container Linux Configuration. Here is a small example of how to do it: + + +```yaml +variant: flatcar +version: 1.0.0 +storage: + files: + - path: /etc/flatcar/update.conf + overwrite: true + mode: 0644 + contents: + inline: | + GROUP=stable + SERVER=http://your.nebraska.host:port/v1/update/ +``` + +In addition to the default `stable`, `beta` and `alpha` groups, you can also create and use **custom groups** for greater control over the updates. + + +```yaml +variant: flatcar +version: 1.0.0 +storage: + files: + - path: /etc/flatcar/update.conf + overwrite: true + mode: 0644 + contents: + inline: | + GROUP=ab51a000-02dc-4fc7-a6b0-c42881c89856 + SERVER=http://your.nebraska.host:port/v1/update/ +``` + +**Note**: The sample Nebraska containers provided use the `port 8000` by default (**plain HTTP, no SSL**). Please adjust the update URL setup in your servers to match your Nebraska deployment. + +**Note**: You must not combine the special `locksmith:` CLC section with the above CLC `files:` section (or a similar one with `/etc/coreos/update.conf`) because it results in a conflict where only one entry wins. + +More documentation for the `update.conf` file is [on the Flatcar website](https://www.flatcar.org/docs/latest/setup/releases/update-conf/). + +## Existing machines + +To update the update server in existing instances please edit `/etc/flatcar/update.conf` and update the `SERVER` value (and optionally `GROUP` if needed): + + SERVER=https://your.nebraska.host/v1/update/ + +In may take a few minutes to see an update request coming through. If you want to see it sooner, you can force it running this command: + + update_engine_client -update + +## Setting a machine alias + +The machine alias is an additional name an instance can use when registering in Nebraska. +Since the field is supplied by the instance itself, it's not necessarily unique and can contain arbitrary data. +In the instance list it is shown instead of the instance ID, while on the instance page it is shown in addition to the instance ID. + +To add a machine alias to your Flatcar Container Linux instance, you can edit the `/etc/flatcar/update.conf` file or create it on deployment through a Container Linux Configuration: + +```yaml +variant: flatcar +version: 1.0.0 +storage: + files: + - path: /etc/flatcar/update.conf + mode: 0644 + overwrite: true + contents: + inline: | + MACHINE_ALIAS="myhost a.b.c" +``` + +The `MACHINE_ALIAS` value can be used without quotes when it contains no whitespace. +For dynamic contents like the IP address, you may write the value through a script: + +``` +sudo sed -i "/MACHINE_ALIAS=.*/d" /etc/flatcar/update.conf +echo "MACHINE_ALIAS=\"$(hostname) ${MY_IP_ADDR}\"" | sudo tee -a /etc/flatcar/update.conf +``` + +## Flatcar Container Linux packages in Nebraska + +Nebraska is able to periodically poll the public Flatcar Container Linux update servers and create new packages to update the corresponding channels. So if Nebraska is connected to the internet, new packages will show up automatically for the official Flatcar Container Linux. This functionality is optional, and turned off by default. If you +prefer to use it, you should pass the option `-enable-syncer=true` when running Nebraska. + +Notice that by default Nebraska only stores metadata about the Flatcar Container Linux updates, not the updates payload. This means that the updates served to your instances contain instructions to download the packages payload from the public Flatcar Container Linux update servers directly, so your servers need access to the Internet to download them. + +### Hosting synchronized packages + +It is also possible to host the Flatcar Container Linux packages payload in Nebraska. In this case, in addition to get the packages metadata, Nebraska will also download the package payload itself so that it can serve it to your instances when serving updates. + +This functionality is turned off by default. So to make Nebraska host the Flatcar Container Linux packages payload, the following options have to be passed to it: + + nebraska -host-flatcar-packages=true -flatcar-packages-path=/PATH/TO/STORE/PACKAGES -nebraska-url=http://your.Nebraska.host:port + +### Overriding synchronized packages' URLs + +Some users may choose to host their own packages elsewhere (i.e. without using the +host function explained above), and thus it is desired to synchronize the packages +from upstream but giving them a custom URL for the actual update download. + +This can be achieved by setting the `syncer-packages-url` CLI option. This should +be set as a URL, and any `{{VERSION}}` and `{{ARCH}}` keywords in the URL will be +replaced by the packages' version and arch, respectively. + +For example: +```bash +nebraska -enable-syncer=true -syncer-packages-url=https://mysepcialstorage.io/flatcar/{{ARCH}}/{{VERSION}} +``` + +## Managing updates for your own applications + +In addition to managing updates for Flatcar Container Linux, you can use Nebraska for other applications as well. + +In the `updaters/lib` directory there are some sample helpers that can be useful to create your own updaters that talk to Nebraska or even embed them into your own applications. + +In the `updaters/examples` you'll find a sample minimal application built using [grace](https://github.com/facebookgo/grace) that is able to update itself using Nebraska in a graceful way. diff --git a/content/docs/latest/nebraska/migrate-to-older-version.md b/content/docs/latest/nebraska/migrate-to-older-version.md new file mode 100644 index 00000000..700dd702 --- /dev/null +++ b/content/docs/latest/nebraska/migrate-to-older-version.md @@ -0,0 +1,21 @@ +--- +title: Migrating to older version of Nebraska. +weight: 11 +--- + + +When migrating down to an older version of Nebraska one might have to rollback database migrations that were applied as a part of the release. + +First get the migrations that were part of the version that you want to downgrade to using the following command. + +> git show :./backend/pkg/api/db/migrations | tail -n1 + +For example to find the migrations that were part of `2.4.0` release clone the Nebraska repo and run. + +> git show 2.4.0:../backend/pkg/api/db/migrations | tail -n1 +> +> 0013_add_stats_indexes.sql + +To rollback to 0013_add_stats_indexes.sql use the following command + +> /nebraska/nebraska --rollback-db-to 0013 diff --git a/content/docs/latest/nebraska/nebraska-logo.svg b/content/docs/latest/nebraska/nebraska-logo.svg new file mode 100644 index 00000000..ccef1571 --- /dev/null +++ b/content/docs/latest/nebraska/nebraska-logo.svg @@ -0,0 +1,107 @@ + + + + + + image/svg+xml + + + + + + + + + + + + + + + + + + + + + + diff --git a/content/docs/latest/setup/releases/switching-channels.md b/content/docs/latest/setup/releases/switching-channels.md index 47a82982..626c00ac 100644 --- a/content/docs/latest/setup/releases/switching-channels.md +++ b/content/docs/latest/setup/releases/switching-channels.md @@ -29,8 +29,8 @@ By default, Flatcar uses the public update server `public.update.flatcar-linux.n It promotes the new releases for each channel at the same time they are published. If you need more control about the update rollout, you can have a look at the possible [reboot strategies and manual update methods](update-strategies). The other alternative is running your own update server which allows you to control the update rollout over your fleet and even divide it into groups that have different rollout policies and release versions. -The [Nebraska](nebraska) Open Source project implements the update server and is also used for our public instance. -More on it below and on the Nebraska [docs site](nebraska-docs). +The [Nebraska][nebraska] Open Source project implements the update server and is also used for our public instance. +More on it below and on the Nebraska [docs site][nebraska-docs]. ## Customizing channel configuration @@ -95,7 +95,7 @@ $ sudo flatcar-update --to-version "$VER" ## Use a personal update server -When setting up your own [Nebraska](nebraska) update server you will be able to point your Flatcar machines to fetch its updates from it. +When setting up your own [Nebraska][nebraska] update server you will be able to point your Flatcar machines to fetch its updates from it. Nebraska's web interface allows to create custom groups that are used to specify update rollout policies, and custom channels that specify the Flatcar version. Multiple groups can point to the same channel. The Nebraska web interface also gives an overview about the machines and their update status. @@ -116,7 +116,7 @@ SERVER=http://your.nebraska.host:port/v1/update/ GROUP=myproduction ``` -More specifics about Nebraska can be found on its [docs site](nebraska-docs). +More specifics about Nebraska can be found on its [docs site][nebraska-docs]. ## Debugging @@ -142,5 +142,5 @@ cat /usr/share/flatcar/update.conf Note: while a manual channel switch is in progress, `/usr/share/flatcar/update.conf` shows the channel for the current OS while `/etc/flatcar/update.conf` shows the one for the next update. -[nebraska]: https://github.com/kinvolk/nebraska/ -[nebraska-docs]: https://kinvolk.io/docs/nebraska/latest +[nebraska]: https://github.com/flatcar/nebraska/ +[nebraska-docs]: https://flatcar.org/docs/latest/nebraska/ diff --git a/content/docs/latest/setup/releases/update-conf.md b/content/docs/latest/setup/releases/update-conf.md index f7b1794d..15856342 100644 --- a/content/docs/latest/setup/releases/update-conf.md +++ b/content/docs/latest/setup/releases/update-conf.md @@ -9,7 +9,7 @@ aliases: - ../../clusters/management/update-conf --- -Flatcar Container Linux uses [`update_engine`][update_engine] to check and fetch new updates from the [Nebraska Update Service](https://github.com/kinvolk/nebraska). +Flatcar Container Linux uses [`update_engine`][update_engine] to check and fetch new updates from the [Nebraska Update Service](../../nebraska). ## Location