diff --git a/charts/testkube-enterprise/docs/README.md b/charts/testkube-enterprise/docs/README.md deleted file mode 100644 index bf62a89f3..000000000 --- a/charts/testkube-enterprise/docs/README.md +++ /dev/null @@ -1,467 +0,0 @@ - - -**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* - -- [Testkube Enterprise Helm Chart Installation and Usage Guide](#testkube-enterprise-helm-chart-installation-and-usage-guide) - - [Prerequisites](#prerequisites) - - [Configuration](#configuration) - - [Docker images](#docker-images) - - [License](#license) - - [Online License](#online-license) - - [Offline License](#offline-license) - - [Ingress](#ingress) - - [Configuration](#configuration-1) - - [Domain](#domain) - - [TLS](#tls) - - [Auth](#auth) - - [Metrics](#metrics) - - [Invitations](#invitations) - - [Invitations via email](#invitations-via-email) - - [Auto-accept invitations](#auto-accept-invitations) - - [Bring Your Own Infra](#bring-your-own-infra) - - [MongoDB](#mongodb) - - [NATS](#nats) - - [MinIO](#minio) - - [Dex](#dex) - - [Installation](#installation) - - [Minimal setup](#minimal-setup) - - [Production setup](#production-setup) - - [FAQ](#faq) - - - -# Testkube Enterprise Helm Chart Installation and Usage Guide - -Welcome to the Testkube Enterprise Helm chart installation and usage guide. -This comprehensive guide provides step-by-step instructions for installing and utilizing the Testkube Enterprise Helm chart. -Testkube Enterprise is a cutting-edge Kubernetes-native testing platform designed to optimize your testing and quality assurance processes with enterprise-grade features. - -## Prerequisites - -Before you proceed with the installation, please ensure that you have the following prerequisites in place: -* Kubernetes cluster (version 1.21+) -* [Helm](https://helm.sh/docs/intro/quickstart/) (version 3+) -* [cert-manager](https://cert-manager.io/docs/installation/) (version 1.11+) - used for TLS certificate management -* [NGINX Controller](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/) (version v1.8+) - used for Ingress configuration -* (OPTIONAL) [Prometheus Operator](https://github.com/prometheus-operator/prometheus-operator) (version 0.49+) - used for metrics collection -* Own a public/private domain for creating Ingress rules -* License Key and/or License File (if offline access is required) - -**NOTE** -While it is possible to use custom TLS certificates for the Testkube Enterprise API and Dashboard, -we strongly recommend using `cert-manager` for easier certificate management. - -## Configuration - -### Docker images - -**DEPRECATION NOTICE**: As of November 2023, Testkube Enterprise Docker images are publicly accessible. -You only need to follow the steps in this section if you wish to re-publish the images to your private Docker registry; -otherwise, you may skip this section. - -To begin, ensure that you have access to the Testkube Enterprise API & Dashboard Docker images. -You can either request access from your Testkube representative or upload the Docker image tarball artifacts to a private Docker registry. - -Next, create a secret to store your Docker registry credentials: -```bash -kubectl create secret docker-registry testkube-enterprise-registry \ - --docker-server= \ - --docker-username= \ - --docker-password= \ - --docker-email= \ - --namespace=testkube-enterprise -``` - -Make sure to configure the image pull secrets in your `values.yaml` file: -```helm -global: - imagePullSecrets: - - name: testkube-enterprise-registry -``` - -### License - -Select the appropriate license type for your environment. - -For air-gaped & firewalled environments, we offer an option to use an [Offline License](#offline-license) for enhanced security. -An **Offline License** consists of a **License Key** and **License File**. - -If your environment has internet access, you can use an [Online License](#online-license), which only requires the **License Key**. - -#### Online License - -If your environment has internet access, you can use an **Online License**, which only requires the **License Key**, -and can be provided as a Helm parameter or Kubernetes secret. - -To provide the **License Key** as a Helm parameter, use the following configuration: -```helm -global: - enterpriseLicenseKey: -``` - -To provide the **License Key** as a Kubernetes secret, first we need to create a secret with the required field. -Run the following command to create the secret either from a file or from a literal: - -From file: -```bash -kubectl create secret generic testkube-enterprise-license \ - --from-file=LICENSE_KEY= \ - --namespace=testkube-enterprise -``` -From literal: -```bash -kubectl create secret generic testkube-enterprise-license \ - --from-literal=LICENSE_KEY= \ - --namespace=testkube-enterprise -``` -And then use the following Helm chart configuration: -```helm -global: - enterpriseLicenseSecretRef: -``` - -#### Offline License - -For an **Offline License**, supply both the **License Key** and **License File** as either Kubernetes secrets or Helm parameters. -Using secrets is safer, as it prevents exposing sensitive license information in Helm chart values. - -The Kubernetes secret needs to contain 2 entries: `license.lic` and `LICENSE_KEY`. -To create the secret with the **License Key** and **License File**, run the following command: -```bash -kubectl create secret generic testkube-enterprise-license \ - --from-literal=LICENSE_KEY= \ - --from-file=license.lic= \ - --namespace=testkube-enterprise -``` - -After creating the secret, use the following Helm chart configuration: -```helm -global: - enterpriseOfflineAccess: true - licenseFileSecret: testkube-enterprise-license -``` - -Alternatively, you can provide the **License File** as a Helm parameter: -```helm -global: - licenseKey: - licenseFile: -``` - -### Ingress - -Testkube Enterprise requires the NGINX Controller to configure and optimize its protocols. -NGINX is the sole supported Ingress Controller, and is essential for Testkube Enterprise's operation. - -We highly recommend installing Testkube Enterprise with Ingress enabled. -This requires a valid domain (public or private) with a valid TLS certificate. -Ingresses are enabled and created by default. - -To disable Ingress creation, adjust the following values accordingly. Note that you must then manually configure the API & Dashboard services to maintain accessibility: -```helm -global: - ingress: - enabled: false - -testkube-cloud-api: - api: - tls: - serveHTTPS: false -``` - -#### Configuration - -To ensure the reliable functioning of gRPC and Websockets protocols, Testkube Enterprise is locked in with NGINX Ingress Controller. - -Below are current configurations per Ingress resource which ensure Testkube Enterprise protocols work performant and reliably. -It is not recommended to change any of these settings! - -gRPC Ingress annotations: -```kubernetes -annotations: - nginx.ingress.kubernetes.io/proxy-body-size: 8m - nginx.ingress.kubernetes.io/client-header-timeout: "10800" - nginx.ingress.kubernetes.io/client-body-timeout: "10800" -``` - -Websockets Ingress annotations: -```kubernetes -annotations: - nginx.ingress.kubernetes.io/proxy-read-timeout: "3600" - nginx.ingress.kubernetes.io/proxy-send-timeout: "3600" -``` - -If you want to use a different Ingress Controller, we kindly ask you to reach out and discuss with our support team. - -#### Domain - -Testkube Enterprise requires a domain (public or internal) under which it will expose the following services: - -| Subdomain | Service | Endpoint | -|---------------------------------|------------------|----------------------------------| -| `dashboard.` | Dashboard UI | `testkube-enterprise-ui:8080` | -| `api.` | REST API | `testkube-enterprise-api:9443` | -| `agent.` | gRPC API | `testkube-enterprise-api:8443` | -| `websockets.` | WebSockets API | `testkube-enterprise-api:9443` | -| `storage.` | Storage API | `testkube-enterprise-minio:9000` | - -#### TLS - -For best performance, TLS should be terminated at application level (Testkube Enterprise API) instead of NGINX/Ingress level because -gRPC and Websockets protocols perform significantly better when HTTP2 protocol is used end-to-end. -Note that NGINX, by default, downgrades the HTTP2 protocol to HTTP1.1 when the backend service is using an insecure port. - -If `cert-manager` (check the [Prerequisites](#prerequisites) for installation guide) is installed in your cluster, it should be configured to issue certificates for the configured domain by using `Issuer` or `ClusterIssuer` resource. -Testkube Enterprise Helm chart needs the following config in that case: -```helm -global: - certificateProvider: "cert-manager" - certManager: - issuerRef: -``` - -By default, Testkube Enterprise uses a `ClusterIssuer` `cert-manager` resource, that can be changed by setting the `testkube-cloud-api.api.tls.certManager.issuerKind` field to `Issuer`. - -If `cert-manager` is not installed in your cluster, valid TLS certificates (for API & Dashboard) which cover the following subdomains need to be provided: -* API (tls secret name is configured with `testkube-cloud-api.api.tls.tlsSecret` field) - * `api.` - * `agent.` - * `websockets.` -* Dashboard (TLS secret name is configured with `testkube-cloud-ui.ingress.tlsSecretName` field) - * `dashboard.` -Also, `global.certificateProvider` should be set to blank (""). -```helm -global: - certificateProvider: "" -``` - -#### Custom certificates - -In order to use custom certificates, first a secret needs to be created with the following entries: -* `tls.crt` - the certificate -* `tls.key` - the private key -* `ca.crt` - the CA certificate (if the certificate is not self-signed) - -If certificate-based authentication is required, the custom certificates need to be configured in the following places: -* Enterprise API - * If `MINIO_ENDPOINT` is set to an exposed URL, then the following Helm values need to be configured: - - The following Helm parameter needs to be enabled to inject the custom certificate into MinIO `testkube-cloud-api.minio.certSecret.enabled: true` - - If the certificate is not self-signed, the CA cert needs to be injected also by enabling the Helm parameter `testkube-cloud-api.minio.mountCACertificate: true` - - Custom certificate verification can also be skipped by setting `testkube-cloud-api.minio.skipVerify: true` - * If `MINIO_ENDPOINT` uses the Kubernetes DNS record (`testkube-enterprise-minio..svc.cluster.local:9000`), `AGENT_STORAGE_HOSTNAME` should be set to point to the exposed storage URL -* Agent - * Agent API - - If the Enterprise API is configured to use certificate-based authentication or is using a certificate signed by a custom CA, the Agent API needs to be configured to use the same certificates by pointing `testkube-api.cloud.tls.certificate.secretRef` to the Kubernetes secret which contains the certificates - - Custom certificate verification can also be skipped by setting `testkube-api.cloud.tls.skipVerify: true` - * Storage - - The following Helm parameter needs to be enabled to inject the custom certificate into MinIO `testkube-api.storage.certSecret.enabled: true` - - If the certificate is not self-signed, the CA cert needs to be injected also by enabling the Helm parameter `testkube-cloud-api.minio.mountCACertificate: true` - - Custom certificate verification can also be skipped by setting `testkube-api.storage.skipVerify: true` - -### Auth - -Testkube Enterprise utilizes [Dex](https://dexidp.io/) for authentication & authorization. -For detailed instruction on configuring Dex, please refer to the [auth.md](./auth.md) document. - -Dex requires persisting state to perform various tasks such as track refresh tokens, preventing replays, and rotating keys. -This document is a summary of the storage configurations supported by dex. - -Default storage type for Dex is Kubernetes CRDs. This is the recommended storage type for most users. - -In order to use a different storage type, you need to configure the `dex.storage` field in the `values.yaml` file. -```helm -storage: - type: sqlite3 - config: - file: /var/dex/dex.db -``` - -If the `dex.storage` field is not configured, it will default to the Kubernetes CRDs storage type. - -More info on various storage options can be found in the [Dex Storage](https://dexidp.io/docs/storage/) documentation. - -### Metrics - -Testkube Enterprise exposes Prometheus metrics on the `/metrics` endpoint and uses a `ServiceMonitor` resource to expose them to Prometheus. -In order for this to work, you need to have `Prometheus Operator` installed in your cluster so that the `ServiceMonitor` resource can be created. - - -Use the following configuration to enable metrics: -```helm -testkube-cloud-api: - prometheus: - enabled: true -``` - -### Invitations - -Testkube Enterprise allows you to invite users to Organizations and Environments within Testkube, granting them specific roles and permissions. - -There are two supported invitation modes: `email` and `auto-accept`. -Use `email` to send an invitation for the user to accept, and `auto-accept` to automatically add users without requiring acceptance. - -#### Invitations via email - -If `testkube-cloud-api.api.inviteMode` is set to `email`, Testkube Enterprise will send emails when a user gets invited to -an Organization or an Environment, and in that case SMTP settings need to be configured in the API Helm chart. - -```helm -testkube-cloud-api: - api: - email: - fromEmail: "example@gmail.com" - fromName: "Example Invitation" - inviteMode: email - smtp: - host: - port: - username: - password: - # password can also be referenced by using the `passwordSecretRef` field which needs to contain the key SMTP_PASSWORD - # passwordSecretRef: -``` - -#### Auto-accept invitations - -If `testkube-cloud-api.api.inviteMode` is set to `auto-accept`, Testkube Enterprise will automatically add users to -Organizations and Environments when they get invited. - -```helm -testkube-cloud-api: - inviteMode: auto-accept -``` - -## Bring Your Own Infra - -Testkube Enterprise supports integrating with existing infrastructure components such as MongoDB, NATS, Dex, etc. - -### MongoDB - -Testkube Enterprise uses MongoDB as a database for storing all the data. -By default, it will install a MongoDB instance using the Bitnami MongoDB Helm chart. - -If you wish to use an existing MongoDB instance, you can configure the following values: -```helm -mongodb: - enabled: false - -testkube-cloud-api: - api: - mongo: - dsn: -``` - -### NATS - -Testkube Enterprise uses NATS as a message broker for communication between API and Agents. - -If you wish to use an existing NATS instance, you can configure the following values: -```helm -nats: - enabled: false - -testkube-cloud-api: - api: - nats: - uri: -``` - -### MinIO - -Testkube Enterprise uses MinIO as a storage backend for storing artifacts. - -If you wish to use an existing MinIO instance, you can configure the following values: -```helm -testkube-cloud-api: - minio: - enabled: false - api: - minio: {} # check out the `testkube-cloud-api.api.minio` block in the values.yaml for all available settings -``` - -### Dex - -Testkube Enterprise uses Dex as an identity provider. - -If you wish to use an existing Dex instance, you can configure the following values: -```helm -global: - dex: - issuer: -dex: - enabled: false -testkube-cloud-api: - api: - oauth: {} # check out the `testkube-cloud-api.api.oauth` block in the values.yaml for all available settings -``` - -## Installation - -1. Add our Testkube Enterprise Helm registry: - ```bash - helm repo add testkubeenterprise https://kubeshop.github.io/testkube-cloud-charts - ``` -2. Create a `values.yaml` with preferred configuration -3. Run `helm install testkube-enterprise testkubeenterprise/testkube-enterprise -f values.yaml --namespace testkube-enterprise` - -**IMPORTANT** -The Bitnami MongoDB Helm chart does not work reliably on ARM architectures. If you are installing MongoDB using this chart, you need to use an ARM compatible image: -```helm -mongodb: - image: - repository: zcube/bitnami-compat-mongodb - tag: "6.0.5" -``` - -### Minimal setup - -This is a minimal setup which will install a development Testkube Enterprise cluster with the following components: -* Testkube Enterprise API -* Testkube Enterprise Dashboard -* MongoDB -* NATS -* Dex - -This setup should not be used in production environments. For a more advanced setup please refer to the [Production Setup](#production-setup) section. - -Following configuration can be used for a minimal development setup of Testkube Enterprise: -```helm -global: - domain: - imagePullSecrets: - - name: - licenseKey: - ingress: - enabled: false - -dex: - configTemplate: - additionalConfig: | - enablePasswordDB: true - staticPasswords: - - email: - hash: - username: -``` - -### Production setup - -For best performance and reliability, users should follow this official setup guide and make sure each section is properly configured. - -1. Configure DNS records as described in the [Domain](#domain) section -2. Configure TLS certificates as described in the [TLS](#tls) section -3. Configure Dex as described in the [Auth](#auth) section -4. Configure Ingress as described in the [Ingress](#ingress) section -5. Configure Metrics as described in the [Metrics](#metrics) section -6. Configure Invitations as described in the [Invitations](#invitations) section -7. Configure BYOI components as described in the [Bring Your Own Infra](#bring-your-own-infra) section -8. Install Testkube Enterprise as described in the [Installation](#installation) section - -## FAQ - -Q: Testkube Enterprise API is crashing (pod is in `Error`/`CrashLoopBackOff` state) with the following error: -``` -panic: license file is invalid -``` -A: Make sure the license file ends with a newline character. -There should be a new line after the `-----END LICENSE FILE-----` line in the license file. diff --git a/charts/testkube-enterprise/docs/auth.md b/charts/testkube-enterprise/docs/auth.md deleted file mode 100644 index 1b531d176..000000000 --- a/charts/testkube-enterprise/docs/auth.md +++ /dev/null @@ -1,99 +0,0 @@ -## Configure Testkube Enterprise with Identity Providers - -You can configure Testkube Enterprise to authenticate users using different identity providers such as Azure AD, Google, Okta, and OIDC. To do this, you need to update the `additionalConfig` field in the Helm chart values with the appropriate configuration for each identity provider. - -For a list of all supported identity providers, see [Connectors](https://dexidp.io/docs/connectors/). - -Examples below show how to configure Testkube Enterprise with each identity provider by editing the `dex.configTemplate.additionalConfig` field in the Helm chart values. - -### Quickstart - -For a quickstart, or if you do not have an identity provider, you can configure Testkube Enterprise to use static users. -See [Static Users](#static-users). - -### Static Users - -To configure Testkube Enterprise with static users, add the following configuration to the additionalConfig field: - -```yaml -additionalConfig: | - enablePasswordDB: true - staticPasswords: - - email: - hash: - username: -``` - -Replace ``, ``, and `` with your actual values. - -### Azure AD - -To configure Testkube Enterprise with Azure AD, add the following configuration to the `additionalConfig` field: - -```yaml -additionalConfig: | - connectors: - - type: azuread - id: azuread - name: Azure AD - config: - clientID: - clientSecret: - redirectURI: -``` - -Replace , , and with your actual Azure AD configuration values. - -### Google - -To configure Testkube Enterprise with Google, add the following configuration to the additionalConfig field: - -```yaml -additionalConfig: | - connectors: - - type: google - id: google - name: Google - config: - clientID: - clientSecret: - redirectURI: -``` - -Replace , , and with your actual Google configuration values. - -### Okta - -To configure Testkube Enterprise with Okta, add the following configuration to the additionalConfig field: - -```yaml -additionalConfig: | - connectors: - - type: okta - id: okta - name: Okta - config: - issuerURL: - clientID: - clientSecret: - redirectURI: -``` - -Replace , , , and with your actual Okta configuration values. - -### OIDC - -To configure Testkube Enterprise with an OIDC provider, add the following configuration to the additionalConfig field: - -```yaml -additionalConfig: | - connectors: - - type: oidc - id: oidc - name: OIDC - config: - issuerURL: - clientID: - clientSecret: - redirectURI: -```