Skip to content

Quality Assurance Process

Jump to bottom
Stephanos Ioannidis edited this page Sep 26, 2024 · 5 revisions

DISCLAIMER

This is a preliminary document explaining what should be done before releasing a new version of Zephyr SDK.

Note that the process documented here is not formal nor is it strictly enforced; it is only intended to serve as a recommendation for the SDK release managers. There may be numerous exceptions and deviations made for each SDK release, depending on the nature of the release and the amount of changes involved.

Overview

The Release Quality Assurance Process shall consist of, at minimum, the following components:

  • Integration Test
  • Footprint Validation

Integration Test

The purpose of an integration test is to ensure that the current and, if applicable, previous Zephyr codebases are compatible with an SDK release and all Zephyr tests and samples can be successfully built using it.

An integration test shall:

  • build all Zephyr tests and samples on the default platforms.
  • run all Zephyr tests and samples on the emulate-able platforms.
  • begin when the first Release Candidate (RC) is published.
  • be performed using the latest Zephyr development branch (main) and, if applicable, using all compatible previous Zephyr release tags.
  • perform coverage test (to ensure that gcov integration is functional).

The recommended strategy for performing an integration test is as follows:

  1. Integrate the latest Release Candidate (RC) release into the CI and User Docker images.
  2. Update the Continuous Integration (CI) workflows of the zephyr-testing repository to use the updated CI Docker image.
  3. Build and run Zephyr tests and samples using the Twister workflow.

Footprint Validation

The purpose of the footprint validation step is to ensure that the changes introduced in a Zephyr SDK release does not result in an unintentional increase in the footprint of the binaries produced by the included toolchains and libraries.

The footprint validation step shall validate the following footprint changes:

  • GCC Code Generation Footprint Change

    • to detect any footprint increase due to the changes introduced in the GCC C/C++ code generator
    • shall validate one or more widely used optimisation levels.
    • shall validate all supported architectures.

  • Newlib Footprint Change

    • to detect any footprint increase due to the changes introduced in the Newlib codebase as well as the GCC C code generator used for compiling the Newlib.
    • shall validate both the "full" and the "nano" variants of the Newlib.
    • should validate one or more widely used architectures.

  • Picolibc Footprint Change

    • to detect any footprint increase due to the changes introduced in the Picolibc codebase as well as the GCC C code generator used for compiling the Picolibc.
    • should validate one or more widely used architectures.

  • libstdc++ Footprint Change

    • to detect any footprint increase due to the changes introduced in the libstdc++ codebase as well as the GCC C/C++ code generator used for compiling the libstdc++ library.
    • should validate one or more widely used architectures.

Wiki Home

Clone this wiki locally