Skip to content

Commit

Permalink
docs: rust: Add description of Rust documentation test as KUnit ones
Browse files Browse the repository at this point in the history
Rust documentation tests are automatically converted into KUnit
tests. The commit adding this feature

commit a66d733 ("rust: support running Rust documentation tests as KUnit ones")

from Miguel has a very nice commit message with a lot details
for this. To not 'hide' that just in a commit message, pick the main
parts of it and add it to the documentation. And add a short info
how to enable this. While adding this, improve the structure of
the sections.

Reviewed-by: David Gow <[email protected]>
Reviewed-by: Alice Ryhl <[email protected]>
Co-developed-by: Miguel Ojeda <[email protected]>
Signed-off-by: Miguel Ojeda <[email protected]>
Signed-off-by: Dirk Behme <[email protected]>
Reviewed-by: Trevor Gross <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
[ Fixed unordered list rendering, rewrapped text and made headers
  consistent with the other documents in `rust/`. ]
Signed-off-by: Miguel Ojeda <[email protected]>
  • Loading branch information
dirkbehme authored and ojeda committed Feb 18, 2024
1 parent e3db8c1 commit 2392e2a
Showing 1 changed file with 119 additions and 8 deletions.
127 changes: 119 additions & 8 deletions Documentation/rust/testing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,25 @@
Testing
=======

There are the tests that come from the examples in the Rust documentation
and get transformed into KUnit tests. These can be run via KUnit. For example
via ``kunit_tool`` (``kunit.py``) on the command line::
This document contains useful information how to test the Rust code in the
kernel.

There are two sorts of tests:

- The KUnit tests.
- The ``#[test]`` tests.

The KUnit tests
---------------

These are the tests that come from the examples in the Rust documentation. They
get transformed into KUnit tests.

Usage
*****

These tests can be run via KUnit. For example via ``kunit_tool`` (``kunit.py``)
on the command line::

./tools/testing/kunit/kunit.py run --make_options LLVM=1 --arch x86_64 --kconfig_add CONFIG_RUST=y

Expand All @@ -14,11 +30,106 @@ Documentation/dev-tools/kunit/index.rst for the general KUnit documentation
and Documentation/dev-tools/kunit/architecture.rst for the details of kernel
built-in vs. command line testing.

Additionally, there are the ``#[test]`` tests. These can be run using
the ``rusttest`` Make target::
To use these KUnit doctests, the following must be enabled::

CONFIG_KUNIT
Kernel hacking -> Kernel Testing and Coverage -> KUnit - Enable support for unit tests
CONFIG_RUST_KERNEL_DOCTESTS
Kernel hacking -> Rust hacking -> Doctests for the `kernel` crate

in the kernel config system.

KUnit tests are documentation tests
***********************************

These documentation tests are typically examples of usage of any item (e.g.
function, struct, module...).

They are very convenient because they are just written alongside the
documentation. For instance:

.. code-block:: rust
/// Sums two numbers.
///
/// ```
/// assert_eq!(mymod::f(10, 20), 30);
/// ```
pub fn f(a: i32, b: i32) -> i32 {
a + b
}
In userspace, the tests are collected and run via ``rustdoc``. Using the tool
as-is would be useful already, since it allows verifying that examples compile
(thus enforcing they are kept in sync with the code they document) and as well
as running those that do not depend on in-kernel APIs.

For the kernel, however, these tests get transformed into KUnit test suites.
This means that doctests get compiled as Rust kernel objects, allowing them to
run against a built kernel.

A benefit of this KUnit integration is that Rust doctests get to reuse existing
testing facilities. For instance, the kernel log would look like::

KTAP version 1
1..1
KTAP version 1
# Subtest: rust_doctests_kernel
1..59
# rust_doctest_kernel_build_assert_rs_0.location: rust/kernel/build_assert.rs:13
ok 1 rust_doctest_kernel_build_assert_rs_0
# rust_doctest_kernel_build_assert_rs_1.location: rust/kernel/build_assert.rs:56
ok 2 rust_doctest_kernel_build_assert_rs_1
# rust_doctest_kernel_init_rs_0.location: rust/kernel/init.rs:122
ok 3 rust_doctest_kernel_init_rs_0
...
# rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
ok 59 rust_doctest_kernel_types_rs_2
# rust_doctests_kernel: pass:59 fail:0 skip:0 total:59
# Totals: pass:59 fail:0 skip:0 total:59
ok 1 rust_doctests_kernel

Tests using the `? <https://doc.rust-lang.org/reference/expressions/operator-expr.html#the-question-mark-operator>`_
operator are also supported as usual, e.g.:

.. code-block:: rust
/// ```
/// # use kernel::{spawn_work_item, workqueue};
/// spawn_work_item!(workqueue::system(), || pr_info!("x"))?;
/// # Ok::<(), Error>(())
/// ```
The tests are also compiled with Clippy under ``CLIPPY=1``, just like normal
code, thus also benefitting from extra linting.

In order for developers to easily see which line of doctest code caused a
failure, a KTAP diagnostic line is printed to the log. This contains the
location (file and line) of the original test (i.e. instead of the location in
the generated Rust file)::

# rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150

Rust tests appear to assert using the usual ``assert!`` and ``assert_eq!``
macros from the Rust standard library (``core``). We provide a custom version
that forwards the call to KUnit instead. Importantly, these macros do not
require passing context, unlike those for KUnit testing (i.e.
``struct kunit *``). This makes them easier to use, and readers of the
documentation do not need to care about which testing framework is used. In
addition, it may allow us to test third-party code more easily in the future.

A current limitation is that KUnit does not support assertions in other tasks.
Thus, we presently simply print an error to the kernel log if an assertion
actually failed. Additionally, doctests are not run for nonpublic functions.

The ``#[test]`` tests
---------------------

Additionally, there are the ``#[test]`` tests. These can be run using the
``rusttest`` Make target::

make LLVM=1 rusttest

This requires the kernel ``.config`` and downloads external repositories.
It runs the ``#[test]`` tests on the host (currently) and thus is fairly
limited in what these tests can test.
This requires the kernel ``.config`` and downloads external repositories. It
runs the ``#[test]`` tests on the host (currently) and thus is fairly limited in
what these tests can test.

0 comments on commit 2392e2a

Please sign in to comment.