Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sys/shell: reduce overhead of XFA shell commands #20958

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

maribu
Copy link
Member

@maribu maribu commented Nov 6, 2024

Contribution description

We do not need to add an array of pointers to the shell commands, just an array of shell commands is sufficient. This reduced the overhead of XFA by sizeof(void *) per command.

Testing procedure

$ make BOARD=arduino-uno flash term -C examples/default
[...]

2024-11-06 18:27:11,666 # main(): This is RIOT! (Version: 2024.10-devel-388-gf4b2ef-sys/shell/xfa-reduce-overhead)
2024-11-06 18:27:11,686 # Welcome to RIOT!
help
2024-11-06 18:27:25,324 # help
2024-11-06 18:27:25,361 # Command              Description
2024-11-06 18:27:25,402 # ---------------------------------------
2024-11-06 18:27:25,459 # pm                   interact with layered PM subsystem
2024-11-06 18:27:25,525 # ps                   Prints information about running threads.
2024-11-06 18:27:25,594 # saul                 interact with sensors and actuators using SAUL
2024-11-06 18:27:25,648 # version              Prints current RIOT_VERSION
2024-11-06 18:27:25,684 # reboot               Reboot the node
> ps
2024-11-06 18:27:27,196 # ps
2024-11-06 18:27:27,302 # 	pid | name                 | state    Q | pri | stack  ( used) ( free) | base addr  | current     
2024-11-06 18:27:27,405 # 	 1 | idle                 | pending  Q |  15 |    128 (   90) (   38) |      0x3f2 |      0x41a 
2024-11-06 18:27:27,507 # 	 2 | main                 | running  Q |   7 |    644 (  296) (  348) |      0x472 |      0x5fc 
2024-11-06 18:27:27,581 # 	   | SUM                  |            |     |    772 (  386) (  386)
> 2024-11-06 18:27:28,874 # Exiting Pyterm
$ make BOARD=same54-xpro flash term -C examples/default
[...]
> help
2024-11-06 18:29:29,540 # help
2024-11-06 18:29:29,543 # Command              Description
2024-11-06 18:29:29,546 # ---------------------------------------
2024-11-06 18:29:29,551 # ifconfig             Configure network interfaces
2024-11-06 18:29:29,557 # txtsnd               Sends a custom string as is over the link layer
2024-11-06 18:29:29,562 # pm                   interact with layered PM subsystem
2024-11-06 18:29:29,567 # ps                   Prints information about running threads.
2024-11-06 18:29:29,572 # rtc                  control RTC peripheral interface
2024-11-06 18:29:29,578 # saul                 interact with sensors and actuators using SAUL
2024-11-06 18:29:29,582 # version              Prints current RIOT_VERSION
2024-11-06 18:29:29,585 # reboot               Reboot the node
> ps
2024-11-06 18:29:31,316 # ps
2024-11-06 18:29:31,324 # 	pid | name                 | state    Q | pri | stack  ( used) ( free) | base addr  | current     
2024-11-06 18:29:31,333 # 	 - | isr_stack            | -        - |   - |    512 (  172) (  340) | 0x20000000 | 0x200001c8
2024-11-06 18:29:31,342 # 	 1 | main                 | running  Q |   7 |   1536 (  700) (  836) | 0x200002c0 | 0x20000724 
2024-11-06 18:29:31,350 # 	 2 | pktdump              | bl rx    _ |   6 |   1472 (  468) ( 1004) | 0x20001048 | 0x2000155c 
2024-11-06 18:29:31,359 # 	 3 | sam0_eth             | bl anyfl _ |   2 |    896 (  276) (  620) | 0x20000a5c | 0x20000d1c 

And native will be tested by the CI.

Issues/PRs references

@github-actions github-actions bot added the Area: sys Area: System label Nov 6, 2024
@maribu maribu added Type: cleanup The issue proposes a clean-up / The PR cleans-up parts of the codebase / documentation CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Nov 6, 2024
@maribu
Copy link
Member Author

maribu commented Nov 6, 2024

Ah, I should also show that this does actually bring down .text.

This PR:

$ make BOARD=samr21-xpro RIOT_CI_BUILD=1 -C examples/default 
make: Entering directory '/home/[email protected]/Repos/software/RIOT/master/examples/default'
Building application "default" for "samr21-xpro" with CPU "samd21".

   text	  data	   bss	   dec	   hex	filename
  49180	   168	  6196	 55544	  d8f8	/home/[email protected]/Repos/software/RIOT/master/examples/default/bin/samr21-xpro/default.elf
make: Leaving directory '/home/[email protected]/Repos/software/RIOT/master/examples/default'

And in master:

make BOARD=samr21-xpro RIOT_CI_BUILD=1 -C examples/default
make: Entering directory '/home/[email protected]/Repos/software/RIOT/master/examples/default'
Building application "default" for "samr21-xpro" with CPU "samd21".

   text	  data	   bss	   dec	   hex	filename
  49220	   168	  6196	 55584	  d920	/home/[email protected]/Repos/software/RIOT/master/examples/default/bin/samr21-xpro/default.elf
make: Leaving directory '/home/[email protected]/Repos/software/RIOT/master/examples/default'

@riot-ci
Copy link

riot-ci commented Nov 6, 2024

Murdock results

FAILED

55a1a7a DROPME: fix build with rust

Success Failures Total Runtime
19 1 9217 01m:27s
Build failures (1)
Application Target Toolchain Runtime (s) Worker
tests/periph/wdt atmega256rfr2-xpro gnu 0.28 breeze

Artifacts

@maribu maribu requested a review from benpicco November 6, 2024 17:33
@maribu
Copy link
Member Author

maribu commented Nov 6, 2024

Interesting: On native64 an 8 byte gap is added between XFA members. Looks like some padding gone wrong in the linker script or so.

@maribu maribu force-pushed the sys/shell/xfa-reduce-overhead branch from f4b2ef1 to 9434532 Compare November 6, 2024 20:07
@github-actions github-actions bot added Area: network Area: Networking Area: tests Area: tests and testing framework Area: core Area: RIOT kernel. Handle PRs marked with this with care! Area: drivers Area: Device drivers Area: OTA Area: Over-the-air updates Area: examples Area: Example Applications labels Nov 6, 2024
@maribu
Copy link
Member Author

maribu commented Nov 6, 2024

Turned out that the failure of native64 are a bug in XFA. #20960 fixes that bug. This now depends on and includes #20960.

@maribu maribu added the State: waiting for other PR State: The PR requires another PR to be merged first label Nov 6, 2024
@maribu maribu force-pushed the sys/shell/xfa-reduce-overhead branch from 9434532 to c9240ac Compare November 6, 2024 20:21
@github-actions github-actions bot added Platform: ARM Platform: This PR/issue effects ARM-based platforms Platform: ESP Platform: This PR/issue effects ESP-based platforms Area: cpu Area: CPU/MCU ports labels Nov 6, 2024
maribu added a commit to maribu/rust-riot-wrappers that referenced this pull request Nov 6, 2024
maribu added a commit to maribu/rust-riot-wrappers that referenced this pull request Nov 7, 2024
@maribu maribu force-pushed the sys/shell/xfa-reduce-overhead branch from c106c72 to 7ebf793 Compare November 7, 2024 10:03
Comment on lines -286 to +305
XFA_USE_CONST(shell_command_xfa_t*, shell_commands_xfa); \
XFA_USE_CONST(shell_command_xfa_t, shell_commands_xfa_v2); \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to rename this? I would have expected this name to be an implementation detail.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is an implementation detail, but the implementation of shell XFA is split across two repos: This repo contains the C implementation, https://github.com/RIOT-OS/rust-riot-wrappers contains the rust implementation.

If the rust impl would live here as well, I would have just adapted both in lock-step and kept the symbol name. But with the current state of affair, there is a benefit to be able to implement the rust side in a way that it will work with both variants of RIOT. (E.g. so that we can update the rust side while using a stable release on the C side.) See RIOT-OS/rust-riot-wrappers#134 for why this is beneficial for the rust side.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, thanks for the explanation, makes sense!

This command does the same as `help`, but provides a machine readable
JSON rather than a human readable table. It is only provided when the
(pseudo-)module `shell_json` is used.
This increases the robustness of the test by not relying on the
order shell commands are printed in. At least for XFA based shell
commands, there is no guarantee in which order they will be shown in
the help.
We do not need to add an array of pointers to the shell commands, just
an array of shell commands is sufficient. This reduced the overhead of
XFA by `sizeof(void *)` per command.
@maribu maribu force-pushed the sys/shell/xfa-reduce-overhead branch from 7ebf793 to 55a1a7a Compare November 7, 2024 21:01
@github-actions github-actions bot added Area: build system Area: Build system and removed Platform: ARM Platform: This PR/issue effects ARM-based platforms Area: network Area: Networking Area: core Area: RIOT kernel. Handle PRs marked with this with care! Area: drivers Area: Device drivers Area: OTA Area: Over-the-air updates Platform: ESP Platform: This PR/issue effects ESP-based platforms Area: cpu Area: CPU/MCU ports Area: examples Area: Example Applications labels Nov 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: build system Area: Build system Area: sys Area: System Area: tests Area: tests and testing framework CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR State: waiting for other PR State: The PR requires another PR to be merged first Type: cleanup The issue proposes a clean-up / The PR cleans-up parts of the codebase / documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants