Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_models: add a test that fuzzes the tx messages #32425

Open
sshane opened this issue May 14, 2024 · 1 comment · May be fixed by #33587
Open

test_models: add a test that fuzzes the tx messages #32425

sshane opened this issue May 14, 2024 · 1 comment · May be fixed by #33587

Comments

@sshane
Copy link
Contributor

sshane commented May 14, 2024

Currently we compare the car states from panda and openpilot to ensure that both parse CAN messages with the same 1) validity checks and 2) correctness/no bugs. That let us fix quite a few mismatches in behavior between the two: #30443

Now we want to fuzz the tx messages as well, to discover bugs related to controlsAllowed in panda, and general tx safety. Here is one such mismatch that we should aim to catch: commaai/panda#1948

See the first PR linked above for rx to know where to start. Bounty is for a merged PR that adds fuzz testing for sent openpilot messages that

  1. detects mismatches in tx (sent messages) logic between openpilot and panda,
  2. extensible (generic, and easy to expand with more state),
  3. as performant as our current fuzzy testing, and
  4. covers as many cases as possible (catches a few bugs along the way, if there are any).
@markopetkovic96
Copy link

@sshane Based on the rx example, should the tx messages be similar to the rx messages? Create a packet, transmit can messages (via safety_tx_hook), update openpilot, and compare it with the previous panda state?

@adeebshihadeh adeebshihadeh changed the title [$500 bounty] test_models: add a test that fuzzes the tx messages test_models: add a test that fuzzes the tx messages Jul 7, 2024
@bongbui321 bongbui321 linked a pull request Sep 19, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Locked
2 participants