WIP [$500 bounty] test_models: add a test that fuzzes the tx messages #32577
Conversation
|
Thanks for contributing to openpilot! In order for us to review your PR as quickly as possible, check the following:
|
selfdrive/car/tests/test_models.py
Outdated
| @st.composite | ||
| def car_control_strategy(draw): |
There was a problem hiding this comment.
you need to have openpilot generate these CAN messages, we want to ensure that panda doesn't block anything openpilot will try to send to start to this test
There was a problem hiding this comment.
oh ok so it should somehow get those CAN messages from state_control in the Control class from controlsd.py with CarStates obtained with fuzzed data
selfdrive/car/tests/test_models.py
Outdated
| now_nanos = DT_CTRL * 1e9 | ||
| CI = self.CarInterface(self.CP, self.CarController, self.CarState) | ||
|
|
||
| if self.CP.carName == "honda": |
There was a problem hiding this comment.
this test should know as little about the cars/brands as possible
selfdrive/car/tests/test_models.py
Outdated
| if self.CP.carName == "honda": | ||
| if ((addr == 0xE4 or addr == 0x194) and (CC.latActive and act.steerOutputCan != 0)) \ | ||
| or (addr == 0x296 and bus == bus_buttons and CC.cruiseControl.cancel): | ||
| self.assertFalse(tx_ok, "issue TX") |
There was a problem hiding this comment.
same, should not hard code addresses here. should also be a test that asserts true
|
@sshane CC messages are now generated by controlsd that's fed with the data required via hypothesis |
|
This should really follow the same structure as the rx fuzzy test, going to close. Feel free to retry if you have a simpler approach. This should be simple and provide a high leverage amount of coverage, without starting processes or manually defining message structs (we have https://github.com/commaai/openpilot/blob/master/selfdrive/test/fuzzy_generation.py for that) Again we should start by asserting that any message the car controller generates should be accepted (tx=1) by panda safety. |
Concerning bounty #32425
Generating tests for
CarControlmessages and check iftxis False whencontrols_allowedif FalseLmk if I'm on the right path @sshane