CANParser: remove dependence on cereal

[deanlee]

resolve #825

[deanlee]

So we still need to support parsing dynamic capnp in parser.cc if the DYNAMIC_CAPNP is defined?

Also parser_pyx.pyx still depends on cereal, do we need to move it to openpiot? Or just leave it as it is?

[adeebshihadeh]

The goal of this is to make opendbc a real independent submodule,, and that means getting rid of the cereal dependence entirely.

I'm imagining we have a very fast function that converts from cereal's CAN struct to opendbc's. Then, we can wrap it in Cython for openpilot and PlotJuggler can also use it.

[deanlee]

got it

[adeebshihadeh]

@deanlee want to finish this one up?

[deanlee]

y, but it might take a little while.

[deanlee]

@adeebshihadeh : Done. there is no loss in performance.
PlotJuggler needs to be modified after this to use the new api.

[deanlee]

@adeebshihadeh removed all dependencies on cereal. the CAN frame uses the same format as CanPacker: [address, 0, dat, source]

[deanlee]

@sshane the extra conversion steps make it (commaai/openpilot#32009) more than twice as slow as the master.

this branch

6000 CAN packets, 10 runs
263.47 mean ms, 367.93 max ms, 258.64 min ms, 37.38 std ms
0.0506 mean ms / CAN packet

master

6000 CAN packets, 10 runs
125.61 mean ms, 130.85 max ms, 118.84 min ms, 2.78 std ms
0.0209 mean ms / CAN packet

[sshane]

Seeing similar results here. I think this is acceptable, but would be nice to speed it up in the future. Before #1039 I'm seeing

6000 CAN packets, 10 runs
295.08 mean ms, 303.76 max ms, 292.93 min ms, 2.99 std ms
0.0492 mean ms / CAN packet

on this PR I'm seeing

6000 CAN packets, 10 runs
214.52 mean ms, 216.95 max ms, 212.54 min ms, 1.13 std ms
0.0358 mean ms / CAN packet

and on current master I'm seeing

6000 CAN packets, 10 runs
97.72 mean ms, 98.73 max ms, 96.72 min ms, 0.70 std ms
0.0163 mean ms / CAN packet

[sshane]

@deanlee is this ready now? Once it's in we can really start on commaai/openpilot#32630!

[sshane]

BTW here's a list of things we need to remove from selfdrive/car, do you know if we have Python logging in opendbc yet? commaai/openpilot#32726

[deanlee]

Y, it's ready.

BTW here's a list of things we need to remove from selfdrive/car, do you know if we have Python logging in opendbc yet? https://github.com/commaai/openpilot/pull/32726BTW here's a list of things we need to remove from selfdrive/car, do you know if we have Python logging in opendbc yet? https://github.com/commaai/openpilot/pull/32726BTW here's a list of things we need to remove from selfdrive/car, do you know if we have Python logging in opendbc yet? commaai/openpilot#32726

There shouldn't be any, but I'm not very sure right now

[sshane]

LGTM, thanks for the PR!

[sshane]

@deanlee want to update PlotJuggler?

[deanlee]

I keep encountering the following error when trying to clone the PlotJuggler:

Cloning into 'PlotJuggler'...
remote: Enumerating objects: 18341, done.
remote: Counting objects: 100% (4032/4032), done.
remote: Compressing objects: 100% (428/428), done.
error: RPC failed; curl 56 GnuTLS recv error (-9): Error decoding the received TLS packet.
fatal: the remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed

I'll attempt it again later.