Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix capture for big segments. #308

Draft
wants to merge 2 commits into
base: develop
Choose a base branch
from
Draft

Conversation

jmichelp
Copy link
Contributor

Current logic was keeping all traces in memory instead of freeing each
segment when possible.

Current logic was keeping all traces in memory instead of freeing each
segment when possible.
@jmichelp
Copy link
Contributor Author

Oops, don't merge yet. It seems the fix is not complete yet

@jmichelp
Copy link
Contributor Author

jmichelp commented Feb 15, 2021

I don't know if this is the expected behaviour for Chipwhisperer.

When I run a long capture, if I follow the given examples, I get the following simplified code:

project = cw.create_project(...)
for _ in range(...):
  trace = cw.capture_trace(...)
  if trace:
    project.traces.append(trace)
project.save()

There are 2 issues:

  1. I can't control the length of each segment because there's no official API to set it
  2. all traces and all segments stay in memory until project.save() is called to trigger Project.saveTraceManager() is called.

The first commit was trying to solve this situation by only keeping the current segment in memory but segments are created by calling Segments.new() which copies an empty TraceContainer without initialising the configuration object. This is fixed by calling Project.get_new_trace_segment() instead.

@jmichelp jmichelp marked this pull request as draft March 4, 2021 23:50
@colinoflynn
Copy link
Contributor

On projects - it was always a bit messy for larger captures. We've been looking at changing around some of that, as even ourselves we just use larger NumPy arrays when it gets to longer captures for examples (and saving/segmenting them in the capture loop).

It might be easier to rebuild from something more modern (we had looked into Zarr here) rather than fixing the current code...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants