Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[shortfin] Fix mobilenet_server example by filling mapped storage. #241

Merged
merged 3 commits into from
Oct 3, 2024

Conversation

ScottTodd
Copy link
Member

I saw this error trying to run this example:

RuntimeError: Async exception on <Worker 'device-hostcpu:0:0@0'>): '_shortfin_default.lib.array.storage' object has no attribute 'data'

I looked around for and API that would allow for copying from array.array (https://docs.python.org/3/library/array.html) into shortfin's array.storage (https://shortfin.readthedocs.io/en/latest/reference.html#shortfin_default.lib.array.storage) and only found map().

Quite possible that I'm missing something, or there is an easier interface that isn't fully plumbed through to Python and the documentation yet.

Logs of this running: https://gist.github.com/ScottTodd/da8c1d6805584490c46e4d2086404a6b

@ScottTodd ScottTodd marked this pull request as ready for review October 1, 2024 21:53
@@ -40,7 +40,12 @@ async def run(self):
# just writing to the backing storage is the best we have API
# support for. Generally, APIs on storage should be mirrored onto
# the array.
self.host_staging.storage.data = request.raw_image_data
# TODO: Easier to use API for writing into the storage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grrg. Yeah, these low level transfer methods need an upgrade. Up until the most recent commit, I didn't have the device level metadata I needed to determine the right way to load data on to the device (i.e. map, transfer, chunk, etc). Now that I have that, need to design a more ergonomic API.

With that said, I believe that mapping.fill(buffer) will do what you want if the buffer is the size of the target (i.e. it will copy x1 vs tile). There are some tests that exercise that.

So I think you can say:

with self.host_staging.storage.map(write=True, discard=True) as m:
  m.fill(request.raw_image_data)

(assuming sizes line up)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, fill works. I saw that API and thought it would only work for a pattern of bytes, based on my read the docs: https://shortfin.readthedocs.io/en/latest/reference.html#shortfin_default.lib.array.storage.fill (also seems like docs are missing for the result of map()... should make sure we have cross reference links between all the types).

@ScottTodd ScottTodd changed the title [shortfin] Fix mobilenet_server example by writing bytes directly. [shortfin] Fix mobilenet_server example by filling mapped storage. Oct 3, 2024
@ScottTodd ScottTodd merged commit 8e074b4 into nod-ai:main Oct 3, 2024
7 of 8 checks passed
@ScottTodd ScottTodd deleted the shortfin-example-io branch October 3, 2024 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants