Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Account for bytes in Python 3 #39

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Account for bytes in Python 3 #39

wants to merge 1 commit into from

Conversation

neilmb
Copy link
Contributor

@neilmb neilmb commented Apr 6, 2018

This accounts (somewhat) for the difference between bytes and strings in Python 3. It fixes #38 in particular.

It also adds python 3 to the Vagrant VM for developers and defines a make test3 target for running the tests specifically under that python 3.

@neilmb neilmb requested a review from b4hand April 6, 2018 18:17

# if we stored a bytes object, decode it into a string
try:
obj = obj.decode('ascii')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think instead of this you should change the mode for download_file to wb so that the file object will be in binary mode as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish that was the fix. The test failure that needed fixing was on https://github.com/seomoz/s3po/blob/master/test/test_connection.py#L29 where it checks the result of self.conn.download. For the memory backend, download() grabs the actual python object that is stored, so when upload_file runs with mode rb, the value that the memory backend stores is bytes.

https://github.com/seomoz/s3po/blob/master/s3po/backends/memory.py#L20 is trying to write those bytes to into fobj which is a StringIO from https://github.com/seomoz/s3po/blob/master/s3po/connection.py#L79.

I'm totally not sure what the "right way" to handle the memory backend is in the face of string vs. bytes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the default for StringIO is to use the system locale via this method:

locale.getpreferredencoding(False)

I think that's preferable to specifying ascii since that could throw away bytes greater than 0x7f.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Python3 needs mode rb in upload_file
2 participants