Getting OSError in s3 when permission for kms:Decrypt are missing

[mayk0gan]

Problem description

Be sure your description clearly answers the following questions:

• What are you trying to achieve?
• What is the expected result?
• What are you seeing instead?

Steps/code to reproduce the problem

In order for us to be able to solve your problem, we have to be able to reproduce it on our end.
Without reproducing the problem, it is unlikely that we'll be able to help you.

Include full tracebacks, logs and datasets if necessary.
Please keep the examples minimal (minimal reproducible example).

In AWS S3, when attempting to access an object for which one lacks "kms:Decrypt" permissions, instead of receiving the expected indicative exception type, a general OSError is encountered. This circumstance makes it challenging to effectively catch and identify this particular type of error.

Traceback:

 with smart_open.open(
 File \"/usr/lib/python3.8/site-packages/smart_open/smart_open_lib.py\", line 224, in open
 binary = _open_binary_stream(uri, binary_mode, transport_params)
 File \"/usr/lib/python3.8/site-packages/smart_open/smart_open_lib.py\", line 400, in _open_binary_stream
 fobj = submodule.open_uri(uri, mode, transport_params)
 File \"/usr/lib/python3.8/site-packages/smart_open/s3.py\", line 224, in open_uri
 return open(parsed_uri['bucket_id'], parsed_uri['key_id'], mode, **kwargs)
 File \"/usr/lib/python3.8/site-packages/smart_open/s3.py\", line 291, in open
 fileobj = Reader(
 File \"/usr/lib/python3.8/site-packages/smart_open/s3.py\", line 574, in __init__
 self.seek(0)
 File \"/usr/lib/python3.8/site-packages/smart_open/s3.py\", line 666, in seek
 self._current_pos = self._raw_reader.seek(offset, whence)
 File \"/usr/lib/python3.8/site-packages/smart_open/s3.py\", line 417, in seek
 self._open_body(start, stop)
 File \"/usr/lib/python3.8/site-packages/smart_open/s3.py\", line 438, in _open_body
 response = _get(
 File \"/usr/lib/python3.8/site-packages/smart_open/s3.py\", line 338, in _get
 raise wrapped_error from error
OSError: unable to access bucket: 'BUCKET_NAME' key: 'key/path/1.gz' version: None error: An error occurred (AccessDenied) when calling the GetObject operation: User: arn:aws:sts::ACCOUNT_ID:assumed-role/role-name/session-name is not authorized to perform: kms:Decrypt on resource: arn:aws:kms:us-east-1:ACCOUNT_ID:key/123456-1231231-3123123123-123123 because no identity-based policy allows the kms:Decrypt action

I suspect that this is the relevant area:

Versions

Please provide the output of:

import platform, sys, smart_open
print(platform.platform())
print("Python", sys.version)
print("smart_open", smart_open.__version__)

macOS-12.4-arm64-arm-64bit
Python 3.8.16 (default, Dec  7 2022, 01:24:57)
[Clang 14.0.0 (clang-1400.0.29.202)]
smart_open 6.3.0

Checklist

Before you create the issue, please make sure you have:

• Described the problem clearly
• Provided a minimal reproducible example, including any required data
• Provided the version numbers of the relevant software

[mayk0gan]

btw, the same happens with "NoSuchKey":

 with smart_open.open(
 File \"/usr/lib/python3.8/site-packages/smart_open/smart_open_lib.py\", line 224, in open
 binary = _open_binary_stream(uri, binary_mode, transport_params)
 File \"/usr/lib/python3.8/site-packages/smart_open/smart_open_lib.py\", line 400, in _open_binary_stream
 fobj = submodule.open_uri(uri, mode, transport_params)
 File \"/usr/lib/python3.8/site-packages/smart_open/s3.py\", line 224, in open_uri
 return open(parsed_uri['bucket_id'], parsed_uri['key_id'], mode, **kwargs)
 File \"/usr/lib/python3.8/site-packages/smart_open/s3.py\", line 291, in open
 fileobj = Reader(
 File \"/usr/lib/python3.8/site-packages/smart_open/s3.py\", line 574, in __init__
 self.seek(0)
 File \"/usr/lib/python3.8/site-packages/smart_open/s3.py\", line 666, in seek
 self._current_pos = self._raw_reader.seek(offset, whence)
 File \"/usr/lib/python3.8/site-packages/smart_open/s3.py\", line 417, in seek
 self._open_body(start, stop)
 File \"/usr/lib/python3.8/site-packages/smart_open/s3.py\", line 438, in _open_body
 response = _get(
 File \"/usr/lib/python3.8/site-packages/smart_open/s3.py\", line 338, in _get
 raise wrapped_error from error
OSError: unable to access bucket: 'BUCKET' key: 'key/path/1.json' version: None error: An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist

[ddelange]

you can isolate specific errors as follows:

try:
    your_open_fn(key)
except OSError as exc:
    msg = str(exc)
    if 'PermissionDenied' in msg:
        # special handling
    else:
        raise # default handling

[mayk0gan]

you can isolate specific errors as follows:

try:
    your_open_fn(key)
except OSError as exc:
    msg = str(exc)
    if 'PermissionDenied' in msg:
        # special handling
    else:
        raise # default handling

thanks @ddelange, that's what I'm doing currently - but it isn't a very "pythonic" way.
Is there a way to raise the original exception, instead of OSError?

[ddelange]

smart_open uses raise from so the original exception will be accessible via __cause__ https://stackoverflow.com/a/71514843/5511061