Test failures on Fedora Rawhide

[opoplawski]

SUMMARY

Attempting to build the collection on Fedora Rawhide we are seeing test failures.

ISSUE TYPE

• Bug Report

COMPONENT NAME

test_backend_openssl_cli.py
test_time.py

ANSIBLE VERSION

ansible [core 2.16.12]
  config file = /home/orion/.ansible.cfg
  configured module search path = ['/home/orion/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.13/site-packages/ansible
  ansible collection location = /home/orion/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.13.0 (main, Oct  8 2024, 00:00:00) [GCC 14.2.1 20240912 (Red Hat 14.2.1-4)] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True

COLLECTION VERSION

2.22.2

CONFIGURATION

CONFIG_FILE() = /home/orion/.ansible.cfg
EDITOR(env: EDITOR) = vim
GALAXY_SERVER_LIST(/home/orion/.ansible.cfg) = ['release_galaxy']
PAGER(env: PAGER) = less

OS / ENVIRONMENT

Fedora Rawhide

STEPS TO REPRODUCE

/usr/lib/rpm/ansible_collection.py test --python-interpreter /usr/bin/python3 --local

EXPECTED RESULTS

No test failures

ACTUAL RESULTS

Loading collection metadata from /home/orion/fedora/ansible-collection-community-crypto/ansible-collection-community-crypto-2.22.2-build/community.crypto-2.22.2/galaxy.yml
Running: ('ansible-test', 'units', '--python-interpreter', '/usr/bin/python3', '--local')

Unit test modules with Python 3.13
============================= test session starts ==============================
platform linux -- Python 3.13.0, pytest-8.3.3, pluggy-1.5.0
rootdir: /tmp/tmpyv_e91xd/ansible_collections/community/crypto
configfile: ../../../../../usr/lib/python3.13/site-packages/ansible_test/_data/pytest/config/default.ini
plugins: hypothesis-6.104.2, libtmux-0.37.0, flaky-3.8.1, asyncio-0.23.6, rerunfailures-14.0, forked-1.6.0, requests-mock-1.12.1, anyio-3.7.1, openfiles-0.6.0, pyfakefs-5.6.0, vcr-1.0.2, arraydiff-0.6.1, xprocess-1.0.2, mock-3.14.0, remotedata-0.4.1, mpi-0.6, xdist-3.6.1, timeout-2.3.1, doctestplus-1.2.1, localserver-0.9.0.post0
asyncio: mode=Mode.STRICT
created: 6/6 workers
6 workers [76 items]

........................................................................ [ 94%]
....                                                                     [100%]
- generated xml file: /tmp/tmpyv_e91xd/ansible_collections/community/crypto/tests/output/junit/python3.13-modules-units.xml -
============================== 76 passed in 1.91s ==============================
Unit test module_utils with Python 3.13
============================= test session starts ==============================
platform linux -- Python 3.13.0, pytest-8.3.3, pluggy-1.5.0
rootdir: /tmp/tmpyv_e91xd/ansible_collections/community/crypto
configfile: ../../../../../usr/lib/python3.13/site-packages/ansible_test/_data/pytest/config/default.ini
plugins: hypothesis-6.104.2, libtmux-0.37.0, flaky-3.8.1, asyncio-0.23.6, rerunfailures-14.0, forked-1.6.0, requests-mock-1.12.1, anyio-3.7.1, openfiles-0.6.0, pyfakefs-5.6.0, vcr-1.0.2, arraydiff-0.6.1, xprocess-1.0.2, mock-3.14.0, remotedata-0.4.1, mpi-0.6, xdist-3.6.1, timeout-2.3.1, doctestplus-1.2.1, localserver-0.9.0.post0
asyncio: mode=Mode.STRICT
created: 6/6 workers
6 workers [339 items]

................F.................F..................................... [ 21%]
...s.....s....s...ss...s..s.ss...s....F..s.....s....s......s.......s.... [ 42%]
........................................................................ [ 63%]
........................................................................ [ 84%]
.........FFF...F.F.................................                      [100%]
=================================== FAILURES ===================================
____________ test_interpolate_timestamp[start1-end1-0.5-expected1] _____________
[gw3] linux -- Python 3.13.0 /usr/bin/python3

start = {'day': 1, 'hour': 0, 'minute': 0, 'month': 1, ...}
end = {'day': 1, 'hour': 1, 'minute': 0, 'month': 1, ...}, percentage = 0.5
expected = {'day': 1, 'hour': 0, 'minute': 30, 'month': 1, ...}

    @pytest.mark.parametrize("start, end, percentage, expected", TEST_INTERPOLATE_TIMESTAMP)
    def test_interpolate_timestamp(start, end, percentage, expected):
        module = MagicMock()
        backend = OpenSSLCLIBackend(module, openssl_binary='openssl')
        ts_start = backend.get_utc_datetime(**start)
        ts_end = backend.get_utc_datetime(**end)
        ts_expected = backend.get_utc_datetime(**expected)
        timestamp = backend.interpolate_timestamp(ts_start, ts_end, percentage)
>       assert ts_expected == timestamp
E       assert datetime.datetime(2024, 1, 1, 0, 30) == datetime.datetime(2024, 1, 1, 7, 30)

tests/unit/plugins/module_utils/acme/test_backend_openssl_cli.py:122: AssertionError
____________ test_interpolate_timestamp[start2-end2-1.0-expected2] _____________
[gw3] linux -- Python 3.13.0 /usr/bin/python3

start = {'day': 1, 'hour': 0, 'minute': 0, 'month': 1, ...}
end = {'day': 1, 'hour': 1, 'minute': 0, 'month': 1, ...}, percentage = 1.0
expected = {'day': 1, 'hour': 1, 'minute': 0, 'month': 1, ...}

    @pytest.mark.parametrize("start, end, percentage, expected", TEST_INTERPOLATE_TIMESTAMP)
    def test_interpolate_timestamp(start, end, percentage, expected):
        module = MagicMock()
        backend = OpenSSLCLIBackend(module, openssl_binary='openssl')
        ts_start = backend.get_utc_datetime(**start)
        ts_end = backend.get_utc_datetime(**end)
        ts_expected = backend.get_utc_datetime(**expected)
        timestamp = backend.interpolate_timestamp(ts_start, ts_end, percentage)
>       assert ts_expected == timestamp
E       assert datetime.datetime(2024, 1, 1, 1, 0) == datetime.datetime(2024, 1, 1, 8, 0)

tests/unit/plugins/module_utils/acme/test_backend_openssl_cli.py:122: AssertionError
____________ test_interpolate_timestamp[start0-end0-0.0-expected0] _____________
[gw2] linux -- Python 3.13.0 /usr/bin/python3

start = {'day': 1, 'hour': 0, 'minute': 0, 'month': 1, ...}
end = {'day': 1, 'hour': 1, 'minute': 0, 'month': 1, ...}, percentage = 0.0
expected = {'day': 1, 'hour': 0, 'minute': 0, 'month': 1, ...}

    @pytest.mark.parametrize("start, end, percentage, expected", TEST_INTERPOLATE_TIMESTAMP)
    def test_interpolate_timestamp(start, end, percentage, expected):
        module = MagicMock()
        backend = OpenSSLCLIBackend(module, openssl_binary='openssl')
        ts_start = backend.get_utc_datetime(**start)
        ts_end = backend.get_utc_datetime(**end)
        ts_expected = backend.get_utc_datetime(**expected)
        timestamp = backend.interpolate_timestamp(ts_start, ts_end, percentage)
>       assert ts_expected == timestamp
E       assert datetime.datetime(2024, 1, 1, 0, 0) == datetime.datetime(2024, 1, 1, 7, 0)

tests/unit/plugins/module_utils/acme/test_backend_openssl_cli.py:122: AssertionError
_______________________ test_epoch_seconds[0-timestamp0] _______________________
[gw2] linux -- Python 3.13.0 /usr/bin/python3

seconds = 0
timestamp = {'day': 1, 'hour': 0, 'microsecond': 0, 'minute': 0, ...}

    @pytest.mark.parametrize("seconds, timestamp", TEST_EPOCH_SECONDS)
    def test_epoch_seconds(seconds, timestamp):
        ts_wo_tz = datetime.datetime(**timestamp)
>       assert seconds == get_epoch_seconds(ts_wo_tz)
E       assert 0 == 25200.0
E        +  where 25200.0 = get_epoch_seconds(datetime.datetime(1970, 1, 1, 0, 0))

tests/unit/plugins/module_utils/test_time.py:298: AssertionError
_____________________ test_epoch_seconds[1e-06-timestamp1] _____________________
[gw4] linux -- Python 3.13.0 /usr/bin/python3

seconds = 1e-06
timestamp = {'day': 1, 'hour': 0, 'microsecond': 1, 'minute': 0, ...}

    @pytest.mark.parametrize("seconds, timestamp", TEST_EPOCH_SECONDS)
    def test_epoch_seconds(seconds, timestamp):
        ts_wo_tz = datetime.datetime(**timestamp)
>       assert seconds == get_epoch_seconds(ts_wo_tz)
E       assert 1e-06 == 25200.000001
E        +  where 25200.000001 = get_epoch_seconds(datetime.datetime(1970, 1, 1, 0, 0, 0, 1))

tests/unit/plugins/module_utils/test_time.py:298: AssertionError
_____________________ test_epoch_seconds[0.001-timestamp2] _____________________
[gw0] linux -- Python 3.13.0 /usr/bin/python3

seconds = 0.001
timestamp = {'day': 1, 'hour': 0, 'microsecond': 1000, 'minute': 0, ...}

    @pytest.mark.parametrize("seconds, timestamp", TEST_EPOCH_SECONDS)
    def test_epoch_seconds(seconds, timestamp):
        ts_wo_tz = datetime.datetime(**timestamp)
>       assert seconds == get_epoch_seconds(ts_wo_tz)
E       assert 0.001 == 25200.001
E        +  where 25200.001 = get_epoch_seconds(datetime.datetime(1970, 1, 1, 0, 0, 0, 1000))

tests/unit/plugins/module_utils/test_time.py:298: AssertionError
____________________ test_epoch_seconds[3691.2-timestamp3] _____________________
[gw2] linux -- Python 3.13.0 /usr/bin/python3

seconds = 3691.2
timestamp = {'day': 1, 'hour': 1, 'microsecond': 200000, 'minute': 1, ...}

    @pytest.mark.parametrize("seconds, timestamp", TEST_EPOCH_SECONDS)
    def test_epoch_seconds(seconds, timestamp):
        ts_wo_tz = datetime.datetime(**timestamp)
>       assert seconds == get_epoch_seconds(ts_wo_tz)
E       assert 3691.2 == 28891.2
E        +  where 28891.2 = get_epoch_seconds(datetime.datetime(1970, 1, 1, 1, 1, 31, 200000))

tests/unit/plugins/module_utils/test_time.py:298: AssertionError
_____________________ test_epoch_to_seconds[timestamp0-62] _____________________
[gw4] linux -- Python 3.13.0 /usr/bin/python3

timestamp = datetime.datetime(1970, 1, 1, 0, 1, 2), expected_seconds = 62

    @pytest.mark.parametrize("timestamp, expected_seconds", TEST_EPOCH_TO_SECONDS)
    def test_epoch_to_seconds(timestamp, expected_seconds):
>       assert expected_seconds == get_epoch_seconds(timestamp)
E       assert 62 == 25262.0
E        +  where 25262.0 = get_epoch_seconds(datetime.datetime(1970, 1, 1, 0, 1, 2))

tests/unit/plugins/module_utils/test_time.py:311: AssertionError
=============================== warnings summary ===============================
tests/unit/plugins/module_utils/acme/test_backend_openssl_cli.py::test_interpolate_timestamp[start1-end1-0.5-expected1]
tests/unit/plugins/module_utils/acme/test_backend_openssl_cli.py::test_interpolate_timestamp[start2-end2-1.0-expected2]
tests/unit/plugins/module_utils/acme/test_backend_openssl_cli.py::test_interpolate_timestamp[start0-end0-0.0-expected0]
  /tmp/tmpyv_e91xd/ansible_collections/community/crypto/plugins/module_utils/time.py:92: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
    return datetime.datetime.utcfromtimestamp(timestamp)

tests/unit/plugins/module_utils/acme/test_backend_openssl_cli.py::test_now
tests/unit/plugins/module_utils/test_time.py::test_get_now_datetime
  /tmp/tmpyv_e91xd/ansible_collections/community/crypto/plugins/module_utils/time.py:49: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
    return datetime.datetime.utcnow()

tests/unit/plugins/module_utils/openssh/test_cryptography.py::test_default_key_params[dsa-None-None-None]
  /tmp/tmpyv_e91xd/ansible_collections/community/crypto/tests/unit/plugins/module_utils/openssh/test_cryptography.py:151: CryptographyDeprecationWarning: SSH DSA key support is deprecated and will be removed in a future release
    pair = OpensshKeypair.generate(keytype=keytype, size=size, passphrase=passphrase, comment=comment)

tests/unit/plugins/module_utils/openssh/test_cryptography.py::test_default_key_params[dsa-None-None-None]
  /tmp/tmpyv_e91xd/ansible_collections/community/crypto/tests/unit/plugins/module_utils/openssh/test_cryptography.py:153: CryptographyDeprecationWarning: SSH DSA key support is deprecated and will be removed in a future release
    pair = OpensshKeypair.generate(keytype=keytype, size=size, passphrase=passphrase, comment=comment)

tests/unit/plugins/module_utils/openssh/test_cryptography.py::test_keypair_comparison
  /tmp/tmpyv_e91xd/ansible_collections/community/crypto/tests/unit/plugins/module_utils/openssh/test_cryptography.py:351: CryptographyDeprecationWarning: SSH DSA key support is deprecated and will be removed in a future release
    assert OpensshKeypair.generate() != OpensshKeypair.generate(keytype='dsa')

tests/unit/plugins/module_utils/openssh/test_cryptography.py::test_keypair_comparison
  /tmp/tmpyv_e91xd/ansible_collections/community/crypto/tests/unit/plugins/module_utils/openssh/test_cryptography.py:363: CryptographyDeprecationWarning: SSH DSA key support is deprecated and will be removed in a future release
    'pair': OpensshKeypair.generate(keytype='dsa', passphrase='change_me'.encode('UTF-8')),

tests/unit/plugins/module_utils/openssh/test_cryptography.py::test_keypair_comparison
  /tmp/tmpyv_e91xd/ansible_collections/community/crypto/plugins/module_utils/openssh/cryptography.py:560: CryptographyDeprecationWarning: SSH DSA keys are deprecated and will be removed in a future release.
    privatekey = privatekey_loader(

tests/unit/plugins/module_utils/openssh/test_cryptography.py::test_keypair_comparison
  /tmp/tmpyv_e91xd/ansible_collections/community/crypto/plugins/module_utils/openssh/cryptography.py:615: CryptographyDeprecationWarning: SSH DSA keys are deprecated and will be removed in a future release.
    publickey = publickey_loader(

tests/unit/plugins/module_utils/openssh/test_cryptography.py::test_keypair_comparison
  /tmp/tmpyv_e91xd/ansible_collections/community/crypto/tests/unit/plugins/module_utils/openssh/test_cryptography.py:380: CryptographyDeprecationWarning: SSH DSA key support is deprecated and will be removed in a future release
    loaded_dsa_key = OpensshKeypair.load(path=keys['dsa']['filename'], passphrase='change_me'.encode('UTF-8'))

tests/unit/plugins/module_utils/openssh/test_cryptography.py::test_keypair_comparison
  /tmp/tmpyv_e91xd/ansible_collections/community/crypto/tests/unit/plugins/module_utils/openssh/test_cryptography.py:383: CryptographyDeprecationWarning: SSH DSA key support is deprecated and will be removed in a future release
    loaded_dsa_key.update_passphrase('change_me_again'.encode('UTF-8'))

tests/unit/plugins/module_utils/openssh/test_cryptography.py::test_keypair_comparison
  /tmp/tmpyv_e91xd/ansible_collections/community/crypto/tests/unit/plugins/module_utils/openssh/test_cryptography.py:386: CryptographyDeprecationWarning: SSH DSA key support is deprecated and will be removed in a future release
    loaded_dsa_key.update_passphrase('change_me'.encode('UTF-8'))

tests/unit/plugins/module_utils/openssh/test_cryptography.py::test_valid_user_key_params[dsa-1024-change_me-comment]
  /tmp/tmpyv_e91xd/ansible_collections/community/crypto/tests/unit/plugins/module_utils/openssh/test_cryptography.py:169: CryptographyDeprecationWarning: SSH DSA key support is deprecated and will be removed in a future release
    pair = OpensshKeypair.generate(keytype=keytype, size=size, passphrase=passphrase, comment=comment)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: /tmp/tmpyv_e91xd/ansible_collections/community/crypto/tests/output/junit/python3.13-module_utils-units.xml -
=========================== short test summary info ============================
SKIPPED [15] tests/unit/plugins/module_utils/crypto/test_asn1.py:78: unconditional skip
FAILED tests/unit/plugins/module_utils/acme/test_backend_openssl_cli.py::test_interpolate_timestamp[start1-end1-0.5-expected1]
FAILED tests/unit/plugins/module_utils/acme/test_backend_openssl_cli.py::test_interpolate_timestamp[start2-end2-1.0-expected2]
FAILED tests/unit/plugins/module_utils/acme/test_backend_openssl_cli.py::test_interpolate_timestamp[start0-end0-0.0-expected0]
FAILED tests/unit/plugins/module_utils/test_time.py::test_epoch_seconds[0-timestamp0]
FAILED tests/unit/plugins/module_utils/test_time.py::test_epoch_seconds[1e-06-timestamp1]
FAILED tests/unit/plugins/module_utils/test_time.py::test_epoch_seconds[0.001-timestamp2]
FAILED tests/unit/plugins/module_utils/test_time.py::test_epoch_seconds[3691.2-timestamp3]
FAILED tests/unit/plugins/module_utils/test_time.py::test_epoch_to_seconds[timestamp0-62]
============ 8 failed, 316 passed, 15 skipped, 15 warnings in 9.80s ============
FATAL: Command "pytest -r a -n auto --color no -p no:cacheprovider -c /usr/lib/python3.13/site-packages/ansible_test/_data/pytest/config/default.ini --junit-xml /tmp/tmpyv_e91xd/ansible_collections/community/crypto/tests/output/junit/python3.13-module_utils-units.xml --strict-markers --rootdir /tmp/tmpyv_e91xd/ansible_collections/community/crypto --confcutdir /tmp/tmpyv_e91xd/ansible_collections/community/crypto tests/unit/plugins/module_utils/acme/test_backend_cryptography.py tests/unit/plugins/module_utils/acme/test_backend_openssl_cli.py tests/unit/plugins/module_utils/acme/test_challenges.py tests/unit/plugins/module_utils/acme/test_errors.py tests/unit/plugins/module_utils/acme/test_io.py tests/unit/plugins/module_utils/acme/test_orders.py tests/unit/plugins/module_utils/acme/test_utils.py tests/unit/plugins/module_utils/crypto/test_asn1.py tests/unit/plugins/module_utils/crypto/test_cryptography_support.py tests/unit/plugins/module_utils/crypto/test_math.py tests/unit/plugins/module_utils/crypto/test_pem.py tests/unit/plugins/module_utils/openssh/test_certificate.py tests/unit/plugins/module_utils/openssh/test_cryptography.py tests/unit/plugins/module_utils/openssh/test_utils.py tests/unit/plugins/module_utils/test_time.py" returned exit status 1.
Command '('ansible-test', 'units', '--python-interpreter', '/usr/bin/python3', '--local')' returned non-zero exit status 1.

[felixfontein]

Running the tests with standard isolation (Docker) does not exhibit this:

ansible-test units -v --docker --python 3.13 tests/unit/plugins/module_utils/acme/test_backend_openssl_cli.py tests/unit/plugins/module_utils/test_time.py

My guess is that it is related to the timezone of the machine where the tests are run, and in the default Docker container the timezone is likely normalized (UTC maybe?). Does your machine's local time happen to be in a timezone that's UTC-7 right now?

I just tried running the tests without Docker isolation (ansible-test units -v --python 3.12 tests/unit/plugins/module_utils/acme/test_backend_openssl_cli.py tests/unit/plugins/module_utils/test_time.py --requirements), I got similar errors, but with -1 hour instead of +7 hours difference; my local timezone is UTC+2, so for some reason there's a one hour difference, so maybe your timezone is UTC-6 and not UTC-7?

In any case, I have to take a closer look at what's going on...

[felixfontein]

Hmm, it looks like that these failures are due to datetime.datetime.utcfromtimestamp(0).timestamp() not being zero if the machine's timezone is not UTC. The datetime module is pretty bad with implicit timezones, which I guess is the one big reason why a lot of the deprecations show up and why everyone avoids them ;) If I remember correctly the code still needs to work with them for backwards compatibility with other libraries. I haven't looked at that code since ~early May so I'm not sure which parts exactly need this.

I'll try to take a closer look over the next days...

[felixfontein]

Note to self: freezegun should allow to run tests with different time zones. We should definitely use this to avoid this problem, since CI always uses Docker isolation, which always has UTC...

(0.3.10 seems to be the last version with Python 2.6 support, 0.3.15 the last one with Python 2.7 support, from 1.0.0 on it's Python 3 only.)

[opoplawski]

Yeah, I'm running in MDT. export TZ=UTC does not appear to help. We can't really make use of docker in the Fedora builds, but since the koji builders are UTC, it builds fine there. So, not a particularly critical issue for us, but does break local builds. Thanks for looking into it.

[felixfontein]

I guess that's both the good and bad side when too many systems use UTC ;-)

[felixfontein]

Hmm, it turns out that freezegun has a severe bug in its datetime.datetime.now(tz=xxx) implemenation (spulec/freezegun#348, spulec/freezegun#553). There's another project, time-machine which seems to be better, but even its earliest versions only support Python 3.6 or newer. (We also have Python 3.5, 2.7, and 2.6.)

This is sad :-( I guess I have to kick some tests out that don't work because of that bug, and stick with freezegun...

[felixfontein]

I think I have a fix for this: #810