[6.15.z] Eval CapsuleContent::API Assertion Errors, time delta/format in 'wait_for_sync()'

[Satellite-QE]

Cherrypick of PR: #14066

Problem Statement

4 Failures in recent runs of API::CapsuleContent, for AssertionError:

tests/foreman/api/test_capsulecontent.py:190: in test_positive_checksum_sync
    module_capsule_configured.wait_for_sync()
robottelo/host_helpers/capsule_mixins.py:75: in wait_for_sync
    or datetime.strptime(sync_status['last_sync_time'], '%Y-%m-%d %H:%M:%S UTC')
E   AssertionError

Locally I can see the last_sync_time format from logs as:

nailgun.client - DEBUG - Received HTTP 200 response: {"last_sync_time":"2024-02-13 15:05:10 UTC", ...

Solution

• Split the assertions checking last_sync_time (check if None /Falsey first), then checking length of any in-progress sync tasks, avoiding possibly ambiguous assertion- where last_sync_time has not updated with a recent task, and is still an old sync time/None.
  The prior Assertions with AND, OR, etc fail without showing the compared values. Provided error messages with assertion failure details.
• Apply a one second margin of safety to start_time, due to rounding. Often the assertion fails for one second delta, which is start time recorded and sync done at pretty much the same time, but the seconds from sync_status were rounded to one second before start_time. This is inconsistent, depending on the milliseconds when last_sync_time is recorded.
• From dateutil.parser, use parse(date-time string) method on the start_time, and any found last_sync_time. So when checking the last sync time is newer, parse will account for different formats of the same time. Similar solution to PR Fix Date assertion failure in CI errata:e2e  #13121

Related Issue >> From 'sat-6.15-rhel8-Capsule-Content'

4 Failed api scenarios due to the AssertionError in this method. Inconsistently, others will fail locally for the same error.
6.14.z and 6.15.0 (most recent Build 13, Mar 7 2024, 12:43 AM)

PRT Case

trigger: test-robottelo
pytest: tests/foreman/api/test_capsulecontent.py::TestCapsuleContentManagement

[damoore044]

trigger: test-robottelo
pytest: tests/foreman/api/test_capsulecontent.py::TestCapsuleContentManagement

[Satellite-QE]

PRT Result

Build Number: 6086
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/api/test_capsulecontent.py::TestCapsuleContentManagement --external-logging
Test Result : ===== 6 failed, 14 passed, 1 skipped, 1186 warnings in 5610.67s (1:33:30) ======

[Satellite-QE]

PRT Result

Build Number: 6108
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/api/test_capsulecontent.py::TestCapsuleContentManagement --external-logging
Test Result : =========== 20 passed, 1243 warnings, 1 error in 5632.21s (1:33:52) ============

David- Single Failure is BrokerError: Valid hostname must be provided .... restarting PRT

[damoore044]

trigger: test-robottelo
pytest: tests/foreman/api/test_capsulecontent.py::TestCapsuleContentManagement

[damoore044]

trigger: test-robottelo
pytest: tests/foreman/api/test_capsulecontent.py::TestCapsuleContentManagement

[damoore044]

trigger: test-robottelo
pytest: tests/foreman/api/test_capsulecontent.py::TestCapsuleContentManagement
env:
  ROBOTTELO_server__deploy_arguments__deploy_sat_version: '6.15.0'
  ROBOTTELO_server__deploy_arguments__deploy_snap_version: '10.1'

[damoore044]

trigger: test-robottelo
pytest: tests/foreman/api/test_capsulecontent.py::TestCapsuleContentManagement

[damoore044]

Build [6135] is listed as a failure, but this is because there was an error finding the test report file generated.
The latest forced push just prior to this was only a rebase, no code changes.

From console output we can see that 22/22 API tests (all for this module) passed as expected in 6135:

13:01:28  ============================= test session starts ==============================
13:01:28  platform linux -- Python 3.11.8, pytest-8.1.1, pluggy-1.4.0
13:01:28  Mandatory Requirements Mismatch: paramiko==3.4.0  # Temporary until Broker is back on PyPi docker==7.0.0  # Temporary until Broker is back on PyPi
13:01:28  Optional Requirements Mismatch: manage==0.1.15 pytest-cov==4.1.0 flake8==7.0.0 sphinx-autoapi==3.0.0 redis==5.0.3 sphinx==7.2.6 pre-commit==3.6.2
13:01:28  To update mismatched requirements, run the pytest command with '--update-required-reqs' OR '--update-all-reqs' option.
13:01:28  shared_function enabled - OFF - scope:  - storage: file
13:01:28  rootdir: /opt/app-root/src/robottelo
13:01:28  configfile: pyproject.toml
13:01:28  plugins: fixturecollection-0.1.2, ibutsu-2.2.4, mock-3.12.0, order-1.2.0, reportportal-5.4.0, services-2.2.1, xdist-3.5.0
13:01:33  collected 22 items
...
14:50:44  tests/foreman/api/test_capsulecontent.py ......................          [100%]
14:50:44  ================ 22 passed, 1519 warnings in 6549.87s (1:49:09) ================

[Satellite-QE]

PRT Result

Build Number: 6137
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/api/test_capsulecontent.py::TestCapsuleContentManagement --external-logging
Test Result : ================ 22 passed, 1431 warnings in 5979.77s (1:39:39) ================