Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

selftests: drop job-api test variant that may exceed sqlite3's own limits #6074

Merged
merged 3 commits into from
Dec 2, 2024

Conversation

clebergnu
Copy link
Contributor

The sqlite3 module (and actually, even the non-python binary with the same name), have "interesting" limitations when it comes to the database path.

When the test that creates the journal file, and checks for it, runs as part of an RPM build, the extra directories (/buildir/...) exceed the path size limit.

While it would be possible to check if the "logdir" path, plus the name of the journal database path, would eventually exceed the sqlite3 limits, this logic would be exclusive for one test variant.

IMO it does not justify adding such logic to keep a test that has considerably low value.

The number based naming of the job-api related suites do not provide
clear information on what it is about.  This uses a more meaningful
name for them.

Signed-off-by: Cleber Rosa <[email protected]>
…mits

The sqlite3 module (and actually, even the non-python binary with the
same name), have "interesting" limitations when it comes to the
database path.

When the test that creates the journal file, and checks for it, runs
as part of an RPM build, the extra directories (/buildir/...) exceed
the path size limit.

While it would be possible to check if the "logdir" path, plus the
name of the journal database path, would eventually exceed the sqlite3
limits, this logic would be exclusive for one test variant.

IMO it does not justify adding such logic to keep a test that has
considerably low value.

Signed-off-by: Cleber Rosa <[email protected]>
@clebergnu clebergnu added this to the 109 - Codename TBD milestone Nov 27, 2024
@clebergnu clebergnu self-assigned this Nov 27, 2024
Copy link

codecov bot commented Nov 27, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 54.32%. Comparing base (f452aa5) to head (b80f298).
Report is 9 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6074      +/-   ##
==========================================
- Coverage   54.47%   54.32%   -0.16%     
==========================================
  Files         202      202              
  Lines       21892    21892              
==========================================
- Hits        11925    11892      -33     
- Misses       9967    10000      +33     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@richtja
Copy link
Contributor

richtja commented Nov 27, 2024

Hi @clebergnu, can you please add a reference to the failure? I wasn't able to find any CI failure which would be caused by sqlite3 limitations. Thank you.

@clebergnu
Copy link
Contributor Author

/packit copr-build

@clebergnu
Copy link
Contributor Author

This is one example of the limitation causing a failure during an RPM build:

[stdlog] 2024-11-27 20:07:04,543 avocado.core.status.repo repo             L0065 DEBUG| Task "1-examples/tests/true" started message: "{'status': 'started', 'time': 45051.478570847, 'output_dir': '/builddir/avocado/job-results/job-2024-11-27T20.03-b45e70e/test-results/job-api-5-10-_builddir_build_BUILD_python-avocado-108.0-build_avocado-4e86b09ab23531f034c837829b5ac683bfd97213_selftests_check.py_JobAPIFeaturesTest.test_check_file_exists_run.journal.enabled-True/tmp_dir1s_4y4jb/1-_builddir_build_BUILD_python-avocado-108.0-build_avocado-4e86b09ab23531f034c837829b5ac683bfd97213_selftests_check.py_JobAPIFeaturesTest.test_check_file_exists_run.journal.enabled-True/job-2024-11-27T20.07-7f64ff1/test-results/1-examples_tests_true', 'id': '1-examples/tests/true'}"
[stdlog] 2024-11-27 20:07:04,551 avocado.app human            L0089 DEBUG|  (1/1) examples/tests/true: STARTED
[stdlog] 2024-11-27 20:07:04,552 avocado.app extension_manage L0223 ERROR| Error running method "start_test" of plugin "journal": unable to open database file
[stdlog] 2024-11-27 20:07:04,552 avocado.app stacktrace       L0041 ERROR| 
[stdlog] 2024-11-27 20:07:04,552 avocado.app stacktrace       L0043 ERROR| Reproduced traceback from: /builddir/build/BUILD/python-avocado-108.0-build/BUILDROOT/usr/lib/python3.13/site-packages/avocado/core/extension_manager.py:229
[stdlog] 2024-11-27 20:07:04,554 avocado.app stacktrace       L0050 ERROR| Traceback (most recent call last):
[stdlog] 2024-11-27 20:07:04,554 avocado.app stacktrace       L0050 ERROR|   File "/builddir/build/BUILD/python-avocado-108.0-build/BUILDROOT/usr/lib/python3.13/site-packages/avocado/plugins/journal.py", line 118, in start_test
[stdlog] 2024-11-27 20:07:04,554 avocado.app stacktrace       L0050 ERROR|     self.lazy_init_journal(state)
[stdlog] 2024-11-27 20:07:04,554 avocado.app stacktrace       L0050 ERROR|     ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
[stdlog] 2024-11-27 20:07:04,554 avocado.app stacktrace       L0050 ERROR|   File "/builddir/build/BUILD/python-avocado-108.0-build/BUILDROOT/usr/lib/python3.13/site-packages/avocado/plugins/journal.py", line 77, in lazy_init_journal
[stdlog] 2024-11-27 20:07:04,554 avocado.app stacktrace       L0050 ERROR|     self._init_journal(state["job_logdir"])
[stdlog] 2024-11-27 20:07:04,554 avocado.app stacktrace       L0050 ERROR|     ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
[stdlog] 2024-11-27 20:07:04,555 avocado.app stacktrace       L0050 ERROR|   File "/builddir/build/BUILD/python-avocado-108.0-build/BUILDROOT/usr/lib/python3.13/site-packages/avocado/plugins/journal.py", line 66, in _init_journal
[stdlog] 2024-11-27 20:07:04,555 avocado.app stacktrace       L0050 ERROR|     self.journal = sqlite3.connect(self.journal_path)
[stdlog] 2024-11-27 20:07:04,555 avocado.app stacktrace       L0050 ERROR|                    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
[stdlog] 2024-11-27 20:07:04,555 avocado.app stacktrace       L0050 ERROR| sqlite3.OperationalError: unable to open database file
[stdlog] 2024-11-27 20:07:04,555 avocado.app stacktrace       L0051 ERROR| 

During RPM builds on Fedora 42 (really rawhide at this time), the RPM
package dependency system provides MarkupSafe > 3.0.0 (currently
3.0.2).  So, we remove the pinning, or else, the build would fail due
to a version requirement conflict.

Signed-off-by: Cleber Rosa <[email protected]>
Copy link
Contributor

@richtja richtja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @clebergnu, thank you for provided example, the previous changes LGTM. But I think that the last commit b80f298 is not related to this PR, am I right?

@clebergnu
Copy link
Contributor Author

Hi @clebergnu, thank you for provided example, the previous changes LGTM. But I think that the last commit b80f298 is not related to this PR, am I right?

It's not... but it was necessary to include it to prove that the CI can be fixed by this (inclusive). I can send them in separate PRs, but, I would advise against it because we'd still have CI failures on both cases.

Copy link
Contributor

@richtja richtja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No problem, thanks for the explanation.

@richtja richtja merged commit 91752d4 into avocado-framework:master Dec 2, 2024
59 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done 109
Development

Successfully merging this pull request may close these issues.

2 participants