Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[24.0] Fix History Dataset Association creation so that hid is always set #18036

Merged

Conversation

mvdbeek
Copy link
Member

@mvdbeek mvdbeek commented Apr 21, 2024

We already had test coverage for this, but we were not running the strict checks on new items. b1333e0 should be the missing puzzle piece to prevent this entirely in testing, however we might discover more bugs in the process.

xref #18014:

so I'm surprised we're not hitting

raise Exception(f"HistoryDatasetAssociation {self} without hid, this is not valid")

Significantly extends test coverage by also checking new items for strict rules, and also fixes serialization of Dataset Collection Elements for collection elements that aren't populated yet.

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

mvdbeek added 2 commits April 21, 2024 21:10
I think that's the missing piece to find all the instances where we
might still be committing HDAs without HIDs, and I think we might have
a similar bug for DatasetCollectionElements.
Otherwise we may end up with a history item that either temporarily or
permanently lacks a hid.

This could currently be encountered via implicit conversion that set
`visible=False` in the when calling `datatype.convert_dataset`.
@mvdbeek mvdbeek added kind/bug area/database Galaxy's database or data access layer labels Apr 21, 2024
@mvdbeek mvdbeek force-pushed the implicit_conversion_no_hid_fix branch from 5d6e057 to badbd3a Compare April 22, 2024 09:52
@mvdbeek mvdbeek changed the title [24.0] Fix missing HID for some implicit conversion routes [24.0] Fix History Dataset Association creation so that hid is always set Apr 22, 2024
@mvdbeek mvdbeek marked this pull request as ready for review April 22, 2024 09:55
@github-actions github-actions bot added this to the 24.1 milestone Apr 22, 2024
@mvdbeek mvdbeek requested a review from a team April 22, 2024 10:39
@mvdbeek mvdbeek force-pushed the implicit_conversion_no_hid_fix branch from 3196da4 to cedc441 Compare April 22, 2024 14:23
log.debug("Using strict flush checks")

def before_flush(session, flush_context, instances):
for obj in session.new:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't that what versioned_objects_strict() does? If so, why not do something like this:

for obj in versioned_objects_strict(session.new):
    pass

Copy link
Member Author

@mvdbeek mvdbeek Apr 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It creates a version, which doesn't make sense for new objects.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yes, I could do that, though it's not related to versioning objects, which is what threw my off in your comment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And then, if I'm not mistaken, you wouldn't need this if/then that checks os.environ and contains 2 similar function definitions, and just use the existing logic, I think? i.e., setting versioned_objects = versioned_objects_strict would still work?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think one is better than the other with the exception this one is slightly better since we don't run the os.environ check in the module space ?

@mvdbeek mvdbeek force-pushed the implicit_conversion_no_hid_fix branch from cedc441 to 121f27e Compare April 22, 2024 15:20
@jdavcs jdavcs merged commit b97825b into galaxyproject:release_24.0 Apr 23, 2024
52 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/database Galaxy's database or data access layer kind/bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants