Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How do we deal with a Unit's unpinned references to components when those components are soft-deleted? #219

Closed
ormsbee opened this issue Aug 30, 2024 · 8 comments
Assignees
Labels
arch Architecture data model Anything relating to the relational models or more abstract "model" concepts around Learning Core.

Comments

@ormsbee
Copy link
Contributor

ormsbee commented Aug 30, 2024

Is it worth adding more metadata to the Published table to cover use cases where something has been soft deleted (e.g. last_version_before_deletion)? This came to mind when I was thinking about what happens if we use the "a null version pointer means take the latest draft or published" convention and then soft-delete+publish a Component that was being referenced. Do we always want to carry over the deletion, or do we sometimes just want to lock to the last valid version?

This information is derivable from PublishLogRecord (find where the entry for this PublishableEntity gets its new_version set to null). But that's slower to access for the "get a big list of components" scenarios.

(This came up tangentially in this comment w.r.t. how Slots in Units might change over time, e.g. someone deletes a Component from a pool of possibilities to be randomly selected from.)

@ormsbee
Copy link
Contributor Author

ormsbee commented Aug 30, 2024

FYI @kdmccormick, @bradenmacdonald

@bradenmacdonald
Copy link
Contributor

This came to mind when I was thinking about what happens if we use the "a null version pointer means take the latest draft or published" convention and then soft-delete+publish a Component that was being referenced. Do we always want to carry over the deletion, or do we sometimes just want to lock to the last valid version?

Can I get some clarity on this?

Lib component Alpha version 1 is published
Lib component Alpha version 2 is published
That Alpha component is used in a course, with a null reference that means "take the latest published version", and version 2 is copied into the course.
Lib component Alpha version 3 is published, and then immediately (or perhaps some time later) is soft-deleted
Now when the course author goes to apply updates from the library, are you saying we should be able to pull in the soft deleted (but last published) version 3 as an update to replace the existing version 2 in the course? If so, I don't think that matters.

If I'm understanding correctly, I don't think we need to work too hard to support this case. I think users would understand if they're now stuck on version 2 even though it wasn't the last published version, because the whole thing has been deleted upstream. And if it were really important to upgrade to version 3, the library author could un-delete it.

@ormsbee
Copy link
Contributor Author

ormsbee commented Aug 30, 2024

@bradenmacdonald: I think I'm going to close this one as unnecessary anyway, but to give more clarity:

This is mostly relevant to relationships between Components and Units within a Library and eventually within a Course, not really across a Library -> Course borrowing in the short/medium term.

The "let the version float" convention we were talking about–where UnitVersions only store the components that are in them and leave the component_versions unpinned–works for controlling version explosion where every tiny change to every Component in a course cascades up with fully realized new versions. The scenarios where new UnitVersions are actually created would be constrained to changes to the Unit itself, and not its referenced Components:

  1. When Unit metadata is updated, e.g. the title of the Unit.
  2. When a Component is added to a Unit.
  3. When the ordering of Components within a Unit changes.
  4. When we decide to lock a Unit's Component to a specific version (not really a current use case, but we might do this for archiving a course later).
  5. When we remove a Component from a Unit.

But in that data model, it's possible that one of those Components that the Unit references becomes soft-deleted. If that happens, we can get into a situation where the Unit has effectively lost that Component (floating to the draft or published version will bring back a null entry).

So I was briefly thinking that we would also store the last published version before it was deleted, and that might be useful in scenarios like keeping a SlotVariant's links working in the "delete a thing from the random pool" use case. But after a bit of reflection, I think it's better if Units (and other container types) listen for when one of their child elements are deleted and force a new UnitVersion to be created when that happens.

@ormsbee
Copy link
Contributor Author

ormsbee commented Aug 30, 2024

Oh wait, but even if the new UnitVersion is right in that case and accounts for the deleted Component, the previous UnitVersions will all have the null references. Unless creating a new UnitVersion somehow freezes the ComponentVersion links of the previous UnitVersions? Or maybe it could store the whole range?

  • component
  • start_component_version
  • end_component_version

So then end_component_version being null does the free-floating thing, but it gets locked to a specific version when a new UnitVersion is created?

I think I need to kick this around more...

@ormsbee ormsbee changed the title Track Last Published Version? How do we deal with a Unit's unpinned references to components when those components are soft-deleted? Aug 30, 2024
@ormsbee
Copy link
Contributor Author

ormsbee commented Aug 31, 2024

Unfortunately, that idea above about locking when a new version is created would complicate publishing and reverts. For example, if a new version of a Unit is created but not published, it would be wrong to lock the end_component_version for the previous, published version. Because that would mean that the old Unit version (that's still live and published) wouldn't get the latest versions of its Components from that point forward. So it fails for the scenario where you edit a Unit and then publish a Component that the Unit uses without publishing the Unit itself.

I'm assuming that Components can live in multiple Units. If Components associated with Units are always copies + references (like they are between libraries and courses), then this becomes simpler. Maybe that's the angle to go with...

@ormsbee
Copy link
Contributor Author

ormsbee commented Sep 1, 2024

Okay, I was sketching this out in a notebook over Korean BBQ this afternoon, and I think it's a lot simpler than I was worried it would be. The simpler representation where we just have the nullable foreign key to the version should still work, and reverts should still be simple, as long as:

  1. We force the creation of a new UnitVersion when a Component's draft is deleted, for any Units that reference that Component; and...
  2. We make sure that any Units that have a reference to the Component are published at the same time as the Component's deletion is published. (Or before. Just can't let the Component publish first.)

I thought (2) was going to be hard, but at the moment the only way to publish a delete is to publish everything (because you can't access a "publish" button on a deleted thing), so the Units would come along for the ride. But if we ever do give a separate "publish the deletes" button, we should make sure the referencing Units get published as well.

(I think this means we should have a "Trash" page for things that are going to be deleted in the next publish–but maybe that can be covered by a generic "preview list of things that will be published if you press this button". It would also be nice to be able to "undo" a deletion without reverting everything to the published state.)

@ormsbee ormsbee self-assigned this Sep 1, 2024
@ormsbee ormsbee added arch Architecture data model Anything relating to the relational models or more abstract "model" concepts around Learning Core. labels Sep 1, 2024
@ormsbee
Copy link
Contributor Author

ormsbee commented Sep 2, 2024

Oh wait, it works if you're only ever looking at the state of things that are published, but it wouldn't necessarily work for a situation where you had something that was linking directly to an older version–say you had Student state that you wanted to fix to a SlotVersionVariant... So I think we do want to do a post-publish freezing of the end_version from null to set it to the last published version before deletion.

That would complicate two things:

  • Reverts (not just discarding the current draft, but actually reverting to a previously published version), because we'd need to encode enough information to make it unpinned again after the revert.
  • Scale if we need to update references that are per-student (say the randomization of problems for a given Student), and there are a hundred thousand of them.

Though there could be workarounds here too. Reverts would need a little extra housekeeping and complexity, but it's doable.

We could bypass issues around scaling if we make the convention that any container type made for a specific user must have all the versions pinned from the start. This is feasible for a few reasons:

  1. We don't worry about version explosion because in the case of something like the randomized choices, we're only maintaining a single one for any given user. We'd just overwrite it.
  2. Having the versions pinned means that we shouldn't have to worry about anything getting deleted.
  3. We can opportunistically check for updates when the user is interacting with it, or during some async process. We'd never need to amend the records as part of the normal publish process itself.

@ormsbee
Copy link
Contributor Author

ormsbee commented Oct 9, 2024

The latest version of this is being captured in #240 and I'm closing this Issue in favor of that one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arch Architecture data model Anything relating to the relational models or more abstract "model" concepts around Learning Core.
Projects
None yet
Development

No branches or pull requests

2 participants