Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure FSRS parameter mismatch issue has a cross-compatibility plan #17236

Closed
2 of 4 tasks
mikehardy opened this issue Oct 11, 2024 · 16 comments
Closed
2 of 4 tasks

Ensure FSRS parameter mismatch issue has a cross-compatibility plan #17236

mikehardy opened this issue Oct 11, 2024 · 16 comments

Comments

@mikehardy
Copy link
Member

Checked for duplicates?

  • This issue is not a duplicate

Does it also happen in the desktop version?

  • This bug does not occur in the latest version of Anki Desktop

What are the steps to reproduce this bug?

This is mostly a tracking issue to make sure Anki Desktop has a resolution for cross-compatiblity between old and new anki versions when FSRS parameter count changes with 24.10

Expected behaviour

different versions should be cross-compatible

Debug info

currently they are not

(Optional) Anything else you want to share?

No response

Research

  • I have checked the manual and the FAQ and could not find a solution to my issue
  • (Optional) I have confirmed the issue is not resolved in the latest alpha release (instructions)
@BrayanDSO
Copy link
Member

Pinging @dae

My previous idea was to sync two parameters list in 24.10, one with 17 parameters, compatible with old clients, under the current key, and one with 19 parameters under a new key.

If the user optimizes in a newer version, the last two parameters can be stripped and synced under the old key so things keep working

If the user optimizes in an older version, Anki can detect it and re-optimize if necessary.

@user1823
Copy link
Contributor

user1823 commented Oct 12, 2024

Automatically converting FSRS 4.5 parameters into FSRS 5 parameters is easy and the latest beta of Anki already does that.

However, converting FSRS 5 parameters into FSRS 4.5 parameters is difficult because quantifying the effect of the short-term reviews for making necessary adjustments to the other parameters is difficult.

If the aim is just preventing an error, Anki/AnkiWeb can just strip off the last 2 parameters and make some adjustments to w4 and w5 (inverse operations to those mentioned in ankitects/anki#3463 (comment)). But, I doubt that the converted parameters would perform better than the parameters optimised on an old client.

So, I think that if someone uses both old and new clients, they should optimize the parameters only on the old client until they can update all of their clients to the new backend.

With Brayan's suggestion, I am not sure how Anki will obtain the FSRS 4.5 parameters. If they are NOT obtained by performing some operations on the FSRS 5 parameters, the idea might be worth implementing.

@david-allison
Copy link
Member

david-allison commented Oct 12, 2024

We're going to get review bombed if we don't handle the following cases in a reasonable manner:

  • AnkiDroid > Anki Desktop
    • AnkiDroid auto-updates and the user does not want to update Anki due to addon compat. Anki Desktop is broken
  • AnkiDroid < Anki Desktop
    • A user does not want to/cannot update AnkiDroid (e.g. due to regressions, or their device no longer being supported). AnkiDroid is broken

UX

The current error provides the user neither explanation nor context (what is a parameter), nor action to take to fix the issue

Screenshot_20241012_053756_AnkiDroid.jpg

Even if we have a suboptimal solution, we & Anki Desktop need to provide the user with a reasonable UX to enable them to continue to review

Our Release Schedule

In terms of pending releases:

  • AnkiDroid 2.19: 24.06 - 17 params
  • AnkiDroid 2.20: 24.10 - 19 params

2.20 will be out quickly after 2.19, we know we need it to be stable for when Anki releases, and will accept fewer risky changes now GSoC is over and development has slowed

@dae
Copy link
Contributor

dae commented Oct 12, 2024

Things don't break until the user explicitly reoptimizes after an update, so I think the "user has no idea what happened" cases are going to be somewhat limited in scope. It's also not going to save us from the inevitable questions about heterogeneous FSRS versions giving different scheduling results for the same card. But I seem to be in a minority here, so will be happy to accept a PR that changes things. Similar to Brayan's suggestion, how does the following sound?

  • fsrs5-specific field added to deck config; optimizations stored there
  • fsrs5, when fetching params (during review, and in the deck options screen), falls back via 5Params ?? 4Params ?? defaults
  • fsrs5 would only write to the new field, so older clients can retain their old params

@user1823
Copy link
Contributor

Similar to Brayan's suggestion, how does the following sound?

Sounds GREAT to me.

@mikehardy
Copy link
Member Author

Agreed, that sounds like a solid forwards / backwards-compatible plan. Paves the way as a style for fsrs vN+1 etc etc where there are perhaps truly breaking things

@mikehardy
Copy link
Member Author

For the now even smaller group of future people that will be on heterogenous versions of FSRS and will notice the different scheduling behavior with the same card and will care enough, we can put a FAQ in perhaps that explains what happens if you are on different FSRS versions, and whatever the current client vs FSRS versions are. A trivial / easy thing to do.

Then for AnkIDroid at least we could add the FSRS version to our About page info "anki" line (currently says like anki 24.06.03 / d678e393 could say anki 24.06.03 / fsrs4 / d678e393 - but these are pulled dynamically or at build time so will move along nicely in future without ongoing maintenance) to help that be a self-service answer for people and then be done with it?

@BrayanDSO
Copy link
Member

BrayanDSO commented Oct 12, 2024

so, the cross-compatibility stuff needs to be done at Anki's repo, and the FSRS version notice here?

Although what version name of FSRS should be used? FSRS-X or the version at fsrs-rs? Or both? (I don't get the naming logic here)

Where's @Expertium

@mikehardy
Copy link
Member Author

(note: the version display is a nice-to-have in my opinion, to help any concerned + confused users self-help themselves if they become really preoccupied about scheduling behavior of cards on heterogenous FSRS clients. I personally think it's good because I also think it should be easy to do, but just handling the cross-compat case with non-error degradation vs errors up in anki is the thing I'm really excited about)

@Expertium
Copy link

Then for AnkIDroid at least we could add the FSRS version to our About page info "anki" line (currently says like anki 24.06.03 / d678e393 could say anki 24.06.03 / fsrs4 / d678e393

https://forums.ankiweb.net/t/display-fsrs-version-in-the-debug-info/50236?u=expertium

I have previously proposed that + adding the FSRS version in the debug info, Dae wasn't excited about it.

Similar to Brayan's suggestion, how does the following sound?

  • fsrs5-specific field added to deck config; optimizations stored there
  • fsrs5, when fetching params (during review, and in the deck options screen), falls back via 5Params ?? 4Params ?? defaults
  • fsrs5 would only write to the new field, so older clients can retain their old params

If it doesn't require any extra actions from the user and is - for all intents and purposes - invisible, that would be really good.

@david-allison
Copy link
Member

david-allison commented Oct 13, 2024

I think the "user has no idea what happened" cases are going to be somewhat limited in scope.

We're getting user support queries about this just from the beta:

https://www.reddit.com/r/Anki/comments/1g2tmvk/ankidroid_does_not_work_pc_seems_fine/


I'm happy with the above solution to use two fields for parameters

I'd want to update AnkiDroid to handle the "parameter mismatch" case and provide a "reoptimize" button at the very least

If we can do it efficiently, we would want to detect that a newer/larger set of parameters exists, so we can be both forwards and backwards compatible

@user1823
Copy link
Contributor

If we can do it efficiently, we would want to detect that a newer/larger set of parameters exists, so we can be both forwards and backwards compatible

I think it would be better to show a Re-optimize button whenever the user has invalid parameters. So, we don't need to know what the future/new parameters look like. IMO, this change should be made in Anki Desktop too.

@mikehardy
Copy link
Member Author

Anki-Android-Backend has been published with the FSRS crate version available as FSRS_VERSION in the backend BuildConfig, so anyone that wants to make a PR in this repo to add it to debug info (example) / about screen (example) can give it a go

@user1823
Copy link
Contributor

user1823 commented Oct 22, 2024

FYI: Damien has already made the promised change (separate fields for FSRS 5 parameters) in Anki in ankitects/anki@c45fa51 and an Anki beta has been released with the change.

@david-allison
Copy link
Member

Thanks. Voting to close as resolved.

The following still needs to be integrated, should be a separate issue if we agree to close this

@mikehardy
Copy link
Member Author

Agreed - thanks all, it may be a mountain out of a molehill but I do believe users will appreciate this even if they didn't know it was done, there will be people using incompatible versions for years for whatever reason and they'll be handled gracefully

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants