Local history trashed after upgrade

[dpetroff]

I recently upgraded to the June 2023 version from... honestly I'm not sure, I think I was up to date, or one release behind at most.

My local history appears to have been trashed. Every revision from before a certain date for every file I have checked so far is giving me this error message when I try to compare to local history:

I have my local history configured to not be deleted, ever, so as you can see even the histories of files that haven't been changed for months have been trashed.

[mickaelistria]

In Help > About Eclipse IDE > Installation history, you can see info about the installations. Can you please check what was the version of Eclipse Platfom before your upgrade (you'll need to expand a few items to see it)?
What happens if you reopen the same workspace with a previous revision? Is the data accessible again or does it seem lost?

[dpetroff]

Somehow I forgot about that tab. It was an upgrade from Eclipse Platform 4.25.0.v20220824-2203 to Eclipse Platform 4.28.0.v20230525-0719.

I've tried installing the 2022-09 version again but it makes no difference so that data seems lost. It may be worth highlighting that the upgrade happened on 09/06/2023 and the local history breakage seems to be around 29/06/2023.

This might suggest that the upgrade was not the cause, at least not immediately, but I can't think of anything else that might be relevant. Looking at the log in the workspace, there's nothing noteworthy, just the usual SWT/jface/key binding/action handler noise all the way from early June until the point when I started trying to interact with the inaccessible local history yesterday.

I've noticed that I can restore versions of actually deleted files from before 29/06/2023 as far back as they have local history using the "Restore from Local History" option in the Package Explorer just fine, it's just files that were never deleted that show this error message about deletion.

[dpetroff]

This is actually much worse than I thought. My history is being trashed continuously. I went to check something today and all the previously accessible entries from the pictures above have been trashed now too:

From the timestamps, it somewhat looks like it might have reverted to the default 7 day retention period, but this is what my workspace preferences have been all along, so it shouldn't be deleting anything, ever:

[dpetroff]

I have since switched to a fresh workspace and do not have the problem there. Still, it would be good to get to the bottom of this if someone has any idea where to look.

[dpetroff]

It's happening again after I recently upgraded from the March (4.31.0.v20240229-1022) to the June (4.32.0.v20240601-0610) release:

@mickaelistria, is there any other info I can provide to help diagnose the problem?

[mickaelistria]

Ideally, some steps to reproduce from scratch, eg

1. I start from SDK x.y.z
2. I create a file, modify, save
3. I check that file history has my modification
4. I run Help > Install New Software > ... to upgrade to SDK x.y+1.0
5. I check history
   EXPECTED: former modification accessible
   WAS: former modification not accessible

[dpetroff]

If only it was that simple. I really wish I could give you steps to reproduce. I'm literally just daily driving the IDE. More recently I did a

1. fresh install of the March latest release (4.31.0.v20240229-1022). I started with eclipse installer in advanced mode -> Eclipse IDE for Java and DSL Developers, Latest Release (2024-03) -> next, etc., finish
2. Daily drive until next release
3. Help -> Perform Setup Tasks... -> Finish
4. Update finises, restart, version is now June (4.32.0.v20240601-0610), workspace upgrade prompt -> OK
5. I check history
   EXPECTED: former modification accessible
   WAS: former modification not accessible
6. Keep daily driving same workspace
7. History keeps trashing. Today I started eclipse and saw:
   
   Why did it delete everything prior to 15:20 yesterday? No clue. Most I might have done is switch git branches, but certainly not across any commits where the file is deleted.

[dpetroff]

Some other things that shouldn't but could be related: did Replace with -> HEAD Revision, and maybe also from the Git Staging view or the console. But I do this stuff all the time, and it never becomes a problem until the workspace version migration gets triggered.

I should add that I was specifically watching the local history while I was doing some heavy editing yesterday and it was all there at the end of the day. It was only when I started eclipse again today that it decided it can't open anything from before 15:20 yesterday.

[dpetroff]

OK, new detail. After some more careful observation, I have noticed that my history is being killed after exactly 50 versions (not counting the latest one), which is the default limit. My preferences (as exported from eclipse) are:

...
/instance/org.eclipse.core.net/org.eclipse.core.net.hasMigrated=true
/instance/org.eclipse.core.resources/description.applyfilestatepolicy=false
/instance/org.eclipse.core.resources/encoding=UTF-8
/instance/org.eclipse.core.resources/version=1
/instance/org.eclipse.debug.core/org.eclipse.debug.core.PREF_BREAKPOINT_MANAGER_ENABLED_STATE=true
...

What this looks like in the preferences window is

What I expect this to mean is "do not limit my local history ever" but it seems that the disabled "Maximum entries per file" option is nevertheless active:

From this state above I'm able to

1. Check "Limit history size"
2. Set "Maximum entries per file" to 3
3. Uncheck "Limit history size"
4. Apply and close
5. Restart
6. Only 3+1 versions remain
   

I can't reproduce this in a fresh unmigrated workspace.

[dpetroff]

The file type I initially noticed the problem with was an .xtend file, and after checking in with the xtext project, I can also add that it is reproducible with other file types like .txt and .xml but not .java. I've tried editing the file types with the "Generic Text Editor" and that didn't change the fact that .java is not affected and the other file types are, so it doesn't seem to be something the editors are doing but something else that makes .java extension special to local history. It might help to know why is the .java file type special from a local history point of view

I can try to remove any proprietary information from my broken workspace and attach it for debugging but that might take some time. I can reproduce the bug in new files just fine so it would be especially helpful if you can tell me how to make sure that all the proprietary files are wiped from my local history since I can't send the workspace if the local history contains any of the company IP.

[Bananeweizen]

The local history is not meant as a version control replacement, but rather to allow a human to undo errors, at least for a short amount of time after realizing the error. If you want an endless history for comparing revisions, use a version control system (hint: there is a really good git integration in eclipse). If the local history would be left without automated cleaning, the performance of the workspace would degrade a lot, as there might be hundreds of thousands of history files. If you still want to use local history instead, enable the limitation and set all values to their respective maximum (e.g. 10000 revisions).

[dpetroff]

I think the workaround you're suggesting is besides the point. A bug exists, does it not? Why should the issue be closed instead of maybe being picked up by someone more interested in picking it up in the community at some point?

[mickaelistria]

Do you confirm that the bug is that "local history entries are limited despite preference being disabled" ? If so, please reopen and retitle the issue.

[merks]

I had a short look the other day and at a glance the guards seem to be in place. I don’t have time to invest though. It seems like something that someone could investigate and contribute a solution.

https://github.com/eclipse-platform/.github/blob/main/CONTRIBUTING.md#creating-an-eclipse-development-environment

[Bananeweizen]

The bug is not that history is removed with the checkbox unchecked. The bug is rather the checkbox label being wrong. It should just be changed to "Override default cleanup configuration". It works as intended (by always doing cleanup, either with small default values or user provided values). Disabling it completely is not something that should be introduced now, as it will just lead to users complaining about horribly slow workspaces. I've seen installations with 6 digit numbers of files just in the local history folder, and on Windows that makes it notably slower. I therefore override these limitations in all our company Oomph setups to enforce the default values.

[mickaelistria]

The bug is rather the checkbox label being wrong

OK, so it's a UI issue on the preference page (I think the text should just not be greyed out if it always apply, the existence of the checkbox is questionable). Can you please retitle it and reopen?