Update the FlightLog (WIP)

[JonBooth78]

User facing changes:

• Consolidate the three tabs of the flightlog into a single page with all entries on it.
• Add an options bar to the right hand side to toggle on and off individual types of log entry from the displayed list so you can still see what you’d like
• When pruning the flightlog for space reasons, never remove any elements that have a custom annotation/note added by the user
• Split system flight log entries into a separate arrival and destination entry if there are other entries that come chronologically between them
• Add the ability to order the flightlog oldest to newest as well as the default newest to oldest
• Remove the spurious log on game start of the player docking at their starting station
• Add the option showing the current player stats to the flightlog at the top
• Add the ability to export the currently displayed flightlog to a file, located in the userdata folder based on the player’s name, either as plain text or HTML

Potentially controversial changes I’m aware of:

• Exposed lua io library to our scripts. However, I have removed the functionality to open processes and only allow files to be opened and manipulated within our game userdata folder and nowhere else on the host system to keep our sandbox safe. If this isn’t acceptable, fair enough, I’ll have to build a slightly different custom function binding to allow opening those files.
• The way the RTTI works for the various flightlog types; I think this can be neater.
• This version of the flightlog will end up with very slightly larger savefiles.
• The way I’ve scaled the two main columns (the toolbar and the flightlog)
• This is very much showing my working in the order of commits; a rebase and tidy up of commits may make sense as some things were a bit transient (an extra tab with the export options for example)

Things I need to complete

• Localization

Things I’ve tested:

• Loading existing games
• Saving games and reloading them with the new flightlog format
• Starting a new game
• Jumping between systems
• Landing at spaceports
• Custom flightlog entries and adding custom annotations to other entries

Things I need to test:

• Playing some more
• ???

Things I plan to add later:

• Buttons to add the location of flightlog entries to the current system navigation route
• Automatic flight log entries when a ship is killed by the player (and a filter a filter for)
• Automatic flight log entries when the player changes their ship (and a filter for these)
• Automatic flight log entries when the player changes the ship equipment (and filter for these)
• Automatic flight log entry when a screenshot is taken and some way to view it from the log (as long as it’s still on disc)

[impaktor]

Very cool! Combining all three tabs was my original intent, so nice to see this in place now.

If I understand correctly, if html isn't ticked, it will export to txt? Ideally use some ImGui element that shows both options and have them be mutually exclusive.

This version of the flightlog will end up with very slightly larger savefiles.

Without breaking backwards compatibility?

I was going to complain about the "Save" text on the button, but note we're already sinners regarding this in the option screen.
(I wonder if "export" is better (or "save to"), since technically the flight log is saved when you save the game).

If you want to play around more: I also think the save/export functionality is slightly "immersion breaking", so I wonder if it could be less "front and center", like a button in the corner, opens a new window with export options.

Not sure I understood the pruning part. Is there a function / method for pruning? When is this done?

[JonBooth78]

One overnight thought; I should do the pruning on system entries to consolidate them together if a filtered list is requested, although that leads to some questions about where to store any user annotations that are then edited/added.

Pruning of the flightlog is done when a certain maximum number of entries are reached (was 1000 each for system entries and starport entries). Then when you add a new one, essentially the oldest one is deleted. With this change, once the total number of 3000 is reached the oldest is removed. I need to test that code though, adding it to my TODO list.

I think I used the "Save" text simply because it was already localized :-)

[JonBooth78]

Re backward compatibility; old save files will load. Old versions of the game will not load new savefiles made with this version. AFAIK we have no way of storing upgrade/downgrade scripts such that we can load them into an older version of the game to load a newer savefile.

I'm sure we can move that save button somewhere else :-)

[impaktor]

I'm sure we can move that save button somewhere else :-)

I'll defer UI decision to @nozmajner (TL;DR: Basically: should the export flight log to txt/html say "save" / "export" / icon?), and does the button break immersion.

For localization: note, you need only update the en.json file, transifex does the rest.

[sturnclaw]

Update the fightlog (WIP)

First rule of fightlog is we don't talk about fightlog.

The way the RTTI works for the various flightlog types; I think this can be neater.

Provisionally not a problem, though I may have suggestions once I fully review the code.

This version of the flightlog will end up with very slightly larger savefiles.

Also not a problem, savefiles are compressed. Just avoid writing large quantities of fully-translated strings when you could write the i18n key instead and you should be fine. As long as saves stay < 500kb there shouldn't be any issues.

The way I’ve scaled the two main columns (the toolbar and the flightlog)

I feel like this could be accomplished with a popover menu of some sort, but I don't think that's currently exposed to Lua at the moment. I can look into adding a binding for that given I implemented it during my Editor work.

This is very much showing my working in the order of commits; a rebase and tidy up of commits may make sense as some things were a bit transient (an extra tab with the export options for example)

No worries, we expect a rebase before final merge to clean up the history anyways.

Re backward compatibility; old save files will load. Old versions of the game will not load new savefiles made with this version.

This is expected behavior. We do not provide forwards compatibility; it is expected that savefiles created with a newer version of Pioneer are not loadable in an older version.

---

On to the meat of the PR - I quite like what I'm seeing here! The layout has a minor case of hyperverticality, in that there's a huge amount of horizontal space left unused. It's also a little confusing to read, as your brain wants to read arrival -> departure but they're in inverse order unless you're reading the log bottom-to-top, which isn't clearly indicated.

I'm not sure it's actually useful to show the player's current status as a persistent log entry, given that the information is always visible on the Personal Info screen anyways. You could move that info (along with the player's avatar) into the right hand column, but I'm not sure that really makes sense unless we consolidated Personal Info and Flight Log together (and provided a mechanism to tweak the player's face somewhere in the UI.

I'm very happy with the automatic consolidation of arrival/departure entries in the log. If you have time and are willing, I'd suggest you might consider adding in a new log entry type that tracks the player's purchases while at a station - I've had a partially-baked branch to do that (flight-log-improvement which includes some other prototype stuff that's maybe not useful) but never was able to fully realize the design with the time I had available.

I had originally conceived the design in the screenshot based on a spreadsheet Nozmajner was using to track game progress and profit; while it's non-trivial to fully track cargo items etc. to calculate profit, the basic design I think is solid and provides additional utility for the player. The layout of the table might be worth using to visualize all log entries in the flight log to look at the total lifetime profit of the player, for example.

Regarding the technical implementation, the prototype-quality API I had involved passing around raw strings for the description of Flight Log entries; it might be better to provide an interface to register new flight log "row types" if you decide to adopt that code. That could then decouple the responsibility for formatting from the code that pushes entry data into the log.

[JonBooth78]

It's also a little confusing to read, as your brain wants to read arrival -> departure but they're in inverse order unless you're reading the log bottom-to-top, which isn't clearly indicated.

I swapped these when the log is displayed 'latest first', as I thought it's cleaner to show the departure first then, as it comes after the arrival, this way all the dates in the log are in the 'correct' chronological order. However, when it's displayed 'oldest first' they swap back. I'm in no way attached to this though :-)

The personal info is really only useful when exporting, as I think then it is handy to see. So maybe I'll just always add it to an export and never display it. Would allow me to clean up the code too, so I think that's not too bad an idea.

The commerce section is cool. I'll take a look at that :-) Maybe then I'll want a CSV export option!!!

Yeah, the GUI is pretty sparse. I expect as a few more options for filtering get added that right hand bar will do more but having it pop-up/pop over would be good. Right now, unless the player adds long annotations or custom text there isn't much there.

However, I'm now thinking that a table layout might be good too, with columns for the various elements. A lot of the types of log entry have a bunch of commonality and factoring that into the design might also help with code design too.

Lots to think about. Thanks all.

[sturnclaw]

Some nitpicks while you're thinking about the direction to take the PR:

• The FlightLog file is getting long enough to begin splitting into multiple files. It most likely makes sense with the amount of changes (in terms of preservation of git history) to move it to its own module folder (e.g. modules/FlightLog/FlightLog.lua) and split the FlightLogEntry file out from it.
• Similarly, the export serializers for the flight log should not be implemented in a pigui file; that begins to stray into major separation-of-concerns / spaghetti-code territory. It's also better for flight log export to be a method in the Flight Log module proper so that code outside of the UI can access it.
• We prefer not to use the Lua goto functionality, even though it's been used in a few places in the code. If you're able to make existing code cleaner and easier to follow by removing gotos "while you're there", I'd appreciate it. 😄

[JonBooth78]

Nitpicks are good, it is how I'll learn!

Just been thinking about this and I am leaning into the concept that a flat flightlog is the wrong one.

Instead, the flightlog is a hierarchical, ordered, collection of events.

System events contain starport events, which can contain trading events etc.

Can then be displayed as a flat log if required but also helps with some of the conceptual problems, gluing together events etc...

I'll think about that some more...

[bszlrd]

Regarding immersion, I think an Export button wouldn't be out of place. Maybe "Export Ship Log".
@Web-eWorks refers to this sheet/database of mine:

I could imagine an UI similar to this I think. Relatively thin rows, which are expandable to show the details. I might have a bit of time this evening to play/brainstorm with some mockups, if interested.

Hmm, the ability to export to CSV might be interesting too. Don't know how feasible it would be though

[impaktor]

I'm not sure it's actually useful to show the player's current status as a persistent log entry, given that the information is always visible on the Personal Info screen anyways

I had the same thought.

Other than that, I have no more thoughts, than be ware of feature creep. Might be good to merge this asap, and then work on a v2 later (e.g. for commodity history).

[JonBooth78]

Thanks all.

I've got a bit more tidy up to go, and I'm not sure I've got all the foundations laid quite right before merging. There is a little structural change I'd like to make that could change the save file format.
I'd hate to then have to keep around some code to load a very transient version forever.

I'll try and get it into shape soon.

[sturnclaw]

@JonBooth78 for your worries about savefiles, the upcoming February release is going to be a "savebump" release, which means we will intentionally be breaking backwards compatibility as part of the release. Don't feel the need to worry about maintaining compatibility with flightlog data at this point, especially with a series of PRs in-flight. We make no guarantees that the master branch is save compatible during a development cycle, so if you have to change the data model between now and a month from now there's no problem with that.

I do appreciate the intention of maintaining savefile compatibility, just want to notify you that it's about to be moot anyways when #5622 is merged.

[Gliese852]

I think this problem, as well as the one that is repaired by the previous line in a similar style, should not be repaired in this way. We just need to not generate a docking event when the game starts.

[sturnclaw]

If you're able to find a way to cleanly plumb that through the SpaceStation API, I'd love to merge something to that regard. This is definitely a horrible hack, as you can see from the code fragment 😄

[JonBooth78]

Yeah, I totally agree. I didn't want to get into too many tendrils of changes here but did want to fix the immediate issue with the flightlog.

Perhaps I should create a bug and then can resolve this separately and remove the hacks?

[Gliese852]

I think yes, this problem should be solved separately. I would like to note that in #5619 the "state machine" that controls docking changes quite a lot, so I think it’s worth postponing the solution to this problem until the fate of the specified PR is decided.

[sturnclaw]

I will get to reviewing that PR sometime hopefully today.

[sturnclaw]

Regarding the new FileSystem.lua - I'd recommend implementing the whole thing in C++ instead of having it partially in Lua. In general, I like how the system works, though I'd prefer the root filesystem to be specified via a user:// or data:// protocol-portion of the path rather than passing it as a separate parameter (this can be easily handled via starts_with_ci("user://", path) in C++).

I'd be happy to put together an implementation of the API I'm envisioning if you're interested; I think this discussion is getting a bit beyond the initial purpose of the PR and the scope of "good first contribution".

[JonBooth78]

All good. I actually started with an implementation like that, but then saw we had that type convention for the file/directory enumeration so thought it would be good to follow.

For consistency, should that also change to use the same uri schema, then we can drop the constant altogether.

I'm happy to keep going, but will spin this out into a separate pr

I'm trying to balance a few things here...

1. A desire to the use as much as Lua io as possible as then we don't have to maintain it, it's familiar to anyone new coming who knows Lua and (for people like me) it's well documented for those not familiar with Lua.
2. A desire not to modify the Lua third party, which means I have that ugly hook to change parameters and then call the existing open function
3. A smaller desire to keep that ugly hook just in the sandbox 😁

Anyway, unless you have a burning desire and get to it first, I'll pull this out of here into a stand alone or and get onto it soon enough.

[JonBooth78]

OK for anyone watching, I just force pushed this same tree rebased on the successful merge of the Lua IO changes, which oddly hasn't shown up yet in github, and will start tidying up from here (before taking it further).

[JonBooth78]

This has now been replaced by #5666, as I made this PR no longer accept more changes from my branch. Sorry.