v0.7.2 - Reading Lists, CBL Support, OPDS Flattening and More! (#1959) · milindpatel63/Kavita@006e67a

Commit

v0.7.2 - Reading Lists, CBL Support, OPDS Flattening and More! (Karea…

…dita#1959)

* WebP Covers + Series Detail Enhancements (#1652)

* Implemented save covers as webp. Reworked screen to provide more information up front about webp and what browsers can support it.

* cleaned up pages to use compact numbering and made compact numbering expand into one decimal place (20.5K)

* Fixed an issue with adding new device

* If a book has an invalid language set, drop the language altogether rather than reading in a corrupted entry.

* Ensure genres and tags render alphabetically.

Improved support for partial volumes in Comic parser.

* Ensure all people, tags, collections, and genres are in alphabetical order.

* Moved some code to Extensions to clean up code.

* More unit tests

* Cleaned up release year filter css

* Tweaked some code in all series to make bulk deletes cleaner on the UI.

* Trying out want to read and unread count on series detail page

* Added Want to Read button for series page to make it easy to see when something is in want to read list and toggle it.

Added tooltips instead of title to buttons, but they don't style correctly.

Added a continue point under cover image.

* Code smells

* Bump versions by dotnet-bump-version.

* Fixed Series Relations Schema (#1654)

* Bump loader-utils from 2.0.2 to 2.0.3 in /UI/Web

Bumps [loader-utils](https://github.com/webpack/loader-utils) from 2.0.2 to 2.0.3.
- [Release notes](https://github.com/webpack/loader-utils/releases)
- [Changelog](https://github.com/webpack/loader-utils/blob/v2.0.3/CHANGELOG.md)
- [Commits](https://github.com/webpack/loader-utils/compare/v2.0.2...v2.0.3)

---
updated-dependencies:
- dependency-name: loader-utils
dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* Fixed is want to read coming back as a string and not working correctly.

* Changed from to Continue to be more explicit

* Added the first migration which exports data as a csv in temp/. This is the backup in case data is lost in the migration.

* Note for later

* Fixed the migration for the series relation so when deleting any series on any edge of the relationship, the SeriesRelation row deletes.

* Change buttons back to titles on series detail page

* Wrote the code to import relations from the backup.

* Added an additional version check to avoid file io on migration.

* Code cleanup

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump versions by dotnet-bump-version.

* Fresh Nightly Installs Work (#1659)

* Bump loader-utils from 2.0.3 to 2.0.4 in /UI/Web

Bumps [loader-utils](https://github.com/webpack/loader-utils) from 2.0.3 to 2.0.4.
- [Release notes](https://github.com/webpack/loader-utils/releases)
- [Changelog](https://github.com/webpack/loader-utils/blob/v2.0.4/CHANGELOG.md)
- [Commits](https://github.com/webpack/loader-utils/compare/v2.0.3...v2.0.4)

---
updated-dependencies:
- dependency-name: loader-utils
dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* Removed a very old cgecj from Nov 2021 when data/appsettings was moved to config/

* Added some notes about migration

* Removed a file that shouldn't have been there.

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump versions by dotnet-bump-version.

* Library Settings Modal + New Library Settings (#1660)

* Bump loader-utils from 2.0.3 to 2.0.4 in /UI/Web

---
updated-dependencies:
- dependency-name: loader-utils
dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* Fixed want to read button on series detail not performing the correct action

* Started the library settings. Added ability to update a cover image for a library.

Updated backup db to also copy reading list (and now library) cover images.

* Integrated Edit Library into new settings (not tested) and hooked up a wizard-like flow for new library.

* Fixed a missing update event in backend when updating a library.

* Disable Save when form invalid. Do inline validation on Library name when user types to ensure the name is valid.

* Trim library names before you check anything

* General code cleanup

* Implemented advanced settings for library (include in dashboard, search, recommended) and ability to turn off folder watching for individual libraries.

Refactored some code to streamline perf in some flows.

* Removed old components replaced with new modal

* Code smells

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump versions by dotnet-bump-version.

* UX Alignment and bugfixes (#1663)

* Refactored the design of reading list page to follow more in line with list view. Added release date on the reading list items, if it's set in underlying chapter.

Fixed a bug where reordering the list items could sometimes not update correctly with drag and drop.

* Removed a bug marker that I just fixed

* When generating library covers, make them much smaller as they are only ever icons.

* Fixed library settings not showing the correct image.

* Fixed a bug where duplicate collection tags could be created.

Fixed a bug where collection tag normalized title was being set to uppercase.

Redesigned the edit collection tag modal to align with new library settings and provide inline name checks.

* Updated edit reading list modal to align with new library settings modal pattern. Refactored the backend to ensure it flows correctly without allowing duplicate names.

Don't show Continue point on series detail if the whole series is read.

* Added some more unit tests around continue point

* Fixed a bug on series detail when bulk selecting between volume and chapters, the code which determines which chapters are selected didn't take into account mixed layout for Storyline tab.

* Refactored to generate an OpenAPI spec at root of Kavita. This will be loaded by a new API site for easy hosting.

Deprecated EnableSwaggerUi preference as after validation new system works, this will be removed and instances can use our hosting to hit their server (or run a debug build).

* Test GA

* Reverted GA and instead do it in the build step. This will just force developers to commit it in.

* GA please work

* Removed redundant steps from test since build already does it.

* Try another GA

* Moved all test actions into initial build step, which should drastically cut down on time. Only run sonar if the secret is present (so not for forks). Updated build requirements for develop and stable docker pushes.

* Fixed env variable

* Okay not possible to do secrets in if statement

* Fixed the build step to output the openapi.json where it's expected.

* Fixed GA (#1664)

* Bump versions by dotnet-bump-version.

* Applied new _components layout structure to Kavita. All except manga as there is an open PR that drastically changes that module. (#1666)

* [Experimental] Split Renderers - Double & Double (Manga) fixes (#1667)

* Updated swiper and some packages for reported security issues

* Fixed reading lists promotion not working

* Refactor RenameFileForCopy to use iterative recursion, rather than functional.

* Ensured that bookmarks are fetched and ordered by Created date.

* Fixed a bug where bookmarks were coming back in the correct order, but due to filenames, would not sort correctly.

* Default installs to Debug log level given errors users have and Debug not being too noisy

* Added jumpbar to bookmarks page

* Now added jumpbar to bookmarks

* Refactored some code into pipes and added some debug messaging for prefetcher

* Try loading next and prev chapter's first/last page to cache so it renders faster

* Updated GetImage to do a bound check on max page.

Fixed a critical bug in how manga reader updates image elements src to prefetch/load pages. I was not creating a new reference which broke Angular's ability to update DOM on changes.

* Refactored the image setting code to use a single method which tries to use a cached image always.

* Refactored code to use getPage which favors cache and simplifies image creation code

* Started the work to split the canvas renderer into it's own component

* Refactored a lot of common methods into a service for the reader to support the upcoming renderer split

* Moved components to nested folder. Refactored more code to streamline image sending to child renderer.

Added notes across the code to help streamline flow of data and who owns what.

* Swapped out SQLite for Memory, but the one from hangfire. Added DisableConcurrentExecution on ProcessChange to avoid duplication when multiple threads execute at once.

* Basic split right to left is working with canvas renderer

* Left to right and right to left now work

* Fixed a bug where pagesplitoption wasn't being updated when modifying menu

* Canvas rendering still has a bug with switching between right to left -> left to right on the re-render, it will choose a bad state. All else works fine with it.

* Updated canvas renderer to implement the ImageRenderer interface

* Canvas renderer is done

* Setup single renderer. Need to figure out how to share CSS between renderers and also share some global stuff, like image height.

* Refactored code so that image-container is within the renderers themselves. Still broken in scaling, but working towards a solution.

* Added double click to shortcut menu

* Moved image containers within the renderers

* Pushing up for Robbie

* nothing new

* Move common css to a single scss file

* More css consolidation

* Fixed a npe in isWideImage

* Refactored page updates to renderers to include max pages. Rewrote most of renderer into observables.

* Moved bookmark for second page to double renderer

* Started hooking in double renderer renderPage()

* Fixed height scaling, but now canvas renderer is broken again

* Fixed a bug with canvas renderer not moving to next page. Streamlined the code for getting page amounts from the dfferent renderers

* Added double click to bookmark for canvas

* Stashing the code and taking a break

* Nothing much, buffer is still broken

* Got double renderer to render at least one page

* Double renderer now has access to 5 images at any time, so it can make appropriate decisions on when to render double pages.

* Fixed up double rendererer moving backward page calc

* Forward logic seems to be working

* Cleaned up dead code after testing

* Moved a few loggers in folder watching to trace

* Everything seems to work fine, time to do double manga renderer

* Moved some css around and added the reverse double component

* Only execute renderer's pipes when in the correct mode

* Still working on double renderer

* Fixed scaling issues on double

* Updating double logic

- Fixed: Fixed an issue where a second page would render when current page was wide.

* Hooked up double renderer

* Made changes but not sure if im making progress

* double manga fixes

* Claned some of robbies code

* Fixing last page bug

* Library Settings Modal + New Library Settings (#1660)

* Bump loader-utils from 2.0.3 to 2.0.4 in /UI/Web

---
updated-dependencies:
- dependency-name: loader-utils
dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* Fixed want to read button on series detail not performing the correct action

* Started the library settings. Added ability to update a cover image for a library.

Updated backup db to also copy reading list (and now library) cover images.

* Integrated Edit Library into new settings (not tested) and hooked up a wizard-like flow for new library.

* Fixed a missing update event in backend when updating a library.

* Disable Save when form invalid. Do inline validation on Library name when user types to ensure the name is valid.

* Trim library names before you check anything

* General code cleanup

* Implemented advanced settings for library (include in dashboard, search, recommended) and ability to turn off folder watching for individual libraries.

Refactored some code to streamline perf in some flows.

* Removed old components replaced with new modal

* Code smells

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump versions by dotnet-bump-version.

* UX Alignment and bugfixes (#1663)

* Refactored the design of reading list page to follow more in line with list view. Added release date on the reading list items, if it's set in underlying chapter.

Fixed a bug where reordering the list items could sometimes not update correctly with drag and drop.

* Removed a bug marker that I just fixed

* When generating library covers, make them much smaller as they are only ever icons.

* Fixed library settings not showing the correct image.

* Fixed a bug where duplicate collection tags could be created.

Fixed a bug where collection tag normalized title was being set to uppercase.

Redesigned the edit collection tag modal to align with new library settings and provide inline name checks.

* Updated edit reading list modal to align with new library settings modal pattern. Refactored the backend to ensure it flows correctly without allowing duplicate names.

Don't show Continue point on series detail if the whole series is read.

* Added some more unit tests around continue point

* Fixed a bug on series detail when bulk selecting between volume and chapters, the code which determines which chapters are selected didn't take into account mixed layout for Storyline tab.

* Refactored to generate an OpenAPI spec at root of Kavita. This will be loaded by a new API site for easy hosting.

Deprecated EnableSwaggerUi preference as after validation new system works, this will be removed and instances can use our hosting to hit their server (or run a debug build).

* Test GA

* Reverted GA and instead do it in the build step. This will just force developers to commit it in.

* GA please work

* Removed redundant steps from test since build already does it.

* Try another GA

* Fixed env variable

* Okay not possible to do secrets in if statement

* Fixed the build step to output the openapi.json where it's expected.

* Fixed GA (#1664)

* Bump versions by dotnet-bump-version.

* Applied new _components layout structure to Kavita. All except manga as there is an open PR that drastically changes that module. (#1666)

* Fixed typeahead and updated manga reader to new layout structure

* Fixed book reader fonts lookups

* Fixed up some build issues

* Fixed a bad import of css image

* Some cleanup and rewrote how we log out data.

* Renderer can be null on first load when performing some work.

* Library Settings Modal + New Library Settings (#1660)

* Bump loader-utils from 2.0.3 to 2.0.4 in /UI/Web

---
updated-dependencies:
- dependency-name: loader-utils
dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* Fixed want to read button on series detail not performing the correct action

* Started the library settings. Added ability to update a cover image for a library.

Updated backup db to also copy reading list (and now library) cover images.

* Integrated Edit Library into new settings (not tested) and hooked up a wizard-like flow for new library.

* Fixed a missing update event in backend when updating a library.

* Disable Save when form invalid. Do inline validation on Library name when user types to ensure the name is valid.

* Trim library names before you check anything

* General code cleanup

* Implemented advanced settings for library (include in dashboard, search, recommended) and ability to turn off folder watching for individual libraries.

Refactored some code to streamline perf in some flows.

* Removed old components replaced with new modal

* Code smells

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* UX Alignment and bugfixes (#1663)

* Refactored the design of reading list page to follow more in line with list view. Added release date on the reading list items, if it's set in underlying chapter.

Fixed a bug where reordering the list items could sometimes not update correctly with drag and drop.

* Removed a bug marker that I just fixed

* When generating library covers, make them much smaller as they are only ever icons.

* Fixed library settings not showing the correct image.

* Fixed a bug where duplicate collection tags could be created.

Fixed a bug where collection tag normalized title was being set to uppercase.

Redesigned the edit collection tag modal to align with new library settings and provide inline name checks.

* Updated edit reading list modal to align with new library settings modal pattern. Refactored the backend to ensure it flows correctly without allowing duplicate names.

Don't show Continue point on series detail if the whole series is read.

* Added some more unit tests around continue point

* Fixed a bug on series detail when bulk selecting between volume and chapters, the code which determines which chapters are selected didn't take into account mixed layout for Storyline tab.

* Refactored to generate an OpenAPI spec at root of Kavita. This will be loaded by a new API site for easy hosting.

Deprecated EnableSwaggerUi preference as after validation new system works, this will be removed and instances can use our hosting to hit their server (or run a debug build).

* Test GA

* Reverted GA and instead do it in the build step. This will just force developers to commit it in.

* GA please work

* Removed redundant steps from test since build already does it.

* Try another GA

* Fixed env variable

* Okay not possible to do secrets in if statement

* Fixed the build step to output the openapi.json where it's expected.

* Applied new _components layout structure to Kavita. All except manga as there is an open PR that drastically changes that module. (#1666)

* Post merge cleanup

* Again moving the file

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Robbie Davis <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump versions by dotnet-bump-version.

* Basic Stats (#1673)

* Refactored ResponseCache profiles into consts

* Refactored code to use an extension method for getting user library ids.

* Started server statistics, added a charting library, and added a table sort column (not finished)

* Refactored code and have a fully working example of sortable headers. Still doesn't work with default sorting state, will work on that later.

* Implemented file size, but it's too expensive, so commented out.

* Added a migration to provide extension and length/size information in the DB to allow for faster stat apis.

* Added the ability to force a library scan from library settings.

* Refactored some apis to provide more of a file breakdown rather than just file size.

* Working on visualization of file breakdown

* Fixed the file breakdown visual

* Fixed up 2 visualizations

* Added back an api for member names, started work on top reads

* Hooked up the other library types and username/days.

* Preparing to remove top reads and refactor into Top users

* Added LibraryId to AppUserProgress to help with complex lookups.

* Added the new libraryId hook into some stats methods

* Updated api methods to use libraryId for progress

* More places where LibraryId is needed

* Added some high level server stats

* Got a ton done on server stats

* Updated default theme (dark) to be the default root variables. This will allow user themes to override just what they want, rather than maintain their own css variables.

* Implemented a monster query for top users by reading time. It's very slow and can be cleaned up likely.

* Hooked up top reads. Code needs a big refactor. Handing off for Robbie treatment and I'll switch to User stats.

* Implemented last 5 recently read series (broken) and added some basic css

* Fixed recently read query

* Cleanup the css a bit, Robbie we need you

* More css love

* Cleaned up DTOs that aren't needed anymore

* Fixed top readers query

* When calculating top readers, don't include read events where nothing is read (0 pages)

* Hooked up the date into GetTopUsers

* Hooked top readers up with days and refactored and cleaned up componets not used

* Fixed up query

* Started on a day by day breakdown, but going to take a break from stats.

* Added a temp task to run some migration manually for stats to work

* Ensure OPDS-PS uses new libraryId for progress reporting

* Fixed a code smell

* Adding some styling

* adding more styles

* Removed some debug stuff from user stats

* Bump qs from 6.5.2 to 6.5.3 in /UI/Web

Bumps [qs](https://github.com/ljharb/qs) from 6.5.2 to 6.5.3.
- [Release notes](https://github.com/ljharb/qs/releases)
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/qs/compare/v6.5.2...v6.5.3)

---
updated-dependencies:
- dependency-name: qs
dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* Tweaked some code for bad data cases

* Refactored a chapter lookup to remove un-needed Volume join in 5 places across the code.

* API push

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Robbie Davis <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump versions by dotnet-bump-version.

* Hooked up the API layer to be able to extract images from PDF again for Tachiyomi explicitly (#1686)

* Bump versions by dotnet-bump-version.

* OPDS Enhancements (#1687)

* Bump express from 4.17.2 to 4.18.2 in /UI/Web

Bumps [express](https://github.com/expressjs/express) from 4.17.2 to 4.18.2.
- [Release notes](https://github.com/expressjs/express/releases)
- [Changelog](https://github.com/expressjs/express/blob/master/History.md)
- [Commits](https://github.com/expressjs/express/compare/4.17.2...4.18.2)

---
updated-dependencies:
- dependency-name: express
dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump decode-uri-component from 0.2.0 to 0.2.2 in /UI/Web

Bumps [decode-uri-component](https://github.com/SamVerschueren/decode-uri-component) from 0.2.0 to 0.2.2.
- [Release notes](https://github.com/SamVerschueren/decode-uri-component/releases)
- [Commits](https://github.com/SamVerschueren/decode-uri-component/compare/v0.2.0...v0.2.2)

---
updated-dependencies:
- dependency-name: decode-uri-component
dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump qs and express in /UI/Web

Bumps [qs](https://github.com/ljharb/qs) and [express](https://github.com/expressjs/express). These dependencies needed to be updated together.

Updates `qs` from 6.5.3 to 6.11.0
- [Release notes](https://github.com/ljharb/qs/releases)
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/qs/compare/v6.5.3...v6.11.0)

Updates `express` from 4.17.2 to 4.18.2
- [Release notes](https://github.com/expressjs/express/releases)
- [Changelog](https://github.com/expressjs/express/blob/master/History.md)
- [Commits](https://github.com/expressjs/express/compare/4.17.2...4.18.2)

---
updated-dependencies:
- dependency-name: qs
dependency-type: indirect
- dependency-name: express
dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* Added genre and authors to Series level, added summary to volume and chapter level.

Force order on reading list title as Chunky enforces their own sort order and doesn't respect the spec.

* Moved all the reading list formatting logic to the backend. This allows us to re-use the UI logic for OPDS streams.

* Fixed a broken unit test

* Code smells

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump versions by dotnet-bump-version.

* Epub Table of Generation fixes for Sigil (#1689)

* Fixed generating table of contents where key lookup could fail with how Sigil packs the epubs.

* Tweaked Kavita's fallback ToC generation (when one doesn't exist in the epub) to also use CoalesceKey.

* Code smells

* Bump versions by dotnet-bump-version.

* File Dimension API (#1690)

* Implemented an api for getting file dimensions for a given chapter. This is for CDisplayEx integration. This might be usable in Double Renderer.

* Added the cached filename for new API

* Bump versions by dotnet-bump-version.

* Send Non books to your Devices (#1691)

* Only restrict non-epub/pdf for Kindle devices on Send To.

* Removed restriction to email non-epub/pdfs to devices.

* Bump versions by dotnet-bump-version.

* Misc UI Tweaks (#1692)

* Added a timeAgo pipe which shows live updates for a few areas.

* Fixed some wording on stats page. Changed Total People count to just work on distinct names and not count multiple for different roles.

* Tweaked the compact number so it only shows one decimal

* Fixed a bug

* Bump versions by dotnet-bump-version.

* Reader Refactor Part 2 (#1694)