Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WJ-1283] Change time serialization format #2128

Merged
merged 13 commits into from
Oct 8, 2024
Merged

Conversation

emmiegit
Copy link
Member

@emmiegit emmiegit commented Oct 7, 2024

By emitting RFC-3339 format datetimes, we can more easily interpret the information in JS with only Date.parse() instead of needing custom date processing functions. This requires a #[serde(with)] annotation on OffsetDateTime fields.

Example of output from page_get:

{
    "page_id": 16,
    "page_created_at": "2024-10-07T15:26:56.938072Z",
    "page_updated_at": null,
    "page_deleted_at": null,
    "page_revision_count": 1,
    "site_id": 1,
    "page_category_id": 7,
    "page_category_slug": "_default",
    "discussion_thread_id": null,
    "revision_id": 16,
    "revision_type": "create",
    "revision_created_at": "2024-10-07T15:26:56.938072Z",
    "revision_number": 0,
    "revision_user_id": 2,
    "wikitext": null,
    "compiled_html": null,
    "compiled_at": "2024-10-07T15:27:02.3674Z",
    "compiled_generator": "ftml v1.27.0",
    "revision_comments": "",
    "hidden_fields": [],
    "title": "Welcome to your new Wikijump instance!",
    "alt_title": null,
    "slug": "start",
    "tags": [],
    "rating": -77,
    "layout": "wikijump"
}

In order to help test this, I added a new info API method which consolidates some of the various information methods we had before. This produces a JSON object with the fields instead of returning it as unstructured text.

Example of info response:

{
    "package": {
        "name": "deepwell",
        "description": "DEEPWELL - Wikijump API provider and database manager",
        "license": "AGPL-3.0-or-later",
        "repository": "https://github.com/scpwiki/wikijump/tree/develop/deepwell",
        "version": "v2024.10.6"
    },
    "compile_info": {
        "built_at": "2024-10-07T15:25:14Z",
        "rustc_version": "rustc 1.81.0 (eeb90cda1 2024-09-04)",
        "endian": "little",
        "target": "x86_64-unknown-linux-gnu",
        "threads": 4,
        "git_commit": null
    },
    "current_time": "2024-10-07T19:39:54.103967726Z",
    "hostname": "0bf53259f0d2",
    "config_path": "/etc/deepwell.toml"
}

@emmiegit emmiegit requested a review from Zokhoi October 7, 2024 19:58
@emmiegit emmiegit self-assigned this Oct 7, 2024
@emmiegit emmiegit force-pushed the WJ-1283-serde-time branch from cb9d1d2 to 75fe3fc Compare October 7, 2024 20:03
Copy link

codecov bot commented Oct 7, 2024

Codecov Report

Attention: Patch coverage is 0% with 19 lines in your changes missing coverage. Please review.

Project coverage is 1.92%. Comparing base (588d308) to head (75fe3fc).
Report is 127 commits behind head on develop.

Files with missing lines Patch % Lines
deepwell/src/endpoints/info.rs 0.00% 11 Missing ⚠️
deepwell/src/info.rs 0.00% 7 Missing ⚠️
deepwell/src/api.rs 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           develop   #2128      +/-   ##
==========================================
+ Coverage     1.89%   1.92%   +0.03%     
==========================================
  Files          137     140       +3     
  Lines         5600    5766     +166     
==========================================
+ Hits           106     111       +5     
- Misses        5494    5655     +161     
Flag Coverage Δ
deepwell 1.92% <0.00%> (+0.03%) ⬆️
Files with missing lines Coverage Δ
deepwell/src/endpoints/misc.rs 0.00% <ø> (ø)
deepwell/src/services/page/structs.rs 0.00% <ø> (ø)
deepwell/src/services/page_query/structs.rs 0.00% <ø> (ø)
deepwell/src/api.rs 0.00% <0.00%> (ø)
deepwell/src/info.rs 40.00% <0.00%> (+2.50%) ⬆️
deepwell/src/endpoints/info.rs 0.00% <0.00%> (ø)

... and 3 files with indirect coverage changes

@emmiegit emmiegit marked this pull request as ready for review October 7, 2024 21:32
@emmiegit emmiegit force-pushed the WJ-1283-serde-time branch from 841ab5c to cc0a6ae Compare October 7, 2024 23:20
@Zokhoi
Copy link
Contributor

Zokhoi commented Oct 8, 2024

The view structs using the Models did not have the #[serde(with)], so they still have the old format:

{
  ...
  "data": {
    "site": {
      "site_id": 1,
      "created_at": "2024-10-08 03:49:45.082887 +00:00:00",
      "updated_at": null,
      "deleted_at": null,
      ...
    },
    ...
    "page": {
      "page_id": 16,
      "created_at": "2024-10-08 03:49:45.082887 +00:00:00",
      "updated_at": "2024-10-08 03:59:08.397184 +00:00:00",
      "deleted_at": null,
      ...

and causes
image

The Option<OffsetDateTime>s are also in the old format:

pub struct GetDeletedPageOutput {
    pub page_id: i64,

    #[serde(with = "time::serde::rfc3339")]
    pub page_created_at: OffsetDateTime,
    pub page_updated_at: Option<OffsetDateTime>,

    #[serde(with = "time::serde::rfc3339")]
    pub page_deleted_at: OffsetDateTime,
[{
  "page_id": 16,
  "page_created_at": "2024-10-08T03:49:45.082887Z",
  "page_updated_at": "2024-10-08 03:59:08.397184 +00:00:00",
  "page_deleted_at": "2024-10-08T04:03:46.039123Z",
  "page_revision_count": 5,
  "site_id": 1,
  "discussion_thread_id": null,
  "hidden_fields": [],
  "title": "Welcome to your new Wikijump instance!",
  "alt_title": "",
  "slug": "start",
  "tags": [],
  "rating": 49
}]

@emmiegit
Copy link
Member Author

emmiegit commented Oct 8, 2024

The view structs using the Models did not have the #[serde(with)], so they still have the old format:

{
  ...
  "data": {
    "site": {
      "site_id": 1,
      "created_at": "2024-10-08 03:49:45.082887 +00:00:00",
      "updated_at": null,
      "deleted_at": null,
      ...
    },
    ...
    "page": {
      "page_id": 16,
      "created_at": "2024-10-08 03:49:45.082887 +00:00:00",
      "updated_at": "2024-10-08 03:59:08.397184 +00:00:00",
      "deleted_at": null,
      ...

and causes image

The Option<OffsetDateTime>s are also in the old format:

pub struct GetDeletedPageOutput {
    pub page_id: i64,

    #[serde(with = "time::serde::rfc3339")]
    pub page_created_at: OffsetDateTime,
    pub page_updated_at: Option<OffsetDateTime>,

    #[serde(with = "time::serde::rfc3339")]
    pub page_deleted_at: OffsetDateTime,
[{
  "page_id": 16,
  "page_created_at": "2024-10-08T03:49:45.082887Z",
  "page_updated_at": "2024-10-08 03:59:08.397184 +00:00:00",
  "page_deleted_at": "2024-10-08T04:03:46.039123Z",
  "page_revision_count": 5,
  "site_id": 1,
  "discussion_thread_id": null,
  "hidden_fields": [],
  "title": "Welcome to your new Wikijump instance!",
  "alt_title": "",
  "slug": "start",
  "tags": [],
  "rating": 49
}]

I added the serialization flags for nullable fields, but I see that the issue with the view service (and why the changes weren't added) is because the timestamps are in the SeaORM entity files. I'll see if I can alter the struct generation for those.

@emmiegit
Copy link
Member Author

emmiegit commented Oct 8, 2024

I've added the attribute manually to all the fields, since it doesn't seem we can automatically add the attribute to time fields using sea-orm-cli generate entity.

@emmiegit emmiegit merged commit bc612e4 into develop Oct 8, 2024
12 checks passed
@emmiegit emmiegit deleted the WJ-1283-serde-time branch October 8, 2024 04:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants