Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace REST with JSON-RPC #1644

Merged
merged 189 commits into from
Oct 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
189 commits
Select commit Hold shift + click to select a range
1624363
Revert "Replace reqwest with ureq."
emmiegit Oct 7, 2023
8923d65
Add tarpc and tokio to crates.
emmiegit Oct 7, 2023
1a6d1d7
Use #[tokio::main] as executor.
emmiegit Oct 7, 2023
5b40147
Remove crossbeam, use tokio queues.
emmiegit Oct 8, 2023
a2d6bed
Replace tarpc with jsonrpsee [sic].
emmiegit Oct 9, 2023
692f394
Start jsonrpsee server setup.
emmiegit Oct 9, 2023
d6c1610
Remove ApiServerState, add ServerState / ServerStateInner.
emmiegit Oct 9, 2023
b8c540e
Add TODO for removing site applications on ban.
emmiegit Oct 9, 2023
dbea954
Use snake_case, not camelCase.
emmiegit Oct 9, 2023
faca07c
Rename context -> state.
emmiegit Oct 9, 2023
1fc2f70
Change all methods to async methods.
emmiegit Oct 9, 2023
8cf83bd
Use register! macro to limit boilerplate.
emmiegit Oct 9, 2023
6e34710
Unwrap double-Arc, add explanatory comment.
emmiegit Oct 9, 2023
aba7982
Implement ping, first real method.
emmiegit Oct 9, 2023
dd1025c
Add version methods to RPC.
emmiegit Oct 9, 2023
0ffa279
Migrate simple misc methods.
emmiegit Oct 11, 2023
2ed2bd2
Add Error::Raw() for jsonrpsee errors.
emmiegit Oct 11, 2023
c25825d
Remove unused error_response().
emmiegit Oct 11, 2023
0fc3177
Migrate locale method.
emmiegit Oct 11, 2023
925fd6b
Migrate translate_strings() method.
emmiegit Oct 11, 2023
e1f9687
Migrate page_view method.
emmiegit Oct 11, 2023
a81c02f
Migrate email validation method.
emmiegit Oct 11, 2023
005b74a
Migrate auth_login.
emmiegit Oct 11, 2023
1853bf4
Migrate auth_logout.
emmiegit Oct 11, 2023
5cbc2d5
Migrate MFA verify.
emmiegit Oct 11, 2023
34b836c
Migrate auth_mfa_setup.
emmiegit Oct 11, 2023
b5430de
Migrate mfa_disable.
emmiegit Oct 11, 2023
07e52f8
Migrate auth_mfa_reset_recovery.
emmiegit Oct 11, 2023
3625791
Migrate auth_session_get.
emmiegit Oct 11, 2023
a1249e9
Migrate auth_session_renew and site_create.
emmiegit Oct 11, 2023
7e2e9e9
Migrate auth_session_get_others.
emmiegit Oct 11, 2023
fba4af5
Temporarily cut down on warnings.
emmiegit Oct 11, 2023
305da87
Migrate auth_session_invalidate_others.
emmiegit Oct 11, 2023
7776b36
Finish migrating site_create.
emmiegit Oct 11, 2023
381b0dc
Migrate site_get.
emmiegit Oct 11, 2023
433d3fc
Migrate site_update.
emmiegit Oct 11, 2023
6459b13
Split out into endpoints/site_domain.rs
emmiegit Oct 11, 2023
573e3d9
Migrate get_site_from_domain().
emmiegit Oct 11, 2023
7c09db7
Migrate custom site domain methods.
emmiegit Oct 11, 2023
5da5bc1
Add TODO for renaming "create".
emmiegit Oct 12, 2023
e8d43e0
Migrate site_member endpoints.
emmiegit Oct 12, 2023
0b771a5
Migrate category_get and category_get_all.
emmiegit Oct 13, 2023
af79482
Migrate text methods.
emmiegit Oct 13, 2023
112d5e7
Migrate page_create, page_get, page_get_direct.
emmiegit Oct 13, 2023
2247ca4
Address some warnings.
emmiegit Oct 13, 2023
32cb7e8
Migrate edit_page.
emmiegit Oct 13, 2023
f178064
Add forgotten transaction commits.
emmiegit Oct 13, 2023
5808a1e
Rename ServiceContext::new() -> ::from_req().
emmiegit Oct 13, 2023
8807d47
Rename ServiceContext::from_raw() -> ::new().
emmiegit Oct 13, 2023
9e56fe9
Trial of transaction() closure.
emmiegit Oct 13, 2023
ac126ab
Add replacement register! macro, with transaction + ctx.
emmiegit Oct 13, 2023
d3f15cf
Migrate not_implemented().
emmiegit Oct 13, 2023
08333fa
Add note comment for transaction lifecycle.
emmiegit Oct 13, 2023
f7e6870
Migrate text_get.
emmiegit Oct 13, 2023
bfae0e2
Migrate misc methods to register2!
emmiegit Oct 13, 2023
05c2318
Migrate locale methods to register2!
emmiegit Oct 13, 2023
f4594b0
Migrate page_view to register2!
emmiegit Oct 13, 2023
ac6a846
Condense last line of text_get().
emmiegit Oct 13, 2023
4d335a9
Migrate auth methods.
emmiegit Oct 14, 2023
669c7f4
Migrate validate_email().
emmiegit Oct 14, 2023
fce5693
Migrate page methods.
emmiegit Oct 14, 2023
ec6628c
Migrate site and domain methods.
emmiegit Oct 14, 2023
f839582
Migrate site_member methods.
emmiegit Oct 14, 2023
79a7f85
Migrate page category methods.
emmiegit Oct 14, 2023
d390ae4
Remove unused legacy register! macro.
emmiegit Oct 14, 2023
5a6fe68
Rename register2! -> register! macro.
emmiegit Oct 14, 2023
6d0f1dc
Make ServiceContext cloneable.
emmiegit Oct 14, 2023
6cb2e9e
Use &ServiceContext across everything.
emmiegit Oct 14, 2023
c0cff7e
Migrate page_delete.
emmiegit Oct 14, 2023
e9b2a43
Migrate page_move.
emmiegit Oct 14, 2023
bca6a53
Migrate page_rerender.
emmiegit Oct 14, 2023
fd36e5e
Migrate page_restore.
emmiegit Oct 14, 2023
7ffae9f
Create details vs non-details structs.
emmiegit Oct 14, 2023
1d974ae
Mgirate parent_relationships_get.
emmiegit Oct 15, 2023
fc4503b
Migrate parent_get.
emmiegit Oct 15, 2023
b8594a9
Add get_optional_XXX() method in interaction macro generation.
emmiegit Oct 15, 2023
328c3b0
Move site_from_domain method.
emmiegit Oct 15, 2023
0a1890f
Use optional, return null if missing.
emmiegit Oct 15, 2023
bd77913
Rename endpoints/site_domain.rs -> domain.rs
emmiegit Oct 15, 2023
e2968e0
Migrate parent methods.
emmiegit Oct 15, 2023
761e149
Rename "delete" methods to "remove".
emmiegit Oct 15, 2023
e491bdd
Migrate vote methods.
emmiegit Oct 15, 2023
4e73fb2
Migrate message methods.
emmiegit Oct 15, 2023
20acba9
Migrate bot_user methods.
emmiegit Oct 15, 2023
533d098
Migrate page_revision methods.
emmiegit Oct 15, 2023
3f7d354
Add interaction migration comment.
emmiegit Oct 15, 2023
a6acd53
Migrate page link methods.
emmiegit Oct 15, 2023
1efc19e
Migrate stubbed file methods.
emmiegit Oct 15, 2023
67f8ec2
Migrate file methods.
emmiegit Oct 15, 2023
e2ca00c
Move attributes into CreateFile.
emmiegit Oct 15, 2023
f4c3584
Create Bytes struct for serializing to hexadecimal.
emmiegit Oct 15, 2023
c6af178
Add from impls for hash arrays.
emmiegit Oct 15, 2023
54ee228
Return error on invalid hash length.
emmiegit Oct 15, 2023
4862fe2
Remove unused helper, use Bytes for hex.
emmiegit Oct 15, 2023
760f8f0
Add missing Bytes methods.
emmiegit Oct 15, 2023
6c19869
Rename create_file -> upload_file.
emmiegit Oct 15, 2023
b7b07f2
Implement file_upload method.
emmiegit Oct 15, 2023
ada9eda
Implement file_edit method.
emmiegit Oct 15, 2023
51886ab
Implement file_delete method.
emmiegit Oct 16, 2023
b0a8356
Implement file_restore method.
emmiegit Oct 16, 2023
aa557d7
Remove unused warning suppression.
emmiegit Oct 16, 2023
ec4f25a
Update RPC wrapping comments.
emmiegit Oct 16, 2023
d0af099
Implement file_move.
emmiegit Oct 16, 2023
e22bad3
Implement file_hard_delete method.
emmiegit Oct 16, 2023
04a5588
Add special handling for the empty blob.
emmiegit Oct 16, 2023
b657df5
Migrate file_revision_count.
emmiegit Oct 16, 2023
402cbc0
Migrate file_revision_range.
emmiegit Oct 16, 2023
5df7cf8
Migrate file_revision_get.
emmiegit Oct 16, 2023
c4c008a
Migrate file_revisoin_edit.
emmiegit Oct 16, 2023
642cd44
Migrate user_get.
emmiegit Oct 16, 2023
fab9b5f
Migrate user_create method.
emmiegit Oct 16, 2023
61bb6ba
Migrate user_import method.
emmiegit Oct 16, 2023
5ff3ae4
Migrate user_edit method.
emmiegit Oct 16, 2023
3667a9c
Fix user_edit method for avatar updating.
emmiegit Oct 16, 2023
2213038
Migrate user_delete method.
emmiegit Oct 16, 2023
f51029e
Migrate user_add_name_change method.
emmiegit Oct 16, 2023
da9777e
All tide REST endpoints migrated to RPC!
emmiegit Oct 16, 2023
12790cc
Delete now-empty tide web server.
emmiegit Oct 16, 2023
5216589
Migrate page_rollback method.
emmiegit Oct 16, 2023
f9eeedb
Delete tide types.
emmiegit Oct 16, 2023
6b71b0b
Remove unused imports.
emmiegit Oct 16, 2023
4726a00
Add missing endpoints.
emmiegit Oct 16, 2023
20725e8
Remove not_implemented temporary endpoint definition.
emmiegit Oct 16, 2023
3f772d2
Suppress unused method warning in FileRevision.
emmiegit Oct 16, 2023
557db1e
Resolve clippy lints with extra borrow.
emmiegit Oct 16, 2023
95d2d21
Delete tide error handling.
emmiegit Oct 16, 2023
ad0f4fb
Fix comment.
emmiegit Oct 16, 2023
d4ccf80
Remove extra newline.
emmiegit Oct 16, 2023
02546f2
Comment on argon2 error conversion.
emmiegit Oct 16, 2023
d15d8cf
Merge locale sub-error into main error type.
emmiegit Oct 16, 2023
80a507a
Fix auth error filtration.
emmiegit Oct 16, 2023
156096e
Add Error::EmailVerification error case.
emmiegit Oct 16, 2023
a339349
Add Error::FilterRegexInvalid.
emmiegit Oct 16, 2023
fb91854
Add error code distribution.
emmiegit Oct 16, 2023
fa77710
Replace RemoteOperationFailed with S3-specific sub-errors.
emmiegit Oct 16, 2023
31676ab
Split Error::Conflict into sub-errors.
emmiegit Oct 17, 2023
d5cfff1
Delete Error::Exists.
emmiegit Oct 17, 2023
2e511b5
Change find_or_error to macro to allow specialized error values.
emmiegit Oct 17, 2023
1882d4c
Update comments.
emmiegit Oct 17, 2023
53ed97c
Split most of Error::BadRequest into subtypes.
emmiegit Oct 17, 2023
4cd294f
Add Error code docs.
emmiegit Oct 17, 2023
c334490
Use Debug impl for data.
emmiegit Oct 17, 2023
c6f8caa
Fix duplicate error case.
emmiegit Oct 17, 2023
b1a7e6d
Remove duplicate method registration.
emmiegit Oct 17, 2023
711f0ff
Address clippy lints.
emmiegit Oct 17, 2023
6bf2018
Add site_id to file queries.
emmiegit Oct 17, 2023
e58a199
Use struct for file revision.
emmiegit Oct 17, 2023
c24f3c1
Add site_id in file insertion.
emmiegit Oct 17, 2023
adf2529
Update curl invocation for ping.
emmiegit Oct 17, 2023
2d5a094
Remove unused tide-related imports.
emmiegit Oct 17, 2023
3103e82
Fix docs for main.rs
emmiegit Oct 17, 2023
07b5f0e
Delete unused HttpUnwrap trait.
emmiegit Oct 17, 2023
c7360a8
Remove indirect use of tide for dependency of log / femme.
emmiegit Oct 17, 2023
b4fe5e7
Remove tide dependency.
emmiegit Oct 17, 2023
f452b42
Update rust dependencies generally.
emmiegit Oct 17, 2023
090a37e
Bump deepwell version to 2023.10.16
emmiegit Oct 17, 2023
0dd6db4
Add json-rpc-client to framerail dependencies provisionally.
emmiegit Oct 17, 2023
fe42d39
Update deepwell index to ping new JSON-RPC server.
emmiegit Oct 17, 2023
d469968
Add keepalive support to deepwell client.
emmiegit Oct 17, 2023
1544338
Add types, add better deepwell infra.
emmiegit Oct 17, 2023
7b60978
Ran prettier.
emmiegit Oct 17, 2023
3b39ea1
Implement basic data method for better error structured data.
emmiegit Oct 17, 2023
175d99e
Fix svelte import.
emmiegit Oct 17, 2023
7b9851a
Add node-gyp dependencies.
emmiegit Oct 19, 2023
1084c4c
Increase node version to 20.
emmiegit Oct 19, 2023
f8b5f1b
Return raw error objects in conversion.
emmiegit Oct 19, 2023
b2cb61f
Add yield_error endpoint.
emmiegit Oct 19, 2023
6f63ecb
Delete json-rpc-client package.
emmiegit Oct 19, 2023
b05e221
Move runtime dependencies out of dev.
emmiegit Oct 19, 2023
d73bab8
Install json-rpc-2.0, remove node-gyp dependencies.
emmiegit Oct 19, 2023
683bb09
Set up JSONRPCClient.
emmiegit Oct 19, 2023
e692345
Change page view enum variants as snake_case.
emmiegit Oct 19, 2023
e212fff
Fix data subfields to snake_case.
emmiegit Oct 19, 2023
743a957
Run prettier.
emmiegit Oct 19, 2023
5e0805c
Don't serialize hash values.
emmiegit Oct 19, 2023
9f1ea9e
Add port check for database health-check.sh
emmiegit Oct 19, 2023
fb2b286
Add client timeout.
emmiegit Oct 20, 2023
c2e88f8
Don't check HTTP status.
emmiegit Oct 20, 2023
c7b6b57
Move output struct order.
emmiegit Oct 20, 2023
80f75f8
Fix return type.
emmiegit Oct 20, 2023
018d2af
Remove deepwell client timeout.
emmiegit Oct 20, 2023
3a08740
Fix types for pageView.
emmiegit Oct 20, 2023
81af5e4
Move deepwell methods.
emmiegit Oct 20, 2023
a1af950
Add server explanation comment.
emmiegit Oct 20, 2023
6e11b28
Fix syntax error.
emmiegit Oct 20, 2023
c1f76bf
Run prettier.
emmiegit Oct 20, 2023
2fe3b0a
Suppress unused method warnings.
emmiegit Oct 20, 2023
16b1958
Fix tests.
emmiegit Oct 20, 2023
3f34e9c
Bump deepwell version to 2023.10.19
emmiegit Oct 20, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,085 changes: 640 additions & 445 deletions deepwell/Cargo.lock

Large diffs are not rendered by default.

13 changes: 8 additions & 5 deletions deepwell/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ keywords = ["wikijump", "api", "backend", "wiki"]
categories = ["asynchronous", "database", "web-programming::http-server"]
exclude = [".gitignore", ".editorconfig"]

version = "2023.10.6"
version = "2023.10.19"
authors = ["Emmie Maeda <[email protected]>"]
edition = "2021" # this is *not* the same as the current year

Expand All @@ -20,28 +20,32 @@ anyhow = "1"
argon2 = "0.5"
arraystring = "0.3"
async-std = { version = "1", features = ["attributes"] }
async-trait = "0.1"
cfg-if = "1"
clap = "4"
color-backtrace = "0.6"
crossfire = "1.0"
cuid2 = "0.1"
data-encoding = "2"
dotenvy = "0.15"
either = "1"
femme = "2"
filemagic = "0.12"
fluent = "0.16"
ftml = { version = "1.22", features = ["mathml"] }
futures = { version = "0.3", features = ["async-await"], default-features = false }
hex = "0.4"
hex = { version = "0.4", features = ["serde"] }
hostname = "0.3"
intl-memoizer = "0.5"
jsonrpsee = { version = "0.20", features = ["macros", "server"] }
log = "0.4"
notify = { version = "6", optional = true }
once_cell = "1"
otp = { git = "https://github.com/TimDumol/rust-otp" }
paste = "1"
rand = "0.8"
ref-map = "0.1"
regex = "1"
reqwest = { version = "0.11", features = ["json", "rustls-tls"], default-features = false }
rust-s3 = { version = "0.32", features = ["with-async-std"], default-features = false }
sea-orm = { version = "0.12", features = ["sqlx-postgres", "runtime-async-std-rustls", "postgres-array", "macros", "with-json", "with-time"], default-features = false }
sea-query = "0.30"
Expand All @@ -54,12 +58,11 @@ strum = "0.25"
strum_macros = "0.25"
subtle = "2.4"
thiserror = "1"
tide = "0.16"
time = { version = "0.3", features = ["parsing", "serde", "serde-human-readable"], default-features = false }
tiny-keccak = { version = "2", features = ["k12"] }
toml = { version = "0.8", features = ["parse"] }
tokio = { version = "1", features = ["full"] }
typenum = "1"
ureq = { version = "2.8.0", features = ["charset", "json", "tls"] }
unic-langid = "0.9"
unicase = "2"
wikidot-normalize = "0.12"
Expand Down
2 changes: 2 additions & 0 deletions deepwell/migrations/20220906103252_deepwell.sql
Original file line number Diff line number Diff line change
Expand Up @@ -431,6 +431,7 @@ CREATE TABLE file (
from_wikidot BOOLEAN NOT NULL DEFAULT false,
name TEXT NOT NULL,
page_id BIGINT NOT NULL REFERENCES page(page_id),
site_id BIGINT NOT NULL REFERENCES site(site_id),

UNIQUE (page_id, name, deleted_at)
);
Expand All @@ -442,6 +443,7 @@ CREATE TABLE file_revision (
revision_number INTEGER NOT NULL,
file_id BIGINT NOT NULL REFERENCES file(file_id),
page_id BIGINT NOT NULL REFERENCES page(page_id),
site_id BIGINT NOT NULL REFERENCES site(site_id),
user_id BIGINT NOT NULL REFERENCES "user"(user_id),
name TEXT NOT NULL,
s3_hash BYTEA NOT NULL,
Expand Down
1 change: 0 additions & 1 deletion deepwell/scripts/generate-models.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,5 @@ sea-orm-cli generate entity \
--date-time-crate time \
--with-copy-enums \
--with-serde both \
--model-extra-attributes 'serde(rename_all = "camelCase")' \
--database-url postgres://wikijump:wikijump@localhost/wikijump \
--output-dir src/models
Loading
Loading