Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update nostrdb, switch to local relay model #2121

Draft
wants to merge 146 commits into
base: master
Choose a base branch
from
Draft

Update nostrdb, switch to local relay model #2121

wants to merge 146 commits into from

Conversation

jb55
Copy link
Collaborator

@jb55 jb55 commented Apr 4, 2024

This branch updates nostrdb and switches to a local relay model

Things done and left to do:

  • Update nostrdb
  • Switch to nostrdb for note block rendering
  • Switch to local relay for querying and subscriptions

@jb55 jb55 linked an issue Apr 4, 2024 that may be closed by this pull request
@jb55 jb55 force-pushed the nostrdb-update branch from a3d9cb3 to 3a6dcdc Compare April 4, 2024 22:34
@jb55
Copy link
Collaborator Author

jb55 commented Apr 5, 2024

This broke a bunch of tests which I'm fixing now

@jb55
Copy link
Collaborator Author

jb55 commented Apr 5, 2024

One thing I did here is add nostrdb's patch history to our local copy of nostrdb. This ensure we don't lose bisectability when updating nostrdb. I've also removed a lot of overlap between damus' C code and nostrdb, so most of the C code should be in nostrdb now.

@jb55 jb55 linked an issue Apr 8, 2024 that may be closed by this pull request
@jb55 jb55 force-pushed the nostrdb-update branch 3 times, most recently from 10bc4f4 to c476272 Compare April 25, 2024 22:50
This was linked to issues May 6, 2024
jb55 and others added 18 commits August 28, 2024 14:56
When trying to build from rust

Signed-off-by: William Casarin <[email protected]>
Closes: damus-io/nostrdb#21
Signed-off-by: William Casarin <[email protected]>
Right now it's accidently hardcoded.

Fixes: 8376e5bca05c ("add "import -"")
Signed-off-by: William Casarin <[email protected]>
everything will be in here soon
rust doesn't like packed structures, so hide this from bindgen

This also buttons up the API so less things are exposed which is good.

Signed-off-by: William Casarin <[email protected]>
This is the start of our rust library for nostrdb. Implement idiomatic
interfaces for Ndb and NdbConfig.

Changelog-Added: Add initial rust library
Signed-off-by: William Casarin <[email protected]>
otherwise build fails

Signed-off-by: William Casarin <[email protected]>
since it was overwritten when we synced with damus

Signed-off-by: William Casarin <[email protected]>
we will be applying a patch here as well
A lot of this was pulled from core-lightning. Not sure what is actually
needed or not.

Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
since I keep overwriting it by accident

Signed-off-by: William Casarin <[email protected]>
rustyrussell and others added 10 commits August 28, 2024 15:37
Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Changelog-Added: Add method for parsing filter json
Signed-off-by: William Casarin <[email protected]>
I wondered by `make check` was giving strange errors, until I realized it wasn't fully rebuilding.

Also, remove leftover CCAN files I missed previously.

Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Copy the latest, which has parsing fixes.  We make a new explicit
"bolt11_decode_minimal" which doesn't check sigs, rather than neutering
the bolt11_decode logic.

As a bonus, this now correctly parses "LIGHTNING:BECH32..." format
(upper case, with prefix).

Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: William Casarin <[email protected]>
If we make unknown_field simply discard, we can remove decoders and
have them discard those fields.

Now we can cut down struct bolt11 to only the fields needed by
invoice.c, and also speed up parsing a little.

Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: William Casarin <[email protected]>
otherwise rust gets bitchy at as

Signed-off-by: William Casarin <[email protected]>
Sure, this format would be nice, but it's not what the code does.

Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: William Casarin <[email protected]>
This will find strings which match the beginning of other strings,
which seems wrong.

Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: William Casarin <[email protected]>
we are crossing cursors

Signed-off-by: William Casarin <[email protected]>
rustyrussell and others added 13 commits August 29, 2024 12:49
This lets them be updated/bugfixed together.  I just copied them for now,
didn't change anything else.

Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: William Casarin <[email protected]>
It isn't actually in the CCAN module (though it probably should be!).
So it breaks when we update.

Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: William Casarin <[email protected]>
This is the version of CCAN which CLN was using at the time these
were taken.  Unfortunately lots of whitespace has been changed,
but AFAICT no source changes.

Here's the command I ran (with ../ccan checked out to 1ae4c432):

```
make update-ccan CCAN_NEW="alignof array_size build_assert check_type container_of cppmagic likely list mem short_types str structeq take tal tal/str typesafe_cb utf8 endian crypto/sha256"
```

Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Only change for us: CCAN_TAL_NEVER_RETURN_NULL can be defined if
we don't override tal error handling.

Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: William Casarin <[email protected]>
some binding dir stoppers, and configurator
doesn't need to create a copy of the id

Signed-off-by: William Casarin <[email protected]>
This is still kind of broken until queries are switched over to nostrdb.
Will do this next

Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
6 participants