Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Last note per pubkey algo feed #621

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

Last note per pubkey algo feed #621

wants to merge 3 commits into from

Conversation

jb55
Copy link
Contributor

@jb55 jb55 commented Dec 29, 2024

This is damus' first algo feed.

What's left to do

  • Expand MAX_FILTERS in nostrdb from 16 to 1024 (ideally make dynamic)

Optional

  • Switch over to token parser combinator

I noticed I was missing many parts of the codebase related to column serialization. In an effort to have a more direct parsing style, I created a new token parser combinator.

The algo

algos: introduce last_n_per_pubkey_from_tags

This function creates filters for the base our first algo in Damus:

Called "last N note per pubkey". I don't have a better name for it.

This function generates a query in the form:

[
  {"authors": ["author_a"], "limit": 1, "kinds": [1]
, {"authors": ["author_b"], "limit": 1, "kinds": [1]
, {"authors": ["author_c"], "limit": 1, "kinds": [1]
, {"authors": ["author_c"], "limit": 1, "kinds": [1]
  ...
]

Due to an unfortunate restriction currently in nostrdb and strfry, we
can only do about 16 to 20 of these at any given time. I have made
this limit configurable in strfry[1]. I just need to do the same in
nostrdb now.

[1] https://github.com/hoytech/strfry/pull/133

jb55 added 3 commits December 26, 2024 09:38
This function creates filters for the base our first algo in Damus:

Called "last N note per pubkey". I don't have a better name for it.

This function generates a query in the form:

[
  {"authors": ["author_a"], "limit": 1, "kinds": [1]
, {"authors": ["author_b"], "limit": 1, "kinds": [1]
, {"authors": ["author_c"], "limit": 1, "kinds": [1]
, {"authors": ["author_c"], "limit": 1, "kinds": [1]
  ...
]

Due to an unfortunate restriction currently in nostrdb and strfry, we
can only do about 16 to 20 of these at any given time. I have made
this limit configurable in strfry[1]. I just need to do the same in
nostrdb now.

[1] hoytech/strfry#133

Changelog-Added: Add last_n_per_pubkey_from_tags algo function
Signed-off-by: William Casarin <[email protected]>
In an attempt to make our deck serializer more localized,
comprehensible, and less error-prone, we introduce a new parser
combinator based around string tokens.

This replaces the Selection-based intermediary types so that we have a
more direct serialization style.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant