Code

RequestManager for partial replication#5

Code

Merged

staltz merged 94 commits intomainssbc/ssb-replication-scheduler:mainfrom partiallyssbc/ssb-replication-scheduler:partiallyCopy head branch name to clipboardFeb 18, 2022

+2,095-144

Commits

Commits on Oct 8, 2021

• work in progress request manager

  staltz

  committed
  f54698d

• remove self feed ID logic from req-manager

  staltz

  committed
  8ddd5e2

• fix typo in import

  staltz

  committed
  cf5b148

• add configuration rules

  staltz

  committed
  f45bac7

• fix small mistake

  staltz

  committed
  10fe0e4

• renamed "partially" to "partialReplication"

  staltz

  committed
  c0be934

• move DEFAULT_OPTS to index.js

  staltz

  committed
  42d54cc

• update README

  staltz

  committed
  4d36c28

• protect RequestManager from adding duplicates

  staltz

  committed
  03782bc

• update some FIXME comments

  staltz

  committed
  02b632c

• first draft of MetafeedFinder

  staltz

  committed
  15845c1

• add debouncing flush to RequestManager

  staltz

  committed
  bd9fa09

• fix tests and add debouncePeriod opts

  staltz

  committed
  b2a18e5

• update dependencies

  staltz

  committed
  f3ffd1e

• update ssb-subset-ql

  staltz

  committed
  266ce93

• improve RequestManager with partialReplication traversal

  staltz

  committed
  32f688e

• add one code comment

  staltz

  committed
  073d43a

• update deps

  staltz

  committed
  7732163

• fix typos in metafeed-finder

  staltz

  committed
  7c4ef2d

• validate metafeed/announce in metafeed-finder

  staltz

  committed
  e4737f0

• update dev deps

  staltz

  committed
  5f25cf0

• update some code comments

  staltz

  committed
  f5ff2ee

• improve RequestManager's filtering of public msgs

  staltz

  committed
  8d34e51

• refactor MetafeedFinder: pluckFromAnnounceMsg can be trusted

  staltz

  committed
  1f445ca

• for now, self feed ID always fully replicates itself

  staltz

  committed
  30dcb0a

• add more debug() to MetafeedFinder

  staltz

  committed
  89a83c2

• fix usage of getSubset rpc

  staltz

  committed
  41ff783

• add a FIXME comment

  staltz

  committed
  d734877

• fix mistakes in RequestManager

  staltz

  committed
  59aca75

• MetafeedFinder uses metafeeds.find in case of self ID

  staltz

  committed
  c7ec4c6

• tweak a debug in MetafeedFinder

  staltz

  committed
  f571fd9

• remove one debug in MetafeedFinder

  staltz

  committed
  5219fff

• remove some obsolete FIXME comments

  staltz

  committed
  6a926ca

• integration test for partial replication

  staltz

  committed
  047f97f

• remove all FIXME from code

  staltz

  committed
  54da68e

• introduce different templates for each hops level

  staltz

  committed
  382b727

• register EBT format in RequestManager, if needed

  staltz

  committed
  1a8b49d

• do only one database pull-drain in RequestManager

  staltz

  committed
  2987ed0

• protect RequestManager against quick tombstoning

  staltz

  committed
  13b024f

• remove console.logs from tests

  staltz

  committed
  c590b11

• remove leftovers

  staltz

  committed
  191953a

• test and implement reconfigure()

  staltz

  committed
  8354bd8

• refactor some names in RequestManager

  staltz

  committed
  4f9e91f

• refactor RequestManager internals a bit

  staltz

  committed
  77a2818

• avoid unnecessary database scans in RequestManager

  staltz

  committed
  05ace94

• fix README

  staltz

  committed
  1328eb4

• improve integration/block3 test

  staltz

  committed
  488e526

• improve coverage script

  staltz

  committed
  6df9f5a

• add one test utility

  staltz

  committed
  3a48d2f

• RequestManager can handle blocks and unfollows

  staltz

  committed
  5ec114e

• remove a hack from tests now that ssb-ebt was fixed

  staltz

  committed
  2c818cc

• rename ssb-meta-feeds-rpc to ssb-subset-rpc

  staltz

  committed
  1e3fbdc

• refactor the test a bit

  staltz

  committed
  e986c85

• rename the test file

  staltz

  committed
  0ae3fe9

• update ssb-meta-feeds to 0.23.0

  staltz

  committed
  59ab6e3

• add some code comments

  staltz

  committed
  cabf98f

• speed up test/intergration/partial

  staltz

  committed
  47d8be9

• reconfigure() supports opts.debouncePeriod

  staltz

  committed
  e528b5b

• use graphStream only for computing third-party blocks

  staltz

  committed
  e9f1c9f

• hack block3 test for now

  staltz

  committed
  04661a9

• remove some logging

  staltz

  committed
  c27f016

• improve test/integration/partial.js

  staltz

  committed
  0fe5964

• improve test/integration/block3.js

  staltz

  committed
  1ea63f0

• switch from full to partial if found metafeed/announce

  staltz

  committed
  2f6173b

• add debug in req-manager.js

  staltz

  committed
  f550090

• fix flushing boolean in req-manager.js

  staltz

  committed
  9ec50d4

• refactor internally MetafeedFinder if conditions

  staltz

  committed
  9ae9085

• rename some variables in RequestManager

  staltz

  committed
  ab9da47

• fix treatment of live MetafeedFinder stream

  staltz

  committed
  0286fd3

• add start() API and autostart config

  staltz

  committed
  f41464d

• update ssb-meta-feeds to 0.24.0

  staltz

  committed
  c9b4517

• protect MetafeedFinder forEachNeighborPeer against self ID

  staltz

  committed
  8c69236

• refactor RequestManager's _handleBranch()

  staltz

  committed
  1d283be

• tiny renaming in RequestManager _handleBranch

  staltz

  committed
  fbc7205

• detect tombstones and stop replicating those subfeeds

  staltz

  committed
  5be8091

• rename some variables in RequestManager

  staltz

  committed
  3e10ce0

• tweak RequestManager _handleBranch logic

  staltz

  committed
  3342f19

• change how "partial replication enabled" is checked

  staltz

  committed
  48e9df5

• fix RequestManager _findTemplateForHops() corner case logic

  staltz

  committed
  2ee7063

• fix tests for tombstoning

  staltz

  committed
  4a82252

• add a small timer.unref()

  staltz

  committed
  175b70f

• improve pull.drain in MetafeedFinder _flush

  staltz

  committed
  2338e68

• cosmetic refactor of MetafeedFinder period arg

  staltz

  committed
  efd7f43

• MetafeedFinder calls getSubset on newly connected peers

  staltz

  committed
  f547a55

• make this REUSE compliant

  staltz

  committed
  afa3fad

Commits on Oct 12, 2021

• use ssb-ebt 8.0.0

  staltz

  committed
  b3c153a

Commits on Oct 17, 2021

• MetafeedFinder with period zero behaves synchronously

  staltz

  committed
  26f48fb

• MetafeedFinder doesnt flush if there's nothing to flush

  staltz

  committed
  7d296c2

Commits on Feb 17, 2022

• small refactors

  staltz

  committed
  acb09f5

Commits on Feb 18, 2022

• batch metafeed-finder RPC calls

  staltz

  committed
  f238c48

• test: intermediary friend does not forward to blocked

  staltz

  committed
  4d4908b

• remove wrong code comment

  staltz

  committed
  5338f44

• bump several dependencies

  staltz

  committed
  04c35aa

• tweak tests to show "catch all" replication rule

  staltz

  committed
  0a451af