Type doesn't strictly return messages of a particular type

[KyleMaas]

I just ran this on ssb-browser-demo:

SSB.db.query(SSB.db.operators.type("gathering"), SSB.db.operators.toCallback((err, msgs) => { console.log(msgs) }))

This resulted in 37 results. Out of those, 35 had a type of "gathering". However, it also included these two messages:

%xS73eLtWOfsGd+p2ZnmVm9C6heaoGrXkvE76PA1G1nQ=.sha256 => type: "vote"

%1vMUGhJrwE9HtXzF0rdgSNFn+t2EhoxYHG60bvEXMM0=.sha256 => type: "about"

These were the last two messages in the result list. All of the messages before that were of the proper type.

[arj03]

Really strange. I don't see any errors here. I have 13 gatherings:

SSB.db.query(SSB.db.operators.type("gathering"), SSB.db.operators.toCallback((err, msgs) => { console.log(msgs.map(x => x.value.content.type)) }))

[KyleMaas]

Just tried refreshing to see if it was a transient issue. I still get the same result. What's my next step to help debugging this?

(This is on my @MI0aKQls1HIrZlyl4qSAW4ErLqb4xVUN+diSbG0j7Fc=.ed25519 account, by the way.)

[KyleMaas]

Started a new ssb-browser-demo session in a private tab, imported my key (being careful not to do anything which would result in a write and therefor a fork), and tried it again. Now I just get 25 "gathering" messages. So I'd guess that somehow the index got corrupted.

[KyleMaas]

Left my other (main) tab open, BTW, so if you need me to do anything else to collect debugging information, let me know.

[arj03]

You could of course try and remove the indexes and let them rebuild to see if that fixes the problem. I'm pretty sure that works, but then we loose the ability to debug this further other than see if you can reproduce the steps.

It might make sense to create a small sanity check tool that propels the database for these kind of errors. It could check a lot more cases. I'll try to make that

[arj03]

Smart move 😊 I'll see if I can think of a way to show us what the problem might be with your indexes so if you can please leave them for a couple of days

[KyleMaas]

Yeah, no problem. I would guess that clearing and rebuilding the indices would fix it. But, yeah, I kind of hate to try that, because while I can easily do that, that's a lot to ask of normal users. Better to fix the problem than fix the symptom, and right now we've got broken test data to work with.

[staltz]

Just a data point: in some tests in jitdb, if I had a pausing sink of a push-stream, I sometimes got an incorrect number of results, often more results than I expected.

The only way how you may have a pausing sink is if you activated maxCpu. Have you?

[KyleMaas]

Not explicitly, although I don't know if ssb-browser-core sets that, and I don't know where that setting would be located to query it directly.

[staltz]

Check db2 readme, search for maxCpu

[KyleMaas]

grepping through ssb-browser-core, the only hits I get on "maxCpu" are within ssb-db2, so unless there's a default somewhere that enables that, it doesn't look like it should be set.

[arj03]

It's not enabled in browser-core so unless you did something special that shouldn't be the case. As for the bipf errors as well, I was wondering if you are doing multiple taps or something, because things have been really stable for me. I have never seen those error from bipf.

[KyleMaas]

I do have a second tab open right now, and do that sometimes as I find possible bugs/features I want to investigate later. I hadn't considered that that might be corrupting things, but I guess I could see how that's possible.

Are ssb-db2 writes not thread-safe/atomic, then?

[arj03]

That probably explains it then. The state is not shared between the tabs, so yeah that is a classical multiple writers with no locking and that causes really wierd bugs ;-)

[KyleMaas]

Ah, so writes are not thread-safe/atomic/transactional? I would have expected they would be.

[KyleMaas]

And at this point, is that a safe enough assumption of the cause to justify me clearing this out and rebuilding my profile?

[arj03]

Yes, I would be really interested to hear if you can reproduce any of the bipf or type bugs with just a single tab.

[KyleMaas]

Okay. Usually those errors have only shown up after leaving an ssb-browser-demo tab open for several days.

Nothing we need from this tab before I clear it all away?

[arj03]

We should be good. I havn't seen those kind of bugs and I usually have the browser open for several days as well, at least between laptop sleep mode. This is one of those classical cases where I know there is probably an error so I avoid it and you don't know that so you don't know better and we should support it because it is very natural. Just need the time. We'll get there :-)

[KyleMaas]

Hey, I have no problem being the PEBKAC that causes a README revision. :)

[arj03]

Have you been able to reproduce this after you did the multiple tabs thing in browser?

[KyleMaas]

Sorry for the delayed response - been a little busy. Because of that busyness, I haven't been active enough with ssb-browser-demo to really know if this is fixed yet.

[arj03]

No worries @KyleMaas, was just going through old issues. At least now people will know a bit better what these are about if they stumble upon them :)