test(gossipsub): Topic Membership Tests

[shashankshampi]

Test Plan: https://www.notion.so/Gossipsub-651e02d4d7894bb2ac1e4edb55f3192d

Topic Membership Tests:

Test added: (TC6.6: Verify the handling of subscription changes when the underlying pubsub framework sends SUBSCRIBE and UNSUBSCRIBE events.)

1. Simulate the SUBSCRIBE event and check proper handling in the mesh and gossipsub structures
2. This test will simulate an UNSUBSCRIBE event and check if the topic is removed from the relevant data structures but remains in gossipsub
3. This test ensures that multiple topics can be subscribed to and unsubscribed from, with proper initialization of the topic structures.
4. This test ensures that the number of subscriptions does not exceed the limit set in the GossipSub parameters

Part 2:

TC6.1: Verify that peers can correctly join a topic using JOIN(topic).
TC6.2: Ensure that peers can correctly leave a topic using LEAVE(topic).
TC6.5: Multiple peers join and leave topic simultaneously

[diegomrsantos]

how about testgossiptopicmembership.nim?

[diegomrsantos]

We follow conventional commits for the PR title. In this case, it will start with test(gossipsub):

[shashankshampi]

how about testgossiptopicmembership.nim?

I am adding other tests in the same file related to Topic Membership Tests for Join topic, leave topic, and mesh updated on peer subscribing and leaving topic.

Earlier suggestion looks fine but else will modify as per your suggestion

[shashankshampi]

We follow conventional commits for the PR title. In this case, it will start with test(gossipsub):

Thanks. Will follow same in upcoming PRs

[diegomrsantos]

how about testgossiptopicmembership.nim?

I am adding other tests in the same file related to Topic Membership Tests for Join topic, leave topic, and mesh updated on peer subscribing and leaving topic.

Earlier suggestion looks fine but else will modify as per your suggestion

I'm not sure if gossip membership makes sense.

[shashankshampi]

---

Note: #1205 (review)

I would recommend not using the subscribe system you're using here. The reason being it's misleading, makes you think stuff works differently than it actually does. That behaves more like a mock of sorts, than the actual expected behaviour.
I may have said that too late because I didn't wanna bother you too much, but I've realised it's quite misleading. I suggest you follow the approach I use in my tests. You can check them out here: https://github.com/vacp2p/nim-libp2p/pull/1184/files, for example in messages are not sent back to source or forwarding peer. You might need to check several test cases, as some of them are slightly more complex than others. Checking some of them might be key to get the idea.

I believe you are making some assumptions in how information flows which are not correct.
E.g.: You are calling unsubscribe multiple times on the same node on the same topic on the same unsubscribed-handler, which means nothing's going to be unsubscribed.
I suggest checking the provided docs and/or code and/or requesting a deeper explanation if that's not clear.

Following from 2. Be careful not to conform tests to the results, but the other way around. If results are not as expected you should double check your test does what it intends. If it does, then that's a signal there might be something wrong in the codebase. But absolutely never adapt the assertions just because.

EDIT: Please, apply this suggestions (if and whenever they fit) into the other PR as well). Thanks! :)

---

I have updated these tests as per your suggestions to use peer and common handler and subscribe and unsubscribe with multiple peers to topic

Test updated:

1. "handle JOIN topic and mesh is updated"
2. "handle LEAVE topic and mesh is updated"
3. "multiple peers join and leave topic simultaneously"

Methods created:

1. commonSubscribe
2. commonUnsubscribe

Common handler:
voidTopicHandler

cc: @fbarbu15 @AlejandroCabeza

[shashankshampi]

addressed review comment

[shashankshampi]

All done

[AlejandroCabeza]

What's the difference between this case and the handle SUBSCRIBE to the topic one?

[shashankshampi]

The first test is primarily about verifying mesh behavior after joining.
The second test is broader and checks both mesh and gossipsub structures to ensure proper subscription handling.

In short, the second test provides a more thorough validation by checking both the mesh and gossipsub structures, while the first test is specifically focused on the mesh when peers join a topic.

[AlejandroCabeza]

I think you might be able to condense the two cases into one.

Are they two different cases in the test plans?

[shashankshampi]

There were 6 TC on documentation, and I followed the same.
Let me know if this is not required now will remove it

[shashankshampi]

Addressed all the review comments as per request.

[shashankshampi]

The first test is primarily about verifying mesh behavior after joining.
The second test is broader and checks both mesh and gossipsub structures to ensure proper subscription handling.

In short, the second test provides a more thorough validation by checking both the mesh and gossipsub structures, while the first test is specifically focused on the mesh when peers join a topic.

[AlejandroCabeza]

I think it'd be a good addition to this PR to also follow the Given-When-Then formula used in the rest of the codebase. It documents the case by high-level blocks, making it easier to understand for external eyes.

You can check other PRs and or test files for examples.
Also, here's an example using one of your implemented cases:

  asyncTest "handle SUBSCRIBE to the topic":
    # Given 5 gossipsub nodes
    let
      numberOfNodes = 5
      topic = "test-topic"
      nodes = generateNodes(numberOfNodes, gossip = true)

    await allFuturesThrowing(nodes.mapIt(it.switch.start()))

    # When all of them are connected and subscribed to the same topic
    await subscribeNodes(nodes)
    for node in nodes:
      node.subscribe(topic, voidTopicHandler)

    await sleepAsync(2 * DURATION_TIMEOUT)

    # Then their related attributes should reflect that
    for node in nodes:
      let currentGossip = GossipSub(node)

      check currentGossip.topics.contains(topic)
      check currentGossip.gossipsub[topic].len() == numberOfNodes - 1
      check currentGossip.mesh[topic].len() == numberOfNodes - 1

    await allFuturesThrowing(nodes.mapIt(allFutures(it.switch.stop())))

[shashankshampi]

asyncTest "handle SUBSCRIBE to the topic":
    # Given 5 gossipsub nodes
    let
      numberOfNodes = 5
      topic = "test-topic"
      nodes = generateNodes(numberOfNodes, gossip = true)

    await allFuturesThrowing(nodes.mapIt(it.switch.start()))

    # When all of them are connected and subscribed to the same topic
    await subscribeNodes(nodes)
    for node in nodes:
      node.subscribe(topic, voidTopicHandler)

    await sleepAsync(2 * DURATION_TIMEOUT)

    # Then their related attributes should reflect that
    for node in nodes:
      let currentGossip = GossipSub(node)

      check currentGossip.topics.contains(topic)
      check currentGossip.gossipsub[topic].len() == numberOfNodes - 1
      check currentGossip.mesh[topic].len() == numberOfNodes - 1

    await allFuturesThrowing(nodes.mapIt(allFutures(it.switch.stop())))

Done. Added documentation as per Given When Then

[AlejandroCabeza]

I'd remove the second part here. Leave only: When nodes subscribe to multiple topics, as that's what you're doing here, and later you're handling the unsubscribe part.

[shashankshampi]

done

[AlejandroCabeza]

Here you don't need to handle the if gossipSub.topics.len < gossipSub.topicsHigh:. That's precisely the situation that we want to test, and that subscribe should be handling:

Suggested change

	for topic in topicNames:
	if gossipSub.topics.len < gossipSub.topicsHigh:
	gossipSub.subscribe(
	topic,
	proc(topic: string, data: seq[byte]): Future[void] {.async.} =
	discard
	,
	)
	else:
	# Then the subscription count should not exceed the limit
	check gossipSub.topics.len == gossipSub.topicsHigh
	for topic in topicNames:
	gossipSub.subscribe(
	topic,
	proc(topic: string, data: seq[byte]): Future[void] {.async.} =
	discard
	,
	)

[shashankshampi]

done

[AlejandroCabeza]

I would check the gossipsub.topics.len against gossipSubParams, which is the limit you defined here. That would make the check accurate.

Also, maybe it'd be interesting to connect the nodes and check for mesh and gossipsub population.

[shashankshampi]

done

[AlejandroCabeza]

Isn't join and subscribe the same thing?

[shashankshampi]

fixed

[AlejandroCabeza]

The first half of this should be a bit up, because the connection and subscription is done in the lines above.

[shashankshampi]

done

[AlejandroCabeza]

The subscription is done above, on the subscribe. This is just a synchronous check to turn the test deterministic: Make sure all nodes are connected between themselves.

[shashankshampi]

done

[shashankshampi]

Done all as per request

[shashankshampi]

done

[shashankshampi]

done

[shashankshampi]

done

[shashankshampi]

fixed

[shashankshampi]

done

[shashankshampi]

done

[AlejandroCabeza]

Please, do these checks as exhaustive as possible :)

[shashankshampi]

all 3 validations is already there. Please help with any validation you think is missed out

[AlejandroCabeza]

Same here (re: exhaustiveness).

[shashankshampi]

all three validations are already there. Please help with any validation you think is missed out