Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Subscriber Connectivity Tests #924

Merged
merged 56 commits into from
Feb 15, 2023
Merged

Conversation

AndyTWF
Copy link
Contributor

@AndyTWF AndyTWF commented Jan 18, 2023

Adds test cases for the main requirements of #918 to test subscriber behaviour when there are network connectivity issues.

Note, some tests are skipped due to the skipTest property on the fault - to run all tests you can either set the property to false for whichever test you want to run, or remove the assumeFalse call in the test classes setup method.

Current test failure status is below, many of the tests that hang may be related to ably/ably-java#474, which we can confirm once the fix is merged.

Fault before starting subscriber:

TCP Connection Refused - timeout waiting for Ably to connect, as per publisher
TCP Connection Unresponsive - timeout waiting for Ably to connect, as per publisher
Attach unresponsive - throws Exception re attach timeout
Disconnect with failed resume - exception thrown
Enter failed with non-fatal nack - exception thrown
Disconnect and suspend - timeout waiting for Ably to connect
Enter unresponsive - hang

After starting/before stopping subscriber:

TCP Connection Refused - hang
TCP Connection Unresponsive - hang
Detach unresponsive - throws exception re detach timeout
Disconnect and suspend - hang
Reenter on resume failed - hang
Enter unresponsive - hang

Tests checking we get location and publisher updates after a fault:

TCP Connection Unresponsive - hang
Disconnect and suspend - hang

Add helper functions to start and stop the subscriber, parameterize the
test cases etc.
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka January 18, 2023 15:29 Inactive
Base automatically changed from 865-layer-7-proxy to main January 19, 2023 11:29
And fix the other tests
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka January 19, 2023 11:38 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka January 19, 2023 11:46 Inactive
Move ably publishing connection setup into helper methods
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka January 19, 2023 12:38 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka January 19, 2023 13:26 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka January 19, 2023 14:18 Inactive
@AndyTWF AndyTWF changed the title WIP: Subscriber Connectivity Tests Subscriber Connectivity Tests Jan 19, 2023
They're currently skipped
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka January 19, 2023 14:38 Inactive
@AndyTWF AndyTWF self-assigned this Jan 19, 2023
@AndyTWF AndyTWF added testing Includes all kinds of automated tests, the way that we run them and the infrastructure around them. important labels Jan 19, 2023
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka January 19, 2023 15:10 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka January 19, 2023 16:03 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka January 19, 2023 16:50 Inactive
This is due to an issue with KTOR (see comment in the test setup method).

Likely to be causing all the timeout issues we have at this API level.
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka February 10, 2023 08:08 Inactive
@AndyTWF AndyTWF marked this pull request as ready for review February 10, 2023 08:36
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka February 10, 2023 10:59 Inactive
Copy link
Contributor

@davyskiba davyskiba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall lookgs good :)

To bring into line with #994
This saves us having to do lots of expectations and waits in tests.
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka February 10, 2023 13:48 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka February 10, 2023 13:56 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka February 14, 2023 09:06 Inactive
@KacperKluka KacperKluka removed their request for review February 14, 2023 09:26
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka February 14, 2023 15:18 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/924/dokka February 14, 2023 15:47 Inactive
@AndyTWF AndyTWF merged commit 6b09c9c into main Feb 15, 2023
@AndyTWF AndyTWF deleted the 918-susbscriber-connectivity-tests branch February 15, 2023 08:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
important testing Includes all kinds of automated tests, the way that we run them and the infrastructure around them.
Development

Successfully merging this pull request may close these issues.

Add an integration test for network fault coverage to Subscriber component
4 participants