feat: Improve background batch uploading and session management

[einsteinx2]

Summary

The core goal of this change is to reduce the unnecessary uploading of batches when in the background, especially during long running background usage such as music playback.

Over the course of implementing that, it was also necessary to refactor the background session handling as it directly ties into that logic.

Most of the changes are in the MPBackendController class.

Some highlights:

• Session timeout is now uncapped and can be set to any value from 1 second to the max value of NSTimeInterval
• Some dead logic was removed (logic that used to have purpose but was now doing nothing, e.g. if statement always false)
• Batches are no longer uploaded as soon as the app enters the background, instead they are uploaded when background time remaining falls to under 10 seconds. This prevents unnecessary batch uploads when users quickly switch apps or when switching back and forth into a music streaming app to change songs, etc.
• Background session handling now properly accounts for this concept of "active" background usage, e.g. listening to music in the background
• All code related to backgrounding has been cleaned up and refactored and had it's logic validated

Testing Plan

Test Plan:


In all of these test cases, launch app first with a session timeout of 600 seconds (default release timeout) 
and wait for it to settle (Identity call, upload any persisted batches, etc). These batches are NOT counted 
when it says “no batches should upload).


General batch uploading:
------------------------


Name: Test short duration idle behavior 
Steps: 
- Leave app in foreground doing nothing for 2 minutes
Expected:
- No batches should upload
Results:
- iOS: PASS
- tvOS: PASS


Name: Test long duration idle behavior 
Steps: 
- Leave app in foreground doing nothing for 15 minutes
Expected:
- A batch should upload after 10 minutes, stack trace should show source is upload timer
Results:
- iOS: PASS
- tvOS: PASS


Name: Test manual upload does not reset upload timer
Steps: 
- Set the upload timeout to 60 seconds to accelerate testing
- Use dispatch_async to schedule a log of a regular MPEvent after 15 seconds
- Use dispatch_async to schedule an upload after 30 seconds
- Leave app in foreground doing nothing for 60 seconds
Expected:
- A batch should upload after 30 seconds is called
- The upload timer should fire approximately 30 seconds after upload is called 
Results:
- iOS: PASS
- tvOS: PASS


Background batch uploading:
---------------------------


Name: Test short time in background
Steps: 
- Without music playing, enter background 
- Wait 15 seconds
- Enter foreground
Expected:
- No batches should upload
- Background time remaining should count down from 30 seconds
Results:
- iOS: PASS
- tvOS: PASS


Name: Test app slept by OS in background
Steps: 
- Without music playing, enter background
- Wait for 30 seconds
- Enter foreground
Expected:
- A batch should be uploaded after 20 seconds in background (when 10 seconds remain on the background time) and the app should sleep
- Upon re-entering foreground, no batches should upload
Results:
- iOS: PASS
- tvOS: PASS


Name: Test app not slept by OS in background when playing music
Steps: 
- With music playing, enter background
- Wait for 30 seconds
- Enter foreground
Expected:
- No batches should upload upon entering background
- Background time should be a very high number and NOT count down
- Upon re-entering foreground, no batches should upload
Results:
- iOS: PASS
- tvOS: PASS


Name: Test that the background time remaining begins to count down after music pauses and app is correctly slept by OS
Steps: 
- With music playing, enter background
- Wait 10 seconds
- Pause the music using OS media controls
- Wait for app to be slept by OS
- Enter foreground
Expected:
- No batches should upload upon entering background
- The remaining background time should be a very high number and NOT count down
- After pausing, within a few seconds it should begin counting down from 30 seconds
- Upon reaching 10 seconds remaining, a batch should upload and the app should sleep
- Upon re-entering foreground, no batches should upload
Results:
- iOS: PASS
- tvOS: PASS


Name: Test that the background time remaining begins to count down after music finishes playing naturally and app is correctly slept by OS
Steps: 
- With music playing, enter background
- Wait for music to end on it’s own
- Wait for app to be slept by OS
- Enter foreground
Expected:
- No batches should upload upon entering background
- The remaining background time should be a very high number and NOT count down
- After finishing, within a few seconds it should begin counting down from 30 seconds
- Upon reaching 10 seconds remaining, a batch should upload and the app should sleep
- Upon re-entering foreground, no batches should upload
Results
- iOS: PASS
- tvOS: PASS



Session handling:
-----------------


Name: Test session timeout behavior when app is backgrounded for a short time
Steps: 
- Enter background 
- Wait 15 seconds 
- Enter foreground
Expected: 
- No new sessions should be created
- No batches should upload
- Session backgroundTime should increase by the number of seconds since app entered background
- Session interruptions should increment by 1
Results:
- iOS: PASS
- tvOS: PASS


Name: Test session timeout behavior when app is backgrounded while playing music to prevent sleep
Steps: 
- With music playing, enter background 
- Wait 70 seconds
- Enter foreground
Expected:
- New session should be created after timeout expires
- A batch should upload when session ends
- Old session end time should be time of last event (in this case the AST event)
- When app enters foreground no batch should upload (will upload later when upload timer fires or app is slept)
- When app enters foreground no new session should be created, existing session should continue
- Old session backgroundTime should be 0 seconds since endTime is actually when we backgrounded and a higher number would make a negative foregroundTime, aka the additional background time is not actually part of the session since the endTime is before that
- Old session foregroundTime should be ~10 or however long the app was open before entering background, it should not be a negative value
- Old session should have 1 interruption
- New session should have 0 interruptions
- New session backgroundTime and foregroundTime should be 0 (backgroundTime is 0 because it's only used to calculate foreground time, and not used directly, and anything higher than 0 would result in inaccurate foreground time calculation)
Results:
- iOS: PASS
- tvOS: PASS


Name: Test multiple session timeout behavior when app is backgrounded while playing music to prevent sleep
Steps: 
- With music playing, enter background 
- Wait 130 seconds
- Enter foreground
Expected:
- 2 new sessions should be created
- A batch should upload when each session ends
- Both session end times should be time of last event (in this case the AST event and then the Session Start event)
- When app enters foreground no batch should upload (will upload later when upload timer fires or app is slept)
- When app enters foreground no new session should be created (until session timeout expires again), existing session should continue
- 1st old session backgroundTime should be 0 seconds since endTime is actually when we backgrounded and a higher number would make a negative foregroundTime, aka the additional background time is not actually part of the session since the endTime is before that
- 1st old session foregroundTime should be ~10 or however long the app was open before entering background, it should not be a negative value
- 1st old session should have 1 interruption
- 2nd old session backgroundTime should be 0 seconds (this is fine as we don’t actually report this value only use it to calculate foregroundTime)
- 2nd old session foregroundTime should be 0 seconds as it was never in the foreground
- 2nd old session should have 0 interruptions
- New session should have 0 interruptions
- New session backgroundTime should 0 seconds
Results:
- iOS: PASS
- tvOS: PASS


Name: Test session timeout behavior when timeout occurs while app is sleeping 
Steps: 
- Enter background 
- Wait 70 seconds
- Enter foreground
Expected:
- A batch should upload when the app is slept by the OS
- Upon entering foreground, session timeout should expire, the session should end, and a new session should be created
- A batch should upload when session ends
- Old session end time should be time of last event (in this case the AST entering background event)
- Old session should have 1 interruption
- New session should have 0 interruptions
- New session start time should be the time should be when the app entered the foreground
Results:
- iOS: PASS
- tvOS: PASS


Name:  Test session timeout reset when event is fired in background
Steps: 
- Use dispatch_after to call logEvent after 40 seconds
- With music playing, enter background
- Wait another 100 seconds
Expected:
- The session should end 60 seconds after logging the event (90 seconds after entering the background)
Results:
- iOS: PASS
- tvOS: PASS


Automatic session management completely disabled:
-------------------------------------------------

In all of these test cases, follow the original instructions but set both `automaticSessionTracking` and `shouldBeginSession` to false in MParticleOptions when initializing the SDK.


Name: Test app launch behavior with manual session management, event logged with default properties
Steps: 
- Launch the app without manually calling beginSession
- Log a regular MPEvent after 10 seconds
Expected:
- No session should be created at launch
- A session should be created after logging the event
Results:
- iOS: PASS
- tvOS: PASS


Name: Test app launch behavior with manual session management, event logged with shouldBeginSession = false
Steps: 
- Launch the app without manually calling beginSession
- Log a regular MPEvent after 10 seconds with event.shouldBeginSession = false
Expected:
- No session should be created at launch
- No session should be created after logging the event
Results:
- iOS: PASS
- tvOS: PASS


Name: Test that session timeout does nothing with manual session management
Steps: 
- Call beginSession on app launch
- With music playing, enter background 
- Wait 70 seconds
- Enter foreground
Expected:
- A session should be created on app launch
- The session should not be ended in the background 
- The same session should continue when re-entering the foreground
Results:
- iOS: PASS
- tvOS: PASS


Name: Test manual session management on app launch, backgrounding, and foregrounding
Steps: 
- Call beginSession on appDidBecomeActive
- Call endSession in appDidEnterBackground
- Enter background
- Wait 10 seconds
- Enter foreground
Expected:
- A session should be created on app launch
- The session should be ended in the background
- No batch should upload when session ends (this is a separate code path from when sessions end automatically (this behavior can be changed in a future story, out of scope for this story)
- A new session should be created when returning to foreground 
Results:
- iOS: PASS
- tvOS: PASS

Reference Issue (For mParticle employees only. Ignore if you are an outside contributor)

• Closes https://go.mparticle.com/work/SQDSDKS-5883

[BrandonStalnaker]

LGTM

[samdozor]

Looks great! Non-blocking questions left inline.

[samdozor]

Just checking - I see we're calling waitForKitsAndUpload (which will make batches) within beginUploadTimer, which is called within applicationDidBecomeActive - does an app become "inactive" only after the countdown timer goes to zero? If so, then this is fine.

or does the app not become inactive

[einsteinx2]

@samdozor I’m not sure I fully understand the question, but for clarification waitForKitsAndUpload is being called in the upload timer’s even handler not inside beginUploadTimer, so it’s just there to trigger the upload when the timer fires. The upload timer is only used while the app is active and inactive, so every X seconds (600 by default, aka 10 minutes) this will fire, whether we’re active or inactive. We only disable it when the app is put to sleep by the OS so it doesn’t automatically fire if we wake up later.

[samdozor]

is this needed? since we'll call it again within beginUploadTimer?

[einsteinx2]

@samdozor I actually explicitly added this in based on notes from one of our conversations. :)

I believe the thinking was since the session is ending a batch is mandatory anyway, so it was better to just upload it immediately than to wait since it wouldn't matter for the "events per batch" metric anyway. But you're correct it's not strictly necessary as it will be uploaded on the next timeout (if the app doesn't crash or and isn't force killed by the user).

[mparticle-automation]

🎉 This PR is included in version 8.17.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀