Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing issue with automated ride hail getting simulations stuck #3766

Draft
wants to merge 9 commits into
base: develop
Choose a base branch
from

Conversation

HoneyTauOverTwo
Copy link
Collaborator

@HoneyTauOverTwo HoneyTauOverTwo commented Aug 15, 2023

There is an issue with AV RH which was discovered on GEMINI but later was proven to happen on develop as well, although less frequently.

The test "Ride Hail Electric vehicles" should "pick chargers choosing smaller DrivingTimeCost." of ElectricVehicleChargingBehaviorTest sometimes gets stuck on CI. Investigation led to the conclusion that, at least for this particular test, the issue only appears when running multiple beam simulations in parallel (which is the case for CI).


This change is Reviewable

…-stuck-parallelism

# Conflicts:
#	src/test/scala/beam/agentsim/agents/ElectricVehicleChargingBehaviorTest.scala
@HoneyTauOverTwo
Copy link
Collaborator Author

There is another test related to electric vehicles, ChargingSpec, that got stuck on CI recently on a PR that had only Python changes.

This one does not have any ride hail vehicles. I am investigating whether the issue is the same with AV RH and this one.

…ecified classes. The default was kept as 60s.
@HoneyTauOverTwo
Copy link
Collaborator Author

Investigation led to the conclusion that stuck finder was killing agents that were not actually stuck.
This was leaving the simulation in a looping state where it was expecting answers from the agents killed, leading to the simulation getting stuck.

I have doubled Beamville's stuckAgentDetection timeout from 20s to 40s to specified classes, the default was kept as 60s as a first guess of the timeout needed. I'll be performing tests and updating this value if needed.

A new issue #3798 was created related to these findings.

@HoneyTauOverTwo
Copy link
Collaborator Author

HoneyTauOverTwo commented Oct 11, 2023

test!

Doubling the stuckfinder timeout seems to not be enough for it to not get stuck on CI.

Modifies stuck finder code to log the maximum timeout observed on CI.
@HoneyTauOverTwo
Copy link
Collaborator Author

test!

@HoneyTauOverTwo
Copy link
Collaborator Author

HoneyTauOverTwo commented Oct 11, 2023

test!

Got stuck on ChargingSpec

…ve the logs and see if ChargingSpec will pass.
@HoneyTauOverTwo
Copy link
Collaborator Author

test!

1 similar comment
@HoneyTauOverTwo
Copy link
Collaborator Author

test!

@HoneyTauOverTwo
Copy link
Collaborator Author

HoneyTauOverTwo commented Oct 18, 2023

test!

Did not get stuck on CI but somehow failed unrelated tests:

beam.agentsim.agents.RideHailAgentSpec.A RideHailAgent should drive around when I tell him to
beam.agentsim.agents.RideHailAgentSpec.A RideHailAgent should let me interrupt it and tell it to cancel its job
beam.agentsim.agents.RideHailAgentSpec.A RideHailAgent won't let me cancel its job after it has picked up passengers

@HoneyTauOverTwo
Copy link
Collaborator Author

HoneyTauOverTwo commented Oct 18, 2023

test!

ElectricVehicleChargingBehaviorTest got stuck as usual.

…eously.

Testing showed this could be a source of locking as well.
@HoneyTauOverTwo
Copy link
Collaborator Author

test!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Simulation stuckness - Usage of Automated RH vehicles leads to stuckness in some cases.
1 participant