vrx_2023 phase3_challenge

Phase 3 - VRX Challenge

The process and content of the Phase 3 will be very similar to the Phase 2: Dress Rehearsal.

What to expect

• The VRX 2023 Task Descriptions and Technical Guide, available at the VRX 2023 Competition website, contain all the details about the tasks and about the environmental envelope expected in the unseen competition scenarios.
• Just as in Phase 2, team performance will be assessed by running the team submission on multiple trials of each task, where each trial instantiates a specific task scenario consistent with the general task description.
• The submission process will be the same as Phase 2.

Preparing your system

To ensure that your system can adapt to previously-unseen scenarios, teams should test their system at least against all released sample tasks.

As in phase 2, you can prepare for this phase using the 3 example worlds we have provided for each task in VRX 2023. These example worlds are roughly representative of the task configurations and environmental conditions specified in the task documentation. The Phase 2 Practice Worlds tutorial tutorial explains how these worlds are labeled and how to use them to practice for the competition.

For the evaluation of your Phase 3 submissions, the solutions will be run through new configurations that are not released prior to the Phase 3 deadline, to exercise the solutions ability to deal with new configurations and conditions.

Submission process

To submit an entry, you must do the following:

• Define compliant thruster and component configurations for your WAMV.
• Create a solution to one or more tasks.
• Create a Docker image containing your solution upload it to Docker Hub.
• Create the required files described in the submission format section (below);
• Follow the instructions in the submission process tutorial to submit these files.

Submission format

We expect to receive three files from each competitor prior to this event:

• dockerhub_image.txt: Contains the name of the image to be pulled from DockerHub.
• thruster_config.yaml: Defines the WAM-V thruster configuration (see tutorial for examples).
• component_config.yaml: Defines the WAM-V component configuration (see tutorial for examples).

Important:

• If you're using a private DockerHub repository, please grant access to the virtualrobotx DockerHub user. Otherwise we will not be able to evaluate your submission.
• Follow the WAMV Compliance Tutorial to ensure your WAM-V configuration complies with the competition guidelines.

Submission testing

We highly recommend that each team test their solution and the docker image by following the Testing Your Submission* wiki. This will allow teams to reproduce the evaluation and assessment system prior to submission.

Submission process notes

Before submitting your pull request we recommend that you validate your submission and evaluate it to verify that it works as expected. See the following tutorials for a walk through of this process:

• How to validate your submission.
• How to run a trial and score yourself.

Once you submit the pull request, the VRX technical team will do two things before merging (accepting) the submission:

1. Check that the WAM-V thruster and component configuration complies with the configuration constraints described in the VRX Technical Guide
2. Check that the DockerHub image is accessible by the virtualrobotx DockerHub user.

Once those two requirements are met, the pull request will be merged and your submission is considered ready.

Important dates

Date	Description
2023, November 1, 23:59 PDT	Deadline for submitting your solution
2023, November 2, 23:59 PDT	Deadline for submitting a correction
2023, November 16	Results published

note on submitting a correction: we allocate an extra day in the schedule for fixing any problems related with the content of the submission files (e.g.: a noncompliant component or thruster configuration, a typo pointing to the dockerhub image, a problem with dockerhub permissions not letting open robotics to download your solution image). teams are not allowed to modify the dockerhub image after the deadline for submitting a solution.

Late Submissions

As a rule, late submissions will not be accepted. All teams must submit the required elements by the solution deadline. The technical team does reserve the right to accept late submissions if the submission was delayed due to unforeseen problems with the submission process. Acceptance of any submissions after the solution deadline will be at the discretion of the VRX Technical Team.

Note about Docker Optional Arguments

Some features of docker require extra command line arguments. If your team would like to specify an extra command line argument when executing your submitted docker image, please let the VRX Technical Team know as soon as possible by creating a new issue in the issue tracker. We will deal with these requests on a case by case basis. We will make every effort to be as flexible as we can with the team solutions while providing a fair competition for all.

We do not expect many teams to want or need the flexibility to add additional command line arguments, but in cases were it can allow teams to innovate without giving them a competitive advantage, we want to work to make that happen.

Back: Phase 2 Practice Worlds	Top: VRX Tutorials	Next: Submission Process