You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This issue comes as a followup to #987, as the due preparations are done, and everything is ready for the batch executions.
I did 5 runs with 5 folds at RNM=0.05 , so 25 calibrations in total. We were unsure whether the cross validation methods were working, but the results are consistent with the ones I presented in #987 during our last meeting, so its fair to assume cross validation is working as intended.
Results
NOTE : See results in the first comment instead, these are valid but there is only 1 set of results per modality.
Collection # ,Averages
RMS (pix) ,7.697367999999999
X err (pix) ,4.387492
Y err (pix) ,3.302896
rgb_rgb_evaluation.csv
Collection # ,Averages
RMS (pix) ,5.042992
X err (pix) ,2.8674880000000003
Y err (pix) ,3.302876
Trans (mm) ,18.659332
Rot (deg) ,0.978508
rgb_depth_evaluation.csv
Collection # ,Averages
RMS (pix) ,5.481612
X err (pix) ,3.202924
Y err (pix) ,2.532076
rgb_lidar_evaluation.csv
Collection # ,Averages
RMS (pix) ,5.841672
X err (pix) ,3.762896
Y err (pix) ,2.073256
X StDev (pix) ,2.9350959999999997
Y StDev (pix) ,1.999436
Data.yml and jinja template
I will put these here just as a reference if anything regarding the batch execution wasn't clear. This section is really detailed so feel free to skip.
data.yml
## █████╗ ████████╗ ██████╗ ███╗ ███╗# ██╔══██╗╚══██╔══╝██╔═══██╗████╗ ████║# ███████║ ██║ ██║ ██║██╔████╔██║# ██╔══██║ ██║ ██║ ██║██║╚██╔╝██║# __ ██║ ██║ ██║ ╚██████╔╝██║ ╚═╝ ██║ _# / _| ╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═╝ | |# | |_ _ __ __ _ _ __ ___ _____ _____ _ __| | __# | _| '__/ _` | '_ ` _ \ / _ \ \ /\ / / _ \| '__| |/ /# | | | | | (_| | | | | | | __/\ v v / (_) | | | <# |_| |_| \__,_|_| |_| |_|\___| \_/\_/ \___/|_| |_|\_\# https://github.com/lardemua/atom# this yml file contains variables to be used in conjunction with batch.yml# Auxiliary variables, to be used to render other fields in the template.yml.j2 filepackage_path: "package://zau_calibration"dataset_path : "${ATOM_DATASETS}/zau/dataset_filtered/dataset_corrected_with_odometry_and_depth_and_rgb_and_pattern_poses_filtered.json"# DATASET : "${ATOM_DATASETS}/zau/dataset_filtered/dataset_corrected_with_odometry_and_depth_and_rgb_and_pattern_poses_filtered.json"DATASET_TRAIN : "${ATOM_DATASETS}/zau/dataset_filtered/dataset_corrected_with_odometry_and_depth_and_rgb_and_pattern_poses_filtered_train.json"DATASET_TRAINED: "${ATOM_DATASETS}/zau/dataset_filtered/atom_calibration.json"DATASET_TEST : "${ATOM_DATASETS}/zau/dataset_filtered/dataset_corrected_with_odometry_and_depth_and_rgb_and_pattern_poses_filtered_test.json"RESULTS_PATH : "${ATOM_DATASETS}/zau/results"CSF : "lambda x: True"RNM : 0.05TOL : 1e-3# Runs are repetitions of the experiments for gathering statistically significant resultsruns: [1,2,3,4,5]cross_validation:
type: "stratified-k-fold"n_splits: 5# Number of foldstrain_size: # Percentage of the dataset used for training, only used in StratifiedShuffleSplit# Experiments are executions with a set of input parametersexperiments:
- {name: zau_fixed_odometry}
This issue comes as a followup to #987, as the due preparations are done, and everything is ready for the batch executions.
I did 5 runs with 5 folds at RNM=0.05 , so 25 calibrations in total. We were unsure whether the cross validation methods were working, but the results are consistent with the ones I presented in #987 during our last meeting, so its fair to assume cross validation is working as intended.
Results
NOTE : See results in the first comment instead, these are valid but there is only 1 set of results per modality.
calibration_errors.csv
rgb_lidar_evaluation.csv
rgb_rgb_evaluation.csv
rgb_depth_evaluation.csv
rgb_lidar_evaluation.csv
Data.yml and jinja template
I will put these here just as a reference if anything regarding the batch execution wasn't clear. This section is really detailed so feel free to skip.
data.yml
template.yml.j2
Each evaluation script is a adaptation of the scripts developed in the prior testing phases, in #987.
Here is one example, for
rgb_rgb_batch_evaluations
The text was updated successfully, but these errors were encountered: