diff --git a/.gitignore b/.gitignore index fbcc55ed..dda3a973 100644 --- a/.gitignore +++ b/.gitignore @@ -6,3 +6,5 @@ venv *.swp __pycache__/ + +tmp diff --git a/docs/dicts/s3gw.dict b/docs/dicts/s3gw.dict index 0a8d144b..843a8d32 100644 --- a/docs/dicts/s3gw.dict +++ b/docs/dicts/s3gw.dict @@ -64,3 +64,6 @@ certmanager errno schedulable Backoff +zeroload +emptydb +deathtype diff --git a/docs/research/ha/RATIONALE.md b/docs/research/ha/RATIONALE.md index 00bd243f..6766631f 100644 --- a/docs/research/ha/RATIONALE.md +++ b/docs/research/ha/RATIONALE.md @@ -17,7 +17,7 @@ - [Notes on testing s3gw within K8s](#notes-on-testing-s3gw-within-k8s) - [EXIT-1, 10 measures](#exit-1-10-measures) - [EXIT-0, 10 measures](#exit-0-10-measures) - - [Tested Scenarios - radosgw focused](#tested-scenarios---radosgw-focused) + - [Tested Scenarios - radosgw-restart](#tested-scenarios---radosgw-restart) - [regular\_localhost\_zeroload\_emptydb](#regular_localhost_zeroload_emptydb) - [segfault\_localhost\_zeroload\_emptydb](#segfault_localhost_zeroload_emptydb) - [regular\_localhost\_load\_fio\_64\_write](#regular_localhost_load_fio_64_write) @@ -25,6 +25,10 @@ - [400K objects - measures done with the WAL file zeroed](#400k-objects---measures-done-with-the-wal-file-zeroed) - [800K objects - measures done with the WAL file still to be processed (size 32G)](#800k-objects---measures-done-with-the-wal-file-still-to-be-processed-size-32g) - [regular-localhost-incremental-fill-5k](#regular-localhost-incremental-fill-5k) + - [scale\_deployment\_0\_1-k3s3nodes\_zeroload\_emptydb](#scale_deployment_0_1-k3s3nodes_zeroload_emptydb) + - [Tested Scenarios - S3-workload during s3gw Pod outage](#tested-scenarios---s3-workload-during-s3gw-pod-outage) + - [PutObj-100ms-ClusterIp](#putobj-100ms-clusterip) + - [PutObj-100ms-Ingress](#putobj-100ms-ingress) We want to investigate what *High Availability* - HA - means for a project like the s3gw. @@ -523,7 +527,7 @@ at least for the cases when the process exits with zero. Anyway, this behavior limits the number of measures we can collect and thus is preventing us to compute decent statistics on restart timings using Deployments. -## Tested Scenarios - radosgw focused +## Tested Scenarios - radosgw-restart When we test a scenario we are interested in collecting `radosgw`'s restart events; for each of those we measure the following metrics: @@ -539,63 +543,85 @@ events; for each of those we measure the following metrics: From these 2 metrics, we produce also a derived metric: `frontend_up_main_delta`, that is just the arithmetic difference between `to_frontend_up` and `to_main`. -For each scenario tested we collect a set of 100 measures. -For each scenario tested we produce 5 artifacts: +For each scenario tested we collect a set of measures. +For each scenario tested we produce a set of artifacts: -- deathtype-environment-description_stats_TS.json +- `*_stats.json` - It is the `json` file containing all the measures done for a scenario. It also contains some key statistics. -- deathtype-environment-description_raw_TS.svg +- `*_raw.svg` - It is the plot containing the all the charts for the measures: - `to_main` - `to_frontend_up` - `frontend_up_main_delta` - The ordinate axis is the `ID` of the restart event. - This is the natural order in which the restart events occurred. + On the X axis there are the restart event's `ID`s. + They follow the temporal order of the restart events. -- deathtype-environment-description_percentiles_to_main_TS.svg +- `*_percentiles_to_main.svg` - It is the plot containing the percentile graph for the `to_main` metric. -- deathtype-environment-description_percentiles_to_fup_TS.svg +- `*_percentiles_to_fup.svg` - It is the plot containing the percentile graph for the `to_frontend_up` metric. -- deathtype-environment-description_percentiles_fup_main_delta_TS.svg +- `*_percentiles_fup_main_delta.svg` - It is the plot containing the percentile graph for the `frontend_up_main_delta` metric. -Each file has a pattern name, where: +The file name, normally, contains some information such as: -- deathtype: is the way the `radosgw` process is asked to die: - - `exit0` - - `exit1` - - `segfault` - - `regular` +- deathtype: the way the `radosgw` process is asked to die: -- environment: is the environment where the scenario is tested: - - `localhost` - - `k8s` + - `exit0` - the process is asked to immediately exit with `exit(0)` + - `exit1` - the process is asked to immediately exit with `exit(1)` + - `segfault` - the process is asked to trigger a `segmentation fault` + - `regular` - the process is asked to exit with the ordered shutdown procedure + +- environment: the environment where the scenario is tested: + + - `localhost/host-path-volume` + - `k8s/k3d/k3s ... /host-path-volume` + - `k8s/k3d/k3s ... /LH-volume` - description: is a key description of the scenario - TS: this is just a timestamp of when the artifacts were produced ### regular_localhost_zeroload_emptydb +- restart-type: `regular` +- env: `localhost/host-path-volume` +- load: `zero-empty-db` +- #measures: `100` + + ||| |---|---| || | + ### segfault_localhost_zeroload_emptydb +- restart-type: `segfault` +- env: `localhost/host-path-volume` +- load: `zero-empty-db` +- #measures: `100` + + ||| |---|---| || | + ### regular_localhost_load_fio_64_write +- restart-type: `regular` +- env: `localhost/host-path-volume` +- load: `fio` +- #measures: `100` + `fio` configuration: ```ini @@ -612,33 +638,86 @@ http_host=localhost:7480 filename=/workload-1/obj1 numjobs=8 rw=write -size=128m +size=64m bs=1m ``` + ||| |---|---| || | + ### regular_localhost_zeroload_400_800Kdb #### 400K objects - measures done with the WAL file zeroed +- restart-type: `regular` +- env: `localhost/host-path-volume` +- load: `zero-400K-db` +- #measures: `100` + + ||| |---|---| || | + #### 800K objects - measures done with the WAL file still to be processed (size 32G) +- restart-type: `regular` +- env: `localhost/host-path-volume` +- load: `zero-800K-db` +- #measures: `100` + + ||| |---|---| || | + ### regular-localhost-incremental-fill-5k -Between every restart there is an interposed `PUT` of 5K objects, +- restart-type: `regular` +- env: `localhost/host-path-volume` +- load: `5K-incremental-800K-db` +- #measures: `100` + +Between every restart there is an interposed `PUT-Object` sequence, each of 5K objects; the sqlite db initially contained 800K objects. + ||| |---|---| || | + + +### scale_deployment_0_1-k3s3nodes_zeroload_emptydb + +- restart-type: `scale_deployment_0_1` +- env: `virtual-machine/k3s-3-nodes/LH-volume` +- load: `zero-empty-db` +- #measures: `300` + +The test has been conducted in 3 blocks, each of 100 restarts. +Each restart in a block is constrained to occur on a specific node. +The schema is the following: + +1. taint all nodes but `node-1` +2. trigger 100 pod restarts +3. taint all nodes but `node-2` +4. trigger 100 pod restarts +5. taint all nodes but `node-3` +6. trigger 100 pod restarts + + +||| +|---|---| +|| | + + +## Tested Scenarios - S3-workload during s3gw Pod outage + +### PutObj-100ms-ClusterIp + +### PutObj-100ms-Ingress diff --git a/docs/research/ha/measurements/regular_localhost_zeroload_400_800Kdb/README.md b/docs/research/ha/measurements/regular_localhost_zeroload_400_800Kdb/README.md index 23f155f3..0a79385e 100644 --- a/docs/research/ha/measurements/regular_localhost_zeroload_400_800Kdb/README.md +++ b/docs/research/ha/measurements/regular_localhost_zeroload_400_800Kdb/README.md @@ -3,9 +3,9 @@ 1. 400K measures done with the WAL file zeroed 2. 800K measures done with the WAL file still to be processed (size 32G) -``` +```shell giuseppe ➜ ~/developer/s3gw-ha/wd (main ✗) $ du -sh . -35G +35G -rw-r--r-- 1 giuseppe giuseppe 798M Sep 12 15:11 s3gw.db -rw-r--r-- 1 giuseppe giuseppe 64M Sep 12 15:11 s3gw.db-shm @@ -14,9 +14,9 @@ giuseppe ➜ ~/developer/s3gw-ha/wd (main ✗) $ du -sh . after 1 restart -``` +```shell giuseppe ➜ ~/developer/s3gw-ha/wd (main ✗) $ du -sh . -2.4G . +2.4G . -rw-r--r-- 1 giuseppe giuseppe 798M Sep 12 15:12 s3gw.db -rw-r--r-- 1 giuseppe giuseppe 32K Sep 12 15:15 s3gw.db-shm diff --git a/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_S3WL_RTT_raw.svg b/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_S3WL_RTT_raw.svg new file mode 100644 index 00000000..b26ee7b0 --- /dev/null +++ b/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_S3WL_RTT_raw.svg @@ -0,0 +1,511 @@ + + + + + +RTT S3Workload: s3wl-putobj-100ms-ClusterIp +Time +0.00 +25000.00 +50000.00 + + + + + + + + + + + + + +RTT: ms + +0 +1000 +2000 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_percentiles_fup_main_delta.svg b/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_percentiles_fup_main_delta.svg new file mode 100644 index 00000000..fdf32893 --- /dev/null +++ b/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_percentiles_fup_main_delta.svg @@ -0,0 +1,157 @@ + + + + + +Percentiles - FrontEndUp-Main-Delta (Nearest Rank): s3wl-putobj-100ms-ClusterIp +Percentile +20 +60 +100 + + + + + + + + + + + + +Duration: ms + +39 +41 +43 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +to-fronted-up + + diff --git a/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_percentiles_to_fup.svg b/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_percentiles_to_fup.svg new file mode 100644 index 00000000..db739329 --- /dev/null +++ b/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_percentiles_to_fup.svg @@ -0,0 +1,164 @@ + + + + + +Percentiles - FrontEndUp (Nearest Rank): s3wl-putobj-100ms-ClusterIp +Percentile +20 +60 +100 + + + + + + + + + + + + +Duration: ms + +625.00 +675.00 +725.00 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +to-fronted-up + + diff --git a/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_percentiles_to_main.svg b/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_percentiles_to_main.svg new file mode 100644 index 00000000..05939aab --- /dev/null +++ b/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_percentiles_to_main.svg @@ -0,0 +1,164 @@ + + + + + +Percentiles - Main (Nearest Rank): s3wl-putobj-100ms-ClusterIp +Percentile +20 +60 +100 + + + + + + + + + + + + +Duration: ms + +575.00 +625.00 +675.00 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +to-main + + diff --git a/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_raw.svg b/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_raw.svg new file mode 100644 index 00000000..5011d2b1 --- /dev/null +++ b/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_raw.svg @@ -0,0 +1,99 @@ + + + + + +Raw Data Measures: s3wl-putobj-100ms-ClusterIp +Restart ID +0 +4 +8 + + + + + + + + + + + + +Duration: ms + +100 +400 +700 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +to-main + + +to-fronted-up + + +fronted-up-main-delta + + diff --git a/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_stats.json b/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_stats.json new file mode 100644 index 00000000..18e8a6ec --- /dev/null +++ b/docs/research/ha/measurements/s3wl-putobj-100ms-clusterip/1695396383_s3wl-putobj-100ms-ClusterIp_stats.json @@ -0,0 +1,2865 @@ +{ + "series_restart_count": 0, + "series_restart": [ + { + "mark": "s3wl-putobj-100ms-ClusterIp", + "min_to_main": 573, + "max_to_main": 686, + "mean_to_main": 616, + "99p_to_main": 679, + "95p_to_main": 679, + "99pNR_to_main": 686, + "95pNR_to_main": 686, + "min_to_frontend_up": 613, + "max_to_frontend_up": 728, + "mean_to_frontend_up": 658, + "99p_to_frontend_up": 721, + "95p_to_frontend_up": 721, + "99pNR_to_frontend_up": 728, + "95pNR_to_frontend_up": 728, + "min_frontend_up_main_delta": 39, + "max_frontend_up_main_delta": 43, + "mean_frontend_up_main_delta": 40, + "99p_frontend_up_main_delta": 42, + "95p_frontend_up_main_delta": 42, + "99pNR_frontend_up_main_delta": 43, + "95pNR_frontend_up_main_delta": 43, + "data": [ + { + "restart_id": 1, + "duration_to_main": 606, + "duration_to_frontend_up": 648, + "frontend_up_main_delta": 41 + }, + { + "restart_id": 2, + "duration_to_main": 671, + "duration_to_frontend_up": 713, + "frontend_up_main_delta": 41 + }, + { + "restart_id": 3, + "duration_to_main": 606, + "duration_to_frontend_up": 647, + "frontend_up_main_delta": 41 + }, + { + "restart_id": 4, + "duration_to_main": 584, + "duration_to_frontend_up": 625, + "frontend_up_main_delta": 41 + }, + { + "restart_id": 5, + "duration_to_main": 573, + "duration_to_frontend_up": 613, + "frontend_up_main_delta": 40 + }, + { + "restart_id": 6, + "duration_to_main": 592, + "duration_to_frontend_up": 635, + "frontend_up_main_delta": 42 + }, + { + "restart_id": 7, + "duration_to_main": 577, + "duration_to_frontend_up": 620, + "frontend_up_main_delta": 43 + }, + { + "restart_id": 8, + "duration_to_main": 686, + "duration_to_frontend_up": 728, + "frontend_up_main_delta": 41 + }, + { + "restart_id": 9, + "duration_to_main": 673, + "duration_to_frontend_up": 714, + "frontend_up_main_delta": 40 + }, + { + "restart_id": 10, + "duration_to_main": 601, + "duration_to_frontend_up": 641, + "frontend_up_main_delta": 39 + } + ] + } + ], + "series_s3_workload_count": 0, + "series_s3_workload": [ + { + "mark": "s3wl-putobj-100ms-ClusterIp", + "min_RTT": 5, + "max_RTT": 2393, + "mean_RTT": 53, + "99p_RTT": 1241, + "95p_RTT": 94, + "99pNR_RTT": 1298, + "95pNR_RTT": 171, + "data": [ + { + "wl_id": 1, + "start": 1695396328988013233, + "end": 1695396329309034963, + "rtt": 321.021696, + "err_desc": "RequestError: send request failed\ncaused by: Put \"http://s3gw-ha-s3gw-ha.s3gw-ha.svc.cluster.local/test-1/objTest\": dial tcp 10.43.2.234:80: connect: connection refused" + }, + { + "wl_id": 2, + "start": 1695396329309039986, + "end": 1695396331596869185, + "rtt": 2287.829248, + "err_desc": "" + }, + { + "wl_id": 3, + "start": 1695396331596877865, + "end": 1695396331605944516, + "rtt": 9.066752, + "err_desc": "" + }, + { + "wl_id": 4, + "start": 1695396331688168451, + "end": 1695396331699185007, + "rtt": 11.016448, + "err_desc": "" + }, + { + "wl_id": 5, + "start": 1695396331788452810, + "end": 1695396331799305605, + "rtt": 10.852864, + "err_desc": "" + }, + { + "wl_id": 6, + "start": 1695396331888554497, + "end": 1695396331899449707, + "rtt": 10.895104, + "err_desc": "" + }, + { + "wl_id": 7, + "start": 1695396331987734873, + "end": 1695396331998327091, + "rtt": 10.592256, + "err_desc": "" + }, + { + "wl_id": 8, + "start": 1695396332088748333, + "end": 1695396332099656423, + "rtt": 10.90816, + "err_desc": "" + }, + { + "wl_id": 9, + "start": 1695396332187939439, + "end": 1695396332199387649, + "rtt": 11.44832, + "err_desc": "" + }, + { + "wl_id": 10, + "start": 1695396332288621771, + "end": 1695396332299283248, + "rtt": 10.661376, + "err_desc": "" + }, + { + "wl_id": 11, + "start": 1695396332388675842, + "end": 1695396332399429279, + "rtt": 10.753536, + "err_desc": "" + }, + { + "wl_id": 12, + "start": 1695396332488576377, + "end": 1695396332498954371, + "rtt": 10.37824, + "err_desc": "" + }, + { + "wl_id": 13, + "start": 1695396332588198498, + "end": 1695396332599455509, + "rtt": 11.257088, + "err_desc": "" + }, + { + "wl_id": 14, + "start": 1695396332688730779, + "end": 1695396332700008682, + "rtt": 11.277824, + "err_desc": "" + }, + { + "wl_id": 15, + "start": 1695396332788419157, + "end": 1695396332799264524, + "rtt": 10.84544, + "err_desc": "" + }, + { + "wl_id": 16, + "start": 1695396332888548976, + "end": 1695396332898854757, + "rtt": 10.305792, + "err_desc": "" + }, + { + "wl_id": 17, + "start": 1695396332988111510, + "end": 1695396332998994027, + "rtt": 10.882304, + "err_desc": "" + }, + { + "wl_id": 18, + "start": 1695396333088293194, + "end": 1695396333099046719, + "rtt": 10.753536, + "err_desc": "" + }, + { + "wl_id": 19, + "start": 1695396333188354449, + "end": 1695396333199127843, + "rtt": 10.773248, + "err_desc": "" + }, + { + "wl_id": 20, + "start": 1695396333288402684, + "end": 1695396333299224227, + "rtt": 10.821632, + "err_desc": "" + }, + { + "wl_id": 21, + "start": 1695396333388443372, + "end": 1695396333399706183, + "rtt": 11.26272, + "err_desc": "" + }, + { + "wl_id": 22, + "start": 1695396333487964533, + "end": 1695396333499780893, + "rtt": 11.816448, + "err_desc": "" + }, + { + "wl_id": 23, + "start": 1695396333588045433, + "end": 1695396333599032521, + "rtt": 10.987264, + "err_desc": "" + }, + { + "wl_id": 24, + "start": 1695396333688321972, + "end": 1695396333699742197, + "rtt": 11.42016, + "err_desc": "" + }, + { + "wl_id": 25, + "start": 1695396333787995239, + "end": 1695396333798651724, + "rtt": 10.656512, + "err_desc": "" + }, + { + "wl_id": 26, + "start": 1695396333887875502, + "end": 1695396333899033028, + "rtt": 11.157504, + "err_desc": "" + }, + { + "wl_id": 27, + "start": 1695396333988357692, + "end": 1695396333999390678, + "rtt": 11.033088, + "err_desc": "" + }, + { + "wl_id": 28, + "start": 1695396334087686941, + "end": 1695396334098561224, + "rtt": 10.874368, + "err_desc": "" + }, + { + "wl_id": 29, + "start": 1695396334187720966, + "end": 1695396334198551432, + "rtt": 10.830592, + "err_desc": "" + }, + { + "wl_id": 30, + "start": 1695396334287713803, + "end": 1695396334297692428, + "rtt": 9.978624, + "err_desc": "" + }, + { + "wl_id": 31, + "start": 1695396334388136556, + "end": 1695396334399241950, + "rtt": 11.105536, + "err_desc": "" + }, + { + "wl_id": 32, + "start": 1695396334488552412, + "end": 1695396334496371954, + "rtt": 7.81952, + "err_desc": "" + }, + { + "wl_id": 33, + "start": 1695396334588704612, + "end": 1695396334607299662, + "rtt": 18.595072, + "err_desc": "" + }, + { + "wl_id": 34, + "start": 1695396334688620872, + "end": 1695396334699851380, + "rtt": 11.230464, + "err_desc": "" + }, + { + "wl_id": 35, + "start": 1695396334787918743, + "end": 1695396335129619410, + "rtt": 341.700608, + "err_desc": "RequestError: send request failed\ncaused by: Put \"http://s3gw-ha-s3gw-ha.s3gw-ha.svc.cluster.local/test-1/objTest\": dial tcp 10.43.2.234:80: connect: connection refused" + }, + { + "wl_id": 36, + "start": 1695396335129632856, + "end": 1695396337435515553, + "rtt": 2305.88288, + "err_desc": "" + }, + { + "wl_id": 37, + "start": 1695396337435526515, + "end": 1695396337444425920, + "rtt": 8.899584, + "err_desc": "" + }, + { + "wl_id": 38, + "start": 1695396337488594601, + "end": 1695396337498984285, + "rtt": 10.389504, + "err_desc": "" + }, + { + "wl_id": 39, + "start": 1695396337588336601, + "end": 1695396337599905414, + "rtt": 11.568896, + "err_desc": "" + }, + { + "wl_id": 40, + "start": 1695396337688172454, + "end": 1695396337699269744, + "rtt": 11.097088, + "err_desc": "" + }, + { + "wl_id": 41, + "start": 1695396337788569837, + "end": 1695396337800092039, + "rtt": 11.522304, + "err_desc": "" + }, + { + "wl_id": 42, + "start": 1695396337888380810, + "end": 1695396337899115252, + "rtt": 10.734336, + "err_desc": "" + }, + { + "wl_id": 43, + "start": 1695396337988377047, + "end": 1695396337999039090, + "rtt": 10.661888, + "err_desc": "" + }, + { + "wl_id": 44, + "start": 1695396338088320991, + "end": 1695396338099620245, + "rtt": 11.299328, + "err_desc": "" + }, + { + "wl_id": 45, + "start": 1695396338187673414, + "end": 1695396338198832491, + "rtt": 11.15904, + "err_desc": "" + }, + { + "wl_id": 46, + "start": 1695396338288106129, + "end": 1695396338299505071, + "rtt": 11.398912, + "err_desc": "" + }, + { + "wl_id": 47, + "start": 1695396338387702199, + "end": 1695396338398813686, + "rtt": 11.111424, + "err_desc": "" + }, + { + "wl_id": 48, + "start": 1695396338488016588, + "end": 1695396338498992684, + "rtt": 10.976, + "err_desc": "" + }, + { + "wl_id": 49, + "start": 1695396338588255317, + "end": 1695396338599282066, + "rtt": 11.026944, + "err_desc": "" + }, + { + "wl_id": 50, + "start": 1695396338688535631, + "end": 1695396338699532969, + "rtt": 10.997504, + "err_desc": "" + }, + { + "wl_id": 51, + "start": 1695396338788710825, + "end": 1695396338799748493, + "rtt": 11.037696, + "err_desc": "" + }, + { + "wl_id": 52, + "start": 1695396338888047800, + "end": 1695396338899280619, + "rtt": 11.232768, + "err_desc": "" + }, + { + "wl_id": 53, + "start": 1695396338988507789, + "end": 1695396338999097765, + "rtt": 10.589952, + "err_desc": "" + }, + { + "wl_id": 54, + "start": 1695396339088439956, + "end": 1695396339099746234, + "rtt": 11.30624, + "err_desc": "" + }, + { + "wl_id": 55, + "start": 1695396339188031636, + "end": 1695396339198756235, + "rtt": 10.724608, + "err_desc": "" + }, + { + "wl_id": 56, + "start": 1695396339288006368, + "end": 1695396339298731132, + "rtt": 10.724608, + "err_desc": "" + }, + { + "wl_id": 57, + "start": 1695396339388041551, + "end": 1695396339398764170, + "rtt": 10.722816, + "err_desc": "" + }, + { + "wl_id": 58, + "start": 1695396339488035475, + "end": 1695396339499788366, + "rtt": 11.752704, + "err_desc": "" + }, + { + "wl_id": 59, + "start": 1695396339588045519, + "end": 1695396339599583997, + "rtt": 11.538432, + "err_desc": "" + }, + { + "wl_id": 60, + "start": 1695396339687674150, + "end": 1695396339698177284, + "rtt": 10.503168, + "err_desc": "" + }, + { + "wl_id": 61, + "start": 1695396339788502375, + "end": 1695396339799039503, + "rtt": 10.537216, + "err_desc": "" + }, + { + "wl_id": 62, + "start": 1695396339888304230, + "end": 1695396339899452082, + "rtt": 11.148032, + "err_desc": "" + }, + { + "wl_id": 63, + "start": 1695396339988665272, + "end": 1695396339996479476, + "rtt": 7.814144, + "err_desc": "" + }, + { + "wl_id": 64, + "start": 1695396340087827265, + "end": 1695396340098682882, + "rtt": 10.85568, + "err_desc": "" + }, + { + "wl_id": 65, + "start": 1695396340187865756, + "end": 1695396340198605332, + "rtt": 10.739456, + "err_desc": "" + }, + { + "wl_id": 66, + "start": 1695396340288055509, + "end": 1695396340299077087, + "rtt": 11.021568, + "err_desc": "" + }, + { + "wl_id": 67, + "start": 1695396340388383125, + "end": 1695396340399942007, + "rtt": 11.558656, + "err_desc": "" + }, + { + "wl_id": 68, + "start": 1695396340488434604, + "end": 1695396340499482392, + "rtt": 11.04768, + "err_desc": "" + }, + { + "wl_id": 69, + "start": 1695396340588805944, + "end": 1695396340600175266, + "rtt": 11.369472, + "err_desc": "" + }, + { + "wl_id": 70, + "start": 1695396340688507282, + "end": 1695396340698793835, + "rtt": 10.286336, + "err_desc": "" + }, + { + "wl_id": 71, + "start": 1695396340788409217, + "end": 1695396341110609792, + "rtt": 322.200576, + "err_desc": "RequestError: send request failed\ncaused by: Put \"http://s3gw-ha-s3gw-ha.s3gw-ha.svc.cluster.local/test-1/objTest\": dial tcp 10.43.2.234:80: connect: connection refused" + }, + { + "wl_id": 72, + "start": 1695396341110616354, + "end": 1695396342185439110, + "rtt": 1074.822912, + "err_desc": "" + }, + { + "wl_id": 73, + "start": 1695396342185446741, + "end": 1695396342195269373, + "rtt": 9.82272, + "err_desc": "" + }, + { + "wl_id": 74, + "start": 1695396342195272587, + "end": 1695396342204107565, + "rtt": 8.834816, + "err_desc": "" + }, + { + "wl_id": 75, + "start": 1695396342288394762, + "end": 1695396342299006692, + "rtt": 10.611968, + "err_desc": "" + }, + { + "wl_id": 76, + "start": 1695396342387918946, + "end": 1695396342398484975, + "rtt": 10.566144, + "err_desc": "" + }, + { + "wl_id": 77, + "start": 1695396342488622753, + "end": 1695396342499410237, + "rtt": 10.787328, + "err_desc": "" + }, + { + "wl_id": 78, + "start": 1695396342588647155, + "end": 1695396342599984033, + "rtt": 11.33696, + "err_desc": "" + }, + { + "wl_id": 79, + "start": 1695396342688197628, + "end": 1695396342698728764, + "rtt": 10.531072, + "err_desc": "" + }, + { + "wl_id": 80, + "start": 1695396342787949943, + "end": 1695396342794321205, + "rtt": 6.371328, + "err_desc": "" + }, + { + "wl_id": 81, + "start": 1695396342888720749, + "end": 1695396342903969163, + "rtt": 15.24864, + "err_desc": "" + }, + { + "wl_id": 82, + "start": 1695396342988220349, + "end": 1695396342999421285, + "rtt": 11.200768, + "err_desc": "" + }, + { + "wl_id": 83, + "start": 1695396343088766114, + "end": 1695396343099900036, + "rtt": 11.133952, + "err_desc": "" + }, + { + "wl_id": 84, + "start": 1695396343188138552, + "end": 1695396343200049052, + "rtt": 11.910656, + "err_desc": "" + }, + { + "wl_id": 85, + "start": 1695396343288360458, + "end": 1695396343299571986, + "rtt": 11.21152, + "err_desc": "" + }, + { + "wl_id": 86, + "start": 1695396343387740678, + "end": 1695396343398166997, + "rtt": 10.426368, + "err_desc": "" + }, + { + "wl_id": 87, + "start": 1695396343488457919, + "end": 1695396343498951877, + "rtt": 10.493952, + "err_desc": "" + }, + { + "wl_id": 88, + "start": 1695396343588395565, + "end": 1695396343599908454, + "rtt": 11.512832, + "err_desc": "" + }, + { + "wl_id": 89, + "start": 1695396343688184942, + "end": 1695396343699550577, + "rtt": 11.365632, + "err_desc": "" + }, + { + "wl_id": 90, + "start": 1695396343787694233, + "end": 1695396343793844173, + "rtt": 6.149888, + "err_desc": "" + }, + { + "wl_id": 91, + "start": 1695396343888131264, + "end": 1695396343905414694, + "rtt": 17.283328, + "err_desc": "" + }, + { + "wl_id": 92, + "start": 1695396343987675395, + "end": 1695396343998514788, + "rtt": 10.839296, + "err_desc": "" + }, + { + "wl_id": 93, + "start": 1695396344088719498, + "end": 1695396344100295241, + "rtt": 11.575552, + "err_desc": "" + }, + { + "wl_id": 94, + "start": 1695396344188587678, + "end": 1695396344199566166, + "rtt": 10.978304, + "err_desc": "" + }, + { + "wl_id": 95, + "start": 1695396344287853875, + "end": 1695396344298101671, + "rtt": 10.247936, + "err_desc": "" + }, + { + "wl_id": 96, + "start": 1695396344388687907, + "end": 1695396344399797040, + "rtt": 11.10912, + "err_desc": "" + }, + { + "wl_id": 97, + "start": 1695396344487977995, + "end": 1695396344498810794, + "rtt": 10.832896, + "err_desc": "" + }, + { + "wl_id": 98, + "start": 1695396344588060735, + "end": 1695396344599547075, + "rtt": 11.486464, + "err_desc": "" + }, + { + "wl_id": 99, + "start": 1695396344687720884, + "end": 1695396344698479794, + "rtt": 10.758912, + "err_desc": "" + }, + { + "wl_id": 100, + "start": 1695396344788773220, + "end": 1695396344800370243, + "rtt": 11.597056, + "err_desc": "" + }, + { + "wl_id": 101, + "start": 1695396344888581966, + "end": 1695396344900361957, + "rtt": 11.780096, + "err_desc": "" + }, + { + "wl_id": 102, + "start": 1695396344988606074, + "end": 1695396344999764119, + "rtt": 11.158272, + "err_desc": "" + }, + { + "wl_id": 103, + "start": 1695396345087985364, + "end": 1695396345098797226, + "rtt": 10.811904, + "err_desc": "" + }, + { + "wl_id": 104, + "start": 1695396345188155502, + "end": 1695396345199055438, + "rtt": 10.899968, + "err_desc": "" + }, + { + "wl_id": 105, + "start": 1695396345288310548, + "end": 1695396345301215996, + "rtt": 12.905472, + "err_desc": "" + }, + { + "wl_id": 106, + "start": 1695396345388588521, + "end": 1695396345399795438, + "rtt": 11.206912, + "err_desc": "" + }, + { + "wl_id": 107, + "start": 1695396345487966861, + "end": 1695396345497521092, + "rtt": 9.554176, + "err_desc": "" + }, + { + "wl_id": 108, + "start": 1695396345587977169, + "end": 1695396345598604612, + "rtt": 10.627328, + "err_desc": "" + }, + { + "wl_id": 109, + "start": 1695396345687677444, + "end": 1695396345698141562, + "rtt": 10.464, + "err_desc": "" + }, + { + "wl_id": 110, + "start": 1695396345788490858, + "end": 1695396345799412040, + "rtt": 10.921216, + "err_desc": "" + }, + { + "wl_id": 111, + "start": 1695396345887667262, + "end": 1695396345898565162, + "rtt": 10.89792, + "err_desc": "" + }, + { + "wl_id": 112, + "start": 1695396345988763990, + "end": 1695396345999392621, + "rtt": 10.628608, + "err_desc": "" + }, + { + "wl_id": 113, + "start": 1695396346088720666, + "end": 1695396346100816463, + "rtt": 12.095744, + "err_desc": "" + }, + { + "wl_id": 114, + "start": 1695396346188122069, + "end": 1695396346198896327, + "rtt": 10.774272, + "err_desc": "" + }, + { + "wl_id": 115, + "start": 1695396346288189804, + "end": 1695396346299484734, + "rtt": 11.294976, + "err_desc": "" + }, + { + "wl_id": 116, + "start": 1695396346387704891, + "end": 1695396346398746425, + "rtt": 11.041536, + "err_desc": "" + }, + { + "wl_id": 117, + "start": 1695396346487906837, + "end": 1695396346498317415, + "rtt": 10.410496, + "err_desc": "" + }, + { + "wl_id": 118, + "start": 1695396346587871950, + "end": 1695396346911350167, + "rtt": 323.478272, + "err_desc": "RequestError: send request failed\ncaused by: Put \"http://s3gw-ha-s3gw-ha.s3gw-ha.svc.cluster.local/test-1/objTest\": dial tcp 10.43.2.234:80: connect: connection refused" + }, + { + "wl_id": 119, + "start": 1695396346911355899, + "end": 1695396348209908375, + "rtt": 1298.552576, + "err_desc": "" + }, + { + "wl_id": 120, + "start": 1695396348209915228, + "end": 1695396348218737741, + "rtt": 8.822528, + "err_desc": "" + }, + { + "wl_id": 121, + "start": 1695396348288109061, + "end": 1695396348298618191, + "rtt": 10.509056, + "err_desc": "" + }, + { + "wl_id": 122, + "start": 1695396348387713164, + "end": 1695396348398566337, + "rtt": 10.85312, + "err_desc": "" + }, + { + "wl_id": 123, + "start": 1695396348487969422, + "end": 1695396348497715258, + "rtt": 9.745664, + "err_desc": "" + }, + { + "wl_id": 124, + "start": 1695396348588065776, + "end": 1695396348594685032, + "rtt": 6.619136, + "err_desc": "" + }, + { + "wl_id": 125, + "start": 1695396348688092090, + "end": 1695396348696822736, + "rtt": 8.730624, + "err_desc": "" + }, + { + "wl_id": 126, + "start": 1695396348788136908, + "end": 1695396348798822944, + "rtt": 10.685952, + "err_desc": "" + }, + { + "wl_id": 127, + "start": 1695396348888192266, + "end": 1695396348898987315, + "rtt": 10.795008, + "err_desc": "" + }, + { + "wl_id": 128, + "start": 1695396348988313406, + "end": 1695396348998943583, + "rtt": 10.630144, + "err_desc": "" + }, + { + "wl_id": 129, + "start": 1695396349088275856, + "end": 1695396349098901131, + "rtt": 10.62528, + "err_desc": "" + }, + { + "wl_id": 130, + "start": 1695396349188096272, + "end": 1695396349198844384, + "rtt": 10.74816, + "err_desc": "" + }, + { + "wl_id": 131, + "start": 1695396349288121742, + "end": 1695396349299032138, + "rtt": 10.910208, + "err_desc": "" + }, + { + "wl_id": 132, + "start": 1695396349388398118, + "end": 1695396349399365362, + "rtt": 10.967296, + "err_desc": "" + }, + { + "wl_id": 133, + "start": 1695396349488577397, + "end": 1695396349498929299, + "rtt": 10.352128, + "err_desc": "" + }, + { + "wl_id": 134, + "start": 1695396349588108777, + "end": 1695396349599180336, + "rtt": 11.071488, + "err_desc": "" + }, + { + "wl_id": 135, + "start": 1695396349688412589, + "end": 1695396349699113974, + "rtt": 10.701312, + "err_desc": "" + }, + { + "wl_id": 136, + "start": 1695396349788441533, + "end": 1695396349799804436, + "rtt": 11.362816, + "err_desc": "" + }, + { + "wl_id": 137, + "start": 1695396349888035473, + "end": 1695396349898708979, + "rtt": 10.673408, + "err_desc": "" + }, + { + "wl_id": 138, + "start": 1695396349988029477, + "end": 1695396349998631955, + "rtt": 10.602496, + "err_desc": "" + }, + { + "wl_id": 139, + "start": 1695396350087794096, + "end": 1695396350098071578, + "rtt": 10.277376, + "err_desc": "" + }, + { + "wl_id": 140, + "start": 1695396350188444243, + "end": 1695396350199453118, + "rtt": 11.009024, + "err_desc": "" + }, + { + "wl_id": 141, + "start": 1695396350287702001, + "end": 1695396350298322252, + "rtt": 10.62016, + "err_desc": "" + }, + { + "wl_id": 142, + "start": 1695396350388734235, + "end": 1695396350400124401, + "rtt": 11.390208, + "err_desc": "" + }, + { + "wl_id": 143, + "start": 1695396350488323983, + "end": 1695396350499368791, + "rtt": 11.044608, + "err_desc": "" + }, + { + "wl_id": 144, + "start": 1695396350587702667, + "end": 1695396350598340785, + "rtt": 10.63808, + "err_desc": "" + }, + { + "wl_id": 145, + "start": 1695396350688506165, + "end": 1695396350699421290, + "rtt": 10.915072, + "err_desc": "" + }, + { + "wl_id": 146, + "start": 1695396350787683580, + "end": 1695396350798791653, + "rtt": 11.108096, + "err_desc": "" + }, + { + "wl_id": 147, + "start": 1695396350888040350, + "end": 1695396350894236689, + "rtt": 6.196224, + "err_desc": "" + }, + { + "wl_id": 148, + "start": 1695396350988575354, + "end": 1695396351004464763, + "rtt": 15.889408, + "err_desc": "" + }, + { + "wl_id": 149, + "start": 1695396351087735228, + "end": 1695396351098469968, + "rtt": 10.734592, + "err_desc": "" + }, + { + "wl_id": 150, + "start": 1695396351188613602, + "end": 1695396351198627303, + "rtt": 10.013696, + "err_desc": "" + }, + { + "wl_id": 151, + "start": 1695396351287982099, + "end": 1695396351296790835, + "rtt": 8.808704, + "err_desc": "" + }, + { + "wl_id": 152, + "start": 1695396351388085542, + "end": 1695396351398499438, + "rtt": 10.413824, + "err_desc": "" + }, + { + "wl_id": 153, + "start": 1695396351488660353, + "end": 1695396351499125653, + "rtt": 10.46528, + "err_desc": "" + }, + { + "wl_id": 154, + "start": 1695396351588521129, + "end": 1695396351598897010, + "rtt": 10.37568, + "err_desc": "" + }, + { + "wl_id": 155, + "start": 1695396351688269100, + "end": 1695396351699118991, + "rtt": 10.850048, + "err_desc": "" + }, + { + "wl_id": 156, + "start": 1695396351788505778, + "end": 1695396351799658275, + "rtt": 11.152384, + "err_desc": "" + }, + { + "wl_id": 157, + "start": 1695396351888017867, + "end": 1695396351898773815, + "rtt": 10.75584, + "err_desc": "" + }, + { + "wl_id": 158, + "start": 1695396351988365617, + "end": 1695396351999019361, + "rtt": 10.653696, + "err_desc": "" + }, + { + "wl_id": 159, + "start": 1695396352088312936, + "end": 1695396352099215071, + "rtt": 10.902272, + "err_desc": "" + }, + { + "wl_id": 160, + "start": 1695396352188529931, + "end": 1695396352199273339, + "rtt": 10.743296, + "err_desc": "" + }, + { + "wl_id": 161, + "start": 1695396352288587313, + "end": 1695396352299398843, + "rtt": 10.811648, + "err_desc": "" + }, + { + "wl_id": 162, + "start": 1695396352387702706, + "end": 1695396352398229692, + "rtt": 10.526976, + "err_desc": "" + }, + { + "wl_id": 163, + "start": 1695396352488284941, + "end": 1695396352799746450, + "rtt": 311.461632, + "err_desc": "RequestError: send request failed\ncaused by: Put \"http://s3gw-ha-s3gw-ha.s3gw-ha.svc.cluster.local/test-1/objTest\": dial tcp 10.43.2.234:80: connect: connection refused" + }, + { + "wl_id": 164, + "start": 1695396352799754579, + "end": 1695396353888195493, + "rtt": 1088.441088, + "err_desc": "" + }, + { + "wl_id": 165, + "start": 1695396353888202218, + "end": 1695396353897271503, + "rtt": 9.069312, + "err_desc": "" + }, + { + "wl_id": 166, + "start": 1695396353988504502, + "end": 1695396353999272899, + "rtt": 10.768384, + "err_desc": "" + }, + { + "wl_id": 167, + "start": 1695396354088172583, + "end": 1695396354098785988, + "rtt": 10.613504, + "err_desc": "" + }, + { + "wl_id": 168, + "start": 1695396354188032348, + "end": 1695396354199021030, + "rtt": 10.9888, + "err_desc": "" + }, + { + "wl_id": 169, + "start": 1695396354288269110, + "end": 1695396354299011237, + "rtt": 10.742272, + "err_desc": "" + }, + { + "wl_id": 170, + "start": 1695396354388304445, + "end": 1695396354399593435, + "rtt": 11.289088, + "err_desc": "" + }, + { + "wl_id": 171, + "start": 1695396354487698991, + "end": 1695396354493878481, + "rtt": 6.179584, + "err_desc": "" + }, + { + "wl_id": 172, + "start": 1695396354588178417, + "end": 1695396354604784829, + "rtt": 16.606464, + "err_desc": "" + }, + { + "wl_id": 173, + "start": 1695396354688051171, + "end": 1695396354696884381, + "rtt": 8.83328, + "err_desc": "" + }, + { + "wl_id": 174, + "start": 1695396354788182429, + "end": 1695396354798046631, + "rtt": 9.864192, + "err_desc": "" + }, + { + "wl_id": 175, + "start": 1695396354888341907, + "end": 1695396354899286043, + "rtt": 10.944, + "err_desc": "" + }, + { + "wl_id": 176, + "start": 1695396354988595628, + "end": 1695396354999694403, + "rtt": 11.098624, + "err_desc": "" + }, + { + "wl_id": 177, + "start": 1695396355087999860, + "end": 1695396355098807117, + "rtt": 10.807296, + "err_desc": "" + }, + { + "wl_id": 178, + "start": 1695396355188099431, + "end": 1695396355196819840, + "rtt": 8.72064, + "err_desc": "" + }, + { + "wl_id": 179, + "start": 1695396355288133885, + "end": 1695396355298879564, + "rtt": 10.7456, + "err_desc": "" + }, + { + "wl_id": 180, + "start": 1695396355388120471, + "end": 1695396355399334632, + "rtt": 11.21408, + "err_desc": "" + }, + { + "wl_id": 181, + "start": 1695396355488593946, + "end": 1695396355498971141, + "rtt": 10.377216, + "err_desc": "" + }, + { + "wl_id": 182, + "start": 1695396355588323552, + "end": 1695396355599726138, + "rtt": 11.402496, + "err_desc": "" + }, + { + "wl_id": 183, + "start": 1695396355688075988, + "end": 1695396355698838568, + "rtt": 10.762496, + "err_desc": "" + }, + { + "wl_id": 184, + "start": 1695396355788137415, + "end": 1695396355799095766, + "rtt": 10.958336, + "err_desc": "" + }, + { + "wl_id": 185, + "start": 1695396355888413512, + "end": 1695396355899127956, + "rtt": 10.714624, + "err_desc": "" + }, + { + "wl_id": 186, + "start": 1695396355988499356, + "end": 1695396355999180378, + "rtt": 10.680832, + "err_desc": "" + }, + { + "wl_id": 187, + "start": 1695396356088445027, + "end": 1695396356099252883, + "rtt": 10.808064, + "err_desc": "" + }, + { + "wl_id": 188, + "start": 1695396356188490555, + "end": 1695396356199631212, + "rtt": 11.140608, + "err_desc": "" + }, + { + "wl_id": 189, + "start": 1695396356287727638, + "end": 1695396356298681225, + "rtt": 10.953728, + "err_desc": "" + }, + { + "wl_id": 190, + "start": 1695396356387855986, + "end": 1695396356399367898, + "rtt": 11.512064, + "err_desc": "" + }, + { + "wl_id": 191, + "start": 1695396356488510282, + "end": 1695396356500023726, + "rtt": 11.5136, + "err_desc": "" + }, + { + "wl_id": 192, + "start": 1695396356588274367, + "end": 1695396356599062732, + "rtt": 10.788352, + "err_desc": "" + }, + { + "wl_id": 193, + "start": 1695396356688327257, + "end": 1695396356698976584, + "rtt": 10.649344, + "err_desc": "" + }, + { + "wl_id": 194, + "start": 1695396356788251627, + "end": 1695396356798652412, + "rtt": 10.400768, + "err_desc": "" + }, + { + "wl_id": 195, + "start": 1695396356887823317, + "end": 1695396356898465867, + "rtt": 10.642432, + "err_desc": "" + }, + { + "wl_id": 196, + "start": 1695396356988611496, + "end": 1695396356999233198, + "rtt": 10.621696, + "err_desc": "" + }, + { + "wl_id": 197, + "start": 1695396357088513572, + "end": 1695396357099688066, + "rtt": 11.174656, + "err_desc": "" + }, + { + "wl_id": 198, + "start": 1695396357187821695, + "end": 1695396357198305478, + "rtt": 10.483968, + "err_desc": "" + }, + { + "wl_id": 199, + "start": 1695396357288635990, + "end": 1695396357299528168, + "rtt": 10.892288, + "err_desc": "" + }, + { + "wl_id": 200, + "start": 1695396357387712979, + "end": 1695396357398290974, + "rtt": 10.57792, + "err_desc": "" + }, + { + "wl_id": 201, + "start": 1695396357488571700, + "end": 1695396357499782034, + "rtt": 11.210496, + "err_desc": "" + }, + { + "wl_id": 202, + "start": 1695396357588057615, + "end": 1695396357598414743, + "rtt": 10.357248, + "err_desc": "" + }, + { + "wl_id": 203, + "start": 1695396357688730754, + "end": 1695396357699274639, + "rtt": 10.543872, + "err_desc": "" + }, + { + "wl_id": 204, + "start": 1695396357788614174, + "end": 1695396357799731190, + "rtt": 11.117056, + "err_desc": "" + }, + { + "wl_id": 205, + "start": 1695396357887944816, + "end": 1695396357898420129, + "rtt": 10.47552, + "err_desc": "" + }, + { + "wl_id": 206, + "start": 1695396357987761378, + "end": 1695396357998350153, + "rtt": 10.588672, + "err_desc": "" + }, + { + "wl_id": 207, + "start": 1695396358088662864, + "end": 1695396358099900874, + "rtt": 11.238144, + "err_desc": "" + }, + { + "wl_id": 208, + "start": 1695396358188154149, + "end": 1695396358198821238, + "rtt": 10.667008, + "err_desc": "" + }, + { + "wl_id": 209, + "start": 1695396358288573234, + "end": 1695396358571253795, + "rtt": 282.680576, + "err_desc": "RequestError: send request failed\ncaused by: Put \"http://s3gw-ha-s3gw-ha.s3gw-ha.svc.cluster.local/test-1/objTest\": dial tcp 10.43.2.234:80: connect: connection refused" + }, + { + "wl_id": 210, + "start": 1695396358571260087, + "end": 1695396359755581378, + "rtt": 1184.32128, + "err_desc": "" + }, + { + "wl_id": 211, + "start": 1695396359755588523, + "end": 1695396359764200315, + "rtt": 8.611584, + "err_desc": "" + }, + { + "wl_id": 212, + "start": 1695396359788360694, + "end": 1695396359798115639, + "rtt": 9.75488, + "err_desc": "" + }, + { + "wl_id": 213, + "start": 1695396359888391559, + "end": 1695396359898952309, + "rtt": 10.560512, + "err_desc": "" + }, + { + "wl_id": 214, + "start": 1695396359987934215, + "end": 1695396359997697199, + "rtt": 9.763072, + "err_desc": "" + }, + { + "wl_id": 215, + "start": 1695396360088068095, + "end": 1695396360098746390, + "rtt": 10.678272, + "err_desc": "" + }, + { + "wl_id": 216, + "start": 1695396360187909722, + "end": 1695396360198267144, + "rtt": 10.357504, + "err_desc": "" + }, + { + "wl_id": 217, + "start": 1695396360288550595, + "end": 1695396360294500930, + "rtt": 5.950208, + "err_desc": "" + }, + { + "wl_id": 218, + "start": 1695396360387666547, + "end": 1695396360393846240, + "rtt": 6.17984, + "err_desc": "" + }, + { + "wl_id": 219, + "start": 1695396360488112778, + "end": 1695396360504539523, + "rtt": 16.426752, + "err_desc": "" + }, + { + "wl_id": 220, + "start": 1695396360588692629, + "end": 1695396360599883206, + "rtt": 11.190528, + "err_desc": "" + }, + { + "wl_id": 221, + "start": 1695396360688176098, + "end": 1695396360698257420, + "rtt": 10.08128, + "err_desc": "" + }, + { + "wl_id": 222, + "start": 1695396360788616836, + "end": 1695396360799689315, + "rtt": 11.072256, + "err_desc": "" + }, + { + "wl_id": 223, + "start": 1695396360887839348, + "end": 1695396360893755174, + "rtt": 5.915904, + "err_desc": "" + }, + { + "wl_id": 224, + "start": 1695396360987938822, + "end": 1695396361004563476, + "rtt": 16.62464, + "err_desc": "" + }, + { + "wl_id": 225, + "start": 1695396361087821199, + "end": 1695396361098799870, + "rtt": 10.97856, + "err_desc": "" + }, + { + "wl_id": 226, + "start": 1695396361188146954, + "end": 1695396361199178687, + "rtt": 11.031808, + "err_desc": "" + }, + { + "wl_id": 227, + "start": 1695396361288500816, + "end": 1695396361297404224, + "rtt": 8.903424, + "err_desc": "" + }, + { + "wl_id": 228, + "start": 1695396361388698994, + "end": 1695396361399314328, + "rtt": 10.615552, + "err_desc": "" + }, + { + "wl_id": 229, + "start": 1695396361488710665, + "end": 1695396361498925177, + "rtt": 10.2144, + "err_desc": "" + }, + { + "wl_id": 230, + "start": 1695396361588193653, + "end": 1695396361599473067, + "rtt": 11.279616, + "err_desc": "" + }, + { + "wl_id": 231, + "start": 1695396361688605061, + "end": 1695396361699130032, + "rtt": 10.524928, + "err_desc": "" + }, + { + "wl_id": 232, + "start": 1695396361788503069, + "end": 1695396361798939112, + "rtt": 10.436096, + "err_desc": "" + }, + { + "wl_id": 233, + "start": 1695396361888239603, + "end": 1695396361899262177, + "rtt": 11.022592, + "err_desc": "" + }, + { + "wl_id": 234, + "start": 1695396361988580782, + "end": 1695396361999537898, + "rtt": 10.957056, + "err_desc": "" + }, + { + "wl_id": 235, + "start": 1695396362087760043, + "end": 1695396362099109619, + "rtt": 11.349504, + "err_desc": "" + }, + { + "wl_id": 236, + "start": 1695396362188419196, + "end": 1695396362199217220, + "rtt": 10.79808, + "err_desc": "" + }, + { + "wl_id": 237, + "start": 1695396362288520344, + "end": 1695396362299758183, + "rtt": 11.237632, + "err_desc": "" + }, + { + "wl_id": 238, + "start": 1695396362388065217, + "end": 1695396362398861589, + "rtt": 10.796288, + "err_desc": "" + }, + { + "wl_id": 239, + "start": 1695396362488091677, + "end": 1695396362498686736, + "rtt": 10.595072, + "err_desc": "" + }, + { + "wl_id": 240, + "start": 1695396362588086651, + "end": 1695396362597030171, + "rtt": 8.943616, + "err_desc": "" + }, + { + "wl_id": 241, + "start": 1695396362688210263, + "end": 1695396362694342550, + "rtt": 6.13248, + "err_desc": "" + }, + { + "wl_id": 242, + "start": 1695396362788562444, + "end": 1695396362804853688, + "rtt": 16.291328, + "err_desc": "" + }, + { + "wl_id": 243, + "start": 1695396362888107628, + "end": 1695396362900914162, + "rtt": 12.806656, + "err_desc": "" + }, + { + "wl_id": 244, + "start": 1695396362988150482, + "end": 1695396362998886157, + "rtt": 10.735616, + "err_desc": "" + }, + { + "wl_id": 245, + "start": 1695396363088195734, + "end": 1695396363099114509, + "rtt": 10.918656, + "err_desc": "" + }, + { + "wl_id": 246, + "start": 1695396363188439398, + "end": 1695396363198723743, + "rtt": 10.284544, + "err_desc": "" + }, + { + "wl_id": 247, + "start": 1695396363287991872, + "end": 1695396363299059375, + "rtt": 11.067648, + "err_desc": "" + }, + { + "wl_id": 248, + "start": 1695396363388331836, + "end": 1695396363399493807, + "rtt": 11.162112, + "err_desc": "" + }, + { + "wl_id": 249, + "start": 1695396363488706205, + "end": 1695396363497772524, + "rtt": 9.06624, + "err_desc": "" + }, + { + "wl_id": 250, + "start": 1695396363587998073, + "end": 1695396363598982970, + "rtt": 10.98496, + "err_desc": "" + }, + { + "wl_id": 251, + "start": 1695396363688242125, + "end": 1695396363697985903, + "rtt": 9.743616, + "err_desc": "" + }, + { + "wl_id": 252, + "start": 1695396363788262014, + "end": 1695396363799630550, + "rtt": 11.368704, + "err_desc": "" + }, + { + "wl_id": 253, + "start": 1695396363887776021, + "end": 1695396363898565861, + "rtt": 10.789888, + "err_desc": "" + }, + { + "wl_id": 254, + "start": 1695396363987667588, + "end": 1695396363998152748, + "rtt": 10.484992, + "err_desc": "" + }, + { + "wl_id": 255, + "start": 1695396364088451259, + "end": 1695396364099151379, + "rtt": 10.700032, + "err_desc": "" + }, + { + "wl_id": 256, + "start": 1695396364187727547, + "end": 1695396364569896982, + "rtt": 382.169344, + "err_desc": "RequestError: send request failed\ncaused by: Put \"http://s3gw-ha-s3gw-ha.s3gw-ha.svc.cluster.local/test-1/objTest\": dial tcp 10.43.2.234:80: connect: connection refused" + }, + { + "wl_id": 257, + "start": 1695396364569903297, + "end": 1695396364741087079, + "rtt": 171.183616, + "err_desc": "" + }, + { + "wl_id": 258, + "start": 1695396364741092045, + "end": 1695396364748697425, + "rtt": 7.605248, + "err_desc": "" + }, + { + "wl_id": 259, + "start": 1695396364787886352, + "end": 1695396364793762090, + "rtt": 5.875712, + "err_desc": "" + }, + { + "wl_id": 260, + "start": 1695396364888103787, + "end": 1695396364903355359, + "rtt": 15.251712, + "err_desc": "" + }, + { + "wl_id": 261, + "start": 1695396364987939304, + "end": 1695396365000196066, + "rtt": 12.256768, + "err_desc": "" + }, + { + "wl_id": 262, + "start": 1695396365088446308, + "end": 1695396365099536419, + "rtt": 11.090176, + "err_desc": "" + }, + { + "wl_id": 263, + "start": 1695396365187667076, + "end": 1695396365198505203, + "rtt": 10.838016, + "err_desc": "" + }, + { + "wl_id": 264, + "start": 1695396365287725381, + "end": 1695396365299064197, + "rtt": 11.339008, + "err_desc": "" + }, + { + "wl_id": 265, + "start": 1695396365388442312, + "end": 1695396365399260802, + "rtt": 10.81856, + "err_desc": "" + }, + { + "wl_id": 266, + "start": 1695396365488446204, + "end": 1695396365499627902, + "rtt": 11.181568, + "err_desc": "" + }, + { + "wl_id": 267, + "start": 1695396365587777512, + "end": 1695396365598847789, + "rtt": 11.070208, + "err_desc": "" + }, + { + "wl_id": 268, + "start": 1695396365688058462, + "end": 1695396365695733051, + "rtt": 7.674624, + "err_desc": "" + }, + { + "wl_id": 269, + "start": 1695396365788743976, + "end": 1695396365799263594, + "rtt": 10.519552, + "err_desc": "" + }, + { + "wl_id": 270, + "start": 1695396365888548967, + "end": 1695396365899779380, + "rtt": 11.230464, + "err_desc": "" + }, + { + "wl_id": 271, + "start": 1695396365988215626, + "end": 1695396365998418834, + "rtt": 10.203392, + "err_desc": "" + }, + { + "wl_id": 272, + "start": 1695396366088731849, + "end": 1695396366100876697, + "rtt": 12.144896, + "err_desc": "" + }, + { + "wl_id": 273, + "start": 1695396366188008295, + "end": 1695396366194467680, + "rtt": 6.459392, + "err_desc": "" + }, + { + "wl_id": 274, + "start": 1695396366287785219, + "end": 1695396366299741111, + "rtt": 11.955968, + "err_desc": "" + }, + { + "wl_id": 275, + "start": 1695396366388069115, + "end": 1695396366399536251, + "rtt": 11.467008, + "err_desc": "" + }, + { + "wl_id": 276, + "start": 1695396366488704133, + "end": 1695396366497126947, + "rtt": 8.422656, + "err_desc": "" + }, + { + "wl_id": 277, + "start": 1695396366588517993, + "end": 1695396366599946684, + "rtt": 11.428864, + "err_desc": "" + }, + { + "wl_id": 278, + "start": 1695396366688183957, + "end": 1695396366698820650, + "rtt": 10.636544, + "err_desc": "" + }, + { + "wl_id": 279, + "start": 1695396366788300631, + "end": 1695396366799041351, + "rtt": 10.740736, + "err_desc": "" + }, + { + "wl_id": 280, + "start": 1695396366888372047, + "end": 1695396366898619718, + "rtt": 10.24768, + "err_desc": "" + }, + { + "wl_id": 281, + "start": 1695396366987822118, + "end": 1695396366998536319, + "rtt": 10.714112, + "err_desc": "" + }, + { + "wl_id": 282, + "start": 1695396367087762289, + "end": 1695396367098424836, + "rtt": 10.662656, + "err_desc": "" + }, + { + "wl_id": 283, + "start": 1695396367188743359, + "end": 1695396367200043589, + "rtt": 11.300096, + "err_desc": "" + }, + { + "wl_id": 284, + "start": 1695396367288405635, + "end": 1695396367299024793, + "rtt": 10.619136, + "err_desc": "" + }, + { + "wl_id": 285, + "start": 1695396367388280421, + "end": 1695396367398968218, + "rtt": 10.688, + "err_desc": "" + }, + { + "wl_id": 286, + "start": 1695396367488178476, + "end": 1695396367499160335, + "rtt": 10.981888, + "err_desc": "" + }, + { + "wl_id": 287, + "start": 1695396367588539157, + "end": 1695396367599577766, + "rtt": 11.03872, + "err_desc": "" + }, + { + "wl_id": 288, + "start": 1695396367687798699, + "end": 1695396367700303440, + "rtt": 12.504576, + "err_desc": "" + }, + { + "wl_id": 289, + "start": 1695396367788647994, + "end": 1695396367800136255, + "rtt": 11.488256, + "err_desc": "" + }, + { + "wl_id": 290, + "start": 1695396367888519361, + "end": 1695396367899828696, + "rtt": 11.309312, + "err_desc": "" + }, + { + "wl_id": 291, + "start": 1695396367988029108, + "end": 1695396367998702262, + "rtt": 10.673152, + "err_desc": "" + }, + { + "wl_id": 292, + "start": 1695396368087970922, + "end": 1695396368099403311, + "rtt": 11.432448, + "err_desc": "" + }, + { + "wl_id": 293, + "start": 1695396368187689986, + "end": 1695396368198501806, + "rtt": 10.811904, + "err_desc": "" + }, + { + "wl_id": 294, + "start": 1695396368288710002, + "end": 1695396368299960493, + "rtt": 11.250688, + "err_desc": "" + }, + { + "wl_id": 295, + "start": 1695396368388287056, + "end": 1695396368399498258, + "rtt": 11.211264, + "err_desc": "" + }, + { + "wl_id": 296, + "start": 1695396368487678336, + "end": 1695396368499264332, + "rtt": 11.585792, + "err_desc": "" + }, + { + "wl_id": 297, + "start": 1695396368588589812, + "end": 1695396368599266172, + "rtt": 10.676224, + "err_desc": "" + }, + { + "wl_id": 298, + "start": 1695396368688560897, + "end": 1695396368700448804, + "rtt": 11.887872, + "err_desc": "" + }, + { + "wl_id": 299, + "start": 1695396368788671871, + "end": 1695396368798507106, + "rtt": 9.835264, + "err_desc": "" + }, + { + "wl_id": 300, + "start": 1695396368888657900, + "end": 1695396368899144923, + "rtt": 10.48704, + "err_desc": "" + }, + { + "wl_id": 301, + "start": 1695396368988491039, + "end": 1695396368999436559, + "rtt": 10.945536, + "err_desc": "" + }, + { + "wl_id": 302, + "start": 1695396369088625763, + "end": 1695396369099615172, + "rtt": 10.989568, + "err_desc": "" + }, + { + "wl_id": 303, + "start": 1695396369187943118, + "end": 1695396369198644409, + "rtt": 10.701312, + "err_desc": "" + }, + { + "wl_id": 304, + "start": 1695396369287728073, + "end": 1695396369298910949, + "rtt": 11.182848, + "err_desc": "" + }, + { + "wl_id": 305, + "start": 1695396369388476140, + "end": 1695396369399561039, + "rtt": 11.0848, + "err_desc": "" + }, + { + "wl_id": 306, + "start": 1695396369487691940, + "end": 1695396369498328341, + "rtt": 10.636288, + "err_desc": "" + }, + { + "wl_id": 307, + "start": 1695396369588737852, + "end": 1695396369600177759, + "rtt": 11.439872, + "err_desc": "" + }, + { + "wl_id": 308, + "start": 1695396369688446469, + "end": 1695396369699339324, + "rtt": 10.8928, + "err_desc": "" + }, + { + "wl_id": 309, + "start": 1695396369788600365, + "end": 1695396369799707036, + "rtt": 11.106816, + "err_desc": "" + }, + { + "wl_id": 310, + "start": 1695396369888123437, + "end": 1695396369898505519, + "rtt": 10.38208, + "err_desc": "" + }, + { + "wl_id": 311, + "start": 1695396369988027252, + "end": 1695396370259525647, + "rtt": 271.498496, + "err_desc": "RequestError: send request failed\ncaused by: Put \"http://s3gw-ha-s3gw-ha.s3gw-ha.svc.cluster.local/test-1/objTest\": dial tcp 10.43.2.234:80: connect: connection refused" + }, + { + "wl_id": 312, + "start": 1695396370259541697, + "end": 1695396371435553739, + "rtt": 1176.012032, + "err_desc": "" + }, + { + "wl_id": 313, + "start": 1695396371435559818, + "end": 1695396371445362972, + "rtt": 9.803008, + "err_desc": "" + }, + { + "wl_id": 314, + "start": 1695396371488674060, + "end": 1695396371499379949, + "rtt": 10.70592, + "err_desc": "" + }, + { + "wl_id": 315, + "start": 1695396371588804155, + "end": 1695396371599569962, + "rtt": 10.765824, + "err_desc": "" + }, + { + "wl_id": 316, + "start": 1695396371688704659, + "end": 1695396371699309543, + "rtt": 10.6048, + "err_desc": "" + }, + { + "wl_id": 317, + "start": 1695396371788711354, + "end": 1695396371798907568, + "rtt": 10.196224, + "err_desc": "" + }, + { + "wl_id": 318, + "start": 1695396371888276510, + "end": 1695396371899756477, + "rtt": 11.480064, + "err_desc": "" + }, + { + "wl_id": 319, + "start": 1695396371987955669, + "end": 1695396371994570979, + "rtt": 6.615296, + "err_desc": "" + }, + { + "wl_id": 320, + "start": 1695396372087674576, + "end": 1695396372095240952, + "rtt": 7.566336, + "err_desc": "" + }, + { + "wl_id": 321, + "start": 1695396372187678073, + "end": 1695396372199241264, + "rtt": 11.563264, + "err_desc": "" + }, + { + "wl_id": 322, + "start": 1695396372288585689, + "end": 1695396372299498211, + "rtt": 10.912512, + "err_desc": "" + }, + { + "wl_id": 323, + "start": 1695396372387720013, + "end": 1695396372398913352, + "rtt": 11.193344, + "err_desc": "" + }, + { + "wl_id": 324, + "start": 1695396372488190450, + "end": 1695396372499330077, + "rtt": 11.139584, + "err_desc": "" + }, + { + "wl_id": 325, + "start": 1695396372588625944, + "end": 1695396372599948414, + "rtt": 11.322368, + "err_desc": "" + }, + { + "wl_id": 326, + "start": 1695396372688161692, + "end": 1695396372698700477, + "rtt": 10.538752, + "err_desc": "" + }, + { + "wl_id": 327, + "start": 1695396372787985678, + "end": 1695396372794416979, + "rtt": 6.431232, + "err_desc": "" + }, + { + "wl_id": 328, + "start": 1695396372888617930, + "end": 1695396372904988112, + "rtt": 16.370176, + "err_desc": "" + }, + { + "wl_id": 329, + "start": 1695396372988205215, + "end": 1695396372999352676, + "rtt": 11.147264, + "err_desc": "" + }, + { + "wl_id": 330, + "start": 1695396373088670370, + "end": 1695396373100029968, + "rtt": 11.359488, + "err_desc": "" + }, + { + "wl_id": 331, + "start": 1695396373188310451, + "end": 1695396373199732119, + "rtt": 11.421696, + "err_desc": "" + }, + { + "wl_id": 332, + "start": 1695396373287991615, + "end": 1695396373299139451, + "rtt": 11.147776, + "err_desc": "" + }, + { + "wl_id": 333, + "start": 1695396373388481856, + "end": 1695396373400249616, + "rtt": 11.767808, + "err_desc": "" + }, + { + "wl_id": 334, + "start": 1695396373488491485, + "end": 1695396373499256591, + "rtt": 10.765056, + "err_desc": "" + }, + { + "wl_id": 335, + "start": 1695396373588631334, + "end": 1695396373600039425, + "rtt": 11.408128, + "err_desc": "" + }, + { + "wl_id": 336, + "start": 1695396373688279648, + "end": 1695396373699530444, + "rtt": 11.250944, + "err_desc": "" + }, + { + "wl_id": 337, + "start": 1695396373787711824, + "end": 1695396373798749855, + "rtt": 11.038208, + "err_desc": "" + }, + { + "wl_id": 338, + "start": 1695396373887894610, + "end": 1695396373899717783, + "rtt": 11.82336, + "err_desc": "" + }, + { + "wl_id": 339, + "start": 1695396373987920714, + "end": 1695396373999317607, + "rtt": 11.396864, + "err_desc": "" + }, + { + "wl_id": 340, + "start": 1695396374088673926, + "end": 1695396374100009162, + "rtt": 11.335168, + "err_desc": "" + }, + { + "wl_id": 341, + "start": 1695396374188314028, + "end": 1695396374200170433, + "rtt": 11.856384, + "err_desc": "" + }, + { + "wl_id": 342, + "start": 1695396374288608718, + "end": 1695396374299275706, + "rtt": 10.667008, + "err_desc": "" + }, + { + "wl_id": 343, + "start": 1695396374388564892, + "end": 1695396374400255570, + "rtt": 11.690496, + "err_desc": "" + }, + { + "wl_id": 344, + "start": 1695396374488414761, + "end": 1695396374497797317, + "rtt": 9.382656, + "err_desc": "" + }, + { + "wl_id": 345, + "start": 1695396374587964815, + "end": 1695396374599372067, + "rtt": 11.407104, + "err_desc": "" + }, + { + "wl_id": 346, + "start": 1695396374688708329, + "end": 1695396374700014741, + "rtt": 11.306496, + "err_desc": "" + }, + { + "wl_id": 347, + "start": 1695396374788378515, + "end": 1695396374799979601, + "rtt": 11.600896, + "err_desc": "" + }, + { + "wl_id": 348, + "start": 1695396374888231350, + "end": 1695396374899750310, + "rtt": 11.518976, + "err_desc": "" + }, + { + "wl_id": 349, + "start": 1695396374987961685, + "end": 1695396374999342994, + "rtt": 11.381504, + "err_desc": "" + }, + { + "wl_id": 350, + "start": 1695396375088626602, + "end": 1695396375101812841, + "rtt": 13.186048, + "err_desc": "" + }, + { + "wl_id": 351, + "start": 1695396375188056108, + "end": 1695396375198871159, + "rtt": 10.814976, + "err_desc": "" + }, + { + "wl_id": 352, + "start": 1695396375288142524, + "end": 1695396375299002823, + "rtt": 10.860288, + "err_desc": "" + }, + { + "wl_id": 353, + "start": 1695396375388269739, + "end": 1695396375399739110, + "rtt": 11.469312, + "err_desc": "" + }, + { + "wl_id": 354, + "start": 1695396375487908607, + "end": 1695396375498529438, + "rtt": 10.620928, + "err_desc": "" + }, + { + "wl_id": 355, + "start": 1695396375587726523, + "end": 1695396375598743832, + "rtt": 11.017216, + "err_desc": "" + }, + { + "wl_id": 356, + "start": 1695396375688182185, + "end": 1695396375699469450, + "rtt": 11.287296, + "err_desc": "" + }, + { + "wl_id": 357, + "start": 1695396375788759245, + "end": 1695396375799914716, + "rtt": 11.155456, + "err_desc": "" + }, + { + "wl_id": 358, + "start": 1695396375888235394, + "end": 1695396375899285777, + "rtt": 11.05024, + "err_desc": "" + }, + { + "wl_id": 359, + "start": 1695396375988457560, + "end": 1695396376294773697, + "rtt": 306.316288, + "err_desc": "RequestError: send request failed\ncaused by: Put \"http://s3gw-ha-s3gw-ha.s3gw-ha.svc.cluster.local/test-1/objTest\": dial tcp 10.43.2.234:80: connect: connection refused" + }, + { + "wl_id": 360, + "start": 1695396376294780288, + "end": 1695396378687984236, + "rtt": 2393.203712, + "err_desc": "" + }, + { + "wl_id": 361, + "start": 1695396378687990973, + "end": 1695396378696786171, + "rtt": 8.795136, + "err_desc": "" + }, + { + "wl_id": 362, + "start": 1695396378788090125, + "end": 1695396378798799115, + "rtt": 10.708992, + "err_desc": "" + }, + { + "wl_id": 363, + "start": 1695396378888053749, + "end": 1695396378897620379, + "rtt": 9.56672, + "err_desc": "" + }, + { + "wl_id": 364, + "start": 1695396378987855694, + "end": 1695396378998980048, + "rtt": 11.12448, + "err_desc": "" + }, + { + "wl_id": 365, + "start": 1695396379088194839, + "end": 1695396379100265485, + "rtt": 12.070656, + "err_desc": "" + }, + { + "wl_id": 366, + "start": 1695396379188619269, + "end": 1695396379199202728, + "rtt": 10.583552, + "err_desc": "" + }, + { + "wl_id": 367, + "start": 1695396379288486112, + "end": 1695396379298946275, + "rtt": 10.46016, + "err_desc": "" + }, + { + "wl_id": 368, + "start": 1695396379388260471, + "end": 1695396379399098777, + "rtt": 10.838528, + "err_desc": "" + }, + { + "wl_id": 369, + "start": 1695396379488468266, + "end": 1695396379499266914, + "rtt": 10.798592, + "err_desc": "" + }, + { + "wl_id": 370, + "start": 1695396379588567183, + "end": 1695396379599336119, + "rtt": 10.768896, + "err_desc": "" + }, + { + "wl_id": 371, + "start": 1695396379688727533, + "end": 1695396379699722626, + "rtt": 10.9952, + "err_desc": "" + }, + { + "wl_id": 372, + "start": 1695396379788070032, + "end": 1695396379797529344, + "rtt": 9.4592, + "err_desc": "" + }, + { + "wl_id": 373, + "start": 1695396379888797472, + "end": 1695396379899384132, + "rtt": 10.586624, + "err_desc": "" + }, + { + "wl_id": 374, + "start": 1695396379988643349, + "end": 1695396379999133054, + "rtt": 10.4896, + "err_desc": "" + }, + { + "wl_id": 375, + "start": 1695396380088448908, + "end": 1695396380099372924, + "rtt": 10.923776, + "err_desc": "" + }, + { + "wl_id": 376, + "start": 1695396380188682953, + "end": 1695396380199685100, + "rtt": 11.002112, + "err_desc": "" + }, + { + "wl_id": 377, + "start": 1695396380288003688, + "end": 1695396380298178902, + "rtt": 10.175232, + "err_desc": "" + }, + { + "wl_id": 378, + "start": 1695396380388483704, + "end": 1695396380398033712, + "rtt": 9.55008, + "err_desc": "" + }, + { + "wl_id": 379, + "start": 1695396380488439713, + "end": 1695396380498975451, + "rtt": 10.53568, + "err_desc": "" + }, + { + "wl_id": 380, + "start": 1695396380588274879, + "end": 1695396380599040237, + "rtt": 10.765312, + "err_desc": "" + }, + { + "wl_id": 381, + "start": 1695396380688318222, + "end": 1695396380698685032, + "rtt": 10.36672, + "err_desc": "" + }, + { + "wl_id": 382, + "start": 1695396380787930484, + "end": 1695396380798694036, + "rtt": 10.763776, + "err_desc": "" + }, + { + "wl_id": 383, + "start": 1695396380887914159, + "end": 1695396380897617203, + "rtt": 9.702912, + "err_desc": "" + }, + { + "wl_id": 384, + "start": 1695396380987752238, + "end": 1695396380997343490, + "rtt": 9.591296, + "err_desc": "" + }, + { + "wl_id": 385, + "start": 1695396381088494486, + "end": 1695396381099344101, + "rtt": 10.849536, + "err_desc": "" + }, + { + "wl_id": 386, + "start": 1695396381188480094, + "end": 1695396381198768830, + "rtt": 10.288896, + "err_desc": "" + }, + { + "wl_id": 387, + "start": 1695396381288181889, + "end": 1695396381297504081, + "rtt": 9.321984, + "err_desc": "" + }, + { + "wl_id": 388, + "start": 1695396381387764980, + "end": 1695396381398364240, + "rtt": 10.599168, + "err_desc": "" + }, + { + "wl_id": 389, + "start": 1695396381487670801, + "end": 1695396381498346575, + "rtt": 10.675712, + "err_desc": "" + }, + { + "wl_id": 390, + "start": 1695396381588747773, + "end": 1695396381598853700, + "rtt": 10.105856, + "err_desc": "" + }, + { + "wl_id": 391, + "start": 1695396381688039311, + "end": 1695396381698857948, + "rtt": 10.81856, + "err_desc": "" + }, + { + "wl_id": 392, + "start": 1695396381788137553, + "end": 1695396381799924866, + "rtt": 11.78752, + "err_desc": "" + }, + { + "wl_id": 393, + "start": 1695396381887902544, + "end": 1695396382253189142, + "rtt": 365.286656, + "err_desc": "RequestError: send request failed\ncaused by: Put \"http://s3gw-ha-s3gw-ha.s3gw-ha.svc.cluster.local/test-1/objTest\": dial tcp 10.43.2.234:80: connect: connection refused" + }, + { + "wl_id": 394, + "start": 1695396382253196147, + "end": 1695396383271519492, + "rtt": 1018.323456, + "err_desc": "" + } + ] + } + ], + "time_unit": "ms" +} \ No newline at end of file diff --git a/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_S3WL_RTT_raw.svg b/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_S3WL_RTT_raw.svg new file mode 100644 index 00000000..d954d32d --- /dev/null +++ b/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_S3WL_RTT_raw.svg @@ -0,0 +1,623 @@ + + + + + +RTT S3Workload: s3wl-putobj-100ms-Ingress +Time +0.00 +25000.00 +50000.00 + + + + + + + + + + + + + + +RTT: ms + +0 +300 +600 +900 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_percentiles_fup_main_delta.svg b/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_percentiles_fup_main_delta.svg new file mode 100644 index 00000000..ffceb1c1 --- /dev/null +++ b/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_percentiles_fup_main_delta.svg @@ -0,0 +1,169 @@ + + + + + +Percentiles - FrontEndUp-Main-Delta (Nearest Rank): s3wl-putobj-100ms-Ingress +Percentile +20 +60 +100 + + + + + + + + + + + + +Duration: ms + +40 +44 +48 +52 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +to-fronted-up + + diff --git a/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_percentiles_to_fup.svg b/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_percentiles_to_fup.svg new file mode 100644 index 00000000..9b01ed18 --- /dev/null +++ b/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_percentiles_to_fup.svg @@ -0,0 +1,157 @@ + + + + + +Percentiles - FrontEndUp (Nearest Rank): s3wl-putobj-100ms-Ingress +Percentile +20 +60 +100 + + + + + + + + + + + + +Duration: ms + +750.00 +1750.00 +2750.00 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +to-fronted-up + + diff --git a/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_percentiles_to_main.svg b/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_percentiles_to_main.svg new file mode 100644 index 00000000..5a62a095 --- /dev/null +++ b/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_percentiles_to_main.svg @@ -0,0 +1,157 @@ + + + + + +Percentiles - Main (Nearest Rank): s3wl-putobj-100ms-Ingress +Percentile +20 +60 +100 + + + + + + + + + + + + +Duration: ms + +750.00 +1750.00 +2750.00 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +to-main + + diff --git a/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_raw.svg b/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_raw.svg new file mode 100644 index 00000000..92aa99d6 --- /dev/null +++ b/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_raw.svg @@ -0,0 +1,97 @@ + + + + + +Raw Data Measures: s3wl-putobj-100ms-Ingress +Restart ID +0 +4 +8 + + + + + + + + + + + + +Duration: ms + +400 +1600 +2800 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +to-main + + +to-fronted-up + + +fronted-up-main-delta + + diff --git a/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_stats.json b/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_stats.json new file mode 100644 index 00000000..8eb24d43 --- /dev/null +++ b/docs/research/ha/measurements/s3wl-putobj-100ms-ingress/1695396145_s3wl-putobj-100ms-Ingress_stats.json @@ -0,0 +1,3635 @@ +{ + "series_restart_count": 0, + "series_restart": [ + { + "mark": "s3wl-putobj-100ms-Ingress", + "min_to_main": 578, + "max_to_main": 2881, + "mean_to_main": 869, + "99p_to_main": 1890, + "95p_to_main": 1890, + "99pNR_to_main": 2881, + "95pNR_to_main": 2881, + "min_to_frontend_up": 625, + "max_to_frontend_up": 2924, + "mean_to_frontend_up": 913, + "99p_to_frontend_up": 1931, + "95p_to_frontend_up": 1931, + "99pNR_to_frontend_up": 2924, + "95pNR_to_frontend_up": 2924, + "min_frontend_up_main_delta": 40, + "max_frontend_up_main_delta": 53, + "mean_frontend_up_main_delta": 43, + "99p_frontend_up_main_delta": 49, + "95p_frontend_up_main_delta": 49, + "99pNR_frontend_up_main_delta": 53, + "95pNR_frontend_up_main_delta": 53, + "data": [ + { + "restart_id": 1, + "duration_to_main": 2881, + "duration_to_frontend_up": 2924, + "frontend_up_main_delta": 42 + }, + { + "restart_id": 2, + "duration_to_main": 600, + "duration_to_frontend_up": 645, + "frontend_up_main_delta": 45 + }, + { + "restart_id": 3, + "duration_to_main": 586, + "duration_to_frontend_up": 628, + "frontend_up_main_delta": 41 + }, + { + "restart_id": 4, + "duration_to_main": 590, + "duration_to_frontend_up": 631, + "frontend_up_main_delta": 40 + }, + { + "restart_id": 5, + "duration_to_main": 649, + "duration_to_frontend_up": 703, + "frontend_up_main_delta": 53 + }, + { + "restart_id": 6, + "duration_to_main": 578, + "duration_to_frontend_up": 625, + "frontend_up_main_delta": 46 + }, + { + "restart_id": 7, + "duration_to_main": 675, + "duration_to_frontend_up": 718, + "frontend_up_main_delta": 42 + }, + { + "restart_id": 8, + "duration_to_main": 899, + "duration_to_frontend_up": 939, + "frontend_up_main_delta": 40 + }, + { + "restart_id": 9, + "duration_to_main": 617, + "duration_to_frontend_up": 663, + "frontend_up_main_delta": 45 + }, + { + "restart_id": 10, + "duration_to_main": 616, + "duration_to_frontend_up": 656, + "frontend_up_main_delta": 40 + } + ] + } + ], + "series_s3_workload_count": 0, + "series_s3_workload": [ + { + "mark": "s3wl-putobj-100ms-Ingress", + "min_RTT": 0, + "max_RTT": 989, + "mean_RTT": 22, + "99p_RTT": 717, + "95p_RTT": 13, + "99pNR_RTT": 718, + "95pNR_RTT": 13, + "data": [ + { + "wl_id": 1, + "start": 1695396089336018488, + "end": 1695396090052560387, + "rtt": 716.541952, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 2, + "start": 1695396090052566077, + "end": 1695396090052810669, + "rtt": 0.244736, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 3, + "start": 1695396090135947169, + "end": 1695396090136587572, + "rtt": 0.640256, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 4, + "start": 1695396090235820127, + "end": 1695396090236381180, + "rtt": 0.561152, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 5, + "start": 1695396090335637940, + "end": 1695396090335958988, + "rtt": 0.321024, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 6, + "start": 1695396090435188996, + "end": 1695396090435553988, + "rtt": 0.365056, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 7, + "start": 1695396090535744310, + "end": 1695396090536063510, + "rtt": 0.319232, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 8, + "start": 1695396090635249381, + "end": 1695396090635536793, + "rtt": 0.287488, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 9, + "start": 1695396090735977864, + "end": 1695396090736516571, + "rtt": 0.538624, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 10, + "start": 1695396090835725399, + "end": 1695396090837636318, + "rtt": 1.91104, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 11, + "start": 1695396090935724925, + "end": 1695396090938031225, + "rtt": 2.306304, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 12, + "start": 1695396091036233649, + "end": 1695396091037328111, + "rtt": 1.0944, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 13, + "start": 1695396091135408300, + "end": 1695396091135820318, + "rtt": 0.411904, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 14, + "start": 1695396091236109678, + "end": 1695396091236429051, + "rtt": 0.319488, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 15, + "start": 1695396091335676096, + "end": 1695396091336283975, + "rtt": 0.607744, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 16, + "start": 1695396091435566163, + "end": 1695396091436414032, + "rtt": 0.847872, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 17, + "start": 1695396091535576896, + "end": 1695396091536126663, + "rtt": 0.549888, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 18, + "start": 1695396091635199183, + "end": 1695396091635686936, + "rtt": 0.48768, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 19, + "start": 1695396091735855101, + "end": 1695396091736940174, + "rtt": 1.085184, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 20, + "start": 1695396091836084607, + "end": 1695396091836665268, + "rtt": 0.580864, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 21, + "start": 1695396091935891609, + "end": 1695396091936301966, + "rtt": 0.410368, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 22, + "start": 1695396092035448227, + "end": 1695396092035898928, + "rtt": 0.45056, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 23, + "start": 1695396092136103448, + "end": 1695396092136641483, + "rtt": 0.538112, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 24, + "start": 1695396092235598246, + "end": 1695396092236529012, + "rtt": 0.93056, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 25, + "start": 1695396092335741160, + "end": 1695396092336783238, + "rtt": 1.042176, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 26, + "start": 1695396092435163565, + "end": 1695396092436054533, + "rtt": 0.89088, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 27, + "start": 1695396092535280521, + "end": 1695396092536127155, + "rtt": 0.846592, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 28, + "start": 1695396092635450439, + "end": 1695396092636576980, + "rtt": 1.126656, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 29, + "start": 1695396092735740658, + "end": 1695396092736795032, + "rtt": 1.054464, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 30, + "start": 1695396092835997620, + "end": 1695396092837045774, + "rtt": 1.048064, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 31, + "start": 1695396092935170564, + "end": 1695396092937853713, + "rtt": 2.683136, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 32, + "start": 1695396093035187110, + "end": 1695396093035622028, + "rtt": 0.434944, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 33, + "start": 1695396093135790606, + "end": 1695396093144050844, + "rtt": 8.260352, + "err_desc": "" + }, + { + "wl_id": 34, + "start": 1695396093235186606, + "end": 1695396093246359578, + "rtt": 11.172864, + "err_desc": "" + }, + { + "wl_id": 35, + "start": 1695396093335653498, + "end": 1695396093347317844, + "rtt": 11.664384, + "err_desc": "" + }, + { + "wl_id": 36, + "start": 1695396093435570932, + "end": 1695396093444776388, + "rtt": 9.205504, + "err_desc": "" + }, + { + "wl_id": 37, + "start": 1695396093536055364, + "end": 1695396093546833059, + "rtt": 10.777856, + "err_desc": "" + }, + { + "wl_id": 38, + "start": 1695396093636140080, + "end": 1695396093647902677, + "rtt": 11.762688, + "err_desc": "" + }, + { + "wl_id": 39, + "start": 1695396093736135777, + "end": 1695396093747343941, + "rtt": 11.208192, + "err_desc": "" + }, + { + "wl_id": 40, + "start": 1695396093835561674, + "end": 1695396093847433096, + "rtt": 11.871488, + "err_desc": "" + }, + { + "wl_id": 41, + "start": 1695396093935656616, + "end": 1695396093946905841, + "rtt": 11.249152, + "err_desc": "" + }, + { + "wl_id": 42, + "start": 1695396094036104130, + "end": 1695396094046653422, + "rtt": 10.549248, + "err_desc": "" + }, + { + "wl_id": 43, + "start": 1695396094135968405, + "end": 1695396094148445647, + "rtt": 12.477184, + "err_desc": "" + }, + { + "wl_id": 44, + "start": 1695396094235553867, + "end": 1695396094243422785, + "rtt": 7.868928, + "err_desc": "" + }, + { + "wl_id": 45, + "start": 1695396094335702789, + "end": 1695396094348893124, + "rtt": 13.1904, + "err_desc": "" + }, + { + "wl_id": 46, + "start": 1695396094436249528, + "end": 1695396094447580933, + "rtt": 11.331328, + "err_desc": "" + }, + { + "wl_id": 47, + "start": 1695396094535878005, + "end": 1695396094546780368, + "rtt": 10.902528, + "err_desc": "" + }, + { + "wl_id": 48, + "start": 1695396094635942149, + "end": 1695396094647772500, + "rtt": 11.830272, + "err_desc": "" + }, + { + "wl_id": 49, + "start": 1695396094736106232, + "end": 1695396094747553113, + "rtt": 11.446784, + "err_desc": "" + }, + { + "wl_id": 50, + "start": 1695396094835832276, + "end": 1695396094847454226, + "rtt": 11.621888, + "err_desc": "" + }, + { + "wl_id": 51, + "start": 1695396094935782649, + "end": 1695396094947046108, + "rtt": 11.263488, + "err_desc": "" + }, + { + "wl_id": 52, + "start": 1695396095035188315, + "end": 1695396095045043767, + "rtt": 9.855488, + "err_desc": "" + }, + { + "wl_id": 53, + "start": 1695396095135228302, + "end": 1695396095146405898, + "rtt": 11.177472, + "err_desc": "" + }, + { + "wl_id": 54, + "start": 1695396095235594548, + "end": 1695396095247582424, + "rtt": 11.987968, + "err_desc": "" + }, + { + "wl_id": 55, + "start": 1695396095335914894, + "end": 1695396095347058822, + "rtt": 11.143936, + "err_desc": "" + }, + { + "wl_id": 56, + "start": 1695396095435292794, + "end": 1695396095445981328, + "rtt": 10.688768, + "err_desc": "" + }, + { + "wl_id": 57, + "start": 1695396095535182476, + "end": 1695396095546121149, + "rtt": 10.938624, + "err_desc": "" + }, + { + "wl_id": 58, + "start": 1695396095635511118, + "end": 1695396095645731117, + "rtt": 10.220032, + "err_desc": "" + }, + { + "wl_id": 59, + "start": 1695396095735932910, + "end": 1695396095746214612, + "rtt": 10.281728, + "err_desc": "" + }, + { + "wl_id": 60, + "start": 1695396095835491938, + "end": 1695396095846841208, + "rtt": 11.349248, + "err_desc": "" + }, + { + "wl_id": 61, + "start": 1695396095936088367, + "end": 1695396095948256055, + "rtt": 12.16768, + "err_desc": "" + }, + { + "wl_id": 62, + "start": 1695396096035510186, + "end": 1695396096047108729, + "rtt": 11.598336, + "err_desc": "" + }, + { + "wl_id": 63, + "start": 1695396096135281978, + "end": 1695396096146992776, + "rtt": 11.710976, + "err_desc": "" + }, + { + "wl_id": 64, + "start": 1695396096235152146, + "end": 1695396096246517997, + "rtt": 11.365888, + "err_desc": "" + }, + { + "wl_id": 65, + "start": 1695396096335943737, + "end": 1695396096347700401, + "rtt": 11.7568, + "err_desc": "" + }, + { + "wl_id": 66, + "start": 1695396096435959925, + "end": 1695396096445773881, + "rtt": 9.814016, + "err_desc": "" + }, + { + "wl_id": 67, + "start": 1695396096536166463, + "end": 1695396096547106950, + "rtt": 10.940672, + "err_desc": "" + }, + { + "wl_id": 68, + "start": 1695396096635293575, + "end": 1695396096646732999, + "rtt": 11.43936, + "err_desc": "" + }, + { + "wl_id": 69, + "start": 1695396096736007354, + "end": 1695396096747475129, + "rtt": 11.467776, + "err_desc": "" + }, + { + "wl_id": 70, + "start": 1695396096835786302, + "end": 1695396096847301844, + "rtt": 11.515648, + "err_desc": "" + }, + { + "wl_id": 71, + "start": 1695396096935517095, + "end": 1695396096947222938, + "rtt": 11.705856, + "err_desc": "" + }, + { + "wl_id": 72, + "start": 1695396097035353860, + "end": 1695396097046909140, + "rtt": 11.555328, + "err_desc": "" + }, + { + "wl_id": 73, + "start": 1695396097135172899, + "end": 1695396097147042114, + "rtt": 11.869184, + "err_desc": "" + }, + { + "wl_id": 74, + "start": 1695396097235237525, + "end": 1695396097245854713, + "rtt": 10.617088, + "err_desc": "" + }, + { + "wl_id": 75, + "start": 1695396097335142237, + "end": 1695396097346106827, + "rtt": 10.964736, + "err_desc": "" + }, + { + "wl_id": 76, + "start": 1695396097435793260, + "end": 1695396098406191783, + "rtt": 970.39872, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 77, + "start": 1695396098406201768, + "end": 1695396098406950995, + "rtt": 0.749056, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 78, + "start": 1695396098436087324, + "end": 1695396098436313815, + "rtt": 0.22656, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 79, + "start": 1695396098535678153, + "end": 1695396098536693077, + "rtt": 1.014784, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 80, + "start": 1695396098635924370, + "end": 1695396098637098937, + "rtt": 1.174528, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 81, + "start": 1695396098735278495, + "end": 1695396098736143746, + "rtt": 0.86528, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 82, + "start": 1695396098835283696, + "end": 1695396098836226259, + "rtt": 0.942592, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 83, + "start": 1695396098935574686, + "end": 1695396098936528490, + "rtt": 0.9536, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 84, + "start": 1695396099035699334, + "end": 1695396099036145485, + "rtt": 0.445952, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 85, + "start": 1695396099135760923, + "end": 1695396099136717080, + "rtt": 0.95616, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 86, + "start": 1695396099236077693, + "end": 1695396099236985358, + "rtt": 0.907776, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 87, + "start": 1695396099336168156, + "end": 1695396099337327102, + "rtt": 1.158912, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 88, + "start": 1695396099435489155, + "end": 1695396099435935193, + "rtt": 0.445952, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 89, + "start": 1695396099535208024, + "end": 1695396099554410111, + "rtt": 19.202048, + "err_desc": "" + }, + { + "wl_id": 90, + "start": 1695396099635813113, + "end": 1695396099647814446, + "rtt": 12.00128, + "err_desc": "" + }, + { + "wl_id": 91, + "start": 1695396099735996707, + "end": 1695396099747494479, + "rtt": 11.497728, + "err_desc": "" + }, + { + "wl_id": 92, + "start": 1695396099835760443, + "end": 1695396099847324201, + "rtt": 11.563776, + "err_desc": "" + }, + { + "wl_id": 93, + "start": 1695396099935706322, + "end": 1695396099947127830, + "rtt": 11.42144, + "err_desc": "" + }, + { + "wl_id": 94, + "start": 1695396100035245054, + "end": 1695396100046447258, + "rtt": 11.202304, + "err_desc": "" + }, + { + "wl_id": 95, + "start": 1695396100135597303, + "end": 1695396100147549667, + "rtt": 11.952384, + "err_desc": "" + }, + { + "wl_id": 96, + "start": 1695396100235878984, + "end": 1695396100247513749, + "rtt": 11.634944, + "err_desc": "" + }, + { + "wl_id": 97, + "start": 1695396100335746838, + "end": 1695396100346939494, + "rtt": 11.192576, + "err_desc": "" + }, + { + "wl_id": 98, + "start": 1695396100435173651, + "end": 1695396100445990607, + "rtt": 10.817024, + "err_desc": "" + }, + { + "wl_id": 99, + "start": 1695396100535119356, + "end": 1695396100545663405, + "rtt": 10.544128, + "err_desc": "" + }, + { + "wl_id": 100, + "start": 1695396100636170061, + "end": 1695396100647898694, + "rtt": 11.72864, + "err_desc": "" + }, + { + "wl_id": 101, + "start": 1695396100736143845, + "end": 1695396100747878420, + "rtt": 11.734528, + "err_desc": "" + }, + { + "wl_id": 102, + "start": 1695396100835139922, + "end": 1695396100846692993, + "rtt": 11.55328, + "err_desc": "" + }, + { + "wl_id": 103, + "start": 1695396100936004532, + "end": 1695396100947534050, + "rtt": 11.529472, + "err_desc": "" + }, + { + "wl_id": 104, + "start": 1695396101035872032, + "end": 1695396101046504270, + "rtt": 10.632192, + "err_desc": "" + }, + { + "wl_id": 105, + "start": 1695396101135686544, + "end": 1695396101148737987, + "rtt": 13.051392, + "err_desc": "" + }, + { + "wl_id": 106, + "start": 1695396101235918399, + "end": 1695396101246852730, + "rtt": 10.934272, + "err_desc": "" + }, + { + "wl_id": 107, + "start": 1695396101336078018, + "end": 1695396101347316900, + "rtt": 11.238912, + "err_desc": "" + }, + { + "wl_id": 108, + "start": 1695396101435680781, + "end": 1695396101446488942, + "rtt": 10.808064, + "err_desc": "" + }, + { + "wl_id": 109, + "start": 1695396101535673656, + "end": 1695396101546772295, + "rtt": 11.098624, + "err_desc": "" + }, + { + "wl_id": 110, + "start": 1695396101636225615, + "end": 1695396101648013501, + "rtt": 11.788032, + "err_desc": "" + }, + { + "wl_id": 111, + "start": 1695396101735486085, + "end": 1695396101746728227, + "rtt": 11.241984, + "err_desc": "" + }, + { + "wl_id": 112, + "start": 1695396101835984753, + "end": 1695396101847505144, + "rtt": 11.520512, + "err_desc": "" + }, + { + "wl_id": 113, + "start": 1695396101935845635, + "end": 1695396101947998757, + "rtt": 12.153088, + "err_desc": "" + }, + { + "wl_id": 114, + "start": 1695396102035248632, + "end": 1695396102045359240, + "rtt": 10.11072, + "err_desc": "" + }, + { + "wl_id": 115, + "start": 1695396102135603774, + "end": 1695396102147741253, + "rtt": 12.137472, + "err_desc": "" + }, + { + "wl_id": 116, + "start": 1695396102235985609, + "end": 1695396102247074477, + "rtt": 11.088896, + "err_desc": "" + }, + { + "wl_id": 117, + "start": 1695396102335283690, + "end": 1695396102346842254, + "rtt": 11.558656, + "err_desc": "" + }, + { + "wl_id": 118, + "start": 1695396102436114705, + "end": 1695396102447566165, + "rtt": 11.451392, + "err_desc": "" + }, + { + "wl_id": 119, + "start": 1695396102535756659, + "end": 1695396102546937860, + "rtt": 11.181312, + "err_desc": "" + }, + { + "wl_id": 120, + "start": 1695396102635124496, + "end": 1695396102646545042, + "rtt": 11.420672, + "err_desc": "" + }, + { + "wl_id": 121, + "start": 1695396102735992929, + "end": 1695396102747188282, + "rtt": 11.195392, + "err_desc": "" + }, + { + "wl_id": 122, + "start": 1695396102835460879, + "end": 1695396102846612901, + "rtt": 11.152128, + "err_desc": "" + }, + { + "wl_id": 123, + "start": 1695396102935889694, + "end": 1695396102947637378, + "rtt": 11.74784, + "err_desc": "" + }, + { + "wl_id": 124, + "start": 1695396103035853236, + "end": 1695396103043483234, + "rtt": 7.629824, + "err_desc": "" + }, + { + "wl_id": 125, + "start": 1695396103135693769, + "end": 1695396103146538977, + "rtt": 10.845184, + "err_desc": "" + }, + { + "wl_id": 126, + "start": 1695396103235746092, + "end": 1695396103243759161, + "rtt": 8.013056, + "err_desc": "" + }, + { + "wl_id": 127, + "start": 1695396103335131970, + "end": 1695396103976973688, + "rtt": 641.841664, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 128, + "start": 1695396103976995668, + "end": 1695396103977725638, + "rtt": 0.730112, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 129, + "start": 1695396104035921703, + "end": 1695396104036790992, + "rtt": 0.869376, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 130, + "start": 1695396104136110575, + "end": 1695396104137036430, + "rtt": 0.925952, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 131, + "start": 1695396104236162429, + "end": 1695396104236475200, + "rtt": 0.312832, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 132, + "start": 1695396104335723246, + "end": 1695396104336542147, + "rtt": 0.818944, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 133, + "start": 1695396104435764524, + "end": 1695396104436883913, + "rtt": 1.119488, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 134, + "start": 1695396104536093677, + "end": 1695396104536877722, + "rtt": 0.784128, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 135, + "start": 1695396104636121244, + "end": 1695396104637009882, + "rtt": 0.888576, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 136, + "start": 1695396104735447815, + "end": 1695396104736469694, + "rtt": 1.021952, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 137, + "start": 1695396104835729460, + "end": 1695396104836723056, + "rtt": 0.993536, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 138, + "start": 1695396104936193480, + "end": 1695396104937281226, + "rtt": 1.087744, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 139, + "start": 1695396105035640943, + "end": 1695396105036676498, + "rtt": 1.035776, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 140, + "start": 1695396105135810157, + "end": 1695396105136748323, + "rtt": 0.93824, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 141, + "start": 1695396105235980965, + "end": 1695396105236882370, + "rtt": 0.901376, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 142, + "start": 1695396105336111657, + "end": 1695396105351868328, + "rtt": 15.7568, + "err_desc": "" + }, + { + "wl_id": 143, + "start": 1695396105435167704, + "end": 1695396105446279258, + "rtt": 11.111424, + "err_desc": "" + }, + { + "wl_id": 144, + "start": 1695396105535708540, + "end": 1695396105546843076, + "rtt": 11.13472, + "err_desc": "" + }, + { + "wl_id": 145, + "start": 1695396105636129555, + "end": 1695396105647295602, + "rtt": 11.165952, + "err_desc": "" + }, + { + "wl_id": 146, + "start": 1695396105735593099, + "end": 1695396105747249632, + "rtt": 11.656448, + "err_desc": "" + }, + { + "wl_id": 147, + "start": 1695396105835597229, + "end": 1695396105847096198, + "rtt": 11.499008, + "err_desc": "" + }, + { + "wl_id": 148, + "start": 1695396105935299996, + "end": 1695396105946386758, + "rtt": 11.086592, + "err_desc": "" + }, + { + "wl_id": 149, + "start": 1695396106035782972, + "end": 1695396106046154750, + "rtt": 10.37184, + "err_desc": "" + }, + { + "wl_id": 150, + "start": 1695396106135395951, + "end": 1695396106147172179, + "rtt": 11.776256, + "err_desc": "" + }, + { + "wl_id": 151, + "start": 1695396106235417206, + "end": 1695396106246638121, + "rtt": 11.220992, + "err_desc": "" + }, + { + "wl_id": 152, + "start": 1695396106336049354, + "end": 1695396106347208017, + "rtt": 11.158528, + "err_desc": "" + }, + { + "wl_id": 153, + "start": 1695396106435478882, + "end": 1695396106447366150, + "rtt": 11.88736, + "err_desc": "" + }, + { + "wl_id": 154, + "start": 1695396106535541870, + "end": 1695396106547203169, + "rtt": 11.661312, + "err_desc": "" + }, + { + "wl_id": 155, + "start": 1695396106635618556, + "end": 1695396106647063287, + "rtt": 11.444736, + "err_desc": "" + }, + { + "wl_id": 156, + "start": 1695396106735256988, + "end": 1695396106746789775, + "rtt": 11.5328, + "err_desc": "" + }, + { + "wl_id": 157, + "start": 1695396106836107929, + "end": 1695396106846864505, + "rtt": 10.756352, + "err_desc": "" + }, + { + "wl_id": 158, + "start": 1695396106936173082, + "end": 1695396106947659763, + "rtt": 11.48672, + "err_desc": "" + }, + { + "wl_id": 159, + "start": 1695396107036102521, + "end": 1695396107047715929, + "rtt": 11.61344, + "err_desc": "" + }, + { + "wl_id": 160, + "start": 1695396107135972960, + "end": 1695396107147152461, + "rtt": 11.17952, + "err_desc": "" + }, + { + "wl_id": 161, + "start": 1695396107235376688, + "end": 1695396107244924892, + "rtt": 9.548288, + "err_desc": "" + }, + { + "wl_id": 162, + "start": 1695396107336150277, + "end": 1695396107347033025, + "rtt": 10.882816, + "err_desc": "" + }, + { + "wl_id": 163, + "start": 1695396107435122601, + "end": 1695396107446569239, + "rtt": 11.446528, + "err_desc": "" + }, + { + "wl_id": 164, + "start": 1695396107535857429, + "end": 1695396107547164676, + "rtt": 11.307264, + "err_desc": "" + }, + { + "wl_id": 165, + "start": 1695396107635388360, + "end": 1695396107647241720, + "rtt": 11.853312, + "err_desc": "" + }, + { + "wl_id": 166, + "start": 1695396107735475600, + "end": 1695396107746981056, + "rtt": 11.505408, + "err_desc": "" + }, + { + "wl_id": 167, + "start": 1695396107836147477, + "end": 1695396107842434151, + "rtt": 6.286592, + "err_desc": "" + }, + { + "wl_id": 168, + "start": 1695396107935641996, + "end": 1695396107947290080, + "rtt": 11.648, + "err_desc": "" + }, + { + "wl_id": 169, + "start": 1695396108035522422, + "end": 1695396108046394335, + "rtt": 10.872064, + "err_desc": "" + }, + { + "wl_id": 170, + "start": 1695396108135634695, + "end": 1695396108146615533, + "rtt": 10.980864, + "err_desc": "" + }, + { + "wl_id": 171, + "start": 1695396108236120841, + "end": 1695396108247643565, + "rtt": 11.522816, + "err_desc": "" + }, + { + "wl_id": 172, + "start": 1695396108335921299, + "end": 1695396108347044035, + "rtt": 11.122688, + "err_desc": "" + }, + { + "wl_id": 173, + "start": 1695396108435238498, + "end": 1695396108441632466, + "rtt": 6.394112, + "err_desc": "" + }, + { + "wl_id": 174, + "start": 1695396108535915131, + "end": 1695396108551555706, + "rtt": 15.640576, + "err_desc": "" + }, + { + "wl_id": 175, + "start": 1695396108635824280, + "end": 1695396108647257282, + "rtt": 11.43296, + "err_desc": "" + }, + { + "wl_id": 176, + "start": 1695396108735697229, + "end": 1695396108747312911, + "rtt": 11.615744, + "err_desc": "" + }, + { + "wl_id": 177, + "start": 1695396108835534431, + "end": 1695396108847112644, + "rtt": 11.578368, + "err_desc": "" + }, + { + "wl_id": 178, + "start": 1695396108935281912, + "end": 1695396108946994286, + "rtt": 11.712256, + "err_desc": "" + }, + { + "wl_id": 179, + "start": 1695396109035264957, + "end": 1695396109046228472, + "rtt": 10.963456, + "err_desc": "" + }, + { + "wl_id": 180, + "start": 1695396109135253989, + "end": 1695396109967148413, + "rtt": 831.894272, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 181, + "start": 1695396109967166303, + "end": 1695396109967951605, + "rtt": 0.785408, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 182, + "start": 1695396110036180527, + "end": 1695396110036985858, + "rtt": 0.805376, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 183, + "start": 1695396110136124277, + "end": 1695396110137077107, + "rtt": 0.952832, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 184, + "start": 1695396110235222579, + "end": 1695396110236101332, + "rtt": 0.878848, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 185, + "start": 1695396110335485478, + "end": 1695396110336380068, + "rtt": 0.89472, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 186, + "start": 1695396110435658101, + "end": 1695396110436737597, + "rtt": 1.079552, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 187, + "start": 1695396110535989588, + "end": 1695396110536712695, + "rtt": 0.7232, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 188, + "start": 1695396110636046746, + "end": 1695396110637110180, + "rtt": 1.063424, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 189, + "start": 1695396110735411714, + "end": 1695396110736291706, + "rtt": 0.879872, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 190, + "start": 1695396110835543654, + "end": 1695396110836423580, + "rtt": 0.880128, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 191, + "start": 1695396110935736033, + "end": 1695396110936858211, + "rtt": 1.122048, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 192, + "start": 1695396111036233259, + "end": 1695396111037214834, + "rtt": 0.981504, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 193, + "start": 1695396111135406499, + "end": 1695396111135796448, + "rtt": 0.389888, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 194, + "start": 1695396111236175944, + "end": 1695396111252631631, + "rtt": 16.45568, + "err_desc": "" + }, + { + "wl_id": 195, + "start": 1695396111335989408, + "end": 1695396111347088389, + "rtt": 11.09888, + "err_desc": "" + }, + { + "wl_id": 196, + "start": 1695396111435246099, + "end": 1695396111446510248, + "rtt": 11.264256, + "err_desc": "" + }, + { + "wl_id": 197, + "start": 1695396111535857768, + "end": 1695396111546509707, + "rtt": 10.65216, + "err_desc": "" + }, + { + "wl_id": 198, + "start": 1695396111635792181, + "end": 1695396111647474278, + "rtt": 11.682048, + "err_desc": "" + }, + { + "wl_id": 199, + "start": 1695396111735792703, + "end": 1695396111746858992, + "rtt": 11.066368, + "err_desc": "" + }, + { + "wl_id": 200, + "start": 1695396111835135624, + "end": 1695396111846799640, + "rtt": 11.663872, + "err_desc": "" + }, + { + "wl_id": 201, + "start": 1695396111935963715, + "end": 1695396111947338782, + "rtt": 11.375104, + "err_desc": "" + }, + { + "wl_id": 202, + "start": 1695396112035622934, + "end": 1695396112046559893, + "rtt": 10.937088, + "err_desc": "" + }, + { + "wl_id": 203, + "start": 1695396112135860655, + "end": 1695396112147528112, + "rtt": 11.667456, + "err_desc": "" + }, + { + "wl_id": 204, + "start": 1695396112235808219, + "end": 1695396112247184983, + "rtt": 11.37664, + "err_desc": "" + }, + { + "wl_id": 205, + "start": 1695396112335544963, + "end": 1695396112346828962, + "rtt": 11.283968, + "err_desc": "" + }, + { + "wl_id": 206, + "start": 1695396112436086724, + "end": 1695396112447167012, + "rtt": 11.080192, + "err_desc": "" + }, + { + "wl_id": 207, + "start": 1695396112535401875, + "end": 1695396112546482736, + "rtt": 11.080704, + "err_desc": "" + }, + { + "wl_id": 208, + "start": 1695396112635654224, + "end": 1695396112646856281, + "rtt": 11.202048, + "err_desc": "" + }, + { + "wl_id": 209, + "start": 1695396112736115113, + "end": 1695396112746833347, + "rtt": 10.718208, + "err_desc": "" + }, + { + "wl_id": 210, + "start": 1695396112836108075, + "end": 1695396112847510878, + "rtt": 11.402752, + "err_desc": "" + }, + { + "wl_id": 211, + "start": 1695396112935830830, + "end": 1695396112947807434, + "rtt": 11.976704, + "err_desc": "" + }, + { + "wl_id": 212, + "start": 1695396113036146695, + "end": 1695396113046914784, + "rtt": 10.768128, + "err_desc": "" + }, + { + "wl_id": 213, + "start": 1695396113135162023, + "end": 1695396113146498835, + "rtt": 11.336704, + "err_desc": "" + }, + { + "wl_id": 214, + "start": 1695396113235716867, + "end": 1695396113246772803, + "rtt": 11.055872, + "err_desc": "" + }, + { + "wl_id": 215, + "start": 1695396113336136068, + "end": 1695396113347422203, + "rtt": 11.286016, + "err_desc": "" + }, + { + "wl_id": 216, + "start": 1695396113435634923, + "end": 1695396113448817732, + "rtt": 13.18272, + "err_desc": "" + }, + { + "wl_id": 217, + "start": 1695396113536096627, + "end": 1695396113547245396, + "rtt": 11.1488, + "err_desc": "" + }, + { + "wl_id": 218, + "start": 1695396113635581147, + "end": 1695396113646669020, + "rtt": 11.087872, + "err_desc": "" + }, + { + "wl_id": 219, + "start": 1695396113736043491, + "end": 1695396113747249289, + "rtt": 11.205888, + "err_desc": "" + }, + { + "wl_id": 220, + "start": 1695396113835485973, + "end": 1695396113847232391, + "rtt": 11.74656, + "err_desc": "" + }, + { + "wl_id": 221, + "start": 1695396113935514403, + "end": 1695396113947164153, + "rtt": 11.649792, + "err_desc": "" + }, + { + "wl_id": 222, + "start": 1695396114035392628, + "end": 1695396114046744242, + "rtt": 11.351808, + "err_desc": "" + }, + { + "wl_id": 223, + "start": 1695396114135951429, + "end": 1695396114147753015, + "rtt": 11.8016, + "err_desc": "" + }, + { + "wl_id": 224, + "start": 1695396114235929064, + "end": 1695396114242302267, + "rtt": 6.37312, + "err_desc": "" + }, + { + "wl_id": 225, + "start": 1695396114335474648, + "end": 1695396114341747650, + "rtt": 6.273024, + "err_desc": "" + }, + { + "wl_id": 226, + "start": 1695396114436116038, + "end": 1695396114452187518, + "rtt": 16.071424, + "err_desc": "" + }, + { + "wl_id": 227, + "start": 1695396114535491171, + "end": 1695396114546351624, + "rtt": 10.860544, + "err_desc": "" + }, + { + "wl_id": 228, + "start": 1695396114635643881, + "end": 1695396114647588058, + "rtt": 11.944192, + "err_desc": "" + }, + { + "wl_id": 229, + "start": 1695396114735886413, + "end": 1695396114747076635, + "rtt": 11.190272, + "err_desc": "" + }, + { + "wl_id": 230, + "start": 1695396114835318921, + "end": 1695396114846115749, + "rtt": 10.7968, + "err_desc": "" + }, + { + "wl_id": 231, + "start": 1695396114935503010, + "end": 1695396114946819707, + "rtt": 11.31648, + "err_desc": "" + }, + { + "wl_id": 232, + "start": 1695396115035495730, + "end": 1695396115753617769, + "rtt": 718.121984, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 233, + "start": 1695396115753626711, + "end": 1695396115754191666, + "rtt": 0.564992, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 234, + "start": 1695396115835594843, + "end": 1695396115836579846, + "rtt": 0.985088, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 235, + "start": 1695396115936109123, + "end": 1695396115936569127, + "rtt": 0.460032, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 236, + "start": 1695396116035789480, + "end": 1695396116036703587, + "rtt": 0.91392, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 237, + "start": 1695396116135921803, + "end": 1695396116136426652, + "rtt": 0.504832, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 238, + "start": 1695396116235759978, + "end": 1695396116236616902, + "rtt": 0.857088, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 239, + "start": 1695396116335965532, + "end": 1695396116337057774, + "rtt": 1.092352, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 240, + "start": 1695396116435117378, + "end": 1695396116435585522, + "rtt": 0.468224, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 241, + "start": 1695396116535910784, + "end": 1695396116536947409, + "rtt": 1.036544, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 242, + "start": 1695396116636215531, + "end": 1695396116637236462, + "rtt": 1.020928, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 243, + "start": 1695396116736052642, + "end": 1695396116737164934, + "rtt": 1.11232, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 244, + "start": 1695396116835502034, + "end": 1695396116836536286, + "rtt": 1.03424, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 245, + "start": 1695396116935882622, + "end": 1695396116937008592, + "rtt": 1.126144, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 246, + "start": 1695396117035149254, + "end": 1695396117035584235, + "rtt": 0.434944, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 247, + "start": 1695396117135955167, + "end": 1695396117136248884, + "rtt": 0.293632, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 248, + "start": 1695396117235448370, + "end": 1695396117243788852, + "rtt": 8.34048, + "err_desc": "" + }, + { + "wl_id": 249, + "start": 1695396117336119226, + "end": 1695396117350219312, + "rtt": 14.099968, + "err_desc": "" + }, + { + "wl_id": 250, + "start": 1695396117435391418, + "end": 1695396117447706613, + "rtt": 12.315136, + "err_desc": "" + }, + { + "wl_id": 251, + "start": 1695396117536100562, + "end": 1695396117547055765, + "rtt": 10.955264, + "err_desc": "" + }, + { + "wl_id": 252, + "start": 1695396117635217813, + "end": 1695396117647756687, + "rtt": 12.53888, + "err_desc": "" + }, + { + "wl_id": 253, + "start": 1695396117736162015, + "end": 1695396117747520088, + "rtt": 11.357952, + "err_desc": "" + }, + { + "wl_id": 254, + "start": 1695396117836052893, + "end": 1695396117847647760, + "rtt": 11.594752, + "err_desc": "" + }, + { + "wl_id": 255, + "start": 1695396117935918791, + "end": 1695396117949361810, + "rtt": 13.443072, + "err_desc": "" + }, + { + "wl_id": 256, + "start": 1695396118035735458, + "end": 1695396118048252306, + "rtt": 12.516864, + "err_desc": "" + }, + { + "wl_id": 257, + "start": 1695396118135521695, + "end": 1695396118149200790, + "rtt": 13.679104, + "err_desc": "" + }, + { + "wl_id": 258, + "start": 1695396118235581710, + "end": 1695396118247161530, + "rtt": 11.579904, + "err_desc": "" + }, + { + "wl_id": 259, + "start": 1695396118335615296, + "end": 1695396118347871399, + "rtt": 12.256256, + "err_desc": "" + }, + { + "wl_id": 260, + "start": 1695396118436127392, + "end": 1695396118443658438, + "rtt": 7.531008, + "err_desc": "" + }, + { + "wl_id": 261, + "start": 1695396118536075149, + "end": 1695396118548349707, + "rtt": 12.274432, + "err_desc": "" + }, + { + "wl_id": 262, + "start": 1695396118635793296, + "end": 1695396118649093386, + "rtt": 13.299968, + "err_desc": "" + }, + { + "wl_id": 263, + "start": 1695396118735278813, + "end": 1695396118741856093, + "rtt": 6.577152, + "err_desc": "" + }, + { + "wl_id": 264, + "start": 1695396118835120306, + "end": 1695396118845817980, + "rtt": 10.697472, + "err_desc": "" + }, + { + "wl_id": 265, + "start": 1695396118936116485, + "end": 1695396118947721816, + "rtt": 11.605248, + "err_desc": "" + }, + { + "wl_id": 266, + "start": 1695396119035977237, + "end": 1695396119048110578, + "rtt": 12.133376, + "err_desc": "" + }, + { + "wl_id": 267, + "start": 1695396119135497578, + "end": 1695396119146768248, + "rtt": 11.270656, + "err_desc": "" + }, + { + "wl_id": 268, + "start": 1695396119235934471, + "end": 1695396119245099087, + "rtt": 9.164544, + "err_desc": "" + }, + { + "wl_id": 269, + "start": 1695396119335306828, + "end": 1695396119347004589, + "rtt": 11.69792, + "err_desc": "" + }, + { + "wl_id": 270, + "start": 1695396119435207486, + "end": 1695396119447390407, + "rtt": 12.18304, + "err_desc": "" + }, + { + "wl_id": 271, + "start": 1695396119535747418, + "end": 1695396119548471443, + "rtt": 12.724224, + "err_desc": "" + }, + { + "wl_id": 272, + "start": 1695396119635853817, + "end": 1695396119651675740, + "rtt": 15.821824, + "err_desc": "" + }, + { + "wl_id": 273, + "start": 1695396119735989439, + "end": 1695396119747505103, + "rtt": 11.515648, + "err_desc": "" + }, + { + "wl_id": 274, + "start": 1695396119835912129, + "end": 1695396119848913464, + "rtt": 13.001216, + "err_desc": "" + }, + { + "wl_id": 275, + "start": 1695396119935172073, + "end": 1695396119948210223, + "rtt": 13.03808, + "err_desc": "" + }, + { + "wl_id": 276, + "start": 1695396120035475440, + "end": 1695396120047629109, + "rtt": 12.1536, + "err_desc": "" + }, + { + "wl_id": 277, + "start": 1695396120135870403, + "end": 1695396120148974311, + "rtt": 13.103872, + "err_desc": "" + }, + { + "wl_id": 278, + "start": 1695396120235203709, + "end": 1695396120248002164, + "rtt": 12.798464, + "err_desc": "" + }, + { + "wl_id": 279, + "start": 1695396120335195212, + "end": 1695396120348600768, + "rtt": 13.405696, + "err_desc": "" + }, + { + "wl_id": 280, + "start": 1695396120436011041, + "end": 1695396120446828420, + "rtt": 10.817536, + "err_desc": "" + }, + { + "wl_id": 281, + "start": 1695396120536156748, + "end": 1695396120543839205, + "rtt": 7.68256, + "err_desc": "" + }, + { + "wl_id": 282, + "start": 1695396120636010952, + "end": 1695396120649202380, + "rtt": 13.191424, + "err_desc": "" + }, + { + "wl_id": 283, + "start": 1695396120735499525, + "end": 1695396120747577279, + "rtt": 12.077824, + "err_desc": "" + }, + { + "wl_id": 284, + "start": 1695396120835873957, + "end": 1695396120847436036, + "rtt": 11.561984, + "err_desc": "" + }, + { + "wl_id": 285, + "start": 1695396120936058981, + "end": 1695396121925290534, + "rtt": 989.231616, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 286, + "start": 1695396121925310957, + "end": 1695396121926100420, + "rtt": 0.789504, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 287, + "start": 1695396121935166134, + "end": 1695396121935772298, + "rtt": 0.606208, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 288, + "start": 1695396122036001799, + "end": 1695396122036383720, + "rtt": 0.381952, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 289, + "start": 1695396122135519664, + "end": 1695396122135882320, + "rtt": 0.362496, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 290, + "start": 1695396122236048039, + "end": 1695396122237167834, + "rtt": 1.119744, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 291, + "start": 1695396122335251306, + "end": 1695396122336443673, + "rtt": 1.192448, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 292, + "start": 1695396122435677800, + "end": 1695396122436697199, + "rtt": 1.019392, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 293, + "start": 1695396122535554620, + "end": 1695396122536397587, + "rtt": 0.843008, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 294, + "start": 1695396122635735998, + "end": 1695396122636830766, + "rtt": 1.094656, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 295, + "start": 1695396122736002405, + "end": 1695396122737076978, + "rtt": 1.074688, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 296, + "start": 1695396122835295269, + "end": 1695396122836274838, + "rtt": 0.979712, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 297, + "start": 1695396122935366535, + "end": 1695396122935868100, + "rtt": 0.501504, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 298, + "start": 1695396123035193173, + "end": 1695396123056401874, + "rtt": 21.208832, + "err_desc": "" + }, + { + "wl_id": 299, + "start": 1695396123135651554, + "end": 1695396123146468024, + "rtt": 10.816512, + "err_desc": "" + }, + { + "wl_id": 300, + "start": 1695396123235980325, + "end": 1695396123247154319, + "rtt": 11.174144, + "err_desc": "" + }, + { + "wl_id": 301, + "start": 1695396123335299771, + "end": 1695396123346486859, + "rtt": 11.186944, + "err_desc": "" + }, + { + "wl_id": 302, + "start": 1695396123435861526, + "end": 1695396123447417032, + "rtt": 11.555584, + "err_desc": "" + }, + { + "wl_id": 303, + "start": 1695396123535652777, + "end": 1695396123547235677, + "rtt": 11.58272, + "err_desc": "" + }, + { + "wl_id": 304, + "start": 1695396123635536345, + "end": 1695396123646579489, + "rtt": 11.043072, + "err_desc": "" + }, + { + "wl_id": 305, + "start": 1695396123736034933, + "end": 1695396123747617825, + "rtt": 11.582976, + "err_desc": "" + }, + { + "wl_id": 306, + "start": 1695396123835973744, + "end": 1695396123847886055, + "rtt": 11.912448, + "err_desc": "" + }, + { + "wl_id": 307, + "start": 1695396123935172488, + "end": 1695396123946813401, + "rtt": 11.640832, + "err_desc": "" + }, + { + "wl_id": 308, + "start": 1695396124036124479, + "end": 1695396124047885211, + "rtt": 11.760896, + "err_desc": "" + }, + { + "wl_id": 309, + "start": 1695396124136191249, + "end": 1695396124146867906, + "rtt": 10.676736, + "err_desc": "" + }, + { + "wl_id": 310, + "start": 1695396124236126460, + "end": 1695396124243107284, + "rtt": 6.980864, + "err_desc": "" + }, + { + "wl_id": 311, + "start": 1695396124335278631, + "end": 1695396124343309987, + "rtt": 8.031488, + "err_desc": "" + }, + { + "wl_id": 312, + "start": 1695396124435575323, + "end": 1695396124446712240, + "rtt": 11.137024, + "err_desc": "" + }, + { + "wl_id": 313, + "start": 1695396124536015747, + "end": 1695396124547924134, + "rtt": 11.908352, + "err_desc": "" + }, + { + "wl_id": 314, + "start": 1695396124635186325, + "end": 1695396124647088409, + "rtt": 11.901952, + "err_desc": "" + }, + { + "wl_id": 315, + "start": 1695396124735259239, + "end": 1695396124746818783, + "rtt": 11.55968, + "err_desc": "" + }, + { + "wl_id": 316, + "start": 1695396124836148728, + "end": 1695396124848055312, + "rtt": 11.90656, + "err_desc": "" + }, + { + "wl_id": 317, + "start": 1695396124935211832, + "end": 1695396124946908215, + "rtt": 11.696384, + "err_desc": "" + }, + { + "wl_id": 318, + "start": 1695396125035181549, + "end": 1695396125047074395, + "rtt": 11.892736, + "err_desc": "" + }, + { + "wl_id": 319, + "start": 1695396125135268581, + "end": 1695396125146309830, + "rtt": 11.04128, + "err_desc": "" + }, + { + "wl_id": 320, + "start": 1695396125235646914, + "end": 1695396125246892939, + "rtt": 11.24608, + "err_desc": "" + }, + { + "wl_id": 321, + "start": 1695396125336243245, + "end": 1695396125348029447, + "rtt": 11.78624, + "err_desc": "" + }, + { + "wl_id": 322, + "start": 1695396125435235934, + "end": 1695396125446456454, + "rtt": 11.220736, + "err_desc": "" + }, + { + "wl_id": 323, + "start": 1695396125535776425, + "end": 1695396125547214250, + "rtt": 11.437824, + "err_desc": "" + }, + { + "wl_id": 324, + "start": 1695396125635497789, + "end": 1695396125647657621, + "rtt": 12.16, + "err_desc": "" + }, + { + "wl_id": 325, + "start": 1695396125736004828, + "end": 1695396125747829116, + "rtt": 11.824128, + "err_desc": "" + }, + { + "wl_id": 326, + "start": 1695396125836153822, + "end": 1695396125848245762, + "rtt": 12.091904, + "err_desc": "" + }, + { + "wl_id": 327, + "start": 1695396125935654816, + "end": 1695396125947545333, + "rtt": 11.890432, + "err_desc": "" + }, + { + "wl_id": 328, + "start": 1695396126035909576, + "end": 1695396126047455625, + "rtt": 11.546112, + "err_desc": "" + }, + { + "wl_id": 329, + "start": 1695396126135799448, + "end": 1695396126147348553, + "rtt": 11.548928, + "err_desc": "" + }, + { + "wl_id": 330, + "start": 1695396126235736447, + "end": 1695396126248476205, + "rtt": 12.73984, + "err_desc": "" + }, + { + "wl_id": 331, + "start": 1695396126335772268, + "end": 1695396126348154713, + "rtt": 12.382464, + "err_desc": "" + }, + { + "wl_id": 332, + "start": 1695396126435415824, + "end": 1695396126446227059, + "rtt": 10.811136, + "err_desc": "" + }, + { + "wl_id": 333, + "start": 1695396126535544900, + "end": 1695396126547181819, + "rtt": 11.636992, + "err_desc": "" + }, + { + "wl_id": 334, + "start": 1695396126635529900, + "end": 1695396126648468136, + "rtt": 12.93824, + "err_desc": "" + }, + { + "wl_id": 335, + "start": 1695396126735781619, + "end": 1695396126744669692, + "rtt": 8.888064, + "err_desc": "" + }, + { + "wl_id": 336, + "start": 1695396126835823044, + "end": 1695396127594451085, + "rtt": 758.628096, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 337, + "start": 1695396127594462261, + "end": 1695396127595125438, + "rtt": 0.663296, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 338, + "start": 1695396127635250101, + "end": 1695396127636168319, + "rtt": 0.918016, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 339, + "start": 1695396127735771228, + "end": 1695396127736776917, + "rtt": 1.005824, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 340, + "start": 1695396127836081864, + "end": 1695396127837107780, + "rtt": 1.025792, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 341, + "start": 1695396127935235666, + "end": 1695396127936199084, + "rtt": 0.963584, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 342, + "start": 1695396128035656729, + "end": 1695396128036477018, + "rtt": 0.820224, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 343, + "start": 1695396128135713093, + "end": 1695396128136044351, + "rtt": 0.331264, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 344, + "start": 1695396128235260692, + "end": 1695396128236166949, + "rtt": 0.90624, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 345, + "start": 1695396128335316799, + "end": 1695396128336220335, + "rtt": 0.90368, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 346, + "start": 1695396128435353131, + "end": 1695396128435994628, + "rtt": 0.641536, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 347, + "start": 1695396128535516584, + "end": 1695396128536467829, + "rtt": 0.95104, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 348, + "start": 1695396128635793053, + "end": 1695396128636788404, + "rtt": 0.995328, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 349, + "start": 1695396128736011884, + "end": 1695396128737027097, + "rtt": 1.015296, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 350, + "start": 1695396128835130031, + "end": 1695396128835585629, + "rtt": 0.455424, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 351, + "start": 1695396128936055561, + "end": 1695396128955441446, + "rtt": 19.385856, + "err_desc": "" + }, + { + "wl_id": 352, + "start": 1695396129035691240, + "end": 1695396129046795173, + "rtt": 11.104, + "err_desc": "" + }, + { + "wl_id": 353, + "start": 1695396129136117221, + "end": 1695396129145842546, + "rtt": 9.725184, + "err_desc": "" + }, + { + "wl_id": 354, + "start": 1695396129236181898, + "end": 1695396129244156333, + "rtt": 7.9744, + "err_desc": "" + }, + { + "wl_id": 355, + "start": 1695396129335278276, + "end": 1695396129347157282, + "rtt": 11.878912, + "err_desc": "" + }, + { + "wl_id": 356, + "start": 1695396129435239461, + "end": 1695396129446983254, + "rtt": 11.743744, + "err_desc": "" + }, + { + "wl_id": 357, + "start": 1695396129536138031, + "end": 1695396129546473057, + "rtt": 10.334976, + "err_desc": "" + }, + { + "wl_id": 358, + "start": 1695396129635654345, + "end": 1695396129647695798, + "rtt": 12.041472, + "err_desc": "" + }, + { + "wl_id": 359, + "start": 1695396129735983785, + "end": 1695396129747886826, + "rtt": 11.902976, + "err_desc": "" + }, + { + "wl_id": 360, + "start": 1695396129836160813, + "end": 1695396129847740351, + "rtt": 11.579648, + "err_desc": "" + }, + { + "wl_id": 361, + "start": 1695396129935987722, + "end": 1695396129948107863, + "rtt": 12.120064, + "err_desc": "" + }, + { + "wl_id": 362, + "start": 1695396130035377911, + "end": 1695396130046419347, + "rtt": 11.041536, + "err_desc": "" + }, + { + "wl_id": 363, + "start": 1695396130135828179, + "end": 1695396130147518828, + "rtt": 11.690496, + "err_desc": "" + }, + { + "wl_id": 364, + "start": 1695396130235867784, + "end": 1695396130246764565, + "rtt": 10.89664, + "err_desc": "" + }, + { + "wl_id": 365, + "start": 1695396130336033767, + "end": 1695396130347873068, + "rtt": 11.839232, + "err_desc": "" + }, + { + "wl_id": 366, + "start": 1695396130436136406, + "end": 1695396130447541155, + "rtt": 11.4048, + "err_desc": "" + }, + { + "wl_id": 367, + "start": 1695396130535725323, + "end": 1695396130542457759, + "rtt": 6.732544, + "err_desc": "" + }, + { + "wl_id": 368, + "start": 1695396130635823010, + "end": 1695396130653222604, + "rtt": 17.399552, + "err_desc": "" + }, + { + "wl_id": 369, + "start": 1695396130735447092, + "end": 1695396130747042861, + "rtt": 11.595776, + "err_desc": "" + }, + { + "wl_id": 370, + "start": 1695396130835199334, + "end": 1695396130847422239, + "rtt": 12.222976, + "err_desc": "" + }, + { + "wl_id": 371, + "start": 1695396130935570535, + "end": 1695396130947121413, + "rtt": 11.550976, + "err_desc": "" + }, + { + "wl_id": 372, + "start": 1695396131035275338, + "end": 1695396131046720324, + "rtt": 11.444992, + "err_desc": "" + }, + { + "wl_id": 373, + "start": 1695396131136000953, + "end": 1695396131147445889, + "rtt": 11.444992, + "err_desc": "" + }, + { + "wl_id": 374, + "start": 1695396131235751331, + "end": 1695396131243320692, + "rtt": 7.569152, + "err_desc": "" + }, + { + "wl_id": 375, + "start": 1695396131335688694, + "end": 1695396131347303478, + "rtt": 11.61472, + "err_desc": "" + }, + { + "wl_id": 376, + "start": 1695396131435604075, + "end": 1695396131446834585, + "rtt": 11.23072, + "err_desc": "" + }, + { + "wl_id": 377, + "start": 1695396131536189101, + "end": 1695396131546988287, + "rtt": 10.799104, + "err_desc": "" + }, + { + "wl_id": 378, + "start": 1695396131635295355, + "end": 1695396131648384023, + "rtt": 13.088768, + "err_desc": "" + }, + { + "wl_id": 379, + "start": 1695396131735735766, + "end": 1695396131747515603, + "rtt": 11.77984, + "err_desc": "" + }, + { + "wl_id": 380, + "start": 1695396131835913198, + "end": 1695396131847498065, + "rtt": 11.584768, + "err_desc": "" + }, + { + "wl_id": 381, + "start": 1695396131935741169, + "end": 1695396131947382388, + "rtt": 11.641088, + "err_desc": "" + }, + { + "wl_id": 382, + "start": 1695396132035571564, + "end": 1695396132046886223, + "rtt": 11.314688, + "err_desc": "" + }, + { + "wl_id": 383, + "start": 1695396132136125735, + "end": 1695396132147528074, + "rtt": 11.402496, + "err_desc": "" + }, + { + "wl_id": 384, + "start": 1695396132235708175, + "end": 1695396132247784045, + "rtt": 12.075776, + "err_desc": "" + }, + { + "wl_id": 385, + "start": 1695396132336027161, + "end": 1695396132348366824, + "rtt": 12.339712, + "err_desc": "" + }, + { + "wl_id": 386, + "start": 1695396132435632488, + "end": 1695396132447599382, + "rtt": 11.966976, + "err_desc": "" + }, + { + "wl_id": 387, + "start": 1695396132536079624, + "end": 1695396132548334743, + "rtt": 12.255232, + "err_desc": "" + }, + { + "wl_id": 388, + "start": 1695396132635601108, + "end": 1695396132647620620, + "rtt": 12.019456, + "err_desc": "" + }, + { + "wl_id": 389, + "start": 1695396132736145193, + "end": 1695396133443809751, + "rtt": 707.66464, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 390, + "start": 1695396133443815445, + "end": 1695396133443991190, + "rtt": 0.175872, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 391, + "start": 1695396133535107576, + "end": 1695396133535560539, + "rtt": 0.452864, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 392, + "start": 1695396133635413048, + "end": 1695396133635715256, + "rtt": 0.302336, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 393, + "start": 1695396133735887106, + "end": 1695396133736265433, + "rtt": 0.378368, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 394, + "start": 1695396133835676109, + "end": 1695396133836632982, + "rtt": 0.956928, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 395, + "start": 1695396133935865482, + "end": 1695396133937029846, + "rtt": 1.164288, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 396, + "start": 1695396134035207215, + "end": 1695396134036046454, + "rtt": 0.839168, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 397, + "start": 1695396134135165674, + "end": 1695396134135514312, + "rtt": 0.348672, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 398, + "start": 1695396134235660736, + "end": 1695396134236026975, + "rtt": 0.36608, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 399, + "start": 1695396134335182638, + "end": 1695396134336290038, + "rtt": 1.107456, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 400, + "start": 1695396134435571143, + "end": 1695396134436373416, + "rtt": 0.802304, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 401, + "start": 1695396134535739065, + "end": 1695396134536723007, + "rtt": 0.983808, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 402, + "start": 1695396134636232583, + "end": 1695396134637198855, + "rtt": 0.966144, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 403, + "start": 1695396134735482555, + "end": 1695396134735913307, + "rtt": 0.430592, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 404, + "start": 1695396134835141125, + "end": 1695396134853315106, + "rtt": 18.173952, + "err_desc": "" + }, + { + "wl_id": 405, + "start": 1695396134935667766, + "end": 1695396134947221115, + "rtt": 11.55328, + "err_desc": "" + }, + { + "wl_id": 406, + "start": 1695396135035484735, + "end": 1695396135046893978, + "rtt": 11.409408, + "err_desc": "" + }, + { + "wl_id": 407, + "start": 1695396135135193256, + "end": 1695396135145430334, + "rtt": 10.236928, + "err_desc": "" + }, + { + "wl_id": 408, + "start": 1695396135235659984, + "end": 1695396135241851029, + "rtt": 6.191104, + "err_desc": "" + }, + { + "wl_id": 409, + "start": 1695396135335151816, + "end": 1695396135346368522, + "rtt": 11.21664, + "err_desc": "" + }, + { + "wl_id": 410, + "start": 1695396135435287345, + "end": 1695396135446523516, + "rtt": 11.236096, + "err_desc": "" + }, + { + "wl_id": 411, + "start": 1695396135535813518, + "end": 1695396135546842426, + "rtt": 11.028736, + "err_desc": "" + }, + { + "wl_id": 412, + "start": 1695396135636164975, + "end": 1695396135646711993, + "rtt": 10.5472, + "err_desc": "" + }, + { + "wl_id": 413, + "start": 1695396135736176568, + "end": 1695396135747206809, + "rtt": 11.030272, + "err_desc": "" + }, + { + "wl_id": 414, + "start": 1695396135835366700, + "end": 1695396135846723483, + "rtt": 11.356928, + "err_desc": "" + }, + { + "wl_id": 415, + "start": 1695396135935981463, + "end": 1695396135947654568, + "rtt": 11.673088, + "err_desc": "" + }, + { + "wl_id": 416, + "start": 1695396136035809975, + "end": 1695396136044078057, + "rtt": 8.268032, + "err_desc": "" + }, + { + "wl_id": 417, + "start": 1695396136135191374, + "end": 1695396136146511125, + "rtt": 11.319808, + "err_desc": "" + }, + { + "wl_id": 418, + "start": 1695396136235786740, + "end": 1695396136247260253, + "rtt": 11.473408, + "err_desc": "" + }, + { + "wl_id": 419, + "start": 1695396136335433967, + "end": 1695396136346836264, + "rtt": 11.40224, + "err_desc": "" + }, + { + "wl_id": 420, + "start": 1695396136436093592, + "end": 1695396136447642721, + "rtt": 11.548928, + "err_desc": "" + }, + { + "wl_id": 421, + "start": 1695396136535830162, + "end": 1695396136547000402, + "rtt": 11.170048, + "err_desc": "" + }, + { + "wl_id": 422, + "start": 1695396136635168054, + "end": 1695396136647110223, + "rtt": 11.942144, + "err_desc": "" + }, + { + "wl_id": 423, + "start": 1695396136735224983, + "end": 1695396136746200851, + "rtt": 10.975744, + "err_desc": "" + }, + { + "wl_id": 424, + "start": 1695396136835514075, + "end": 1695396136847004672, + "rtt": 11.49056, + "err_desc": "" + }, + { + "wl_id": 425, + "start": 1695396136935173857, + "end": 1695396136946629277, + "rtt": 11.455488, + "err_desc": "" + }, + { + "wl_id": 426, + "start": 1695396137036005049, + "end": 1695396137047307105, + "rtt": 11.301888, + "err_desc": "" + }, + { + "wl_id": 427, + "start": 1695396137135439235, + "end": 1695396137141878678, + "rtt": 6.439424, + "err_desc": "" + }, + { + "wl_id": 428, + "start": 1695396137236149115, + "end": 1695396137244305456, + "rtt": 8.156416, + "err_desc": "" + }, + { + "wl_id": 429, + "start": 1695396137335553491, + "end": 1695396137347345518, + "rtt": 11.791872, + "err_desc": "" + }, + { + "wl_id": 430, + "start": 1695396137435574008, + "end": 1695396137446351695, + "rtt": 10.7776, + "err_desc": "" + }, + { + "wl_id": 431, + "start": 1695396137535606909, + "end": 1695396137547004157, + "rtt": 11.397376, + "err_desc": "" + }, + { + "wl_id": 432, + "start": 1695396137635164717, + "end": 1695396137646797237, + "rtt": 11.63264, + "err_desc": "" + }, + { + "wl_id": 433, + "start": 1695396137736150694, + "end": 1695396137747916799, + "rtt": 11.766016, + "err_desc": "" + }, + { + "wl_id": 434, + "start": 1695396137835167528, + "end": 1695396137847046224, + "rtt": 11.878656, + "err_desc": "" + }, + { + "wl_id": 435, + "start": 1695396137935268495, + "end": 1695396137947084943, + "rtt": 11.816448, + "err_desc": "" + }, + { + "wl_id": 436, + "start": 1695396138035260820, + "end": 1695396138046726452, + "rtt": 11.465472, + "err_desc": "" + }, + { + "wl_id": 437, + "start": 1695396138135973861, + "end": 1695396138144921118, + "rtt": 8.9472, + "err_desc": "" + }, + { + "wl_id": 438, + "start": 1695396138235165282, + "end": 1695396138246812084, + "rtt": 11.646976, + "err_desc": "" + }, + { + "wl_id": 439, + "start": 1695396138335959203, + "end": 1695396138349369189, + "rtt": 13.409792, + "err_desc": "" + }, + { + "wl_id": 440, + "start": 1695396138435775447, + "end": 1695396138446026323, + "rtt": 10.250752, + "err_desc": "" + }, + { + "wl_id": 441, + "start": 1695396138535463463, + "end": 1695396138546501631, + "rtt": 11.038208, + "err_desc": "" + }, + { + "wl_id": 442, + "start": 1695396138635821111, + "end": 1695396138647576401, + "rtt": 11.755264, + "err_desc": "" + }, + { + "wl_id": 443, + "start": 1695396138735829289, + "end": 1695396138747074864, + "rtt": 11.245568, + "err_desc": "" + }, + { + "wl_id": 444, + "start": 1695396138835378572, + "end": 1695396138843108822, + "rtt": 7.730176, + "err_desc": "" + }, + { + "wl_id": 445, + "start": 1695396138935218997, + "end": 1695396138935674960, + "rtt": 0.455936, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 446, + "start": 1695396139035895048, + "end": 1695396139036345111, + "rtt": 0.450048, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 447, + "start": 1695396139135498754, + "end": 1695396139135992637, + "rtt": 0.493824, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 448, + "start": 1695396139236151987, + "end": 1695396139236522504, + "rtt": 0.370432, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 449, + "start": 1695396139335801514, + "end": 1695396139336689641, + "rtt": 0.888064, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 450, + "start": 1695396139435800124, + "end": 1695396139436246117, + "rtt": 0.445952, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 451, + "start": 1695396139536061312, + "end": 1695396139536489658, + "rtt": 0.428288, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 452, + "start": 1695396139635727819, + "end": 1695396139636752574, + "rtt": 1.024768, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 453, + "start": 1695396139735412631, + "end": 1695396139736244710, + "rtt": 0.832, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 454, + "start": 1695396139835605053, + "end": 1695396139836574008, + "rtt": 0.96896, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 455, + "start": 1695396139935944308, + "end": 1695396139937021850, + "rtt": 1.07776, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 456, + "start": 1695396140035266954, + "end": 1695396140036114876, + "rtt": 0.847872, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 457, + "start": 1695396140135502130, + "end": 1695396140136555086, + "rtt": 1.052928, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 458, + "start": 1695396140235842353, + "end": 1695396140236701955, + "rtt": 0.859648, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 459, + "start": 1695396140336140186, + "end": 1695396140337317905, + "rtt": 1.1776, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 460, + "start": 1695396140435648264, + "end": 1695396140436672679, + "rtt": 1.024512, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 461, + "start": 1695396140535329236, + "end": 1695396140535837848, + "rtt": 0.508672, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 462, + "start": 1695396140635147069, + "end": 1695396140636109822, + "rtt": 0.962816, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 463, + "start": 1695396140735302257, + "end": 1695396140736185521, + "rtt": 0.883456, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 464, + "start": 1695396140835384990, + "end": 1695396140836363755, + "rtt": 0.978688, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 465, + "start": 1695396140935559754, + "end": 1695396140954579719, + "rtt": 19.020032, + "err_desc": "" + }, + { + "wl_id": 466, + "start": 1695396141035834858, + "end": 1695396141046704486, + "rtt": 10.869504, + "err_desc": "" + }, + { + "wl_id": 467, + "start": 1695396141135892804, + "end": 1695396141146405620, + "rtt": 10.512896, + "err_desc": "" + }, + { + "wl_id": 468, + "start": 1695396141235576990, + "end": 1695396141241928323, + "rtt": 6.35136, + "err_desc": "" + }, + { + "wl_id": 469, + "start": 1695396141335149697, + "end": 1695396141353377304, + "rtt": 18.227456, + "err_desc": "" + }, + { + "wl_id": 470, + "start": 1695396141435634348, + "end": 1695396141447244703, + "rtt": 11.610368, + "err_desc": "" + }, + { + "wl_id": 471, + "start": 1695396141535455219, + "end": 1695396141545561353, + "rtt": 10.106112, + "err_desc": "" + }, + { + "wl_id": 472, + "start": 1695396141635424870, + "end": 1695396141646523070, + "rtt": 11.098368, + "err_desc": "" + }, + { + "wl_id": 473, + "start": 1695396141735874709, + "end": 1695396141747090487, + "rtt": 11.215616, + "err_desc": "" + }, + { + "wl_id": 474, + "start": 1695396141835279051, + "end": 1695396141846772035, + "rtt": 11.492864, + "err_desc": "" + }, + { + "wl_id": 475, + "start": 1695396141936085871, + "end": 1695396141947139326, + "rtt": 11.053568, + "err_desc": "" + }, + { + "wl_id": 476, + "start": 1695396142035382338, + "end": 1695396142046098475, + "rtt": 10.71616, + "err_desc": "" + }, + { + "wl_id": 477, + "start": 1695396142135530453, + "end": 1695396142147297808, + "rtt": 11.767296, + "err_desc": "" + }, + { + "wl_id": 478, + "start": 1695396142235466764, + "end": 1695396142246836328, + "rtt": 11.369472, + "err_desc": "" + }, + { + "wl_id": 479, + "start": 1695396142336097528, + "end": 1695396142347700958, + "rtt": 11.603456, + "err_desc": "" + }, + { + "wl_id": 480, + "start": 1695396142435977850, + "end": 1695396142446849323, + "rtt": 10.871552, + "err_desc": "" + }, + { + "wl_id": 481, + "start": 1695396142536103268, + "end": 1695396142546556931, + "rtt": 10.45376, + "err_desc": "" + }, + { + "wl_id": 482, + "start": 1695396142635849471, + "end": 1695396142647708222, + "rtt": 11.858688, + "err_desc": "" + }, + { + "wl_id": 483, + "start": 1695396142736123398, + "end": 1695396142747510047, + "rtt": 11.386624, + "err_desc": "" + }, + { + "wl_id": 484, + "start": 1695396142835751260, + "end": 1695396142846913049, + "rtt": 11.161856, + "err_desc": "" + }, + { + "wl_id": 485, + "start": 1695396142936164857, + "end": 1695396142947747011, + "rtt": 11.582208, + "err_desc": "" + }, + { + "wl_id": 486, + "start": 1695396143036184661, + "end": 1695396143047809071, + "rtt": 11.624448, + "err_desc": "" + }, + { + "wl_id": 487, + "start": 1695396143135993751, + "end": 1695396143146965437, + "rtt": 10.971648, + "err_desc": "" + }, + { + "wl_id": 488, + "start": 1695396143235177713, + "end": 1695396143242335965, + "rtt": 7.158272, + "err_desc": "" + }, + { + "wl_id": 489, + "start": 1695396143335650309, + "end": 1695396143346820408, + "rtt": 11.170048, + "err_desc": "" + }, + { + "wl_id": 490, + "start": 1695396143436140290, + "end": 1695396143449394166, + "rtt": 13.253888, + "err_desc": "" + }, + { + "wl_id": 491, + "start": 1695396143535831659, + "end": 1695396143546794456, + "rtt": 10.962944, + "err_desc": "" + }, + { + "wl_id": 492, + "start": 1695396143636044612, + "end": 1695396143647481403, + "rtt": 11.4368, + "err_desc": "" + }, + { + "wl_id": 493, + "start": 1695396143735763178, + "end": 1695396143746910155, + "rtt": 11.147008, + "err_desc": "" + }, + { + "wl_id": 494, + "start": 1695396143836173049, + "end": 1695396143847173327, + "rtt": 11.00032, + "err_desc": "" + }, + { + "wl_id": 495, + "start": 1695396143935498700, + "end": 1695396143946566716, + "rtt": 11.067904, + "err_desc": "" + }, + { + "wl_id": 496, + "start": 1695396144035868745, + "end": 1695396144046577402, + "rtt": 10.708736, + "err_desc": "" + }, + { + "wl_id": 497, + "start": 1695396144135873317, + "end": 1695396144146668793, + "rtt": 10.79552, + "err_desc": "" + }, + { + "wl_id": 498, + "start": 1695396144236007289, + "end": 1695396144242347919, + "rtt": 6.340864, + "err_desc": "" + }, + { + "wl_id": 499, + "start": 1695396144335564621, + "end": 1695396144354378176, + "rtt": 18.813696, + "err_desc": "" + }, + { + "wl_id": 500, + "start": 1695396144435529435, + "end": 1695396144447111117, + "rtt": 11.581696, + "err_desc": "" + }, + { + "wl_id": 501, + "start": 1695396144535213978, + "end": 1695396144545346248, + "rtt": 10.132224, + "err_desc": "" + }, + { + "wl_id": 502, + "start": 1695396144635611930, + "end": 1695396144647732520, + "rtt": 12.120576, + "err_desc": "" + }, + { + "wl_id": 503, + "start": 1695396144735960006, + "end": 1695396145678921836, + "rtt": 942.961664, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + }, + { + "wl_id": 504, + "start": 1695396145678933819, + "end": 1695396145679680257, + "rtt": 0.746496, + "err_desc": "NotFound: Not Found\n\tstatus code: 404, request id: , host id: " + } + ] + } + ], + "time_unit": "ms" +} \ No newline at end of file diff --git a/docs/research/ha/measurements/scale_deployment_0_1-k3s3nodes-zeroload-emptydb/scale_deployment_0_1-k3s3nodes-zeroload-emptydb_percentiles_fup_main_delta_1695046129.svg b/docs/research/ha/measurements/scale_deployment_0_1-k3s3nodes-zeroload-emptydb/scale_deployment_0_1-k3s3nodes-zeroload-emptydb_percentiles_fup_main_delta_1695046129.svg new file mode 100644 index 00000000..9470597e --- /dev/null +++ b/docs/research/ha/measurements/scale_deployment_0_1-k3s3nodes-zeroload-emptydb/scale_deployment_0_1-k3s3nodes-zeroload-emptydb_percentiles_fup_main_delta_1695046129.svg @@ -0,0 +1,157 @@ + + + + + +Percentiles - FrontEndUp-Main-Delta (Nearest Rank): scale_deployment_0_1-k3s3nodes-zeroload-emptydb +Percentile +20 +60 +100 + + + + + + + + + + + + +Duration: ms + +75.00 +175.00 +275.00 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +to-fronted-up + + diff --git a/docs/research/ha/measurements/scale_deployment_0_1-k3s3nodes-zeroload-emptydb/scale_deployment_0_1-k3s3nodes-zeroload-emptydb_percentiles_to_fup_1695046129.svg b/docs/research/ha/measurements/scale_deployment_0_1-k3s3nodes-zeroload-emptydb/scale_deployment_0_1-k3s3nodes-zeroload-emptydb_percentiles_to_fup_1695046129.svg new file mode 100644 index 00000000..b214c330 --- /dev/null +++ b/docs/research/ha/measurements/scale_deployment_0_1-k3s3nodes-zeroload-emptydb/scale_deployment_0_1-k3s3nodes-zeroload-emptydb_percentiles_to_fup_1695046129.svg @@ -0,0 +1,157 @@ + + + + + +Percentiles - FrontEndUp (Nearest Rank): scale_deployment_0_1-k3s3nodes-zeroload-emptydb +Percentile +20 +60 +100 + + + + + + + + + + + + +Duration: ms + +2500.00 +12500.00 +22500.00 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +to-fronted-up + + diff --git a/docs/research/ha/measurements/scale_deployment_0_1-k3s3nodes-zeroload-emptydb/scale_deployment_0_1-k3s3nodes-zeroload-emptydb_percentiles_to_main_1695046129.svg b/docs/research/ha/measurements/scale_deployment_0_1-k3s3nodes-zeroload-emptydb/scale_deployment_0_1-k3s3nodes-zeroload-emptydb_percentiles_to_main_1695046129.svg new file mode 100644 index 00000000..251626fa --- /dev/null +++ b/docs/research/ha/measurements/scale_deployment_0_1-k3s3nodes-zeroload-emptydb/scale_deployment_0_1-k3s3nodes-zeroload-emptydb_percentiles_to_main_1695046129.svg @@ -0,0 +1,157 @@ + + + + + +Percentiles - Main (Nearest Rank): scale_deployment_0_1-k3s3nodes-zeroload-emptydb +Percentile +20 +60 +100 + + + + + + + + + + + + +Duration: ms + +2500.00 +12500.00 +22500.00 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +to-main + + diff --git a/docs/research/ha/measurements/scale_deployment_0_1-k3s3nodes-zeroload-emptydb/scale_deployment_0_1-k3s3nodes-zeroload-emptydb_raw_1695046129.svg b/docs/research/ha/measurements/scale_deployment_0_1-k3s3nodes-zeroload-emptydb/scale_deployment_0_1-k3s3nodes-zeroload-emptydb_raw_1695046129.svg new file mode 100644 index 00000000..54632299 --- /dev/null +++ b/docs/research/ha/measurements/scale_deployment_0_1-k3s3nodes-zeroload-emptydb/scale_deployment_0_1-k3s3nodes-zeroload-emptydb_raw_1695046129.svg @@ -0,0 +1,960 @@ + + + + + +Raw Data Measures: scale_deployment_0_1-k3s3nodes-zeroload-emptydb +Restart ID +0.00 +250.00 + + + + + + + + +Duration: ms + +2500.00 +12500.00 +22500.00 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +to-main + + +to-fronted-up + + +fronted-up-main-delta + + diff --git a/docs/research/ha/measurements/scale_deployment_0_1-k3s3nodes-zeroload-emptydb/scale_deployment_0_1-k3s3nodes-zeroload-emptydb_stats_1695046129.json b/docs/research/ha/measurements/scale_deployment_0_1-k3s3nodes-zeroload-emptydb/scale_deployment_0_1-k3s3nodes-zeroload-emptydb_stats_1695046129.json new file mode 100644 index 00000000..b4fbfaf4 --- /dev/null +++ b/docs/research/ha/measurements/scale_deployment_0_1-k3s3nodes-zeroload-emptydb/scale_deployment_0_1-k3s3nodes-zeroload-emptydb_stats_1695046129.json @@ -0,0 +1,1832 @@ +{ + "series_count": 0, + "series": [ + { + "mark": "scale_deployment_0_1-k3s3nodes-zeroload-emptydb", + "min_to_main": 318, + "max_to_main": 24111, + "mean_to_main": 1512, + "99p_to_main": 2398, + "95p_to_main": 1935, + "99pNR_to_main": 2398, + "95pNR_to_main": 1935, + "min_to_frontend_up": 434, + "max_to_frontend_up": 24254, + "mean_to_frontend_up": 1646, + "99p_to_frontend_up": 2523, + "95p_to_frontend_up": 2102, + "99pNR_to_frontend_up": 2523, + "95pNR_to_frontend_up": 2102, + "min_frontend_up_main_delta": 71, + "max_frontend_up_main_delta": 287, + "mean_frontend_up_main_delta": 133, + "99p_frontend_up_main_delta": 253, + "95p_frontend_up_main_delta": 204, + "99pNR_frontend_up_main_delta": 253, + "95pNR_frontend_up_main_delta": 204, + "data": [ + { + "restart_id": 1, + "duration_to_main": 763, + "duration_to_frontend_up": 841, + "frontend_up_main_delta": 77 + }, + { + "restart_id": 2, + "duration_to_main": 846, + "duration_to_frontend_up": 921, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 3, + "duration_to_main": 1800, + "duration_to_frontend_up": 1874, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 4, + "duration_to_main": 1608, + "duration_to_frontend_up": 1684, + "frontend_up_main_delta": 76 + }, + { + "restart_id": 5, + "duration_to_main": 1379, + "duration_to_frontend_up": 1460, + "frontend_up_main_delta": 81 + }, + { + "restart_id": 6, + "duration_to_main": 1146, + "duration_to_frontend_up": 1219, + "frontend_up_main_delta": 72 + }, + { + "restart_id": 7, + "duration_to_main": 881, + "duration_to_frontend_up": 955, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 8, + "duration_to_main": 2730, + "duration_to_frontend_up": 2805, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 9, + "duration_to_main": 1232, + "duration_to_frontend_up": 1324, + "frontend_up_main_delta": 92 + }, + { + "restart_id": 10, + "duration_to_main": 683, + "duration_to_frontend_up": 755, + "frontend_up_main_delta": 71 + }, + { + "restart_id": 11, + "duration_to_main": 2030, + "duration_to_frontend_up": 2104, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 12, + "duration_to_main": 1224, + "duration_to_frontend_up": 1295, + "frontend_up_main_delta": 71 + }, + { + "restart_id": 13, + "duration_to_main": 1163, + "duration_to_frontend_up": 1238, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 14, + "duration_to_main": 1193, + "duration_to_frontend_up": 1272, + "frontend_up_main_delta": 78 + }, + { + "restart_id": 15, + "duration_to_main": 646, + "duration_to_frontend_up": 720, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 16, + "duration_to_main": 1657, + "duration_to_frontend_up": 1732, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 17, + "duration_to_main": 1364, + "duration_to_frontend_up": 1439, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 18, + "duration_to_main": 1093, + "duration_to_frontend_up": 1176, + "frontend_up_main_delta": 83 + }, + { + "restart_id": 19, + "duration_to_main": 834, + "duration_to_frontend_up": 907, + "frontend_up_main_delta": 73 + }, + { + "restart_id": 20, + "duration_to_main": 1826, + "duration_to_frontend_up": 1901, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 21, + "duration_to_main": 1821, + "duration_to_frontend_up": 1896, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 22, + "duration_to_main": 1566, + "duration_to_frontend_up": 1647, + "frontend_up_main_delta": 81 + }, + { + "restart_id": 23, + "duration_to_main": 1338, + "duration_to_frontend_up": 1418, + "frontend_up_main_delta": 79 + }, + { + "restart_id": 24, + "duration_to_main": 1131, + "duration_to_frontend_up": 1206, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 25, + "duration_to_main": 1181, + "duration_to_frontend_up": 1255, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 26, + "duration_to_main": 1214, + "duration_to_frontend_up": 1292, + "frontend_up_main_delta": 78 + }, + { + "restart_id": 27, + "duration_to_main": 665, + "duration_to_frontend_up": 744, + "frontend_up_main_delta": 78 + }, + { + "restart_id": 28, + "duration_to_main": 1995, + "duration_to_frontend_up": 2071, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 29, + "duration_to_main": 953, + "duration_to_frontend_up": 1028, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 30, + "duration_to_main": 1182, + "duration_to_frontend_up": 1263, + "frontend_up_main_delta": 81 + }, + { + "restart_id": 31, + "duration_to_main": 1227, + "duration_to_frontend_up": 1311, + "frontend_up_main_delta": 83 + }, + { + "restart_id": 32, + "duration_to_main": 952, + "duration_to_frontend_up": 1033, + "frontend_up_main_delta": 80 + }, + { + "restart_id": 33, + "duration_to_main": 1864, + "duration_to_frontend_up": 1940, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 34, + "duration_to_main": 1824, + "duration_to_frontend_up": 1899, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 35, + "duration_to_main": 1247, + "duration_to_frontend_up": 1335, + "frontend_up_main_delta": 88 + }, + { + "restart_id": 36, + "duration_to_main": 1224, + "duration_to_frontend_up": 1303, + "frontend_up_main_delta": 79 + }, + { + "restart_id": 37, + "duration_to_main": 1239, + "duration_to_frontend_up": 1321, + "frontend_up_main_delta": 81 + }, + { + "restart_id": 38, + "duration_to_main": 970, + "duration_to_frontend_up": 1047, + "frontend_up_main_delta": 77 + }, + { + "restart_id": 39, + "duration_to_main": 1844, + "duration_to_frontend_up": 1934, + "frontend_up_main_delta": 89 + }, + { + "restart_id": 40, + "duration_to_main": 1259, + "duration_to_frontend_up": 1332, + "frontend_up_main_delta": 72 + }, + { + "restart_id": 41, + "duration_to_main": 1831, + "duration_to_frontend_up": 1907, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 42, + "duration_to_main": 1548, + "duration_to_frontend_up": 1627, + "frontend_up_main_delta": 78 + }, + { + "restart_id": 43, + "duration_to_main": 1587, + "duration_to_frontend_up": 1666, + "frontend_up_main_delta": 79 + }, + { + "restart_id": 44, + "duration_to_main": 1322, + "duration_to_frontend_up": 1398, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 45, + "duration_to_main": 1069, + "duration_to_frontend_up": 1147, + "frontend_up_main_delta": 77 + }, + { + "restart_id": 46, + "duration_to_main": 1061, + "duration_to_frontend_up": 1133, + "frontend_up_main_delta": 72 + }, + { + "restart_id": 47, + "duration_to_main": 1084, + "duration_to_frontend_up": 1159, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 48, + "duration_to_main": 510, + "duration_to_frontend_up": 584, + "frontend_up_main_delta": 73 + }, + { + "restart_id": 49, + "duration_to_main": 1493, + "duration_to_frontend_up": 1568, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 50, + "duration_to_main": 1264, + "duration_to_frontend_up": 1342, + "frontend_up_main_delta": 78 + }, + { + "restart_id": 51, + "duration_to_main": 1004, + "duration_to_frontend_up": 1080, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 52, + "duration_to_main": 1818, + "duration_to_frontend_up": 1895, + "frontend_up_main_delta": 77 + }, + { + "restart_id": 53, + "duration_to_main": 1236, + "duration_to_frontend_up": 1311, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 54, + "duration_to_main": 1199, + "duration_to_frontend_up": 1278, + "frontend_up_main_delta": 79 + }, + { + "restart_id": 55, + "duration_to_main": 1243, + "duration_to_frontend_up": 1343, + "frontend_up_main_delta": 99 + }, + { + "restart_id": 56, + "duration_to_main": 718, + "duration_to_frontend_up": 792, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 57, + "duration_to_main": 1737, + "duration_to_frontend_up": 1810, + "frontend_up_main_delta": 72 + }, + { + "restart_id": 58, + "duration_to_main": 1486, + "duration_to_frontend_up": 1560, + "frontend_up_main_delta": 73 + }, + { + "restart_id": 59, + "duration_to_main": 1526, + "duration_to_frontend_up": 1599, + "frontend_up_main_delta": 72 + }, + { + "restart_id": 60, + "duration_to_main": 1282, + "duration_to_frontend_up": 1355, + "frontend_up_main_delta": 73 + }, + { + "restart_id": 61, + "duration_to_main": 991, + "duration_to_frontend_up": 1066, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 62, + "duration_to_main": 1853, + "duration_to_frontend_up": 1926, + "frontend_up_main_delta": 72 + }, + { + "restart_id": 63, + "duration_to_main": 449, + "duration_to_frontend_up": 524, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 64, + "duration_to_main": 1467, + "duration_to_frontend_up": 1548, + "frontend_up_main_delta": 80 + }, + { + "restart_id": 65, + "duration_to_main": 1244, + "duration_to_frontend_up": 1321, + "frontend_up_main_delta": 77 + }, + { + "restart_id": 66, + "duration_to_main": 940, + "duration_to_frontend_up": 1021, + "frontend_up_main_delta": 81 + }, + { + "restart_id": 67, + "duration_to_main": 1825, + "duration_to_frontend_up": 1903, + "frontend_up_main_delta": 77 + }, + { + "restart_id": 68, + "duration_to_main": 1255, + "duration_to_frontend_up": 1331, + "frontend_up_main_delta": 76 + }, + { + "restart_id": 69, + "duration_to_main": 1140, + "duration_to_frontend_up": 1235, + "frontend_up_main_delta": 94 + }, + { + "restart_id": 70, + "duration_to_main": 862, + "duration_to_frontend_up": 937, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 71, + "duration_to_main": 1845, + "duration_to_frontend_up": 1929, + "frontend_up_main_delta": 83 + }, + { + "restart_id": 72, + "duration_to_main": 1230, + "duration_to_frontend_up": 1316, + "frontend_up_main_delta": 86 + }, + { + "restart_id": 73, + "duration_to_main": 1382, + "duration_to_frontend_up": 1461, + "frontend_up_main_delta": 79 + }, + { + "restart_id": 74, + "duration_to_main": 1138, + "duration_to_frontend_up": 1216, + "frontend_up_main_delta": 78 + }, + { + "restart_id": 75, + "duration_to_main": 863, + "duration_to_frontend_up": 937, + "frontend_up_main_delta": 73 + }, + { + "restart_id": 76, + "duration_to_main": 1824, + "duration_to_frontend_up": 1900, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 77, + "duration_to_main": 1860, + "duration_to_frontend_up": 1936, + "frontend_up_main_delta": 76 + }, + { + "restart_id": 78, + "duration_to_main": 1541, + "duration_to_frontend_up": 1618, + "frontend_up_main_delta": 77 + }, + { + "restart_id": 79, + "duration_to_main": 1666, + "duration_to_frontend_up": 1738, + "frontend_up_main_delta": 72 + }, + { + "restart_id": 80, + "duration_to_main": 1406, + "duration_to_frontend_up": 1482, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 81, + "duration_to_main": 1142, + "duration_to_frontend_up": 1217, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 82, + "duration_to_main": 890, + "duration_to_frontend_up": 972, + "frontend_up_main_delta": 82 + }, + { + "restart_id": 83, + "duration_to_main": 1845, + "duration_to_frontend_up": 1923, + "frontend_up_main_delta": 77 + }, + { + "restart_id": 84, + "duration_to_main": 1231, + "duration_to_frontend_up": 1304, + "frontend_up_main_delta": 72 + }, + { + "restart_id": 85, + "duration_to_main": 1352, + "duration_to_frontend_up": 1426, + "frontend_up_main_delta": 73 + }, + { + "restart_id": 86, + "duration_to_main": 1139, + "duration_to_frontend_up": 1213, + "frontend_up_main_delta": 73 + }, + { + "restart_id": 87, + "duration_to_main": 869, + "duration_to_frontend_up": 942, + "frontend_up_main_delta": 73 + }, + { + "restart_id": 88, + "duration_to_main": 1870, + "duration_to_frontend_up": 1949, + "frontend_up_main_delta": 78 + }, + { + "restart_id": 89, + "duration_to_main": 1293, + "duration_to_frontend_up": 1368, + "frontend_up_main_delta": 74 + }, + { + "restart_id": 90, + "duration_to_main": 1172, + "duration_to_frontend_up": 1253, + "frontend_up_main_delta": 80 + }, + { + "restart_id": 91, + "duration_to_main": 918, + "duration_to_frontend_up": 996, + "frontend_up_main_delta": 78 + }, + { + "restart_id": 92, + "duration_to_main": 1833, + "duration_to_frontend_up": 1923, + "frontend_up_main_delta": 89 + }, + { + "restart_id": 93, + "duration_to_main": 1544, + "duration_to_frontend_up": 1619, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 94, + "duration_to_main": 1388, + "duration_to_frontend_up": 1461, + "frontend_up_main_delta": 72 + }, + { + "restart_id": 95, + "duration_to_main": 1116, + "duration_to_frontend_up": 1193, + "frontend_up_main_delta": 77 + }, + { + "restart_id": 96, + "duration_to_main": 1467, + "duration_to_frontend_up": 1552, + "frontend_up_main_delta": 84 + }, + { + "restart_id": 97, + "duration_to_main": 1213, + "duration_to_frontend_up": 1288, + "frontend_up_main_delta": 75 + }, + { + "restart_id": 98, + "duration_to_main": 1251, + "duration_to_frontend_up": 1337, + "frontend_up_main_delta": 85 + }, + { + "restart_id": 99, + "duration_to_main": 1025, + "duration_to_frontend_up": 1101, + "frontend_up_main_delta": 76 + }, + { + "restart_id": 100, + "duration_to_main": 2138, + "duration_to_frontend_up": 2212, + "frontend_up_main_delta": 73 + }, + { + "restart_id": 1, + "duration_to_main": 23217, + "duration_to_frontend_up": 23436, + "frontend_up_main_delta": 219 + }, + { + "restart_id": 2, + "duration_to_main": 1479, + "duration_to_frontend_up": 1626, + "frontend_up_main_delta": 147 + }, + { + "restart_id": 3, + "duration_to_main": 1191, + "duration_to_frontend_up": 1346, + "frontend_up_main_delta": 155 + }, + { + "restart_id": 4, + "duration_to_main": 877, + "duration_to_frontend_up": 1042, + "frontend_up_main_delta": 164 + }, + { + "restart_id": 5, + "duration_to_main": 1555, + "duration_to_frontend_up": 1696, + "frontend_up_main_delta": 140 + }, + { + "restart_id": 6, + "duration_to_main": 1494, + "duration_to_frontend_up": 1638, + "frontend_up_main_delta": 143 + }, + { + "restart_id": 7, + "duration_to_main": 1528, + "duration_to_frontend_up": 1674, + "frontend_up_main_delta": 145 + }, + { + "restart_id": 8, + "duration_to_main": 982, + "duration_to_frontend_up": 1261, + "frontend_up_main_delta": 278 + }, + { + "restart_id": 9, + "duration_to_main": 1437, + "duration_to_frontend_up": 1581, + "frontend_up_main_delta": 144 + }, + { + "restart_id": 10, + "duration_to_main": 1428, + "duration_to_frontend_up": 1571, + "frontend_up_main_delta": 143 + }, + { + "restart_id": 11, + "duration_to_main": 1495, + "duration_to_frontend_up": 1663, + "frontend_up_main_delta": 168 + }, + { + "restart_id": 12, + "duration_to_main": 1089, + "duration_to_frontend_up": 1333, + "frontend_up_main_delta": 243 + }, + { + "restart_id": 13, + "duration_to_main": 1414, + "duration_to_frontend_up": 1590, + "frontend_up_main_delta": 175 + }, + { + "restart_id": 14, + "duration_to_main": 1261, + "duration_to_frontend_up": 1405, + "frontend_up_main_delta": 144 + }, + { + "restart_id": 15, + "duration_to_main": 1367, + "duration_to_frontend_up": 1516, + "frontend_up_main_delta": 148 + }, + { + "restart_id": 16, + "duration_to_main": 1520, + "duration_to_frontend_up": 1672, + "frontend_up_main_delta": 151 + }, + { + "restart_id": 17, + "duration_to_main": 912, + "duration_to_frontend_up": 1089, + "frontend_up_main_delta": 177 + }, + { + "restart_id": 18, + "duration_to_main": 1438, + "duration_to_frontend_up": 1601, + "frontend_up_main_delta": 163 + }, + { + "restart_id": 19, + "duration_to_main": 1681, + "duration_to_frontend_up": 1848, + "frontend_up_main_delta": 167 + }, + { + "restart_id": 20, + "duration_to_main": 1572, + "duration_to_frontend_up": 1723, + "frontend_up_main_delta": 151 + }, + { + "restart_id": 21, + "duration_to_main": 1500, + "duration_to_frontend_up": 1708, + "frontend_up_main_delta": 207 + }, + { + "restart_id": 22, + "duration_to_main": 1390, + "duration_to_frontend_up": 1565, + "frontend_up_main_delta": 174 + }, + { + "restart_id": 23, + "duration_to_main": 1168, + "duration_to_frontend_up": 1331, + "frontend_up_main_delta": 162 + }, + { + "restart_id": 24, + "duration_to_main": 872, + "duration_to_frontend_up": 1045, + "frontend_up_main_delta": 172 + }, + { + "restart_id": 25, + "duration_to_main": 1361, + "duration_to_frontend_up": 1551, + "frontend_up_main_delta": 189 + }, + { + "restart_id": 26, + "duration_to_main": 1411, + "duration_to_frontend_up": 1559, + "frontend_up_main_delta": 147 + }, + { + "restart_id": 27, + "duration_to_main": 1768, + "duration_to_frontend_up": 1920, + "frontend_up_main_delta": 152 + }, + { + "restart_id": 28, + "duration_to_main": 1505, + "duration_to_frontend_up": 1653, + "frontend_up_main_delta": 148 + }, + { + "restart_id": 29, + "duration_to_main": 1365, + "duration_to_frontend_up": 1504, + "frontend_up_main_delta": 139 + }, + { + "restart_id": 30, + "duration_to_main": 1082, + "duration_to_frontend_up": 1290, + "frontend_up_main_delta": 207 + }, + { + "restart_id": 31, + "duration_to_main": 1192, + "duration_to_frontend_up": 1338, + "frontend_up_main_delta": 145 + }, + { + "restart_id": 32, + "duration_to_main": 1916, + "duration_to_frontend_up": 2060, + "frontend_up_main_delta": 143 + }, + { + "restart_id": 33, + "duration_to_main": 1300, + "duration_to_frontend_up": 1444, + "frontend_up_main_delta": 143 + }, + { + "restart_id": 34, + "duration_to_main": 1935, + "duration_to_frontend_up": 2102, + "frontend_up_main_delta": 166 + }, + { + "restart_id": 35, + "duration_to_main": 1475, + "duration_to_frontend_up": 1625, + "frontend_up_main_delta": 149 + }, + { + "restart_id": 36, + "duration_to_main": 1299, + "duration_to_frontend_up": 1471, + "frontend_up_main_delta": 172 + }, + { + "restart_id": 37, + "duration_to_main": 1418, + "duration_to_frontend_up": 1576, + "frontend_up_main_delta": 158 + }, + { + "restart_id": 38, + "duration_to_main": 1168, + "duration_to_frontend_up": 1335, + "frontend_up_main_delta": 166 + }, + { + "restart_id": 39, + "duration_to_main": 1489, + "duration_to_frontend_up": 1690, + "frontend_up_main_delta": 201 + }, + { + "restart_id": 40, + "duration_to_main": 1383, + "duration_to_frontend_up": 1522, + "frontend_up_main_delta": 139 + }, + { + "restart_id": 41, + "duration_to_main": 1406, + "duration_to_frontend_up": 1572, + "frontend_up_main_delta": 165 + }, + { + "restart_id": 42, + "duration_to_main": 1212, + "duration_to_frontend_up": 1377, + "frontend_up_main_delta": 165 + }, + { + "restart_id": 43, + "duration_to_main": 700, + "duration_to_frontend_up": 837, + "frontend_up_main_delta": 136 + }, + { + "restart_id": 44, + "duration_to_main": 1845, + "duration_to_frontend_up": 1983, + "frontend_up_main_delta": 138 + }, + { + "restart_id": 45, + "duration_to_main": 1545, + "duration_to_frontend_up": 1691, + "frontend_up_main_delta": 146 + }, + { + "restart_id": 46, + "duration_to_main": 1598, + "duration_to_frontend_up": 1771, + "frontend_up_main_delta": 172 + }, + { + "restart_id": 47, + "duration_to_main": 1318, + "duration_to_frontend_up": 1515, + "frontend_up_main_delta": 196 + }, + { + "restart_id": 48, + "duration_to_main": 1374, + "duration_to_frontend_up": 1517, + "frontend_up_main_delta": 142 + }, + { + "restart_id": 49, + "duration_to_main": 965, + "duration_to_frontend_up": 1253, + "frontend_up_main_delta": 287 + }, + { + "restart_id": 50, + "duration_to_main": 1289, + "duration_to_frontend_up": 1463, + "frontend_up_main_delta": 174 + }, + { + "restart_id": 51, + "duration_to_main": 1205, + "duration_to_frontend_up": 1359, + "frontend_up_main_delta": 153 + }, + { + "restart_id": 52, + "duration_to_main": 1410, + "duration_to_frontend_up": 1553, + "frontend_up_main_delta": 143 + }, + { + "restart_id": 53, + "duration_to_main": 1191, + "duration_to_frontend_up": 1360, + "frontend_up_main_delta": 168 + }, + { + "restart_id": 54, + "duration_to_main": 960, + "duration_to_frontend_up": 1156, + "frontend_up_main_delta": 195 + }, + { + "restart_id": 55, + "duration_to_main": 1752, + "duration_to_frontend_up": 1891, + "frontend_up_main_delta": 138 + }, + { + "restart_id": 56, + "duration_to_main": 1438, + "duration_to_frontend_up": 1628, + "frontend_up_main_delta": 189 + }, + { + "restart_id": 57, + "duration_to_main": 1100, + "duration_to_frontend_up": 1271, + "frontend_up_main_delta": 171 + }, + { + "restart_id": 58, + "duration_to_main": 1408, + "duration_to_frontend_up": 1581, + "frontend_up_main_delta": 172 + }, + { + "restart_id": 59, + "duration_to_main": 1240, + "duration_to_frontend_up": 1497, + "frontend_up_main_delta": 256 + }, + { + "restart_id": 60, + "duration_to_main": 773, + "duration_to_frontend_up": 935, + "frontend_up_main_delta": 162 + }, + { + "restart_id": 61, + "duration_to_main": 1465, + "duration_to_frontend_up": 1621, + "frontend_up_main_delta": 155 + }, + { + "restart_id": 62, + "duration_to_main": 1399, + "duration_to_frontend_up": 1580, + "frontend_up_main_delta": 181 + }, + { + "restart_id": 63, + "duration_to_main": 887, + "duration_to_frontend_up": 1088, + "frontend_up_main_delta": 201 + }, + { + "restart_id": 64, + "duration_to_main": 1403, + "duration_to_frontend_up": 1584, + "frontend_up_main_delta": 180 + }, + { + "restart_id": 65, + "duration_to_main": 1492, + "duration_to_frontend_up": 1641, + "frontend_up_main_delta": 148 + }, + { + "restart_id": 66, + "duration_to_main": 1454, + "duration_to_frontend_up": 1644, + "frontend_up_main_delta": 189 + }, + { + "restart_id": 67, + "duration_to_main": 1556, + "duration_to_frontend_up": 1713, + "frontend_up_main_delta": 157 + }, + { + "restart_id": 68, + "duration_to_main": 1370, + "duration_to_frontend_up": 1539, + "frontend_up_main_delta": 168 + }, + { + "restart_id": 69, + "duration_to_main": 825, + "duration_to_frontend_up": 956, + "frontend_up_main_delta": 130 + }, + { + "restart_id": 70, + "duration_to_main": 1363, + "duration_to_frontend_up": 1557, + "frontend_up_main_delta": 193 + }, + { + "restart_id": 71, + "duration_to_main": 1227, + "duration_to_frontend_up": 1400, + "frontend_up_main_delta": 173 + }, + { + "restart_id": 72, + "duration_to_main": 1326, + "duration_to_frontend_up": 1469, + "frontend_up_main_delta": 143 + }, + { + "restart_id": 73, + "duration_to_main": 959, + "duration_to_frontend_up": 1178, + "frontend_up_main_delta": 219 + }, + { + "restart_id": 74, + "duration_to_main": 1311, + "duration_to_frontend_up": 1461, + "frontend_up_main_delta": 150 + }, + { + "restart_id": 75, + "duration_to_main": 1379, + "duration_to_frontend_up": 1561, + "frontend_up_main_delta": 181 + }, + { + "restart_id": 76, + "duration_to_main": 1396, + "duration_to_frontend_up": 1529, + "frontend_up_main_delta": 132 + }, + { + "restart_id": 77, + "duration_to_main": 1077, + "duration_to_frontend_up": 1317, + "frontend_up_main_delta": 239 + }, + { + "restart_id": 78, + "duration_to_main": 1365, + "duration_to_frontend_up": 1544, + "frontend_up_main_delta": 179 + }, + { + "restart_id": 79, + "duration_to_main": 1441, + "duration_to_frontend_up": 1576, + "frontend_up_main_delta": 135 + }, + { + "restart_id": 80, + "duration_to_main": 1415, + "duration_to_frontend_up": 1590, + "frontend_up_main_delta": 175 + }, + { + "restart_id": 81, + "duration_to_main": 1067, + "duration_to_frontend_up": 1262, + "frontend_up_main_delta": 194 + }, + { + "restart_id": 82, + "duration_to_main": 1294, + "duration_to_frontend_up": 1419, + "frontend_up_main_delta": 125 + }, + { + "restart_id": 83, + "duration_to_main": 1529, + "duration_to_frontend_up": 1679, + "frontend_up_main_delta": 150 + }, + { + "restart_id": 84, + "duration_to_main": 1418, + "duration_to_frontend_up": 1564, + "frontend_up_main_delta": 145 + }, + { + "restart_id": 85, + "duration_to_main": 1456, + "duration_to_frontend_up": 1595, + "frontend_up_main_delta": 138 + }, + { + "restart_id": 86, + "duration_to_main": 1024, + "duration_to_frontend_up": 1205, + "frontend_up_main_delta": 181 + }, + { + "restart_id": 87, + "duration_to_main": 1441, + "duration_to_frontend_up": 1609, + "frontend_up_main_delta": 168 + }, + { + "restart_id": 88, + "duration_to_main": 1529, + "duration_to_frontend_up": 1674, + "frontend_up_main_delta": 144 + }, + { + "restart_id": 89, + "duration_to_main": 1548, + "duration_to_frontend_up": 1691, + "frontend_up_main_delta": 142 + }, + { + "restart_id": 90, + "duration_to_main": 1396, + "duration_to_frontend_up": 1580, + "frontend_up_main_delta": 183 + }, + { + "restart_id": 91, + "duration_to_main": 980, + "duration_to_frontend_up": 1177, + "frontend_up_main_delta": 196 + }, + { + "restart_id": 92, + "duration_to_main": 1471, + "duration_to_frontend_up": 1625, + "frontend_up_main_delta": 153 + }, + { + "restart_id": 93, + "duration_to_main": 1394, + "duration_to_frontend_up": 1535, + "frontend_up_main_delta": 141 + }, + { + "restart_id": 94, + "duration_to_main": 1725, + "duration_to_frontend_up": 1897, + "frontend_up_main_delta": 171 + }, + { + "restart_id": 95, + "duration_to_main": 1379, + "duration_to_frontend_up": 1547, + "frontend_up_main_delta": 167 + }, + { + "restart_id": 96, + "duration_to_main": 1196, + "duration_to_frontend_up": 1387, + "frontend_up_main_delta": 190 + }, + { + "restart_id": 97, + "duration_to_main": 905, + "duration_to_frontend_up": 1070, + "frontend_up_main_delta": 165 + }, + { + "restart_id": 98, + "duration_to_main": 2028, + "duration_to_frontend_up": 2185, + "frontend_up_main_delta": 157 + }, + { + "restart_id": 99, + "duration_to_main": 830, + "duration_to_frontend_up": 979, + "frontend_up_main_delta": 148 + }, + { + "restart_id": 100, + "duration_to_main": 1872, + "duration_to_frontend_up": 2018, + "frontend_up_main_delta": 145 + }, + { + "restart_id": 1, + "duration_to_main": 24111, + "duration_to_frontend_up": 24254, + "frontend_up_main_delta": 143 + }, + { + "restart_id": 2, + "duration_to_main": 856, + "duration_to_frontend_up": 980, + "frontend_up_main_delta": 124 + }, + { + "restart_id": 3, + "duration_to_main": 1351, + "duration_to_frontend_up": 1482, + "frontend_up_main_delta": 131 + }, + { + "restart_id": 4, + "duration_to_main": 981, + "duration_to_frontend_up": 1127, + "frontend_up_main_delta": 146 + }, + { + "restart_id": 5, + "duration_to_main": 2393, + "duration_to_frontend_up": 2523, + "frontend_up_main_delta": 129 + }, + { + "restart_id": 6, + "duration_to_main": 1799, + "duration_to_frontend_up": 2000, + "frontend_up_main_delta": 200 + }, + { + "restart_id": 7, + "duration_to_main": 1065, + "duration_to_frontend_up": 1266, + "frontend_up_main_delta": 201 + }, + { + "restart_id": 8, + "duration_to_main": 1144, + "duration_to_frontend_up": 1291, + "frontend_up_main_delta": 146 + }, + { + "restart_id": 9, + "duration_to_main": 982, + "duration_to_frontend_up": 1156, + "frontend_up_main_delta": 174 + }, + { + "restart_id": 10, + "duration_to_main": 2144, + "duration_to_frontend_up": 2275, + "frontend_up_main_delta": 130 + }, + { + "restart_id": 11, + "duration_to_main": 1285, + "duration_to_frontend_up": 1434, + "frontend_up_main_delta": 148 + }, + { + "restart_id": 12, + "duration_to_main": 1702, + "duration_to_frontend_up": 1849, + "frontend_up_main_delta": 146 + }, + { + "restart_id": 13, + "duration_to_main": 1327, + "duration_to_frontend_up": 1472, + "frontend_up_main_delta": 144 + }, + { + "restart_id": 14, + "duration_to_main": 1400, + "duration_to_frontend_up": 1527, + "frontend_up_main_delta": 127 + }, + { + "restart_id": 15, + "duration_to_main": 1572, + "duration_to_frontend_up": 1687, + "frontend_up_main_delta": 115 + }, + { + "restart_id": 16, + "duration_to_main": 1427, + "duration_to_frontend_up": 1543, + "frontend_up_main_delta": 115 + }, + { + "restart_id": 17, + "duration_to_main": 869, + "duration_to_frontend_up": 1031, + "frontend_up_main_delta": 162 + }, + { + "restart_id": 18, + "duration_to_main": 1547, + "duration_to_frontend_up": 1688, + "frontend_up_main_delta": 140 + }, + { + "restart_id": 19, + "duration_to_main": 1694, + "duration_to_frontend_up": 1828, + "frontend_up_main_delta": 133 + }, + { + "restart_id": 20, + "duration_to_main": 1363, + "duration_to_frontend_up": 1503, + "frontend_up_main_delta": 140 + }, + { + "restart_id": 21, + "duration_to_main": 1228, + "duration_to_frontend_up": 1393, + "frontend_up_main_delta": 165 + }, + { + "restart_id": 22, + "duration_to_main": 554, + "duration_to_frontend_up": 662, + "frontend_up_main_delta": 108 + }, + { + "restart_id": 23, + "duration_to_main": 1615, + "duration_to_frontend_up": 1788, + "frontend_up_main_delta": 173 + }, + { + "restart_id": 24, + "duration_to_main": 1216, + "duration_to_frontend_up": 1370, + "frontend_up_main_delta": 154 + }, + { + "restart_id": 25, + "duration_to_main": 1296, + "duration_to_frontend_up": 1440, + "frontend_up_main_delta": 143 + }, + { + "restart_id": 26, + "duration_to_main": 1471, + "duration_to_frontend_up": 1627, + "frontend_up_main_delta": 155 + }, + { + "restart_id": 27, + "duration_to_main": 1299, + "duration_to_frontend_up": 1443, + "frontend_up_main_delta": 143 + }, + { + "restart_id": 28, + "duration_to_main": 732, + "duration_to_frontend_up": 839, + "frontend_up_main_delta": 106 + }, + { + "restart_id": 29, + "duration_to_main": 2095, + "duration_to_frontend_up": 2231, + "frontend_up_main_delta": 135 + }, + { + "restart_id": 30, + "duration_to_main": 318, + "duration_to_frontend_up": 434, + "frontend_up_main_delta": 115 + }, + { + "restart_id": 31, + "duration_to_main": 1357, + "duration_to_frontend_up": 1494, + "frontend_up_main_delta": 136 + }, + { + "restart_id": 32, + "duration_to_main": 1210, + "duration_to_frontend_up": 1319, + "frontend_up_main_delta": 108 + }, + { + "restart_id": 33, + "duration_to_main": 1918, + "duration_to_frontend_up": 2069, + "frontend_up_main_delta": 150 + }, + { + "restart_id": 34, + "duration_to_main": 1605, + "duration_to_frontend_up": 1781, + "frontend_up_main_delta": 176 + }, + { + "restart_id": 35, + "duration_to_main": 1684, + "duration_to_frontend_up": 1799, + "frontend_up_main_delta": 114 + }, + { + "restart_id": 36, + "duration_to_main": 1204, + "duration_to_frontend_up": 1354, + "frontend_up_main_delta": 149 + }, + { + "restart_id": 37, + "duration_to_main": 674, + "duration_to_frontend_up": 793, + "frontend_up_main_delta": 118 + }, + { + "restart_id": 38, + "duration_to_main": 2398, + "duration_to_frontend_up": 2513, + "frontend_up_main_delta": 115 + }, + { + "restart_id": 39, + "duration_to_main": 971, + "duration_to_frontend_up": 1104, + "frontend_up_main_delta": 132 + }, + { + "restart_id": 40, + "duration_to_main": 2311, + "duration_to_frontend_up": 2427, + "frontend_up_main_delta": 116 + }, + { + "restart_id": 41, + "duration_to_main": 918, + "duration_to_frontend_up": 1079, + "frontend_up_main_delta": 160 + }, + { + "restart_id": 42, + "duration_to_main": 1314, + "duration_to_frontend_up": 1499, + "frontend_up_main_delta": 185 + }, + { + "restart_id": 43, + "duration_to_main": 1576, + "duration_to_frontend_up": 1707, + "frontend_up_main_delta": 131 + }, + { + "restart_id": 44, + "duration_to_main": 1031, + "duration_to_frontend_up": 1209, + "frontend_up_main_delta": 177 + }, + { + "restart_id": 45, + "duration_to_main": 1926, + "duration_to_frontend_up": 2067, + "frontend_up_main_delta": 140 + }, + { + "restart_id": 46, + "duration_to_main": 626, + "duration_to_frontend_up": 738, + "frontend_up_main_delta": 111 + }, + { + "restart_id": 47, + "duration_to_main": 1450, + "duration_to_frontend_up": 1606, + "frontend_up_main_delta": 156 + }, + { + "restart_id": 48, + "duration_to_main": 1454, + "duration_to_frontend_up": 1577, + "frontend_up_main_delta": 123 + }, + { + "restart_id": 49, + "duration_to_main": 1145, + "duration_to_frontend_up": 1304, + "frontend_up_main_delta": 159 + }, + { + "restart_id": 50, + "duration_to_main": 648, + "duration_to_frontend_up": 782, + "frontend_up_main_delta": 133 + }, + { + "restart_id": 51, + "duration_to_main": 1871, + "duration_to_frontend_up": 2023, + "frontend_up_main_delta": 151 + }, + { + "restart_id": 52, + "duration_to_main": 1539, + "duration_to_frontend_up": 1689, + "frontend_up_main_delta": 149 + }, + { + "restart_id": 53, + "duration_to_main": 1660, + "duration_to_frontend_up": 1840, + "frontend_up_main_delta": 179 + }, + { + "restart_id": 54, + "duration_to_main": 1603, + "duration_to_frontend_up": 1755, + "frontend_up_main_delta": 151 + }, + { + "restart_id": 55, + "duration_to_main": 1682, + "duration_to_frontend_up": 1862, + "frontend_up_main_delta": 180 + }, + { + "restart_id": 56, + "duration_to_main": 1985, + "duration_to_frontend_up": 2192, + "frontend_up_main_delta": 207 + }, + { + "restart_id": 57, + "duration_to_main": 1488, + "duration_to_frontend_up": 1655, + "frontend_up_main_delta": 166 + }, + { + "restart_id": 58, + "duration_to_main": 1685, + "duration_to_frontend_up": 1861, + "frontend_up_main_delta": 175 + }, + { + "restart_id": 59, + "duration_to_main": 1204, + "duration_to_frontend_up": 1324, + "frontend_up_main_delta": 120 + }, + { + "restart_id": 60, + "duration_to_main": 873, + "duration_to_frontend_up": 1044, + "frontend_up_main_delta": 171 + }, + { + "restart_id": 61, + "duration_to_main": 1630, + "duration_to_frontend_up": 1760, + "frontend_up_main_delta": 129 + }, + { + "restart_id": 62, + "duration_to_main": 1501, + "duration_to_frontend_up": 1637, + "frontend_up_main_delta": 136 + }, + { + "restart_id": 63, + "duration_to_main": 1036, + "duration_to_frontend_up": 1172, + "frontend_up_main_delta": 136 + }, + { + "restart_id": 64, + "duration_to_main": 1026, + "duration_to_frontend_up": 1154, + "frontend_up_main_delta": 128 + }, + { + "restart_id": 65, + "duration_to_main": 1650, + "duration_to_frontend_up": 1806, + "frontend_up_main_delta": 156 + }, + { + "restart_id": 66, + "duration_to_main": 1793, + "duration_to_frontend_up": 1941, + "frontend_up_main_delta": 147 + }, + { + "restart_id": 67, + "duration_to_main": 1527, + "duration_to_frontend_up": 1675, + "frontend_up_main_delta": 148 + }, + { + "restart_id": 68, + "duration_to_main": 1716, + "duration_to_frontend_up": 1859, + "frontend_up_main_delta": 142 + }, + { + "restart_id": 69, + "duration_to_main": 1669, + "duration_to_frontend_up": 1851, + "frontend_up_main_delta": 181 + }, + { + "restart_id": 70, + "duration_to_main": 1107, + "duration_to_frontend_up": 1272, + "frontend_up_main_delta": 164 + }, + { + "restart_id": 71, + "duration_to_main": 719, + "duration_to_frontend_up": 852, + "frontend_up_main_delta": 132 + }, + { + "restart_id": 72, + "duration_to_main": 1228, + "duration_to_frontend_up": 1421, + "frontend_up_main_delta": 192 + }, + { + "restart_id": 73, + "duration_to_main": 1065, + "duration_to_frontend_up": 1254, + "frontend_up_main_delta": 189 + }, + { + "restart_id": 74, + "duration_to_main": 1527, + "duration_to_frontend_up": 1743, + "frontend_up_main_delta": 215 + }, + { + "restart_id": 75, + "duration_to_main": 1624, + "duration_to_frontend_up": 1878, + "frontend_up_main_delta": 253 + }, + { + "restart_id": 76, + "duration_to_main": 1304, + "duration_to_frontend_up": 1447, + "frontend_up_main_delta": 142 + }, + { + "restart_id": 77, + "duration_to_main": 1288, + "duration_to_frontend_up": 1489, + "frontend_up_main_delta": 201 + }, + { + "restart_id": 78, + "duration_to_main": 1881, + "duration_to_frontend_up": 2102, + "frontend_up_main_delta": 220 + }, + { + "restart_id": 79, + "duration_to_main": 2032, + "duration_to_frontend_up": 2229, + "frontend_up_main_delta": 197 + }, + { + "restart_id": 80, + "duration_to_main": 1449, + "duration_to_frontend_up": 1613, + "frontend_up_main_delta": 164 + }, + { + "restart_id": 81, + "duration_to_main": 1515, + "duration_to_frontend_up": 1701, + "frontend_up_main_delta": 185 + }, + { + "restart_id": 82, + "duration_to_main": 1406, + "duration_to_frontend_up": 1557, + "frontend_up_main_delta": 150 + }, + { + "restart_id": 83, + "duration_to_main": 2104, + "duration_to_frontend_up": 2307, + "frontend_up_main_delta": 203 + }, + { + "restart_id": 84, + "duration_to_main": 1268, + "duration_to_frontend_up": 1423, + "frontend_up_main_delta": 155 + }, + { + "restart_id": 85, + "duration_to_main": 1598, + "duration_to_frontend_up": 1741, + "frontend_up_main_delta": 142 + }, + { + "restart_id": 86, + "duration_to_main": 1854, + "duration_to_frontend_up": 2015, + "frontend_up_main_delta": 161 + }, + { + "restart_id": 87, + "duration_to_main": 1583, + "duration_to_frontend_up": 1788, + "frontend_up_main_delta": 204 + }, + { + "restart_id": 88, + "duration_to_main": 1812, + "duration_to_frontend_up": 2048, + "frontend_up_main_delta": 236 + }, + { + "restart_id": 89, + "duration_to_main": 1330, + "duration_to_frontend_up": 1480, + "frontend_up_main_delta": 150 + }, + { + "restart_id": 90, + "duration_to_main": 1668, + "duration_to_frontend_up": 1808, + "frontend_up_main_delta": 139 + }, + { + "restart_id": 91, + "duration_to_main": 1451, + "duration_to_frontend_up": 1627, + "frontend_up_main_delta": 175 + }, + { + "restart_id": 92, + "duration_to_main": 1608, + "duration_to_frontend_up": 1753, + "frontend_up_main_delta": 144 + }, + { + "restart_id": 93, + "duration_to_main": 1270, + "duration_to_frontend_up": 1442, + "frontend_up_main_delta": 172 + }, + { + "restart_id": 94, + "duration_to_main": 909, + "duration_to_frontend_up": 1149, + "frontend_up_main_delta": 239 + }, + { + "restart_id": 95, + "duration_to_main": 1607, + "duration_to_frontend_up": 1798, + "frontend_up_main_delta": 190 + }, + { + "restart_id": 96, + "duration_to_main": 1702, + "duration_to_frontend_up": 1900, + "frontend_up_main_delta": 197 + }, + { + "restart_id": 97, + "duration_to_main": 1226, + "duration_to_frontend_up": 1385, + "frontend_up_main_delta": 159 + }, + { + "restart_id": 98, + "duration_to_main": 1733, + "duration_to_frontend_up": 1881, + "frontend_up_main_delta": 147 + }, + { + "restart_id": 99, + "duration_to_main": 1464, + "duration_to_frontend_up": 1634, + "frontend_up_main_delta": 169 + }, + { + "restart_id": 100, + "duration_to_main": 1345, + "duration_to_frontend_up": 1519, + "frontend_up_main_delta": 173 + } + ] + } + ], + "time_unit": "ms" +} \ No newline at end of file