Pre-Processing the Original Dataset

[jiali222333]

Hello, I'd like to learn more specifically about the processing of the Gen4 dataset. Since the label files in the Gen4 dataset have frame rates of 30 and 60, I'd like to know how to preprocess the data for label datasets with a frame rate of 60.

[magehrig]

Hi @jiali222333

You can find the preprocessing script here. The network is trained and evaluated at 20 Hz so the script extracts both the event representations and labels (when available) at that frequency. You have to adapt the code to process the dataset for a different frequency. I can't test this for you right now because I am busy but feel free to check out the code yourself.

[jiali222333]

Hi@magehrig
First of all, thank you very much for your question. I would like to ask if the "track_id" in the label file refers to the order in which the object appears throughout the entire time series? Since my dataset was collected by myself, after using a preprocessing script, I couldn't read the data correctly. Upon inspection, I found that there was an error in extracting the labels during frequency extraction. So I would like to ask specifically, if my data labels are at 60Hz, what should the generated label information look like when training at 20Hz?
I have looked at the data in the "objframe_idx_2_repr_idx" file provided in your dataset.
indices [ 101 121 141 161 181 201 221 241 261 281 301 321 341 361 381 401 421 441 461 481 501 521 541 561 581 601 761 781 801 841 881 901 921 941 961 1021 1041 1141 1161 1181] convert [(91, 102), (111, 122), (131, 142), (151, 162), (171, 182), (191, 202), (211, 222), (231, 242), (251, 262), (271, 282), (291, 302), (311, 322), (331, 342), (351, 362), (371, 382), (391, 402), (411, 422), (431, 442), (451, 462), (471, 482), (491, 502), (511, 522), (531, 542), (551, 562), (571, 582), (591, 602), (751, 762), (771, 782), (791, 802), (831, 842), (871, 882), (891, 902), (911, 922), (931, 942), (951, 962), (1011, 1022), (1031, 1042), (1131, 1142), (1151, 1162), (1171, 1182)]

But in my own data set, the following situation occurred
indices [ 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 73 75 77 79 81 83 85 87 89 91 93 95 97 99 101 103 105 107 109 111 113 115 117 119 121 123 125 127 129 131 133 135 137 139 141 143 145 147 149 151 153 155 157 159 161 163 165 167 169 171 173 175 177 179 181 183 185 187 193 195 197 199 201 203 205 207 209 211 213 215 217 219 221 223 225 227 229 231 233 235 237 239 241 243 245 247 249 251 253 255 257 259 261 263 265 267 269 271 273 275 277 279 281 283 285 287 289 291 293 295 297 299 301 303 305 307 309 311 313 315 317 319 321 323 325 327 329 331 333 335 337 339 341 343 345 347 349 351 353 355 357 359 361 363 365 367 369 371 373 375 377 379 381 383 385 387 389 391 393 395 397 399 401 403 405 407 409 411 413 415 417 419 421 423 425 427 429 431 433 435 437 439 441 443 445 447 449 451 453 455 457 459 463 465 467 469 471 473 475 477 479 483 485 487 489 491 493 495 497 499 501 503 505 507 509 511 513 515 517 519 521 523 525 527 529 531 533 535 537 539 541 543 549 551 553 555 557 559 561 563 565 567 569 571 573 575 577 579 581 583 585 587 589 591 593 595 597 599 601 603 605 607 609 611 613 615 617 619 621 623 625 627 629 631 633 635 637 639 641 643 645 647 649 651 653 655 657 659 661 663 665 667 669 671 673 675 677 679 681 683 685 687 689 691 693 695 697 701 703 705 707 709 711 713 715 717 719 721 723 725 727 729 731 733 735 737 739 741 743 745 747 749 751 753 755 757 759 761 763 765 767 769 771 773 775 777 779 781 783 785 787 789 791 793 795 797 799 801 803 805 807 809 811 813 815 817 819 821 823 825 827] convert [(0, 188), (189, 544), (545, 828)]

so, could you please briefly describe the idea of ​​preprocessing?

[magehrig]

objframe_idx_2_repr_idx is a list that can be used similarly to a dictionary/hash-map that maps the label index (all labels associated with a timestamp represented by an index) to the corresponding representation index. That is
representation_index = objframe_idx_2_repr_idx[label_index]

That means that if you have both the labels at 60Hz and the event representations at 60 Hz objframe_idx_2_repr_idx should be [0, 1, 2, 3, 4, ...]. However, this depends on how exactly your data looks like and the frequency of your event representations. Hope that helps to understand how that works. Btw there is another issue with a similar question. See my answer.