[Benchmark] Add Google Benchmarks & gitaction

[DonghakPark]

Desceiption

This PR enables Google's Benchmark to measure the execution time of out Application, Operation, ..etc)
Since there have been many tasks recently that require performance measurement, creating multiple files under benchmarks/ and automating them through git action would make it easier to check whether there is any change in performance through PRs.

For now it will print log on gitaction like below

Running ./build/benchmarks/benchmark_application/Benchmark_ResNet
Run on (16 X 4800 MHz CPU s)
CPU Caches:
  L1 Data 48 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 512 KiB (x8)
  L3 Unified 16384 KiB (x1)
Load Average: 2.53, 2.17, 1.82
batch_size: 1 data_split: 128 epoch: 10
#1/10 - Training Loss: 4.42886 >> [ Accuracy: 40% - Validation Loss : 9.3356 ]
#2/10 - Training Loss: 4.71029 >> [ Accuracy: 0% - Validation Loss : 28.9189 ]
#3/10 - Training Loss: 5.39561 >> [ Accuracy: 0% - Validation Loss : 29.9747 ]
#4/10 - Training Loss: 4.65303 >> [ Accuracy: 0% - Validation Loss : 12.3451 ]
#5/10 - Training Loss: 6.17309 >> [ Accuracy: 0% - Validation Loss : 13.5961 ]
#6/10 - Training Loss: 6.90443 >> [ Accuracy: 0% - Validation Loss : 11.319 ]
#7/10 - Training Loss: 7.0996 >> [ Accuracy: 0% - Validation Loss : 10.9714 ]
#8/10 - Training Loss: 6.1048 >> [ Accuracy: 0% - Validation Loss : 7.50057 ]
#9/10 - Training Loss: 5.93462 >> [ Accuracy: 0% - Validation Loss : 7.03264 ]
#10/10 - Training Loss: 6.71959 >> [ Accuracy: 0% - Validation Loss : 7.83153 ]
----------------------------------------------------------
Benchmark                Time             CPU   Iterations
----------------------------------------------------------
Test_ResnetFull 24026683559 ns   23243686066 ns            1

[taos-ci]

📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2708. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.

[skykongkong8]

Great work! I really wanted to use google benchmark on the NNTrainer!

• opinions from my end:

1. I think we should discuss how to evaluate the function performances.
   For example, a GEMM benchmark latency / gflops result might differ from time to time.

2. I found a way to get cpu frequency, but not 100% sure. please review

uint64_t get_cpu_freq() {
  unsigned int freq = 0;
  char cur_cpu_name[512];
  int cpu = sched_getcpu();
  snprintf(cur_cpu_name, sizeof(cur_cpu_name),
    "/sys/devices/system/cpu/cpu%d/cpufreq/scaling_cur_freq", cpu);

  FILE* f = fopen(cur_cpu_name, "r");
  if (f != nullptr) {
    if (fscanf(f, "%d", &freq) != 0) {
      fclose(f);
      return uint64_t(freq) * 1000;
    }
    fclose(f);
  }
  return 0;
}

[taos-ci]

cibot: @DonghakPark, The last line of a text file must have a newline character. Please append a new line at the end of the line in benchmarks/benchmark_application/meson.build.

[DonghakPark]

Great work! I really wanted to use google benchmark on the NNTrainer!

• opinions from my end:

1. I think we should discuss how to evaluate the function performances.
   For example, a GEMM benchmark latency / gflops result might differ from time to time.
2. I found a way to get cpu frequency, but not 100% sure. please review

uint64_t get_cpu_freq() {
  unsigned int freq = 0;
  char cur_cpu_name[512];
  int cpu = sched_getcpu();
  snprintf(cur_cpu_name, sizeof(cur_cpu_name),
    "/sys/devices/system/cpu/cpu%d/cpufreq/scaling_cur_freq", cpu);

  FILE* f = fopen(cur_cpu_name, "r");
  if (f != nullptr) {
    if (fscanf(f, "%d", &freq) != 0) {
      fclose(f);
      return uint64_t(freq) * 1000;
    }
    fclose(f);
  }
  return 0;
}

Thank you for your Review,

1. that right !!, we should discuss how to evaluate! --> cpu clock, freq,
2. i will check google benchmarks can use custom measurement

[skykongkong8]

I think benchmark::State state; is doing that role.
According to here, we can do something like:

...
// benchmark::State state; 
auto foo = get_cpu_freq();
state.counters["foo"] = foo;
...

[DonghakPark]

I think benchmark::State state; is doing that role. According to here, we can do something like:

...
// benchmark::State state; 
auto foo = get_cpu_freq();
state.counters["foo"] = foo;
...

that great!!! i will check ASAP

[DonghakPark]

It would be nice to have a benchmark!

I looked it up and it seems like you can measure memory usage too.

benchmark::RegisterMemoryManager(mm.get());

Since nntrainer is a framework that uses less memory, how about adding memory check?

Great!! i will check, and add more measure

& thank you for check typo!!

[taos-ci]

INFO: You can read if there are misspelled characters at our misspelling check report. Please read http://ci.nnstreamer.ai/nntrainer/ci/repo-workers/pr-checker/2708-202410250922020.85318398475647-defd05e542e582c9276316c4263365998267b8b0/report/misspelling_result.txt.

[taos-ci]

cibot: @DonghakPark, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://ci.nnstreamer.ai/nntrainer/ci/repo-workers/pr-checker/2708-202410250922020.85318398475647-defd05e542e582c9276316c4263365998267b8b0/.

[taos-ci]

@DonghakPark, 💯 All CI checkers are successfully verified. Thanks.

[taos-ci]

@DonghakPark, 💯 All CI checkers are successfully verified. Thanks.

[jijoongmoon]

I wonder if this works for Tizen build. I think this should be enabled for specific target. let's say Ubuntu.

[DonghakPark]

I wonder if this works for Tizen build. I think this should be enabled for specific target. let's say Ubuntu.

For now, it only works for Ubuntu 20.04, 22.04
but i will add tizen & Android Test too

[taos-ci]

@DonghakPark, 💯 All CI checkers are successfully verified. Thanks.

[myungjoo]

LGTM. Can't wait for this. Please rebase!

[jijoongmoon]

LGTM