Code

Performance benchmark#483

Code

Merged

HYLcool merged 15 commits intomainmodelscope/data-juicer:mainfrom feat/perf_benchmodelscope/data-juicer:feat/perf_benchCopy head branch name to clipboardDec 6, 2024

+372-55

Commits

Commits on Nov 11, 2024

• * prepare the config files for performance benchmark

  Show description for 04bd012
  HYLcool

  committed
  04bd012

Commits on Nov 12, 2024

• Merge branch 'main' into feat/perf_bench

  Show description for 8104bc8
  HYLcool

  committed
  8104bc8

• + add performance benchmark script and workflow

  HYLcool

  committed
  086886f

• * install and config for wandb

  HYLcool

  committed
  85197e3

Commits on Nov 25, 2024

• Merge branch 'main' into feat/perf_bench

  HYLcool

  committed
  05cfa18

• + set mp start method to 'spawn' for windows in Monitor

  HYLcool

  committed
  0cb8dee

Commits on Nov 26, 2024

• + add a switch arg to control whether to open monitor

  HYLcool

  committed
  20bfdec

Commits on Nov 28, 2024

• Merge branch 'main' into feat/perf_bench

  HYLcool

  committed
  b9f856d

Commits on Dec 3, 2024

• Merge branch 'main' into feat/perf_bench

  HYLcool

  committed
  af852e5

• + add analyzed results to the monitor results as well

  HYLcool

  committed
  2685f26

Commits on Dec 4, 2024

• * optimize the res structure

  Show description for 8b1ccc5
  HYLcool

  committed
  8b1ccc5

Commits on Dec 5, 2024

• * fix bugs in running scripts

  Show description for 92a50e7
  HYLcool

  committed
  92a50e7

• * modify the dif comparison according to the new log format

  HYLcool

  committed
  7b11fa6

• Merge branch 'main' into feat/perf_bench

  HYLcool

  committed
  a5c5597

Commits on Dec 6, 2024

• * update transformers to the latest version to avoid "shape mismatch" error from v4.46.3 (from issue #34990 from transformers)

  HYLcool

  committed
  04b5ead