refactor: avoid downloading tokenizer if truncate
is False
(#1152)
#373
Job | Run time |
---|---|
1s | |
25s | |
29s | |
30s | |
27s | |
28s | |
25s | |
31s | |
25s | |
3m 41s |
truncate
is False
(#1152)
#373
Job | Run time |
---|---|
1s | |
25s | |
29s | |
30s | |
27s | |
28s | |
25s | |
31s | |
25s | |
3m 41s |