-
Notifications
You must be signed in to change notification settings - Fork 84
In memory compilation "benchmark"
Tamás Koczka edited this page Jan 30, 2018
·
1 revision
This is not really a good benchmark of the in-memory compilers, but at least it gives a glance into where it helps.
Testing method: I used Ubuntu 16.04 WSL on my Windows 10 machine.
- for in-memory compilation measurements I ran
./compiler_backend.py
, openedhttp://127.0.0.1:8000/
with the HelloWorld input and modified one of the constant values 7 times. I threw out the first two results (warmup) and took the average of the other 5 measurements - for normal compilation measurements I ran
./compiler_backend.py --noInMemoryCompilation
instead, everything else was the same as previously
There are a lot of problems with this of course, for example the browser only opens 6 TCP connections at once, but this does not count into the compilation time actually, and of course running 6 compilers at the same time wants to use more than 100% CPU, so they will slow down each other.
Also good to know that starting a new process in WSL is really slow. So the speed up would be much less in a native environment...
language | normal compilation |
in-memory compilation |
speed up |
---|---|---|---|
Perl | 749 ms | (*) 610 ms | 1.22x |
Swift | 4799 ms | (*) 3857 ms | 1.25x |
C++ | 7793 ms | (*) 4669 ms | 1.7x |
Go | 4929 ms | (*) 1957 ms | 2.5x |
C# | 6002 ms | 273 ms | 22x |
Ruby | 769 ms | 33 ms | 23x |
Java | 7342 ms | 284 ms | 26x |
JavaScript | 1535 ms | 38 ms | 40x |
TypeScript | 7843 ms | 176 ms | 44x |
Python | 844 ms | 15 ms | 56x |
PHP | 1991 ms | 19 ms | 105x |
(*) Does not support in-memory compilation actually, but still runs faster because other compilers require less resources (eg. CPU), so it has more.