Update benchmark results (#345)

Co-authored-by: gi0baro <[email protected]>
emmett-framework · Jul 7, 2024 · 3ad04ac · 3ad04ac
1 parent b87b0a3
commit 3ad04ac
Show file tree

Hide file tree

Showing 2 changed files with 62 additions and 62 deletions.
diff --git a/benchmarks/README.md b/benchmarks/README.md
@@ -2,10 +2,10 @@
 
 
 
-Run at: Mon 27 May 2024, 06:54    
+Run at: Sun 07 Jul 2024, 16:40    
 Environment: GHA Linux x86_64 (CPUs: 4)    
 Python version: 3.11    
-Granian version: 1.4.0    
+Granian version: 1.5.0    
 
 ## RSGI response types
 
@@ -14,10 +14,10 @@ The "small" response is 4 bytes, the "big" one is 80kbytes.
 
 | Type | Total requests | RPS | avg latency | max latency |
 | --- | --- | --- | --- | --- |
-| bytes small (c512) | 621740 | 41600 | 12.263ms | 152.217ms |
-| str small (c64) | 681620 | 45447 | 1.406ms | 4.134ms |
-| bytes big (c64) | 422346 | 28158 | 2.269ms | 6.492ms |
-| str big (c64) | 431171 | 28749 | 2.221ms | 6.265ms |
+| bytes small (c128) | 451837 | 45231 | 2.821ms | 25.756ms |
+| str small (c128) | 411512 | 41191 | 3.097ms | 23.925ms |
+| bytes big (c64) | 280418 | 28047 | 2.276ms | 5.666ms |
+| str big (c64) | 287929 | 28796 | 2.217ms | 5.905ms |
 
 
 ## Interfaces
@@ -29,15 +29,15 @@ The "echo" request is a 4bytes POST request responding with the same body.
 
 | Request | Total requests | RPS | avg latency | max latency |
 | --- | --- | --- | --- | --- |
-| RSGI bytes (c64) | 676033 | 45074 | 1.417ms | 4.416ms |
-| RSGI str (c64) | 658183 | 43882 | 1.456ms | 4.514ms |
-| RSGI echo (c256) | 618784 | 41351 | 6.174ms | 78.008ms |
-| ASGI bytes (c64) | 661880 | 44129 | 1.448ms | 4.237ms |
-| ASGI str (c64) | 649292 | 43291 | 1.476ms | 4.393ms |
-| ASGI echo (c128) | 368270 | 24568 | 5.198ms | 24.292ms |
-| WSGI bytes (c64) | 601224 | 40086 | 1.594ms | 4.342ms |
-| WSGI str (c64) | 591918 | 39469 | 1.617ms | 3.864ms |
-| WSGI echo (c256) | 543488 | 36308 | 7.03ms | 74.972ms |
+| RSGI bytes (c64) | 413211 | 41321 | 1.546ms | 3.964ms |
+| RSGI str (c256) | 426394 | 42762 | 5.952ms | 77.356ms |
+| RSGI echo (c512) | 375364 | 37760 | 13.47ms | 164.291ms |
+| ASGI bytes (c512) | 422389 | 42441 | 12.0ms | 149.999ms |
+| ASGI str (c512) | 421167 | 42302 | 12.039ms | 136.599ms |
+| ASGI echo (c512) | 219785 | 22084 | 23.061ms | 108.419ms |
+| WSGI bytes (c64) | 379838 | 37992 | 1.68ms | 4.02ms |
+| WSGI str (c128) | 377965 | 37859 | 3.369ms | 18.501ms |
+| WSGI echo (c64) | 350180 | 35024 | 1.823ms | 4.758ms |
 
 
 ## HTTP/2
@@ -46,10 +46,10 @@ Comparison between Granian HTTP versions on RSGI using 4bytes plain text respons
 
 | Request | Total requests | RPS | avg latency | max latency |
 | --- | --- | --- | --- | --- |
-| HTTP/1 [GET] (c128) | 663758 | 44276 | 2.882ms | 41.407ms |
-| HTTP/1 [POST] (c512) | 618784 | 41414 | 12.319ms | 153.226ms |
-| HTTP/2 [GET] (c128) | 623845 | 41629 | 3.068ms | 22.093ms |
-| HTTP/2 [POST] (c256) | 466859 | 31195 | 8.179ms | 107.051ms |
+| HTTP/1 [GET] (c512) | 443970 | 44606 | 11.416ms | 153.453ms |
+| HTTP/1 [POST] (c128) | 364571 | 36507 | 3.493ms | 23.512ms |
+| HTTP/2 [GET] (c64) | 416862 | 41694 | 1.532ms | 6.128ms |
+| HTTP/2 [POST] (c512) | 286831 | 28799 | 17.653ms | 255.309ms |
 
 
 ## File responses
@@ -59,9 +59,9 @@ WSGI is not part of the benchmark since the protocol doesn't implement anything
 
 | Request | Total requests | RPS | avg latency | max latency |
 | --- | --- | --- | --- | --- |
-| RSGI (c512) | 509384 | 34082 | 14.956ms | 109.014ms |
-| ASGI (c128) | 245421 | 16374 | 7.8ms | 19.693ms |
-| ASGI pathsend (c64) | 442263 | 29489 | 2.166ms | 5.212ms |
+| RSGI (c64) | 347783 | 34788 | 1.833ms | 4.785ms |
+| ASGI (c256) | 167350 | 16784 | 15.169ms | 69.669ms |
+| ASGI pathsend (c64) | 299513 | 29954 | 2.132ms | 7.075ms |
 
 
 ### Other benchmarks

diff --git a/benchmarks/vs.md b/benchmarks/vs.md
@@ -4,57 +4,57 @@
 
 ## VS 3rd party comparison
 
-Run at: Mon 27 May 2024, 06:53    
+Run at: Sun 07 Jul 2024, 16:41    
 Environment: GHA Linux x86_64 (CPUs: 4)    
 Python version: 3.11    
-Granian version: 1.4.0    
+Granian version: 1.5.0    
 
 ### ASGI
 
 | Server | Total requests | RPS | avg latency | max latency |
 | --- | --- | --- | --- | --- |
-| Granian Asgi [GET] (c512) | 591533 | 39537 | 12.905ms | 146.272ms |
-| Granian Asgi [POST] (c128) | 365564 | 24389 | 5.234ms | 33.076ms |
-| Uvicorn H11 [GET] (c64) | 119609 | 7975 | 8.01ms | 17.699ms |
-| Uvicorn H11 [POST] (c128) | 107295 | 7158 | 17.839ms | 26.697ms |
-| Uvicorn Httptools [GET] (c128) | 554481 | 36989 | 3.454ms | 21.168ms |
-| Uvicorn Httptools [POST] (c128) | 503214 | 33572 | 3.803ms | 26.156ms |
-| Hypercorn [GET] (c128) | 74857 | 4995 | 25.552ms | 29.311ms |
-| Hypercorn [POST] (c128) | 69404 | 4630 | 27.561ms | 53.538ms |
+| Granian Asgi [GET] (c256) | 430989 | 43263 | 5.887ms | 77.225ms |
+| Granian Asgi [POST] (c128) | 220124 | 22038 | 5.788ms | 29.382ms |
+| Uvicorn H11 [GET] (c128) | 78906 | 7899 | 16.15ms | 27.199ms |
+| Uvicorn H11 [POST] (c64) | 69602 | 6962 | 9.169ms | 20.795ms |
+| Uvicorn Httptools [GET] (c128) | 370778 | 37121 | 3.438ms | 18.315ms |
+| Uvicorn Httptools [POST] (c128) | 337939 | 33833 | 3.772ms | 18.323ms |
+| Hypercorn [GET] (c128) | 47803 | 4789 | 26.6ms | 45.166ms |
+| Hypercorn [POST] (c128) | 43983 | 4404 | 28.926ms | 44.677ms |
 
 
 ### WSGI
 
 | Server | Total requests | RPS | avg latency | max latency |
 | --- | --- | --- | --- | --- |
-| Granian Wsgi [GET] (c256) | 613981 | 41011 | 6.223ms | 67.584ms |
-| Granian Wsgi [POST] (c128) | 536063 | 35770 | 3.571ms | 18.16ms |
-| Gunicorn Gthread [GET] (c64) | 55314 | 3688 | 17.311ms | 20.787ms |
-| Gunicorn Gthread [POST] (c64) | 53340 | 3557 | 17.952ms | 19.994ms |
-| Gunicorn Gevent [GET] (c128) | 93675 | 6250 | 13.768ms | 9868.424ms |
-| Gunicorn Gevent [POST] (c512) | 87480 | 5852 | 18.193ms | 14848.519ms |
-| Uwsgi [GET] (c64) | 108473 | 7232 | 8.835ms | 25.839ms |
-| Uwsgi [POST] (c256) | 107609 | 7187 | 34.991ms | 3368.516ms |
+| Granian Wsgi [GET] (c256) | 379087 | 38024 | 6.7ms | 60.406ms |
+| Granian Wsgi [POST] (c128) | 348084 | 34852 | 3.657ms | 33.45ms |
+| Gunicorn Gthread [GET] (c64) | 36357 | 3637 | 17.546ms | 22.729ms |
+| Gunicorn Gthread [POST] (c64) | 35822 | 3583 | 17.817ms | 20.54ms |
+| Gunicorn Gevent [GET] (c64) | 62750 | 6276 | 8.606ms | 4635.689ms |
+| Gunicorn Gevent [POST] (c64) | 58511 | 5852 | 8.142ms | 7234.595ms |
+| Uwsgi [GET] (c128) | 72025 | 7210 | 17.576ms | 3089.517ms |
+| Uwsgi [POST] (c512) | 71103 | 7144 | 63.435ms | 6929.166ms |
 
 
 ### HTTP/2
 
 | Server | Total requests | RPS | avg latency | max latency |
 | --- | --- | --- | --- | --- |
-| Granian Asgi [GET] (c128) | 547711 | 36541 | 3.496ms | 28.719ms |
-| Granian Asgi [POST] (c64) | 322277 | 21488 | 2.974ms | 6.605ms |
-| Hypercorn [GET] (c256) | 22713 | 1517 | 165.289ms | 593.967ms |
-| Hypercorn [POST] (c64) | 42739 | 2850 | 22.408ms | 66.85ms |
+| Granian Asgi [GET] (c128) | 365863 | 36640 | 3.482ms | 23.411ms |
+| Granian Asgi [POST] (c64) | 195653 | 19570 | 3.263ms | 7.519ms |
+| Hypercorn [GET] (c64) | 30733 | 3074 | 20.728ms | 53.841ms |
+| Hypercorn [POST] (c64) | 27824 | 2783 | 22.921ms | 54.176ms |
 
 
 ### ASGI file responses
 
 | Server | Total requests | RPS | avg latency | max latency |
 | --- | --- | --- | --- | --- |
-| Granian (pathsend) (c64) | 447477 | 29834 | 2.142ms | 5.706ms |
-| Uvicorn H11 (c64) | 118854 | 7924 | 8.063ms | 23.481ms |
-| Uvicorn Httptools (c128) | 310254 | 20696 | 6.172ms | 21.613ms |
-| Hypercorn (c128) | 75197 | 5017 | 25.44ms | 39.018ms |
+| Granian (pathsend) (c64) | 302160 | 30218 | 2.114ms | 4.601ms |
+| Uvicorn H11 (c128) | 79436 | 7952 | 16.032ms | 23.631ms |
+| Uvicorn Httptools (c64) | 205107 | 20514 | 3.112ms | 6.047ms |
+| Hypercorn (c128) | 50181 | 5024 | 25.358ms | 40.241ms |
 
 
 ### Long I/O
@@ -63,18 +63,18 @@ Plain text 4 bytes response comparison simulating *long* I/O waits (10ms and 100
 
 | Server | Total requests | RPS | avg latency | max latency |
 | --- | --- | --- | --- | --- |
-| Granian Rsgi 10ms (c512) | 540476 | 36164 | 14.104ms | 129.672ms |
-| Granian Rsgi 100ms (c512) | 75412 | 5049 | 100.68ms | 166.538ms |
-| Granian Asgi 10ms (c512) | 593469 | 39734 | 12.847ms | 131.563ms |
-| Granian Asgi 100ms (c512) | 75610 | 5060 | 100.593ms | 151.221ms |
-| Granian Wsgi 10ms (c128) | 179570 | 11979 | 10.659ms | 26.765ms |
-| Granian Wsgi 100ms (c512) | 75806 | 5073 | 100.304ms | 169.675ms |
-| Uvicorn Httptools 10ms (c512) | 365947 | 24473 | 20.831ms | 112.223ms |
-| Uvicorn Httptools 100ms (c512) | 75382 | 5042 | 100.861ms | 187.898ms |
-| Hypercorn 10ms (c128) | 75109 | 5012 | 25.458ms | 40.094ms |
-| Hypercorn 100ms (c128) | 74670 | 4982 | 25.604ms | 55.009ms |
-| Gunicorn Gevent 10ms (c128) | 86951 | 5802 | 22.004ms | 55.52ms |
-| Gunicorn Gevent 100ms (c512) | 73113 | 4894 | 103.898ms | 199.96ms |
-| Uwsgi 10ms (c128) | 106857 | 7132 | 17.798ms | 3298.008ms |
-| Uwsgi 100ms (c512) | 108935 | 7288 | 62.962ms | 10635.392ms |
+| Granian Rsgi 10ms (c512) | 380373 | 38279 | 13.306ms | 122.638ms |
+| Granian Rsgi 100ms (c512) | 50025 | 5032 | 100.623ms | 166.445ms |
+| Granian Asgi 10ms (c512) | 398875 | 40067 | 12.705ms | 122.106ms |
+| Granian Asgi 100ms (c512) | 50138 | 5040 | 100.62ms | 154.915ms |
+| Granian Wsgi 10ms (c128) | 111017 | 11114 | 11.476ms | 26.089ms |
+| Granian Wsgi 100ms (c512) | 50226 | 5051 | 100.299ms | 154.812ms |
+| Uvicorn Httptools 10ms (c512) | 246109 | 24748 | 20.544ms | 115.111ms |
+| Uvicorn Httptools 100ms (c512) | 49993 | 5028 | 100.807ms | 183.131ms |
+| Hypercorn 10ms (c128) | 49363 | 4943 | 25.764ms | 41.813ms |
+| Hypercorn 100ms (c128) | 48959 | 4902 | 25.988ms | 51.059ms |
+| Gunicorn Gevent 10ms (c128) | 57255 | 5732 | 22.248ms | 55.119ms |
+| Gunicorn Gevent 100ms (c512) | 46614 | 4685 | 108.258ms | 153.266ms |
+| Uwsgi 10ms (c128) | 72068 | 7215 | 17.557ms | 2037.695ms |
+| Uwsgi 100ms (c256) | 71994 | 7220 | 33.768ms | 6824.297ms |