Skip to content

eishay_jvm_serializers_benchmark

Jump to bottom
温绍锦 edited this page Oct 7, 2022 · 5 revisions

Test Platform

OS:Linux JVM:Oracle Corporation 1.8.0_333 CPU:null os-arch:null Cores (incl HT):4

Disclaimer

This test focusses on en/decoding of a cyclefree data structure, but the featureset of the libraries compared differs a lot:

  • some serializers support cycle detection/object sharing others just write non-cyclic tree structures
  • some include full metadata in serialized output, some don't
  • some are cross platform, some are language specific
  • some are text based, some are binary,
  • some support versioning forward/backward, both, some don't

(See "ToolBehavior":wiki/ToolBehavior) Other test data will yield different results (e.g. adding a non ascii char to every string :-) ). However the results give a raw estimation of library performance.

Serializers (no shared refs)

Benchmarks serializers
  • Only cycle free tree structures. An object referenced twice will be serialized twice.
  • no manual optimizations.
  • schema is known in advance (pre registration or even class generation). (Not all might make use of that)
Ser Time+Deser Time (ns)

Size, Compressed size [light] in bytes

                                      create     ser   deser   total   size  +dfl
jsonb-array/fastjson2/databind            57     416     420     836    227   145
json-array/dsl-json/databind              55     537     575    1112    296   187
kryo-registered-flat                      57     614     551    1165    218   140
fst-flat-pre                              57     560     631    1191    254   168
protostuff                                72     492     717    1209    242   153
jsonb/fastjson2/databind                  57     608     626    1233    366   235
protobuf                                 260     817     433    1250    242   152
json-array/fastjson2/databind             57     601     731    1332    279   168
kryo-auto-flat                            58     726     766    1492    273   184
json/dsl-json/databind                    56     592     956    1548    488   271
json/fastjson2/databind                   57     713    1014    1728    454   247
protobuf/jackson+afterburner/databind     57     867    1045    1911    242   151
thrift-compact                            79    1252     706    1958    243   152
msgpack/databind                          57     848    1122    1970    236   150
thrift                                    78    1390     707    2097    352   201
protobuf/jackson/databind                 56     928    1262    2190    242   151
cbor/jackson/databind                     56     929    1408    2337    398   251
avro-specific                             86    1734    1072    2806    224   136
avro-fastserde-generic                   339    1916     919    2835    224   136
json/jackson/databind                     57    1177    1679    2856    488   271
avro-generic                             339    1952     912    2864    224   136
json/jackson-jr/databind                  56    1326    1580    2906    471   263
avro-fastserde-specific                   77    1829    1256    3085    224   136
json-array/fastjson/databind              58    1307    1916    3224    284   171
json/fastjson/databind                    60    1795    2018    3813    489   271
json/javax-tree/glassfish                982    7367   11184   18551    488   273
java-built-in                             56    4605   20094   24700    892   520
json/json-lib/databind                    58   23418   63855   87273    488   273

Full Object Graph Serializers

Contains serializer(-configurations)
  • supporting full object graph write/read. Object graph may contain cycles. If an Object is referenced twice, it will be so after deserialization.
  • nothing is known in advance, no class generation, no preregistering of classes. Everything is captured at runtime using e.g. reflection.
  • note this usually cannot be used cross language, however JSON/XML formats may enable cross language deserialization.
Ser Time+Deser Time (ns)

Size, Compressed size [light] in bytes

                                   create     ser   deser   total   size  +dfl
protostuff-graph                       73     682     725    1407    242   153
protostuff-graph-runtime               56     854     751    1605    244   154
kryo-registered                        57    1079     870    1949    236   153
kryo-auto                              56    1195    1152    2347    291   198
fst                                    57    1481    1329    2810    319   208
jboss-marshalling-river-ct             57    2044    1615    3659    301   203
hessian                                59    3592    3260    6851    503   319
jboss-marshalling-river                56    3129    5870    8999    608   395
jboss-serialization                    58    5236    5583   10819    935   587
jboss-marshalling-serial               58    6550    7446   13996    859   503
java-built-in-serializer               56    4766   19709   24475    892   520
stephenerialization                    53    5434   19722   25156   1096   522
json/flexjson/databind                 57   10339   16953   27292    506   282

Cross Lang Binary Serializers

Contains serializer(-configurations)
  • Only cycle free tree structures. An object referenced twice will be serialized twice.
  • schema is known in advance (pre registration, intermediate message description languages, class generation).
Ser Time+Deser Time (ns)

Size, Compressed size [light] in bytes

                                   create     ser   deser   total   size  +dfl
jsonb-array/fastjson2/databind         57     416     420     836    227   145
protobuf/protostuff                    72     478     712    1189    242   152
jsonb/fastjson2/databind               57     608     626    1233    366   235
protobuf                              260     817     433    1250    242   152
protobuf/protostuff-runtime            56     679     670    1349    244   153
thrift-compact                         79    1252     706    1958    243   152
msgpack/databind                       57     848    1122    1970    236   150
thrift                                 78    1390     707    2097    352   201
cbor/jackson/databind                  56     929    1408    2337    398   251
avro-specific                          86    1734    1072    2806    224   136
avro-fastserde-generic                339    1916     919    2835    224   136
avro-generic                          339    1952     912    2864    224   136
avro-fastserde-specific                77    1829    1256    3085    224   136
hessian                                59    3592    3260    6851    503   319

XML/JSon Serializers

  • text format based. Usually can be read by anybody. Frequently inline schema inside data.
  • Mixed regarding required preparation, object graph awareness (references).
Ser Time+Deser Time (ns)

Size, Compressed size [light] in bytes

                                   create     ser   deser   total   size  +dfl
json-array/dsl-json/databind           55     537     575    1112    296   187
json-array/fastjson2/databind          57     601     731    1332    279   168
json/dsl-json/databind                 56     592     956    1548    488   271
json/fastjson2/databind                57     713    1014    1728    454   247
json/jackson/databind                  57    1177    1679    2856    488   271
json/jackson-jr/databind               56    1326    1580    2906    471   263
json-array/fastjson/databind           58    1307    1916    3224    284   171
json/fastjson/databind                 60    1795    2018    3813    489   271
json/javax-tree/glassfish             982    7367   11184   18551    488   273
json/flexjson/databind                 57   10339   16953   27292    506   282
json/json-lib/databind                 58   23418   63855   87273    488   273

Manually optimized Serializers

all flavours of manually optimized serializers. Handcoded and hardwired to exactly the benchmark's message structures.
  • illustrates what's possible, at what level generic approaches can be optimized in case
Ser Time+Deser Time (ns)

Size, Compressed size [light] in bytes

                                   create     ser   deser   total   size  +dfl
kryo-manual                            58     403     352     755    213   136
kryo-opt                               56     589     526    1115    215   136
protostuff-manual                      57     465     670    1135    242   153
datakernel                             56     616     539    1155    228   137
wobly                                  36     796     582    1378    254   155
java-manual                            57     836     594    1430    258   151
wobly-compact                          36     846     603    1449    228   143
cbor/jackson/manual                    59     838    1044    1883    389   245
msgpack/manual                         56     882    1125    2007    236   150
jboss-marshalling-river-ct-manual      57    1173    1033    2206    292   171
json/jackson/manual                    56    1114    1240    2354    471   262
avro-fastserde-generic-manual         337    1797     597    2393    224   136
avro-generic-manual                   332    1805     631    2436    224   136
avro-specific-manual                   86    1632     963    2595    224   136
avro-fastserde-specific-manual         77    1770    1235    3005    224   136
jboss-marshalling-river-manual         56    1474    3745    5219    484   245
json/json-smart/manual-tree            56    4168    3014    7182    498   274
json/gson/manual-tree                  56    4184    4381    8565    488   268
json/javax-stream/glassfish            57    5573   10028   15601    471   262

Cost of features

shows performance vs convenience of manually-selected libs.
  • cycle free, schema known at compile time, manual optimization: kryo-manual, msgpack/manual
  • cycle free, schema known at compile time: protostuff, fst-flat-pre, kryo-flat-pre. (note: protostuff uses class generation while the other two just require a list of classes to be written)
  • cycle free, schema UNKNOWN at compile time: fst-flat, kryo-flat, protostuff-runtime, msgpack/databind
  • full object graph awareness, schema UNKNOWN at compile time: fst, kryo.
Ser Time+Deser Time (ns)

Size, Compressed size [light] in bytes

                                   create     ser   deser   total   size  +dfl
kryo-manual                            58     403     352     755    213   136
fst-flat-pre                           57     560     631    1191    254   168
protostuff                             72     492     717    1209    242   153
protostuff-runtime                     57     648     690    1338    244   154
msgpack/databind                       57     848    1122    1970    236   150
fst-flat                               60     887    1103    1990    317   207
msgpack/manual                         56     882    1125    2007    236   150
fst                                    57    1481    1329    2810    319   208

Full data

                                      create     ser   deser   total   size  +dfl
kryo-manual                               58     403     352     755    213   136
jsonb-array/fastjson2/databind            57     416     420     836    227   145
json-array/dsl-json/databind              55     537     575    1112    296   187
kryo-opt                                  56     589     526    1115    215   136
protostuff-manual                         57     465     670    1135    242   153
datakernel                                56     616     539    1155    228   137
kryo-registered-flat                      57     614     551    1165    218   140
protobuf/protostuff                       72     478     712    1189    242   152
fst-flat-pre                              57     560     631    1191    254   168
protostuff                                72     492     717    1209    242   153
jsonb/fastjson2/databind                  57     608     626    1233    366   235
protobuf                                 260     817     433    1250    242   152
json-array/fastjson2/databind             57     601     731    1332    279   168
protostuff-runtime                        57     648     690    1338    244   154
protobuf/protostuff-runtime               56     679     670    1349    244   153
wobly                                     36     796     582    1378    254   155
protostuff-graph                          73     682     725    1407    242   153
java-manual                               57     836     594    1430    258   151
wobly-compact                             36     846     603    1449    228   143
kryo-auto-flat                            58     726     766    1492    273   184
json/dsl-json/databind                    56     592     956    1548    488   271
protostuff-graph-runtime                  56     854     751    1605    244   154
json/fastjson2/databind                   57     713    1014    1728    454   247
cbor/jackson/manual                       59     838    1044    1883    389   245
protobuf/jackson+afterburner/databind     57     867    1045    1911    242   151
kryo-registered                           57    1079     870    1949    236   153
thrift-compact                            79    1252     706    1958    243   152
msgpack/databind                          57     848    1122    1970    236   150
fst-flat                                  60     887    1103    1990    317   207
msgpack/manual                            56     882    1125    2007    236   150
thrift                                    78    1390     707    2097    352   201
protobuf/jackson/databind                 56     928    1262    2190    242   151
jboss-marshalling-river-ct-manual         57    1173    1033    2206    292   171
cbor/jackson/databind                     56     929    1408    2337    398   251
kryo-auto                                 56    1195    1152    2347    291   198
json/jackson/manual                       56    1114    1240    2354    471   262
avro-fastserde-generic-manual            337    1797     597    2393    224   136
avro-generic-manual                      332    1805     631    2436    224   136
avro-specific-manual                      86    1632     963    2595    224   136
avro-specific                             86    1734    1072    2806    224   136
fst                                       57    1481    1329    2810    319   208
avro-fastserde-generic                   339    1916     919    2835    224   136
json/jackson/databind                     57    1177    1679    2856    488   271
avro-generic                             339    1952     912    2864    224   136
json/jackson-jr/databind                  56    1326    1580    2906    471   263
avro-fastserde-specific-manual            77    1770    1235    3005    224   136
avro-fastserde-specific                   77    1829    1256    3085    224   136
json-array/fastjson/databind              58    1307    1916    3224    284   171
jboss-marshalling-river-ct                57    2044    1615    3659    301   203
json/fastjson/databind                    60    1795    2018    3813    489   271
jboss-marshalling-river-manual            56    1474    3745    5219    484   245
hessian                                   59    3592    3260    6851    503   319
json/json-smart/manual-tree               56    4168    3014    7182    498   274
json/gson/manual-tree                     56    4184    4381    8565    488   268
jboss-marshalling-river                   56    3129    5870    8999    608   395
jboss-serialization                       58    5236    5583   10819    935   587
jboss-marshalling-serial                  58    6550    7446   13996    859   503
json/javax-stream/glassfish               57    5573   10028   15601    471   262
json/javax-tree/glassfish                982    7367   11184   18551    488   273
java-built-in-serializer                  56    4766   19709   24475    892   520
java-built-in                             56    4605   20094   24700    892   520
stephenerialization                       53    5434   19722   25156   1096   522
json/flexjson/databind                    57   10339   16953   27292    506   282
json/json-lib/databind                    58   23418   63855   87273    488   273

                                      Effort          Format         Structure  Misc
kryo-manual                           MANUAL_OPT      BINARY         FLAT_TREE  [] complete manual optimization                             
jsonb-array/fastjson2/databind        ZERO_KNOWLEDGE  BIN_CROSSLANG  FLAT_TREE  []                                                          
json-array/dsl-json/databind          CLASSES_KNOWN   JSON           FLAT_TREE  [] JSON array format - all properties without names.        
kryo-opt                              MANUAL_OPT      BINARY         FLAT_TREE  [] manual optimization                                      
protostuff-manual                     MANUAL_OPT      BINARY         FLAT_TREE  [] manual                                                   
datakernel                            MANUAL_OPT      BINARY         FLAT_TREE  [] manually optimized                                       
kryo-registered-flat                  CLASSES_KNOWN   BINARY         FLAT_TREE  [] class registration, no references (typical usage)        
protobuf/protostuff                   CLASSES_KNOWN   BIN_CROSSLANG  FLAT_TREE  [] protobuf + generated code                                
fst-flat-pre                          CLASSES_KNOWN   BINARY         FLAT_TREE  [] fst in unshared mode with preregistered classes          
protostuff                            CLASSES_KNOWN   BINARY         FLAT_TREE  [] generated code                                           
jsonb/fastjson2/databind              ZERO_KNOWLEDGE  BIN_CROSSLANG  FLAT_TREE  []                                                          
protobuf                              CLASSES_KNOWN   BIN_CROSSLANG  FLAT_TREE  []                                                          
json-array/fastjson2/databind         ZERO_KNOWLEDGE  JSON           FLAT_TREE  []                                                          
protostuff-runtime                    ZERO_KNOWLEDGE  BINARY         FLAT_TREE  [] reflection                                               
protobuf/protostuff-runtime           ZERO_KNOWLEDGE  BIN_CROSSLANG  FLAT_TREE  [] protobuf + reflection                                    
wobly                                 MANUAL_OPT      BINARY         FLAT_TREE  []                                                          
protostuff-graph                      CLASSES_KNOWN   BINARY         FULL_GRAPH [] graph + generated code                                   
java-manual                           MANUAL_OPT      BINARY         FLAT_TREE  []                                                          
wobly-compact                         MANUAL_OPT      BINARY         FLAT_TREE  []                                                          
kryo-auto-flat                        ZERO_KNOWLEDGE  BINARY         FLAT_TREE  [] no class registration, no references                     
json/dsl-json/databind                CLASSES_KNOWN   JSON           FLAT_TREE  [] Serializes all properties with exact names.              
protostuff-graph-runtime              ZERO_KNOWLEDGE  BINARY         FULL_GRAPH [] graph + reflection                                       
json/fastjson2/databind               ZERO_KNOWLEDGE  JSON           FLAT_TREE  []                                                          
cbor/jackson/manual                   MANUAL_OPT      BIN_CROSSLANG  FLAT_TREE  []                                                          
protobuf/jackson+afterburner/databind CLASSES_KNOWN   BINARY         FLAT_TREE  []                                                          
kryo-registered                       CLASSES_KNOWN   BINARY         FULL_GRAPH [] class registration, references (typical usage)           
thrift-compact                        CLASSES_KNOWN   BIN_CROSSLANG  FLAT_TREE  []                                                          
msgpack/databind                      CLASSES_KNOWN   BIN_CROSSLANG  FLAT_TREE  [] uses positional (column) layout (instead of Maps std impl uses) to eliminate use of names
fst-flat                              ZERO_KNOWLEDGE  BINARY         FLAT_TREE  [] fst default, but unshared mode                           
msgpack/manual                        MANUAL_OPT      BIN_CROSSLANG  FLAT_TREE  [] uses positional (column) layout (instead of Maps std impl uses) to eliminate use of names
thrift                                CLASSES_KNOWN   BIN_CROSSLANG  FLAT_TREE  []                                                          
protobuf/jackson/databind             CLASSES_KNOWN   BINARY         FLAT_TREE  []                                                          
jboss-marshalling-river-ct-manual     MANUAL_OPT      BINARY         FULL_GRAPH [] full graph preregistered classes, manual optimization    
cbor/jackson/databind                 ZERO_KNOWLEDGE  BIN_CROSSLANG  FLAT_TREE  []                                                          
kryo-auto                             ZERO_KNOWLEDGE  BINARY         FULL_GRAPH [] no class registration, references                        
json/jackson/manual                   MANUAL_OPT      JSON           FLAT_TREE  []                                                          
avro-fastserde-generic-manual         MANUAL_OPT      BIN_CROSSLANG  FLAT_TREE  [VERSIONING_BACKWARD_COMPATIBLE, VERSIONING_FORWARD_COMPATIBLE, VERSIONING_MISMATCH_DETECTION] 
avro-generic-manual                   MANUAL_OPT      BIN_CROSSLANG  FLAT_TREE  [VERSIONING_BACKWARD_COMPATIBLE, VERSIONING_FORWARD_COMPATIBLE, VERSIONING_MISMATCH_DETECTION] 
avro-specific-manual                  MANUAL_OPT      BIN_CROSSLANG  FLAT_TREE  [VERSIONING_BACKWARD_COMPATIBLE, VERSIONING_FORWARD_COMPATIBLE, VERSIONING_MISMATCH_DETECTION] 
avro-specific                         CLASSES_KNOWN   BIN_CROSSLANG  FLAT_TREE  [VERSIONING_BACKWARD_COMPATIBLE, VERSIONING_FORWARD_COMPATIBLE, VERSIONING_MISMATCH_DETECTION] 
fst                                   ZERO_KNOWLEDGE  BINARY         FULL_GRAPH [] default: JDK serialization drop-in-replacement mode      
avro-fastserde-generic                CLASSES_KNOWN   BIN_CROSSLANG  FLAT_TREE  [VERSIONING_BACKWARD_COMPATIBLE, VERSIONING_FORWARD_COMPATIBLE, VERSIONING_MISMATCH_DETECTION] 
json/jackson/databind                 ZERO_KNOWLEDGE  JSON           FLAT_TREE  []                                                          
avro-generic                          CLASSES_KNOWN   BIN_CROSSLANG  FLAT_TREE  [VERSIONING_BACKWARD_COMPATIBLE, VERSIONING_FORWARD_COMPATIBLE, VERSIONING_MISMATCH_DETECTION] 
json/jackson-jr/databind              ZERO_KNOWLEDGE  JSON           FLAT_TREE  []                                                          
avro-fastserde-specific-manual        MANUAL_OPT      BIN_CROSSLANG  FLAT_TREE  [VERSIONING_BACKWARD_COMPATIBLE, VERSIONING_FORWARD_COMPATIBLE, VERSIONING_MISMATCH_DETECTION] 
avro-fastserde-specific               CLASSES_KNOWN   BIN_CROSSLANG  FLAT_TREE  [VERSIONING_BACKWARD_COMPATIBLE, VERSIONING_FORWARD_COMPATIBLE, VERSIONING_MISMATCH_DETECTION] 
json-array/fastjson/databind          ZERO_KNOWLEDGE  JSON           FLAT_TREE  []                                                          
jboss-marshalling-river-ct            CLASSES_KNOWN   BINARY         FULL_GRAPH [] full graph with preregistered classes                    
json/fastjson/databind                ZERO_KNOWLEDGE  JSON           FLAT_TREE  []                                                          
jboss-marshalling-river-manual        MANUAL_OPT      BINARY         FULL_GRAPH [] full graph with manual optimizations                     
hessian                               ZERO_KNOWLEDGE  BIN_CROSSLANG  FULL_GRAPH []                                                          
json/json-smart/manual-tree           MANUAL_OPT      JSON           FLAT_TREE  []                                                          
json/gson/manual-tree                 MANUAL_OPT      JSON           FLAT_TREE  []                                                          
jboss-marshalling-river               ZERO_KNOWLEDGE  BINARY         FULL_GRAPH [] full graph zero knowledge                                
jboss-serialization                   ZERO_KNOWLEDGE  BINARY         FULL_GRAPH []                                                          
jboss-marshalling-serial              ZERO_KNOWLEDGE  BINARY         FULL_GRAPH []                                                          
json/javax-stream/glassfish           MANUAL_OPT      JSON           FLAT_TREE  []                                                          
json/javax-tree/glassfish             ZERO_KNOWLEDGE  JSON           FLAT_TREE  []                                                          
java-built-in-serializer              ZERO_KNOWLEDGE  BINARY         FULL_GRAPH []                                                          
java-built-in                         ZERO_KNOWLEDGE  BINARY         FLAT_TREE  []                                                          
stephenerialization                   ZERO_KNOWLEDGE  BINARY         FULL_GRAPH [] null                                                     
json/flexjson/databind                ZERO_KNOWLEDGE  JSON           FULL_GRAPH []                                                          
json/json-lib/databind                ZERO_KNOWLEDGE  JSON           FLAT_TREE  []
Clone this wiki locally