Benchmarking JVM model classes

This is a quick study of the relative performance of the typical DTO methods generated by Groovy 2.4.3 annotations, Scala 2.11.6 case classes, Kotlin 0.11.91.1 data classes, and Lombok, all running on Oracle Java 8u40. These tests use the Groovy indy jar and compile flag. The Groovy model object is decorated with @CompileStatic.

I'm impatient, so each benchmark is running with 20 warmups, 50 iterations, and 1 fork. The benchmark is sampling for average time in nanoseconds.

To run locally, ./gradlew jmh (it should take 10-15 minutes and the results will be in ./build/reports/jmh/human.txt)

equals()

Benchmark                                   Mode  Samples     Score    Error  Units
c.n.m.EqualsBenchmark.equalsEclipse         avgt       50    30.265 ±  1.203  ns/op
c.n.m.EqualsBenchmark.equalsGroovy          avgt       50    58.579 ±  2.496  ns/op
c.n.m.EqualsBenchmark.equalsIntelliJ        avgt       50    30.113 ±  1.259  ns/op
c.n.m.EqualsBenchmark.equalsKotlin          avgt       50    32.959 ±  1.356  ns/op
c.n.m.EqualsBenchmark.equalsLombok          avgt       50    30.056 ±  1.183  ns/op
c.n.m.EqualsBenchmark.equalsScala           avgt       50    57.706 ±  2.002  ns/op

Note: @danielthomas went deep by decompiling and comparing the bytecodes of Kotlin and Lombok generated equals methods, and they are essentially identical.

Note 2: Groovy 2.3.9 scored over 1000 ns/op on equals() with a similar error margin as below.

hashCode()

Benchmark                                   Mode  Samples     Score    Error  Units
c.n.m.HashCodeBenchmark.hashCodeEclipse     avgt       50    60.223 ±  2.228  ns/op
c.n.m.HashCodeBenchmark.hashCodeGroovy      avgt       50  1263.242 ± 43.068  ns/op
c.n.m.HashCodeBenchmark.hashCodeIntelliJ    avgt       50    61.726 ±  2.769  ns/op
c.n.m.HashCodeBenchmark.hashCodeKotlin      avgt       50    62.117 ±  2.005  ns/op
c.n.m.HashCodeBenchmark.hashCodeLombok      avgt       50    59.100 ±  2.061  ns/op
c.n.m.HashCodeBenchmark.hashCodeScala       avgt       50   130.130 ±  3.023  ns/op

toString()

Benchmark                                   Mode  Samples     Score    Error  Units
c.n.m.ToStringBenchmark.toStringEclipse     avgt       50   441.055 ± 19.371  ns/op
c.n.m.ToStringBenchmark.toStringGroovy      avgt       50  2692.622 ± 92.499  ns/op
c.n.m.ToStringBenchmark.toStringIntelliJ    avgt       50   460.399 ± 32.811  ns/op
c.n.m.ToStringBenchmark.toStringKotlin      avgt       50   412.480 ± 30.187  ns/op
c.n.m.ToStringBenchmark.toStringLombok      avgt       50   493.107 ± 43.294  ns/op
c.n.m.ToStringBenchmark.toStringScala       avgt       50   806.112 ± 78.413  ns/op