There’s a new framework in the Java world. Quarkus. I’ve decided to compare how it behaves in relation to Apache TomEE and Spring Boot. I’m going to measure artefacts size, compilation, test and deploy times while using Java 8 and Maven.

To compare the frameworks I will create an application with a realistic set of features. It will use REST endpoints, Fault Tolerance and an in memory database with JPA. The source code is available on Github at the java-framework-compare project.

The starting point for each of the subprojects:

The Apache TomEE 8.0.0-M2 Microprofile flavour comes from the MicroProfile Starter by selecting Config and Fault Tolerance APIs.

The Spring Boot 2.1.3 application comes from the Spring Initializr page by adding web, jpa and H2 dependencies.

The Quarkus 0.11.0 app starting point is humbler. It comes from the rest-json example bundled in the quarkus-quickstarts bundle.

The baseline

All starter projects have different bundled dependencies. Just for curiosity these are the sizes, build and startup times out of the box, without an application running:

Platform	Build Time (s)	Start Time (s)	Size (MB)
Apache TomEE 8.0.0-M2	5,454	3,789	44
Spring-boot 2.1.3	2,248	2,936	16,7
Quarkus 0.11.0	1,890	0,623	10,5

This is important to later evaluate the impact of our changes to the baseline, in a  realistic project.

The application

To start, I used the implementation coming from the Quarkus rest-json quick-start example. This bundles a set of tests and rest endpoints I was able to successfully port to SpringBoot and TomEE.

The Integration tests are using RestAssured and JUnit 5 Jupiter and are equivalent across the 3 projects. They needed some tweaks on SpringBoot. Getting the random port and configuring the RestAssured serialization was not obvious.

On TomEE I had to use JUnit 4 because Arquillian, the integration test framework commonly used there, does not support JUnit 5 yet.

I’ve added Timeout and Fallback Fault tolerance features to a method. On both Quarkus and TomEE using MicroProfile and to SpringBoot using Netflix’s Hystrix. Hystrix, commonly used in SpringBoot is on maintenance mode I’m not aware of a replacement on the Spring side. As a comparison, the Microprofile Spec has added support for CompletionStage and will be doing work around reactive code soon.

I tried to use JPA with H2 database but couldn’t make it in Quarkus so I also dropped it in the others but kept all dependencies. The documentation is still sketchy and I need more research time.

 

Conclusions

Platform	Build Time (s)	Build Time with tests (s)	Start Time (s)	Size (MB)
Apache TomEE 8.0.0-M2	6,178	15,304	4,993	44
Spring-boot 2.1.3	3,358	13,348	6,799	46,9
Quarkus 0.11.0	2,533	7,153	0,767	23.4

It looks like build time without tests increased by ~1s in all projects.

The build with tests with Quarkus takes half the time from the other 2. Artefact size is also half of the other two. Start time is almost one order of magnitude bellow. This is very significant. Please note that this start time can be further reduced by using native code with GraalVM.

SpringBoot startup time and uber war size increase very significantly (>2x) when you add real functionality to it.

TomEE starts faster than SpringBoot and the uber jar size is very stable for this use case. No Change.

The online resources on Quarkus are rare if compared with TomEE or SpringBoot leading to a lot of trial and error.

 

Further work

Make JPA with H2 work and measure the performance impact.

Add docker and kubernets and measure deployment times in the cloud.

Generate native artefacts for Quarkus and measure the effect.

 

Tech notes

Build time is calculated as the average of 3 consecutive executions of the base project using “mvn clean install -DskipTests”

Start time is calculated as the average of 3 consecutive executions of the generated uber jar/war. In the case of Quarkus it’s the runner+lib folder. Example: “java -jar target/spring-boot-0.0.1-SNAPSHOT.war”

The size is of the generated uber jar/war. In the case of Quakus it’s the runner+lib folder.

All this was executed on a Lenovo Thinkpad T460 with an Intel I7, 32GB ram and 1/3 empty 512GB SSD.

Photo taken on Saturday near Penacova, Portugal