TomEE vs SpringBoot vs Quarkus

moinhos.jpg

 

There’s a new framework in the Java world. Quarkus. I’ve decided to compare how it behaves in relation to Apache TomEE and Spring Boot. I’m going to measure artefacts size, compilation, test and deploy times while using Java 8 and Maven.

To compare the frameworks I will create an application with a realistic set of features. It will use REST endpoints, Fault Tolerance and an in memory database with JPA. The source code is available on Github at the java-framework-compare project.

The starting point for each of the subprojects:

The Apache TomEE 8.0.0-M2 Microprofile flavour comes from the MicroProfile Starter by selecting Config and Fault Tolerance APIs.

The Spring Boot 2.1.3 application comes from the Spring Initializr page by adding web, jpa and H2 dependencies.

The Quarkus 0.11.0 app starting point is humbler. It comes from the rest-json example bundled in the quarkus-quickstarts bundle.

The baseline

All starter projects have different bundled dependencies. Just for curiosity these are the sizes, build and startup times out of the box, without an application running:

Platform Build Time (s) Start Time (s) Size (MB)
Apache TomEE 8.0.0-M2 5,454 3,789 44
Spring-boot 2.1.3 2,248 2,936 16,7
Quarkus 0.11.0 1,890 0,623 10,5

This is important to later evaluate the impact of our changes to the baseline, in a  realistic project.

The application

To start, I used the implementation coming from the Quarkus rest-json quick-start example. This bundles a set of tests and rest endpoints I was able to successfully port to SpringBoot and TomEE.

The Integration tests are using RestAssured and JUnit 5 Jupiter and are equivalent across the 3 projects. They needed some tweaks on SpringBoot. Getting the random port and configuring the RestAssured serialization was not obvious.

On TomEE I had to use JUnit 4 because Arquillian, the integration test framework commonly used there, does not support JUnit 5 yet.

I’ve added Timeout and Fallback Fault tolerance features to a method. On both Quarkus and TomEE using MicroProfile and to SpringBoot using Netflix’s Hystrix. Hystrix, commonly used in SpringBoot is on maintenance mode I’m not aware of a replacement on the Spring side. As a comparison, the Microprofile Spec has added support for CompletionStage and will be doing work around reactive code soon.

I tried to use JPA with H2 database but couldn’t make it in Quarkus so I also dropped it in the others but kept all dependencies. The documentation is still sketchy and I need more research time.

 

Conclusions

Platform Build Time (s) Build Time with tests (s) Start Time (s) Size (MB)
Apache TomEE 8.0.0-M2 6,178 15,304 4,993 44
Spring-boot 2.1.3 3,358 13,348 6,799 46,9
Quarkus 0.11.0 2,533 7,153 0,767 23.4

It looks like build time without tests increased by ~1s in all projects.

The build with tests with Quarkus takes half the time from the other 2. Artefact size is also half of the other two. Start time is almost one order of magnitude bellow. This is very significant. Please note that this start time can be further reduced by using native code with GraalVM.

SpringBoot startup time and uber war size increase very significantly (>2x) when you add real functionality to it.

TomEE starts faster than SpringBoot and the uber jar size is very stable for this use case. No Change.

The online resources on Quarkus are rare if compared with TomEE or SpringBoot leading to a lot of trial and error.

 

Further work

Make JPA with H2 work and measure the performance impact.

Add docker and kubernets and measure deployment times in the cloud.

Generate native artefacts for Quarkus and measure the effect.

 

Tech notes

Build time is calculated as the average of 3 consecutive executions of the base project using “mvn clean install -DskipTests

Start time is calculated as the average of 3 consecutive executions of the generated uber jar/war. In the case of Quarkus it’s the runner+lib folder. Example: “java -jar target/spring-boot-0.0.1-SNAPSHOT.war

The size is of the generated uber jar/war. In the case of Quakus it’s the runner+lib folder.

All this was executed on a Lenovo Thinkpad T460 with an Intel I7, 32GB ram and 1/3 empty 512GB SSD.

Photo taken on Saturday near Penacova, Portugal

7 thoughts on “TomEE vs SpringBoot vs Quarkus

    1. Thanks for the comment.
      You don’t mention the compile times in the project. Do you have some values you can post?
      Mind that compile time is tight to functionality. That’s why I tried to replicate the same behaviour across all platforms.

  1. Thanks for the write up. I noticed a couple of things. Even with your code out of the box, I couldn’t manage to get the startup time of the Boot project above 2.5 seconds. Might be worth double checking.

    Also, neither the Quarkus one nor the Boot one actually make use of JPA in the projects. Boot – as it’s sort of expecting that you *would* use it as you have added it to the classpath in the first place – still bootstraps the JPA infrastructure. If you remove the JPA related dependencies from both projects, I get to Quarkus 0.9s, Boot 2.3s. A big chunk of the Boot startup time is contributed by the bootstrap of the Cloud functionality which we’re aware of and are working on to improve on. Also, if you fancy to give the Boot 2.2 snapshots a spin, switching to that let’s the sample start in 1.4 seconds with Spring Cloud, in 0.9s without that.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.