JSON is the accepted standard these days for transmitting data between servers and web applications. We often don’t think about the JSON libraries we use, but there are some differences between them.
JSON is often used to transport and parse big files. This is a scenario that is common in data processing applications running in Hadoop or Spark clusters. Given the size of these files, you can be looking at significant differences in parsing speed between libraries.
Small files come up all the time as incoming requests at high throughput, and parsing them happens quickly, so the differences in performance may not seem to be a big deal at first. But the differences add up, as often you need to parse lots of small files in rapid succession during times of heavy traffic. Microservices and distributed architectures often use JSON for transporting these kinds of files, as it is the de facto format for web APIs.
Not all JSON libraries perform the same. Picking the right one for your environment can be critical. We ran a benchmark test to see how fast four of the most popular JSON libraries for Java parse different sizes of files. This benchmark can help you decide.
JSON.simple vs GSON vs Jackson vs JSONP For the benchmark tests, we looked at four major JSON libraries for Java: JSON.simple, GSON, Jackson, and JSONP. All of these libraries are popularly used for JSON processing in a Java environment, and were chosen according to their popularity in Github projects. Here are the ones we tested:
We ran a benchmark test on the libraries for both big files and small files. The requirements (and therefore performance) for handling different file sizes are different, as are the environments in which the need to parse these files arise.
The benchmark tested two key scenarios: parsing speed for big files (190 MB) and parsing speed for small files (1 KB). The big files were taken from GitHub. The small files were randomly generated from here the JSON Generator.
For both big and small files, we ran each file 10 times per library. Given the size of the big file, we did 10 iterations per run for each library. Each small file was iterated 10,000 times per run for each library. For the small files test, we didn’t retain the files in memory between iterations and the test was run on a c3.large instance on AWS.
The results for the big file are shown in full below, but I’ve further averaged the results for the small files in the interest of space. To view the extended results, go here. If you want to view the source code for the small files or the libraries, go here.
Big differences here! Depending on the run, Jackson or JSON.simple traded fastest times, with Jackson edging out JSON.simple in aggregate. Looking at the average result across all the test runs, Jackson and JSON.simple come out well ahead on the big file, with JSONP a distant third and GSON far in last place.
Let’s put that in percentage terms. Jackson is the winner in average time across all the runs. Looking at the numbers from two different angles, here are the percentage results:
Those are big differences between the library speeds!
Takeaway: It was a photo finish, but Jackson is your winning library. JSON.simple is a nose behind and the other two are in the rearview mirror.
The table above shows the average of 10 runs for each file, and the total average at the bottom. The tally for fastest library on number of files won is:
Looking at the average result for all the test runs across all the files, GSON is the winner here, with JSON.simple and JSONP taking a distinct second and third place, respectively.
Jackson came in 2nd to last. So despite not being the fastest on any single file, JSON.simple is the second fastest in aggregate. And despite being the fastest on a handful of files, JSONP is well in third place in aggregate.
Jackson is very consistent across all the files, while the other three libraries are occasionally much faster than Jackson, but on some files end up running at about the same speed or even slightly slower.
Let’s put the numbers in percentage terms, again looking at the numbers from two different angles:
Compared to the big file tests, these are smaller differences, but still quite noticeable.
Takeaway: Bad luck for JSON.simple, as it again loses a close race, but GSON is your winner. JSONP is a clear third and Jackson brings up the rear.
Parsing speed isn’t the only consideration when choosing a JSON library, but it is an important one. Upon running this benchmark test, what we found was that there is no one library that blows the others away on parsing speed across all file sizes and all runs. The libraries that performed best for big files suffered for small files and vice versa.
Choosing which library to use on the merit of parsing speed comes down to your environment then.
As far as parsing speed goes, JSONP doesn’t have much to recommend for it in any scenario. It performs poorly for both big and small files compared to other available options. Fortunately, Java 9 is reportedly getting native JSON implementation, which one would imagine is going to be an improvement over the reference implementation.
If you’re concerned about parsing speed for your JSON library, choose Jackson for big files, GSON for small files, and JSON.simple for handling both.
We ran the same benchmarks again but are now using the latest Libraries and ran the tests on Java 11.
The numbers look quite different from when we ran these benchmarks previously. In general the performance increased drastically.
Last benchmarks Jackson was the winner, but this time GSON is by far the fastest, with JSONP being a close second and then followed by Jackson and then JSON.simple last.
And again looking at the numbers from two different angles, here are the percentage results:
GSON is a clear winner.
In the small File Results Gson is still the winner. Clearly beating the other 3 contenders.
Here are the results:
Gson is processing those files faster by almost 200+ ms before the next candidate.
The obvious elephant in the room is that java and the json libraries got faster. Like way faster then back when the first benchmarks were run.
It is also obvious that GSON stepped up big and won both benchmarks for Big and small files. In both cases very clearly.
Enjoyed reading this blog post or have questions or feedback?
Share your thoughts by creating a new topic in the Harness community forum.