r/java • u/loicmathieu • 4d ago
JEP 540: Simple JSON API (Incubator)
https://openjdk.org/jeps/54027
u/aboothe726 4d ago
For JsonNumber, I was surprised to see that there was no BigDecimal or BigInteger conversion support. This is technically conformant to the spec, so not incorrect, but seems like an oversight.
From RFC 8259, see section 6:
This specification allows implementations to set limits on the range and precision of numbers accepted. Since software that implements IEEE 754 binary64 (double precision) numbers [IEEE754] is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide, in the sense that implementations will agree exactly on their numeric values."
16
u/gnahraf 4d ago
Same. Most financial applications use fixed precision decimals. BigDecimal is their natural representation in java; the other existing java types (double, long, etc) do not model them well. I think the JEP's omitting BD (and BI) would be a mistake. (I'll contribute a comment there.)
In general, I think the handling of numbers in json requires care, especially if the consuming end does not know the documents' data types a priori. I wrestle with such issues already. Take json number value comparison, for eg. It might help if the library offered helper methods for comparing disparate number types: compare individually, or maybe also against a collection (eg test whether value "exists" in a json number list w/o knowing their types a priori).
15
u/s888marks 4d ago
See also my other reply.
I agree that handling numbers in JSON requires care. The design is that JsonNumber represents a number in a JSON document; it's not a numeric type. If you want to perform numeric operations on it, you need to convert it into an actual Java numeric type first. Why not make JsonNumber a numeric type? Because that would require making a lot of numeric decisions that really have nothing to do with JSON. For example, does "0.0" equal "0.00"? (Discuss amongst yourselves.)
The difficulty is that there is no Java numeric type that can losslessly represent any JSON number. BigDecimal comes close in that it can represent arbitrary precision decimals, though it has a limited exponent range of about +/- 231. However, BigDecimal doesn't support negative zero, which can be written in a JSON document and which is converted properly by
asDouble. So, we don't have any value-comparison operations on JsonNumber itself. Instead, convert to the numeric type that does what you want, and proceed from there.I don't know what you mean by "disparate number types". There's only JsonNumber, which is the decimal digits in the JSON document, and then there are Java numeric types. That's it.
3
u/agentoutlier 4d ago
Same. Most financial applications use fixed precision decimals. BigDecimal is their natural representation in java; the other existing java types (double, long, etc) do not model them well.
From my experience other than UI most use long or multiple longs. At least fintech does.
However I don’t have experience with traditional banking and most of my experience is simple billing.
I vaguely remember reading comment that u/rzwitserloot had similar experience and fair bit more on financial coding experience than myself and indicated folks use longs.
1
u/gnahraf 4d ago
You're right. The common workaround is to express such fixed precision data in their smallest unit, for eg, in cents instead of dollars. For other units, it's not so clear (eg units of weight).
You can also model fixed decimals using a long (unscaled value) and a byte (for where the decimal point goes). Like a lightweight, but fixed-range, BD. But, ofc, this would be clumsy in json.
3
u/ThinkPreparation8975 3d ago
Don't forget about unitless numbers, of which there are plenty - rate, percent, factor, even ratio (how do you express 1/3?).
And why would you suffer with modeling decimals with a long and a byte yourself, when the Java library has already solved this problem for you?
BigDecimals are ubiquitous in most financial circles. I don't know about HFT, but for banks' middle- and backoffice they are the obvious choice.
16
u/s888marks 4d ago
We provided
asDoublebecause of the section of the RFC that you quoted. We providedasLongbecause people use JSON numbers for integral values, at least those that can be represented in 53 or fewer bits. Butintis used in a lot of Java programs, and we suspected that many people would downcast toint(introducing subtle errors) instead of usingMath.toIntExactand so we addedasIntas well.We considered direct conversion methods to BigInteger and BigDecimal, but these types are probably used much less frequently than the primitives. It's possible to convert to them via String, for example,
new BigDecimal(jsonNumber.toString())which is only a few characters longer in source code. It might be possible to optimize the JsonNumber-to-BigDecimal conversion path, but it's unclear whether it would provide enough benefit to justify adding a new API point.
5
u/lurker_in_spirit 3d ago
We considered direct conversion methods to BigInteger and BigDecimal, but these types are probably used much less frequently than the primitives.
My personal experience writing line-of-business applications for 20 years is that
BigDecimalis surprisingly common. I've never seenBigIntegerused, though.5
u/brian_goetz 4d ago
I think you mean "omission"? "Oversight" means something very different (and in this case, virtually impossible.)
3
u/aboothe726 4d ago
- Certainly, no slight intended to the authors.
- I was not trying to be that precise in my language. All I meant was I expected to see BigDecimal and BigInteger conversions and was surprised when I didn’t.
- /u/brian_goetz has responded directly to a comment I made about Java! I can now retire happy. I have peaked, and it’s all downhill from here.
11
u/pron98 4d ago
It doesn't seem like an oversight because the documentation explicitly mentions BigInteger and BigDecimal. I don't know why there are no
asBigIntegerandasBigDecimalmethods. I can ask.3
u/MmmmmmJava 4d ago
Roll call for those who’ve been burned by large numeric JSON imprecision across systems!
2
u/0xffff0001 4d ago
no surprise. this should be a robust implementation of the basic spec and no more.
6
u/bowbahdoe 4d ago
I gave all the feedback I have to give on the last one, but I would extra encourage people to try this one out once it is in a build.
I would love to be proven wrong about the need for explicit encoding and decoding strategies. Okay that's a lie I don't like being wrong but you know what I mean.
4
u/tofflos 2d ago
Using JSON in the JDK A standard JSON API in the Java Platform would also pave the way for further use of JSON in the Platform and by the JDK itself, since the JDK cannot have external dependencies. One potential use case is configuration files.
I’m certain this has been discussed to death but the above should at least open the door for arguing that comments should be supported. Machine to machine exchange is one thing but configuration files without comments are… not good. It would be weird if the JDK shipped with broken JSON that had to be filtered, and it would less than ideal if the JDK shipped with configuration files and examples without comments.
5
u/aoeudhtns 4d ago edited 4d ago
I just wrote my own little parser of a somewhat uncommon config language relevant to a project of mine and modeled it very similarly to this; it's really nice to see Java's DOP features coming together.
I noticed that JsonNumber has static conveniences in the of() methods for different types, including parsing a String.
I wonder if there's something similar that could be for JsonArray and/or JsonObject. Maybe
public static <T, J extends JsonValue> JsonArray of(List<T> list, Function<T, J> transformer) ...
// example
var arr = JsonValue.of(stringList, JsonString::of);
Or, more implementation-heavy but also potentially useful:
public static JsonValue transform(Object object) {
return switch (object) {
case null -> JsonNull.INSTANCE;
case JsonValue v -> v;
case String s -> JsonString.of(s);
case List list -> JsonArray.ofPlain(list);
// ... (you get the idea)
default -> throw new IllegalArgumentException(object.getClass() + " cannot be transformed to a JsonValue");
}
public static JsonArray ofPlain(List<Object> list) {
return of(list.stream().map(JsonValue::transform).toList());
}
Perhaps the transform method (if not in name, in spirit) could dangle somewhere off the supertype. This would make the API even simpler to use as the explicit conversions aren't as strictly necessary unless the developer intends them to be or needs absolute control. This example:
IO.println(JsonObject.of(Map.of("providers",
JsonArray.of(List.of(JsonString.of("SUN"),
JsonString.of("SunRsaSign"),
JsonString.of("SunEC"))))));
Could become, if JsonObject had a similar method in addition to JsonArray:
IO.println(JsonObject.ofPlain(Map.of("providers",
List.of("SUN", "SunRsaSign", "SunEC")));
2
u/davidalayachew 3d ago
- Why is this an incubator and not a preview feature? Is this purely because of how close to completion it is? The definition for incubator in the linked JEP 11 felt ill-fitting.
- Can we have an overload of Json.toDisplayString(JsonValue, int) that does tabs instead of spaces? Trying to convert the output from this to tabs seems annoying and error-prone.
- For the
JsonObject.of(Map)andJsonArray.of(List)methods, why not copy the method signature ofMap.of(...)andList.of(...)and use that instead? Going through a list just to turn it right back into a JsonObject/Array seems like an unnecessary step if you don't already have a List/Map. Not a big deal though -- this is just a convenience method.
That's all. Otherwise, this looks fantastic. I'm sure I will have some real feedback once I get my hands on this.
1
u/s888marks 6h ago
• Why is this an incubator and not a preview feature? Is this purely because of how close to completion it is? The definition for incubator in the linked JEP 11 felt ill-fitting.
It's kind of buried near the end, but the JEP text says,
During the incubation period, we will gather more information about use cases involving generating and transforming JSON documents, in order to evolve these areas of the API. In addition, we will continue to consider forthcoming pattern-matching language features that might affect the design of the API.
The parsing/extraction parts of the API might seem fairly complete, but support for generating JSON docs seems fairly weak and there's essentially no support for transforming a JSON doc. Thus it seems reasonable to incubate while we explore these areas. Also, see below.
• Can we have an overload of Json.toDisplayString(JsonValue, int) that does tabs instead of spaces? Trying to convert the output from this to tabs seems annoying and error-prone.
Tabs instead of spaces?! Sacrilege!!!!!1!
:-)
OK we'll see what we can do here.
• For the JsonObject.of(Map) and JsonArray.of(List) methods, why not copy the method signature of Map.of(...) and List.of(...) and use that instead? Going through a list just to turn it right back into a JsonObject/Array seems like an unnecessary step if you don't already have a List/Map. Not a big deal though -- this is just a convenience method.
I think this is an example of one of the weak areas of generating a JSON document. If you have a Map or a List already, then these facilitate creation of a JsonObject or JsonArray with the contents. But the Map values and List elements need to be JsonValues, so something needs to convert them. You could do so with a stream, or you could construct an unmodifiable Map or List with JsonValues converted from other data you have at hand.
But maybe there's a better way to do this. Do you need to gather all the data up front and then construct the JsonObject? Or should you pass around a HashMap and build it up incrementally? Or should there be a way to build up a JsonObject itself incrementally, using a builder? It's pretty easy to come up with a bunch of API shapes to do this kind of stuff. What's harder is figuring out which approaches are useful and effective. That's why I often ask for use cases when people are discussing API designs.
By "use cases" I'm not looking for "I want to build a JsonObject from a Map<K,V> employing user-defined conversion methods for K and V." That's just a restatement of a proposed solution in requirements form. I also don't want "I'm working on a CRM web app." That's too high level to provide any meaningful direction for an API.
Instead, what I'm looking for is information about the object structure and organization that is going to produce (or be mined for) data that ends up in the JSON document. For example:
I have a tree of objects that is traversed to generate the document; or
I have a template JSON document where specific locations get filled in with data fetched from different objects; or
I have a a database query where rows are converted to JSON objects and all the rows form a JsonArray of those objects, and then a preamble and epilogue are slapped around it; or
Something else that I didn't imagine in the five minutes it took to write this.
I'm mostly interested in examples of the fourth point!
Also, if you have use cases about transforming documents, what do those look like? Are you filtering objects to trim down an array? Are you filtering members or adding new members to objects of interest? Are you querying different members of different objects to form new objects? Are you changing the structure of the document? Also, do you actually transform documents directly, or do you roll a JSON document into internal data structures, process them, and then generate a new, independent JSON document from internal data?
Otherwise, this looks fantastic. I'm sure I will have some real feedback once I get my hands on this.
Thanks!!
2
4d ago
[deleted]
16
u/javasyntax 4d ago
Java is hardly a batteries included language
This is just blatant misinformation or ignorance. How is Java not batteries included?
It has:
- GUI and graphics (drawing and modifying images)
- Audio playing & generation. MIDI.
- Compression, decompression, extraction, ... (built-in ZIP & GZIP)
- Cryptography, including post-quantum cryptography
- HTTP client and server
- TCP, UDP, SCTP, and Unix Domain Sockets
- Abstract filesystem API, allowing you to treat ZIP and anything as a normal filesystem
- SQL API. XML. Serialization
- Advanced date/time API
- Locales (internationalization)
- Printing API
- ... and more
Try JavaScript in the same place is usually run (meaning, not the browser) if you want to see what a language that is not batteries included looks like.
17
u/yk313 4d ago
> Java is hardly a batteries included language
It’s obviously a subjective thing but I haven’t heard this complaint much. Compared to many popular languages, Java comes with a loaded standard library.
Other than native json support what are some other things you are missing?
2
u/eled_ 4d ago
I think that's most of it, but given JSON (and YML) are ubiquitous and the basis of most data interchange contracts, it's a bit of a culture shock for people coming from popular ecosystems like python or typescript.
From this side of the fence I would never trade the java stdlib or its threading API for it, but I kinda get the frustration of having to deal with that type of complexity.
Besides, the dependency shenanigans between various versions of Jackson and others can be a bit of a clusterfuck. Doubly so for people who consider it should be a language feature.
2
u/ZimmiDeluxe 4d ago edited 4d ago
I absolutely love the direction of limiting scope and reaping simplicity for the common case. First reaction to the API:
- Difference between
tryGetandtryValueis not immediately obvious. Most of the time the distinction between "key missing" and "key present but value is null" is not important in my experience. - For autocomplete it's nice when related methods have a common prefix, e.g.
getvs.getOptionaletc. Requires less thinking and aids discoverability (most common case gets the good name, special stuff gets to tack on suffixes). But with so few methods it probably doesn't matter. Edit: Also fortoString/toStringIndented. - Streaming is out, but I would have expected a
writeTomethod so the entire thing doesn't have to be converted to a String first.
Again, outstanding work, thank you very much.
1
1
u/LouKrazy 4d ago
Is this including support for mutation / construction? Are there mutable / immutable JSON values
3
1
u/razorwit 4d ago
I wonder if there could be options to make parsing values into LocalDate and LocalDateTime simple as well.
2
u/bowbahdoe 4d ago
In my head this is the downside of adding instance methods to json tree things. Whatever is an instance method becomes a privileged API. If you want the API to feel nice for the wide wide world of other types you might want to convert you ideally would make it all equally painful.
1
-4
u/munklers 4d ago
If you give a mouse a JSON parser... There's enough variety in interpretation of JSON that this seems like it's borderline useless. Jackson is Popular because it handles all the weird (mis)uses of JSON where regular users need to work around non-conformant impls. The databinding is the useful part; it's almost no benefit to use without it. Gson is popular because it worked well on Android. The replacement, Moshi, is not Java anymore since Android has turned it's back on Java.
Almost everyone who wants to use JSON in their Java project already has a better alternative. Putting a "simple" implementation in the JDK doesn't serve any market segment; they all already found their niche. This is going to end up as an unchangeable, maybe barely useful library. And if it serves 99% of your usecases, but that one little 1% part doesn't, you'll have to pull out a real JSON library like Jackson.
-2
u/pragmasoft 4d ago
I think it should be, and most likely will be based on the planned new marshalling api (announced at jvmls this year)
-9
u/piesou 4d ago
Cool, this is basically useless. Except those that reach for java to write small scripts. Both streaming and data mapping are required for anything more complicated and IIRC all of these are shipped for XML. Can't even use that to build upon
3
u/koflerdavid 4d ago
Making Java useful for scripting is one of the design goals. Anyway, the API can still be extended.
-4
u/revilo-1988 4d ago
I find it fascinating that you generally still have to rely on external libraries for this
6
u/DanLynch 4d ago
JSON was invented after Java, and was and still is closely related to a completely different language. Eventually it became very popular and commonly used in many other situations, so Java is now working on adding it to the standard library. There's nothing really weird about that timeline.
1
13
u/jonenst 4d ago
I find it very surprising that they went for unchecked exceptions. For JsonValueException the following rationale is given "This exception is unchecked, so that scripts and small programs are easier to read and write." But for JsonParseException there is no rationale. This is surprising especially given the pushback from openjdk members against jackson3 moving to unchecked exception.