Skip to content

Conversation

@paulovap
Copy link
Contributor

Java implementation of optional scalars, following issue #6014.

@paulovap paulovap force-pushed the java_scalar_optionals branch from 442709b to 11636b2 Compare October 25, 2020 19:57
Copy link
Collaborator

@aardappel aardappel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Looks pretty simple so far :)

public ScalarStuff __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; }

public byte justI8() { int o = __offset(4); return o != 0 ? bb.get(o + bb_pos) : 0; }
public Byte maybeI8() { int o = __offset(6); return o != 0 ? new Byte(bb.get(o + bb_pos)) : null; }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bit of a shame this allocates a new object.

I guess the only other way I can see to implement this is to emit 2 methods, e.g. hasMaybeI8 (return a bool) and maybeI8 which works just the same as justI8 with an implied default of 0. That is certainly less elegant, but a lot more FlatBuffer-esque, and emulates the "option" return in other languages.

You're only supposed to call the accessor if you've already guaranteed the has method returns true, but its not the biggest deal if it gets accessed anyway.

This is analogous to other parts of the API in Java, for example where we have a Len method for vectors since it would be too expensive to return an accessor object for them.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course, if Java had structs, or multiple return values, or anything really, it could be solved more nicely.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CasperN opinions?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

idk java or java users well enough, honestly. I prefer to use the local language's canonical representation. However, if Java already promises to be able to read a flatbuffer with few or no allocations, and there are tests enforcing this, then I'd do the less canonical but more performant thing. Not my decision though.

or multiple return values

Do java's tuples allocate too? lol

Copy link
Contributor Author

@paulovap paulovap Oct 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is why I was not very excited to have this feature on Java, it comes with a cost.

Using Optional is even worse because it would allocate the optional object and also trigger autoboxing since generics do not handle primitives.

As for having hasMaybeI8, I think it makes it very confusing and error-prone, as I could see users ending up forgetting to check the field before trying to access it because no attention to the docs was paid.

Java developers are aware that dealing with any sort of optional API-like or handling nullability it implies objects, thus allocation.

I would rather have a big warning "Usage of Optionals causes boxing and performance penalties" on the docs and a tidy API rather than having users suffering on production because they think a field is coming as 0 when actually the back-end is not even sending the field.

In the end, this would leave at least the optional for the user, if want to extract maximum performance and keep your primitives, just don't use optional. It is easy to spot the penalty also, easy to avoid.

But in the end, whatever is your decision, I will abide.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thing is, people may start using optional scalars in their schemas because it is "free" in most languages, and then suddenly run into performance issues when using it from Java.

The "big warning" only helps in Java-first projects.

Java would be the first language to implement this feature in an expensive way.

It's hard to judge, since I am no Java user either, so I have no idea how jarring this API would be. I obviously have a preference for whatever is efficient, and the Java API already has quite a few examples of efficiency instead of good API design, so I don't think this would be anything new.

Anyone else opinions? @dbaileychess ? @vglavnyy ?

And yes, lol @ Java's Optional. "Documentation by Allocation", it's a new programming paradigm :P

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should go with what you have for now and open an issue to introduce an "--unboxed_optional_accessor" flag for java users who need both optional scalars and speed, and are willing to forego the type-system protection. If it turns out people care, they'll hopefully find that issue and convince us to generate hasMaybeI8 and MaybeI8Or0 methods. You can add warnings to the java docs that link to the issue too.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type-system protection

A null pointer is runtime protection, at best :)

An option could be a solution. Still, would like to hear more opinions on what the default should be.

We could have a general --java-slow-but-nice ;) option.. for example, the builder API returns naked ints for offsets which is very unsafe. A slower API could return a MonsterOffset instead :) And like I said, a flatbuffers.Vector class could be nice. etc. etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thing is, people may start using optional scalars in their schemas because it is "free" in most languages, and then suddenly run into performance issues when using it from Java.

After giving some thought, this argument made me change my mind, considering that java users might not have control over the schema and shouldn't pay the costs. So I guess I am more inclined towards emitting the hasMaybe() now.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to leave this open for a little longer to see if anyone wants to come and disagree :P

Java implementation of optional scalars, following issue google#6014.
@paulovap paulovap force-pushed the java_scalar_optionals branch from 11636b2 to d61d98f Compare October 26, 2020 22:29
@aardappel
Copy link
Collaborator

Ok, this seems good to go in, thanks!

@aardappel aardappel merged commit f9a18ea into google:master Oct 27, 2020
}

std::string GenOptionalScalarCheck(FieldDef &field) const {
if (!field.IsScalarOptional()) return "";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is merged, but why check IsScalarOptional twice? Once in the method and one in the calling at Line 686

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants