Skip to content

JSONGenerator/Loader cleanups#5146

Merged
ChrisDodd merged 6 commits into
p4lang:mainfrom
ChrisDodd:cdodd-json
Mar 2, 2025
Merged

JSONGenerator/Loader cleanups#5146
ChrisDodd merged 6 commits into
p4lang:mainfrom
ChrisDodd:cdodd-json

Conversation

@ChrisDodd

Copy link
Copy Markdown
Contributor

In trying to use the JSONGenerator/Loader code for additional things, I found a number of ways in which they are tricky to use and easy to use incorrectly (as well as some ways they are used incorrectly), mostly due to having exposed internals. So I made all those internals private and provided a cleaner interface to be used

  • JSONGenerator now uses emit methods to output values (rather than allowing << arbitrary text which was erro prone), and ensures that objects only contain key-value pairs and vector only contain values (no keys) and things match up properly.
  • JSONLoader now provides is tester methods for checking that values are the expected type, and manages its internal memory with unique_ptr so as to not require the garbage collector.

Comment thread ir/json_parser.h Outdated
Comment thread ir/json_generator.h Outdated
Comment thread ir/json_loader.h Outdated
Comment thread ir/json_parser.cpp Outdated
@fruffy fruffy added the core Topics concerning the core segments of the compiler (frontend, midend, parser) label Feb 25, 2025
Signed-off-by: Chris Dodd <cdodd@nvidia.com>
Comment thread ir/json_parser.h Outdated
- use unqiue_ptr to manage memory so as to not excercise the garbage
  collector as much

Signed-off-by: Chris Dodd <cdodd@nvidia.com>
- fix escaping/reading of special characters in json strings

Signed-off-by: Chris Dodd <cdodd@nvidia.com>
Signed-off-by: Chris Dodd <cdodd@nvidia.com>
json << json.indent << "\"assertions\" : " << p4Assertions;
json.indent--;
json << json.indent << "}\n";
auto state = json.begin_object();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just loudly thinking: (not for this PR) we might do some scope-level RAII objects that would do end_object automagically. Or make begin_object to accept a lambda. So, end_object would be called automatically. I would probably prefer RAII as it might have nesting neater.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point of RAII is to recover from exceptions cleanly. However, an exception when generating json would likely mean the entire json was invalid, so there's not much point in trying to fix up the superficial syntax, so there's not really anything useful for the RAII to recover/clean up.

We could add it (or a lambda) as an alternate syntax, however.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, why cannot state call end_object in its destructor? We'd end with something like this:

{
  auto objectScope = json.begin_object_scope();
  object.emit("checkpoints", checkpoints);
  ...
  object.emit("assertions", p4Assertions);
}

Here objectScope would hold reference to json, so it could close the scope properly on destruction.

or, alternatively, with function object:

{
  json.object([](auto &state) {
    state.emit("checkpoints", checkpoints);
    ...
    state.emit("assertions", p4Assertions);
 });
}

Note sure which variant I like better.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I made the 'state' value returned by begin_object/_vector automatically call end_object/_vector as appropriate when it goes out of scope, if it hasn't been called yet. So now you can have an explicit end call, or you can just leave it off.

Since all these functions are inline in JSONGenerator.h, the overhead should be minimal -- it should be able to optimize away the redundant state changes and checks, and the result should be just a series of ostream outputs.

Comment thread ir/json_generator.h
}
out << "]";
auto t = begin_vector();
for (auto &el : v) emit(el);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we can have emit_as_vector that would iterate over container?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's basically what this is. Perhaps make a templated generate method that can take any iterable type (any type that has iterator, begin(), and end() -- C++20 concepts would be helpful here). But there would be some ambiguity whether a type should be a vector or an object.

@asl asl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Made few small comments about possible future improvements.

Signed-off-by: Chris Dodd <cdodd@nvidia.com>
- the state object will automatically call end_object or end_vector as
  need when it goes out of scope.

Signed-off-by: Chris Dodd <cdodd@nvidia.com>
@ChrisDodd ChrisDodd added this pull request to the merge queue Mar 2, 2025
Merged via the queue into p4lang:main with commit 79d1a85 Mar 2, 2025
@ChrisDodd ChrisDodd deleted the cdodd-json branch March 2, 2025 09:59
@vlstill vlstill added the breaking-change This change may break assumptions of compiler back ends. label Mar 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking-change This change may break assumptions of compiler back ends. core Topics concerning the core segments of the compiler (frontend, midend, parser)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants