Skip to content

Update JSON schema#528

Merged
goneall merged 8 commits intodevelopment/v2.2.1from
updatejsonschema
Sep 9, 2021
Merged

Update JSON schema#528
goneall merged 8 commits intodevelopment/v2.2.1from
updatejsonschema

Conversation

@goneall
Copy link
Copy Markdown
Member

@goneall goneall commented Aug 16, 2021

Update the JSON schema to be consistent with the current JSON example file.

  • Remove the top level Document property
  • Add documentNamespace

Signed-off-by: Gary O'Neall gary@sourceauditor.com

Signed-off-by: Gary O'Neall <gary@sourceauditor.com>
@goneall
Copy link
Copy Markdown
Member Author

goneall commented Aug 16, 2021

Resolves issue #527

@goneall
Copy link
Copy Markdown
Member Author

goneall commented Aug 16, 2021

@SamuraiAku - this PR should resolve the issue, however, when I run it through one of the online JSON schema validators with the current JSON example, it seems that everything validates - even an empty JSON file.

Let me know if you have any ideas on how to improve the JSON file and I'll update the PR.

BTW - This JSON schema file is generated by the SPDX Java tools

Copy link
Copy Markdown

@SamuraiAku SamuraiAku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edit: Nevermind, I was mistyping when searching for the change.

@goneall If you want to remove the Document overall property I guess that's OK. But that's not what I was looking for. We need a field for documentNamespace, section 2.5 of the spec. There is nothing for that now.

Signed-off-by: Gary O'Neall <gary@sourceauditor.com>
@goneall
Copy link
Copy Markdown
Member Author

goneall commented Aug 18, 2021

Added the SPDXID property to the Element classes which includes the SpdxDocument. This will resolve issue #529

@goneall
Copy link
Copy Markdown
Member Author

goneall commented Aug 18, 2021

I plan on improving the detail of the schema substantially over the next week. I'll be posting the improvements to this PR.

Signed-off-by: Gary O'Neall <gary@sourceauditor.com>
@goneall
Copy link
Copy Markdown
Member Author

goneall commented Aug 19, 2021

Added licenseConcluded and licenseDeclared to schema - resolve issue #530

Signed-off-by: Gary O'Neall <gary@sourceauditor.com>
@goneall
Copy link
Copy Markdown
Member Author

goneall commented Aug 19, 2021

Added the missing spdxElementId from relationships to the schema - resolves issue #531

Signed-off-by: Gary O'Neall <gary@sourceauditor.com>
@goneall
Copy link
Copy Markdown
Member Author

goneall commented Aug 19, 2021

Schema updated with required fields which forces validation. Resolved an issue where both fileName and name properties were on the files array - should only be fileName.

@goneall
Copy link
Copy Markdown
Member Author

goneall commented Aug 19, 2021

This schema should be good to go - I'd appreciate any reviews.

@goneall goneall marked this pull request as ready for review August 19, 2021 22:35
Signed-off-by: Gary O'Neall <gary@sourceauditor.com>
@goneall
Copy link
Copy Markdown
Member Author

goneall commented Aug 20, 2021

Renamed property describesPackages to documentDescribes - resolves issue #533

@goneall
Copy link
Copy Markdown
Member Author

goneall commented Aug 23, 2021

@swinslow
Copy link
Copy Markdown
Member

Hi @goneall, is this modifying the existing properties for JSON documents for SPDX 2.2?

I'm asking because @specter25 has been working on the JSON parsing / saving for the Golang tools for SPDX 2.2 as his GSoC project this summer, which has now concluded. Though I believe he has mostly been working from the example JSON file as the definition of SPDX JSON, so if this is just aligning the schema definition to fit with that example file, then hopefully that should be fine.

@goneall
Copy link
Copy Markdown
Member Author

goneall commented Aug 24, 2021

is this modifying the existing properties for JSON documents for SPDX 2.2?

@swinslow The intent is not to modify any of the existing properties. The intent is to make the current schema consistent with the JSON example file - which is our only other documentation on the JSON serialization format.

I did test this schema against the current JSON example using the JSON Schema Lint online tool and it passed.

If you have a chance, it would be helpful to run the output of the golang tools against the same schema. If you find any issues, we can discuss here and decide if it is a schema or tool issue.

@goneall
Copy link
Copy Markdown
Member Author

goneall commented Aug 26, 2021

@SamuraiAku @swinslow - Just pinging both of you for a review of this PR - let me know if you think this is OK to merge. I realize there are still some outstanding issues with the JSON schema this PR doesn't resolve - we can add additional PR's to handle those later.

@davaya
Copy link
Copy Markdown

davaya commented Sep 1, 2021

In order to prevent the schema from validating everything including empty files, it's a good practice to include "additionalProperties": false on every object type. I also include string size limits (I picked 5,000 characters out of the air) for extracted texts - the precise number isn't that important, but an SPDX file with megabytes or gigabytes in one extracted license text is probably not valid.

We should have some invalid SPDX test files to ensure that the schema doesn't pass them - I'm sure there are plenty of examples from the NTIA plugfests, but haven't systematically gone through them yet. Checking for false positives is as important as eliminating false negatives.

A JSON schema generated from my SPDX-v2.2 information model also validates the current JSON example using JSON Schema Lint.

…r lineNumber and offset

Signed-off-by: Gary O'Neall <gary@sourceauditor.com>
@goneall
Copy link
Copy Markdown
Member Author

goneall commented Sep 2, 2021

Thanks @davaya for the review! I added the additionalProperties as suggested. I didn't add the length restrictions since they are not currently included in the spec (something we should probably add).

BTW - it looks like your schema is much more complete. We should consider using your schema in future releases.

@SamuraiAku
Copy link
Copy Markdown

@goneall goneall merged commit a08ffa4 into development/v2.2.1 Sep 9, 2021
@goneall goneall deleted the updatejsonschema branch September 9, 2021 17:53
spiffcs added a commit to anchore/syft that referenced this pull request Sep 14, 2021
latest patch for json schema:
https://github.com/spdx/spdx-spec/blob/development/v2.2.1/schemas/spdx-schema.json

See this pr:
spdx/spdx-spec#528

See this comment:
spdx/spdx-spec#528 (comment)

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
spiffcs added a commit to anchore/syft that referenced this pull request Sep 14, 2021
latest patch for json schema:
https://github.com/spdx/spdx-spec/blob/development/v2.2.1/schemas/spdx-schema.json

See this pr:
spdx/spdx-spec#528

See this comment:
spdx/spdx-spec#528 (comment)

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
spiffcs added a commit to anchore/syft that referenced this pull request Sep 17, 2021
* update spdx22 Document model to include relationships field

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update document and relationship to match current JSON spec
https://github.com/spdx/spdx-spec/blob/development/v2.2.1/schemas/spdx-schema.json
spdx/spdx-spec#528
spdx/spdx-spec#528 (comment)

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update File struct based on SPDX schema

Required fields:
[ "SPDXID", "fileName", "copyrightText", "licenseConcluded" ]
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
)

* update spdx22 Document model to include relationships field

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update document and relationship to match current JSON spec
https://github.com/spdx/spdx-spec/blob/development/v2.2.1/schemas/spdx-schema.json
spdx/spdx-spec#528
spdx/spdx-spec#528 (comment)

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update File struct based on SPDX schema

Required fields:
[ "SPDXID", "fileName", "copyrightText", "licenseConcluded" ]
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants