Remove MetadataType from core package object and normalize JSON metadataType values by wagoodman · Pull Request #1983 · anchore/syft

wagoodman · 2023-07-31T21:47:49Z

This PR:

removes the pkg.Package.MetadataType from the core package model, keeping it as a concern for the syftjson format.
adds a convention around how to name JSON metadataType values, documented in the DEVELOPING.md.
adds a 1:many lookup for json metadataType names to support legacy conversions of JSON payloads into a package metadata struct
adds an adapter layer between the package metadata structs (an implementation detail) and the data shapes that are officially supported in the syft json output. This allows us to rename package metadata structs without needing to break the JSON schema (since today that would change a type definition, and if we were being consistent, would change the name of the MetadataType name). This adapter layer breaks the need for MetadataTypes to be strongly associated with the struct names.
adds a new SYFT_FORMAT_JSON_LEGACY=<bool> (defaulting to false) to the syft application config. This allows folks to be able to fallback to the old JSON metadata type names (and other soon-to-be-breaking changes) to get to a pre-1.0 state of the JSON output.
Migrates the SYFT_TEMPLATE configuration to SYFT_FORMAT_TEMPLATE to be consistent with future format related configurations.
Renames the existing pkg.*Metadata structs to be consistent with the metadata type names (they do not always match exactly).

Doing this necessarily breaks the JSON schema, so it has been rev'd to v12 in this PR.

The downstream grype PR has been drafted: anchore/grype#1423

For a semantic diff of the v11.0.1 vs v12 JSON schema see #1983 (comment) .

Fixes #1844
Fixes #1735

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

…adata Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

github-actions · 2023-07-31T21:54:06Z

Benchmark Test Results

Benchmark results from the latest changes vs base branch

goos: linux%0Agoarch: amd64%0Apkg: github.com/anchore/syft/test/integration%0Acpu: Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz%0A                                                              │ ./.tmp/benchmark-db2153c.txt │%0A                                                              │            sec/op            │%0AImagePackageCatalogers/alpmdb-cataloger-2                                       12.08m ±  1%25%0AImagePackageCatalogers/apkdb-cataloger-2                                        650.3µ ±  4%25%0AImagePackageCatalogers/binary-cataloger-2                                       204.6µ ±  0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                       545.4µ ±  2%25%0AImagePackageCatalogers/dotnet-portable-executable-cataloger-2                   20.82µ ±  0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                             92.67µ ± 14%25%0AImagePackageCatalogers/java-cataloger-2                                         13.26m ±  2%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                         90.32µ ±  1%25%0AImagePackageCatalogers/javascript-package-cataloger-2                           342.0µ ±  1%25%0AImagePackageCatalogers/nix-store-cataloger-2                                    247.2µ ±  2%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                       718.0µ ±  2%25%0AImagePackageCatalogers/portage-cataloger-2                                      411.0µ ±  2%25%0AImagePackageCatalogers/python-package-cataloger-2                               3.167m ±  1%25%0AImagePackageCatalogers/r-package-cataloger-2                                    172.9µ ±  1%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                       469.3µ ±  2%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                                 827.3µ ±  2%25%0AImagePackageCatalogers/sbom-cataloger-2                                         115.8µ ±  0%25%0Ageomean                                                                         449.4µ%0A%0A                                                              │ ./.tmp/benchmark-db2153c.txt │%0A                                                              │             B/op             │%0AImagePackageCatalogers/alpmdb-cataloger-2                                       5.142Mi ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                        202.0Ki ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                       30.57Ki ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                       167.3Ki ± 0%25%0AImagePackageCatalogers/dotnet-portable-executable-cataloger-2                   3.697Ki ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                             9.906Ki ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                         2.817Mi ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                         8.594Ki ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                           92.81Ki ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                    47.63Ki ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                       181.4Ki ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                      118.7Ki ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                              1021.8Ki ± 0%25%0AImagePackageCatalogers/r-package-cataloger-2                                    51.75Ki ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                       179.7Ki ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                                 140.9Ki ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                         14.21Ki ± 0%25%0Ageomean                                                                         99.34Ki%0A%0A                                                              │ ./.tmp/benchmark-db2153c.txt │%0A                                                              │          allocs/op           │%0AImagePackageCatalogers/alpmdb-cataloger-2                                        88.12k ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                         4.136k ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                         848.0 ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                        3.064k ± 0%25%0AImagePackageCatalogers/dotnet-portable-executable-cataloger-2                     132.0 ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                               281.0 ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                          40.51k ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                           228.0 ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                            1.315k ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                      860.0 ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                        3.998k ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                       2.245k ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                                16.34k ± 0%25%0AImagePackageCatalogers/r-package-cataloger-2                                      902.0 ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                        3.954k ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                                  2.393k ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                           394.0 ± 0%25%0Ageomean                                                                          2.038k

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

wagoodman · 2023-08-09T21:04:54Z

Semantic diff for reviewers between the v11 and v12 json schemas BEFORE the struct renames

Code

The python code that generated this list

import json
import difflib

original_schema = "schema/json/schema-11.0.1.json"
new_schema = "schema/json/schema-12.0.0.json"

# {old-type-name: new-type-name}
type_def_mapping = {
 "AlpmMetadata": "arch-alpm-db-record",
 "ApkMetadata": "alpine-apk-db-record",
 "BinaryMetadata": "binary-signature",
 "CocoapodsMetadata": "cocoa-podfile-lock",
 "ConanLockMetadata": "c-conan-lock",
 "ConanMetadata": "c-conan",
 "DartPubMetadata": "dart-pubspec-lock",
 "DotnetPortableExecutableMetadata": "dotnet-portable-executable",
 "DotnetDepsMetadata": "dotnet-deps",
 "DpkgMetadata": "debian-dpkg-db-record",
 "GemMetadata": "ruby-gemspec",
 "GolangBinMetadata": "go-module-binary-buildinfo",
 "GolangModMetadata": "go-module",
 "HackageMetadata": "haskell-hackage-stack",
 "JavaMetadata": "java-archive",
 "KbPackageMetadata": "microsoft-kb-patch",
 "LinuxKernelMetadata": "linux-kernel-archive",
 "LinuxKernelModuleMetadata": "linux-kernel-module",
 "MixLockMetadata": "elixir-mix-lock",
 "NixStoreMetadata": "nix-store",
 "NpmPackageJSONMetadata": "javascript-npm-package",
 "NpmPackageLockJSONMetadata": "javascript-npm-package-lock",
 "PhpComposerJSONMetadata": "php-composer-lock",
 "PortageMetadata": "gentoo-portage-db-record",
 "PythonPackageMetadata": "python-package",
 "PythonPipfileLockMetadata": "python-pipfile-lock",
 "PythonRequirementsMetadata": "python-pip-requirements",
 "RebarLockMetadata": "erlang-rebar-lock",
 "RDescriptionFileMetadata": "r-description",
 "RpmdbFileRecord": "rpm-file-record",
 "RpmMetadata": "redhat-rpm-db-record",
 "RpmdbMetadata": "redhat-rpm-db-record",
 "RpmDBMetadata": "redhat-rpm-db-record",
 "RpmArchiveMetadata": "redhat-rpm-archive",
 "SwiftPackageManagerMetadata": "swift-package-manager-lock",
 "CargoPackageMetadata": "rust-cargo-lock"
}


def main():
 original_type_definitions = extract_type_definitions(original_schema)
 new_type_definitions = extract_type_definitions(new_schema)

 new_names_diffed = set()
 names_with_same_content = set()

 for definition_name, old_def in original_type_definitions.items():
     new_name = get_new_name(definition_name)
     if not new_name:
         new_name = definition_name
     new_def = new_type_definitions.get(new_name, "")
     if not new_def:
         print("Missing definition in new schema: {}".format(definition_name))
         continue

     new_names_diffed.add(new_name)

     # diff the definitions
     diff = difflib.unified_diff(old_def.splitlines(), new_def.splitlines(), fromfile=original_schema, tofile=new_schema)
     diff = "\n".join(diff)
     if diff:
         print("Diff for {}".format(definition_name))
         print(diff)
         print()
     else:
         names_with_same_content.add(definition_name)

 # for all new names not processed, print a warning

 for definition_name, new_def in new_type_definitions.items():
     if definition_name not in new_names_diffed:
         print("Missing equivalent definition in original schema: {}".format(definition_name))

 print(f"Definitions with same content: {len(names_with_same_content)}")
 for name in sorted(list(names_with_same_content)):
     print("  -", name)


def extract_type_definitions(schema_file_path) -> dict[str, str]:
 with open(schema_file_path, "r") as schema_file:
     schema = json.load(schema_file)

 definitions = schema.get("$defs", {})
 type_definitions = {}

 for definition_name, definition in definitions.items():
     # if definition_name in type_def_mapping:
     #     new_name = to_camel_case(type_def_mapping[definition_name])
     #     # print("Renaming {} to {}".format(definition_name, new_name))
     #     definition_name = new_name
     # # else:
     # #     print("No mapping for {}".format(definition_name))
     type_definitions[definition_name] = json.dumps(definition, indent=2, sort_keys=True)

 return type_definitions


def get_new_name(name: str) -> str | None:
 if name in type_def_mapping:
     return to_camel_case(type_def_mapping[name])


def to_camel_case(s: str) -> str:
 s = s.replace("-", "_")
 return ''.join(x.capitalize() or '_' for x in s.split('_'))


if __name__ == "__main__":
 main()

Diff for HackageMetadata
--- schema/json/schema-11.0.1.json

+++ schema/json/schema-12.0.0.json

@@ -1,21 +1,8 @@

 {
   "properties": {
-    "name": {
-      "type": "string"
-    },
     "pkgHash": {
-      "type": "string"
-    },
-    "snapshotURL": {
-      "type": "string"
-    },
-    "version": {
       "type": "string"
     }
   },
-  "required": [
-    "name",
-    "version"
-  ],
   "type": "object"
 }

Diff for Package
--- schema/json/schema-11.0.1.json

+++ schema/json/schema-12.0.0.json

@@ -30,100 +30,109 @@

           "type": "null"
         },
         {
-          "$ref": "#/$defs/AlpmMetadata"
+          "$ref": "#/$defs/AlpineApkDbRecord"
         },
         {
-          "$ref": "#/$defs/ApkMetadata"
+          "$ref": "#/$defs/ArchAlpmDbRecord"
         },
         {
-          "$ref": "#/$defs/BinaryMetadata"
+          "$ref": "#/$defs/BinarySignature"
         },
         {
-          "$ref": "#/$defs/CargoPackageMetadata"
+          "$ref": "#/$defs/CConan"
         },
         {
-          "$ref": "#/$defs/CocoapodsMetadata"
+          "$ref": "#/$defs/CConanLock"
         },
         {
-          "$ref": "#/$defs/ConanLockMetadata"
+          "$ref": "#/$defs/CocoaPodfileLock"
         },
         {
-          "$ref": "#/$defs/ConanMetadata"
+          "$ref": "#/$defs/DartPubspecLock"
         },
         {
-          "$ref": "#/$defs/DartPubMetadata"
+          "$ref": "#/$defs/DebianDpkgDbRecord"
         },
         {
-          "$ref": "#/$defs/DotnetDepsMetadata"
+          "$ref": "#/$defs/DotnetDeps"
         },
         {
-          "$ref": "#/$defs/DotnetPortableExecutableMetadata"
+          "$ref": "#/$defs/DotnetPortableExecutable"
         },
         {
-          "$ref": "#/$defs/DpkgMetadata"
+          "$ref": "#/$defs/ElixirMixLock"
         },
         {
-          "$ref": "#/$defs/GemMetadata"
+          "$ref": "#/$defs/ErlangRebarLock"
         },
         {
-          "$ref": "#/$defs/GolangBinMetadata"
+          "$ref": "#/$defs/GentooPortageDbRecord"
         },
         {
-          "$ref": "#/$defs/GolangModMetadata"
+          "$ref": "#/$defs/GoModule"
         },
         {
-          "$ref": "#/$defs/HackageMetadata"
+          "$ref": "#/$defs/GoModuleBinaryBuildinfo"
         },
         {
-          "$ref": "#/$defs/JavaMetadata"
+          "$ref": "#/$defs/HaskellHackageStack"
         },
         {
-          "$ref": "#/$defs/KbPackageMetadata"
+          "$ref": "#/$defs/HaskellHackageStackLock"
         },
         {
-          "$ref": "#/$defs/LinuxKernelMetadata"
+          "$ref": "#/$defs/JavaArchive"
         },
         {
-          "$ref": "#/$defs/LinuxKernelModuleMetadata"
+          "$ref": "#/$defs/JavascriptNpmPackage"
         },
         {
-          "$ref": "#/$defs/MixLockMetadata"
+          "$ref": "#/$defs/JavascriptNpmPackageLock"
         },
         {
-          "$ref": "#/$defs/NixStoreMetadata"
+          "$ref": "#/$defs/LinuxKernelArchive"
         },
         {
-          "$ref": "#/$defs/NpmPackageJSONMetadata"
+          "$ref": "#/$defs/LinuxKernelModule"
         },
         {
-          "$ref": "#/$defs/NpmPackageLockJSONMetadata"
+          "$ref": "#/$defs/MicrosoftKbPatch"
         },
         {
-          "$ref": "#/$defs/PhpComposerJSONMetadata"
+          "$ref": "#/$defs/NixStore"
         },
         {
-          "$ref": "#/$defs/PortageMetadata"
+          "$ref": "#/$defs/PhpComposerInstalled"
         },
         {
-          "$ref": "#/$defs/PythonPackageMetadata"
+          "$ref": "#/$defs/PhpComposerLock"
         },
         {
-          "$ref": "#/$defs/PythonPipfileLockMetadata"
+          "$ref": "#/$defs/PythonPackage"
         },
         {
-          "$ref": "#/$defs/PythonRequirementsMetadata"
+          "$ref": "#/$defs/PythonPipRequirements"
         },
         {
-          "$ref": "#/$defs/RDescriptionFileMetadata"
+          "$ref": "#/$defs/PythonPipfileLock"
         },
         {
-          "$ref": "#/$defs/RebarLockMetadata"
+          "$ref": "#/$defs/RDescription"
         },
         {
-          "$ref": "#/$defs/RpmMetadata"
+          "$ref": "#/$defs/RedhatRpmArchive"
         },
         {
-          "$ref": "#/$defs/SwiftPackageManagerMetadata"
+          "$ref": "#/$defs/RedhatRpmDbRecord"
+        },
+        {
+          "$ref": "#/$defs/RubyGemspec"
+        },
+        {
+          "$ref": "#/$defs/RustCargoLock"
+        },
+        {
+          "$ref": "#/$defs/SwiftPackageManagerLock"
         }
       ]
     },

Diff for RpmMetadata
--- schema/json/schema-11.0.1.json

+++ schema/json/schema-12.0.0.json

@@ -15,7 +15,7 @@

     },
     "files": {
       "items": {
-        "$ref": "#/$defs/RpmdbFileRecord"
+        "$ref": "#/$defs/RpmFileRecord"
       },
       "type": "array"
     },

Missing equivalent definition in original schema: HaskellHackageStackLock
Missing equivalent definition in original schema: PhpComposerInstalled
Missing equivalent definition in original schema: RedhatRpmArchive
Definitions with same content: 64
  - AlpmFileRecord
  - AlpmMetadata
  - ApkFileRecord
  - ApkMetadata
  - BinaryMetadata
  - CargoPackageMetadata
  - ClassifierMatch
  - CocoapodsMetadata
  - ConanLockMetadata
  - ConanMetadata
  - Coordinates
  - DartPubMetadata
  - Descriptor
  - Digest
  - Document
  - DotnetDepsMetadata
  - DotnetPortableExecutableMetadata
  - DpkgFileRecord
  - DpkgMetadata
  - File
  - FileLicense
  - FileLicenseEvidence
  - FileMetadataEntry
  - GemMetadata
  - GolangBinMetadata
  - GolangModMetadata
  - IDLikes
  - JavaManifest
  - JavaMetadata
  - KbPackageMetadata
  - License
  - LinuxKernelMetadata
  - LinuxKernelModuleMetadata
  - LinuxKernelModuleParameter
  - LinuxRelease
  - Location
  - MixLockMetadata
  - NixStoreMetadata
  - NpmPackageJSONMetadata
  - NpmPackageLockJSONMetadata
  - PhpComposerAuthors
  - PhpComposerExternalReference
  - PhpComposerJSONMetadata
  - PomParent
  - PomProject
  - PomProperties
  - PortageFileRecord
  - PortageMetadata
  - PythonDirectURLOriginInfo
  - PythonFileDigest
  - PythonFileRecord
  - PythonPackageMetadata
  - PythonPipfileLockMetadata
  - PythonRequirementsMetadata
  - RDescriptionFileMetadata
  - RebarLockMetadata
  - Relationship
  - RpmdbFileRecord
  - Schema
  - SearchResult
  - Secrets
  - Source
  - SwiftPackageManagerMetadata
  - licenses

…adata Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

kzantow

I don't see any blocking issues, but left a suggestion about defining the type-to-name mappings for JSON.

Removing approval so this doesn't accidentally get merged until we're ready for it

spiffcs · 2023-08-17T15:44:51Z

@wagoodman I read through everything here and no notes or comments the change makes sense IMO - feel free to merge when you think syft is ready for the major schema bump

…adata Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

…adata Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

there have been enough changes to warrant a review on the new change set

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

…adata Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

kzantow

🎉

…ataType values (anchore#1983) * [wip] Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * distinct the package metadata functions Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * remove metadata type from package core model Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * incorporate review feedback for names Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * add RPM archive metadata and split parser helpers Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * clarify the python package metadata type Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * rename the KB metadata type Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * break hackage and composer types by use case Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * linting fix Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix encoding and decoding for syft-json and cyclonedx Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * bump json schema to 11 Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * update cyclonedx-json snapshots Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * update cyclonedx-xml snapshots Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * update spdx-json snapshots Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * update spdx-tv snapshots Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * update syft-json snapshots Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * correct metadata type in stack yaml parser test Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix bom-ref redactor for cyclonedx-xml Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * add tests for legacy package metadata names Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * regenerate json schema v11 Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix legacy HackageMetadataType reflect type value check Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix linting Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * packagemetadata discovery should account for type shadowing Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix linting Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix cli tests Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * bump json schema version to v12 Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * update json schema to incorporate changes from main Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * add syft-json legacy config option Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * add tests around v11-v12 json decoding Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * add docs for SYFT_JSON_LEGACY Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * rename structs to be compliant with new naming scheme Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> --------- Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

wagoodman added 23 commits July 13, 2023 09:13

[wip]

808ea50

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

distinct the package metadata functions

ddce1a4

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

remove metadata type from package core model

1f08d1f

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

incorporate review feedback for names

863f34c

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

add RPM archive metadata and split parser helpers

48aede3

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

clarify the python package metadata type

cff1a19

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

rename the KB metadata type

48b5051

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

break hackage and composer types by use case

51255ce

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

linting fix

dfe47a0

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

fix encoding and decoding for syft-json and cyclonedx

0fbaa53

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

bump json schema to 11

05c94f7

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

update cyclonedx-json snapshots

3bd1d76

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

update cyclonedx-xml snapshots

756fe65

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

update spdx-json snapshots

1e1a4e6

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

update spdx-tv snapshots

593adbe

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

update syft-json snapshots

fc14a0b

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

correct metadata type in stack yaml parser test

f7855a8

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

fix bom-ref redactor for cyclonedx-xml

8cc06d3

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

add tests for legacy package metadata names

0fef399

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

Merge remote-tracking branch 'origin/main' into normalize-package-met…

6638f0d

…adata Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

regenerate json schema v11

c6fc1fb

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

fix legacy HackageMetadataType reflect type value check

3e96167

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

fix linting

7b7b616

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

packagemetadata discovery should account for type shadowing

e829dc4

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

wagoodman mentioned this pull request Aug 9, 2023

Account for syft package metadata changes anchore/grype#1423

Merged

wagoodman added 2 commits August 9, 2023 17:15

Merge remote-tracking branch 'origin/main' into normalize-package-met…

8a3ddd3

…adata Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

fix linting

f5a8c83

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

wagoodman marked this pull request as ready for review August 9, 2023 21:17

fix cli tests

6aac241

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

kzantow previously approved these changes Aug 11, 2023

View reviewed changes

Comment thread syft/internal/packagemetadata/names.go Outdated

spiffcs previously approved these changes Aug 17, 2023

View reviewed changes

wagoodman added 3 commits September 21, 2023 11:38

Merge remote-tracking branch 'origin/main' into normalize-package-met…

a20281b

…adata Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

bump json schema version to v12

5fe5bca

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

Merge remote-tracking branch 'origin/main' into normalize-package-met…

628bfde

…adata Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

github-actions Bot added json-schema Changes the json schema breaking-change Change is not backwards compatible labels Oct 25, 2023

wagoodman added 2 commits October 25, 2023 10:35

update json schema to incorporate changes from main

ba45503

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

add syft-json legacy config option

ea1bdf0

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

wagoodman added 2 commits October 25, 2023 14:45

add tests around v11-v12 json decoding

8308780

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

add docs for SYFT_JSON_LEGACY

be41414

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

wagoodman requested a review from a team October 25, 2023 19:06

wagoodman mentioned this pull request Oct 25, 2023

Rename Location virtualPath to accessPath #1835

Closed

kzantow reviewed Oct 27, 2023

View reviewed changes

wagoodman added 2 commits October 30, 2023 10:28

rename structs to be compliant with new naming scheme

cf20d29

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

Merge remote-tracking branch 'origin/main' into normalize-package-met…

1d867ac

…adata Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

wagoodman force-pushed the normalize-package-metadata branch from e4a4303 to 1d867ac Compare October 30, 2023 15:03

kzantow approved these changes Oct 30, 2023

View reviewed changes

wagoodman merged commit 1aaa644 into main Oct 30, 2023

wagoodman deleted the normalize-package-metadata branch October 30, 2023 16:12

Porkepix mentioned this pull request Nov 7, 2023

syft 0.95.0 Homebrew/homebrew-core#153559

Merged

kzantow mentioned this pull request May 23, 2024

feat: add config command #2892

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove MetadataType from core package object and normalize JSON metadataType values#1983

Remove MetadataType from core package object and normalize JSON metadataType values#1983
wagoodman merged 36 commits intomainfrom
normalize-package-metadata

wagoodman commented Jul 31, 2023 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 31, 2023 •

edited

Loading

Uh oh!

wagoodman commented Aug 9, 2023 •

edited

Loading

Uh oh!

kzantow left a comment

Uh oh!

Uh oh!

spiffcs commented Aug 17, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kzantow left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

wagoodman commented Jul 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmark Test Results

Uh oh!

wagoodman commented Aug 9, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kzantow left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

spiffcs commented Aug 17, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kzantow left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

wagoodman commented Jul 31, 2023 •

edited

Loading

github-actions Bot commented Jul 31, 2023 •

edited

Loading

wagoodman commented Aug 9, 2023 •

edited

Loading