AMF 5.7.2 Release Notes

@arielmirra

AMF 5.7.2 Release Notes

This release introduces limited (phase 1) support for OpenAPI Specification (OAS) 3.1.

Our goal with this version is to support parsing/transforming/rendering OAS 3.1 APIs correctly, we achieved that goal with all the new fields and validations OAS 3.1 introduces, but we limit the support to just that. More info under Limitations title.

What's New: Limited OpenAPI 3.1 Support

Root Model

New webhooks field: We've added parsing and rendering support for the new webhooks field at root level (for callback mechanisms).
New jsonSchemaDialect field: We've added parsing and rendering support for the new jsonSchemaDialect field at root level.
- NOTE: While it's stored, its functionality (applying it as a default schema) is not yet implemented.
ComponentsObject - New PathItems field in Components: You can now define reusable PathItems objects within the components section, aligning with the new capabilities of OAS 3.1.

New Fields in OAS 3.1 objects

Several core OAS objects have been updated to support new fields and features introduced in OAS 3.1:

ReferenceObject - New summary and description Fields: $ref fields can now include summary and description fields.
DiscriminatorObject - Extensions Support: The discriminator object now correctly parses and stores extensions.
InfoObject - New summary Field: The root-level info object now supports the summary field.
LicenseObject - New identifier Field and Exclusive Fields Validation: The license has been updated with the new identifier field for SPDX license identifiers. Additionally, enhanced validations ensure that the identifier and url/name fields are used exclusively as per the OAS 3.1 specification.
SchemaObject - New $schema Field: The $schema field from OAS 3.1 is now parsed and stored within the SchemaObject.
- NOTE: Similar to jsonSchemaDialect, its functionality (applying it as a default schema) is not yet implemented.
SchemaObject - Removal of nullable Field: In alignment with OAS 3.1's move to JSON Schema Draft 2020-12 (where nullable is replaced by type: ['string', 'null'] for example), the nullable field has been removed from schemas.
SecurityScheme - New Security Scheme Type mutualTLS: AMF now recognizes and supports the new mutualTLS security scheme type introduced in OAS 3.1.

Validation Improvements & Changes

This release includes new validations that follow OAS 3.1 rules:

New Validation - Path Parameter Values: Path parameter values cannot contain the unescaped characters /, ? or #.
New Validation - Server Variable's enum and default Validations: if a Server Variable defines an enum facet (which is not required) it must define a non-empty array, and if the enum facet is defined the default value MUST be defined in the enum's values
Validation Change - OperationObject responses Field is Optional: The responses field within an operation is now considered optional
Validation Change - At least paths, components, or webhooks must be defined: A new validation rule ensures that an OpenAPI document defines at least one of paths, components, or webhooks at the top level.

Limitations

While AMF 5.7.2 offers foundational support for OAS 3.1, it's important to be aware of the following limitations in this release. These areas are currently out of scope for the limited OAS 3.1 implementation:

Conversions between Specs: The ability to convert an API definition from OAS 2.0 or 3.0 to OAS 3.1, or vice-versa, hasn't been fully tested thus is not supported. This release focuses solely on parsing/transforming/rendering OAS 3.1 APIs correctly.
OAS 3.1 Components Asset: Support for the new Components asset type in OAS 3.1 is out of scope.
JSON Schema 2020-12 Support:
- JSON Schema 2020-12 Semantics: The full functional adoption and application of JSON Schema Draft 2020-12 within the AMF model is out of scope. While some related fields are parsed (like $schema and jsonSchemaDialect), their semantic implications on schema validation or transformation within AMF are not active.
- jsonSchemaDialect Application: The jsonSchemaDialect field in the OpenAPIObject is parsed and stored, but AMF does not currently apply its declared dialect as a default schema for subsequent schemas in the document.
- $schema Application: Similarly, the $schema field within OAS 3.1 schemas is parsed and stored, but AMF does not apply its declared schema dialect for the individual schema.

Thank you.

Full OAS 3.1 API

This is a complete OAS 3.1 API showing the new fields and features we've added:

openapi: 3.1.0
info:
  title: OAS 3.1 with all new fields
  version: 1.0.0
  summary: summary of the API
  license:
    name: Apache 2.0
    identifier: Apache-2.0 # THIS IS NEW
jsonSchemaDialect: https://spec.openapis.org/oas/3.1/dialect/base # NEW OBJECT - not being applied
webhooks: # NEW OBJECT
  /newPet:
    post:
      requestBody:
        description: Information about a new pet in the system
        content:
          application/json:
            schema: { }
      responses:
        "200":
          description: Return a 200 status to indicate that the data was received successfully
  testWebhook: # webhooks don't need to start with '/' as pathItems
    $ref: '#/components/pathItems/testEndpoint'

paths:
  /users:
    post:
      operationId: getUsers
      responses:
        '200':
          description: this response description should override the referenced one
          $ref: '#/components/responses/usersListResponse'
      requestBody:
        description: this request description should override the referenced one
        $ref: '#/components/requestBodies/createUserRequest'

  /test:
    $ref: '#/components/pathItems/testEndpoint' # THIS IS NEW

security:
  - openIdConnect:
      - some
  - apikey:
      - something
      - other thing
  - http:
      - something else
  - mutualTLS:
      - other something

components:
  schemas:
    oneOfSchema:
      discriminator:
        x-custom-ann-test: custom ann value # THIS IS NEW
        x-custom-ann: custom ann value # THIS IS NEW
        propertyName: a
      oneOf:
        - type: object
          properties:
            a:
              type: string
        - type: object
          properties:
            b:
              type: string
    User:
      type: object
      properties:
        id:
          type: string
        name:
          type: string
        email:
          type: string
    OtherSchema:
      $schema: https://spec.openapis.org/oas/3.1/dialect/base # NEW OBJECT - not being applied
      type: object
      properties:
        some:
          type: string
    NullPropSchema:
      type: object
      properties:
        nullProp:
          type: null
    NullableSchema:
      anyOf: # THIS REPLACES NULLABLE
        - type: "string"
        - type: "null"

  responses:
    usersListResponse:
      description: this response description should be overridden
      content:
        application/json:
          schema:
            type: array
            items:
              $ref: '#/components/schemas/User'
          examples:
            usersExample:
              summary: this example summary should override the referenced one # THIS IS NEW
              description: this example description should override the referenced one # THIS IS NEW
              $ref: '#/components/examples/usersList'

  requestBodies:
    createUserRequest:
      description: this request description should be overridden
      content:
        application/json:
          schema:
            type: string

  examples:
    usersList:
      summary: this example summary should be overridden
      description: this example description should be overridden
      value:
        - id: "1"
          name: Alice
          email: alice@example.com
        - id: "2"
          name: Bob
          email: bob@example.com

  securitySchemes:
    apikey:
      type: apiKey
      name: someApiKeyName
      in: header
    http:
      type: http
      scheme: bearer
      bearerFormat: JWT
    openIdConnect:
      type: openIdConnect
      description: openIdConnect security scheme
      openIdConnectUrl: https://a.ml/
    mutualTLS: # THIS IS NEW
      type: mutualTLS
      description: mutualTLS security scheme

  pathItems: # NEW OBJECT - not being applied
    testEndpoint: # doesn't need to start with '/' like pathItems
      get:
        responses:
          '200':
            description: A simple string response
            content:
              text/plain:
                schema:
                  type: string

What's Changed

W-10548354: server variables validations in OAS 3.1 by @arielmirra in #2111
W-10548360 - Added new MutualTLS security scheme for OAS 3.1 by @looseale in #2112
W-18215550: restrict which payloads FixFilePayloads applies to by @arielmirra in #2113
W-18217146: Publish 5.7.1 dev by @damianpedra in #2118
W-10548352 - $schema parsing in OAS 3.1 SchemaObject by @looseale in #2119
W-17778257: add warning for undefined path params in OAS by @arielmirra in #2120
W-10548356: remove nullable key from OAS 3.1 shapes by @arielmirra in #2121
W-18052245 - Fixed nested ref resolution to ExternalFragment with com… by @looseale in https://github.com/a...

@looseale

What's Changed

W-17321537 - Adding test to JSON-LD of OAS 3.1. Fixed OAS 3.1 syntax by @looseale in #2083
W-17128842 - Test bug fix - Multiline text with escape character by @leochalub in #2084
W-17397626 - Fix Nexus IQ by @leochalub in #2085
W-17562799: fix header parameter examples not being normalized in OAS 3 by @arielmirra in #2086
W-16952616 - Bump vulnerable version of jackson databind and core by @looseale in #2090
W-16530856: add missing flow and scopes fields in raml to oas2 conversion by @arielmirra in #2094
W-17746653: maybe fix extra character not throwing violation by @arielmirra in #2095
W-16786864 - Fix typings by @looseale in #2096
W-17168388. Fix for nested refs in OAS external fragments. by @jisoldi in #2097
W-17291325 - Skip body part emission in OAS3 if there is no payload by @looseale in #2098
W-17759271: fix raml file upload param to oas by @arielmirra in #2100
W-17369296: fix node shape props propagation in raml unions by @arielmirra in #2101
W-17369296: refactor Oas20SecuritySettingsMapper by @arielmirra in #2102
W-10548235 - OAS 3.1: added jsonSchemaDialect parsing and emission by @looseale in #2103
W-18067178: add AddItemsToArrayType transf. step to OAS2 by @arielmirra in #2104
W-10548359 - OAS 3.1 security settings scopes by @looseale in #2105
W-15609917: fix oas3 request lexicals when missing requestBody by @arielmirra in #2106
W-18012202: fix async message payloads casting by @arielmirra in #2109

Full Changelog: 5.7.0...5.7.1

@arielmirra

Java 21 support

Dependency Changes

Moved to Scala 2.12.20
Moved to SBT 1.10.4
ScalaJS 1.17.0

Behavior Changes

Unified REGEX validation between JVM and JS: now ScalaJS exports the Java Regex library. JS Regex works the same as Java ones.

What's Changed

W-10548463: [OAS 3.1] add ref summary and description by @arielmirra in #2072
W-17016391: Update sonarQube credentials for sfdc sonar by @damianpedra in #2073
W-17142856 - parameter validation missing range by @leochalub in #2074
W-14887298: fix domain extensions component id encoding by @arielmirra in #2075
W-10548497 - LicenseObject - New facet 'identifier' and exclusive fields validation by @leochalub in #2076
W-10548498 - InfoObject - New facet summary by @leochalub in #2077
W-14661042 - Java21 support related changes by @looseale in #2078
W-14661042 - Commenting NexusIQ step after Java21 bump. Pending to fix by @looseale in #2079

Full Changelog: 5.6.4...5.7.0

@arielmirra

What's Changed

W-16343554: make avro record fields required + insert unresolved shapes in invalid avro types by @arielmirra in #2059
W-16899246 - ES Type extensions by @looseale in #2060
W-17005476 - Fix cache pipelines with wrong id by @looseale in #2061
W-10548506: add OAS 3.1 basic support by @arielmirra in #2062
W-16993090: fix OAS component $ref parsing by @arielmirra in #2063
W-10548506: add Oas31 to oas matchers in configurations and tests by @arielmirra in #2064
W-10548504: WebAPI - New facet webhooks by @leochalub in #2065
W-10548462: add annotations support in OAS3.1 discriminator by @arielmirra in #2067

New Contributors

@leochalub made their first contribution in #2065

Full Changelog: 5.6.3...5.6.4

@arielmirra

What's Changed

W-16847560: use the same json schema version mapping in raml and oas by @arielmirra in #2052
W-16712475 - Added parsing option allowing to parse extensions in eve… by @looseale in #2053
W-16930843 - Some dependency bumps by @looseale in #2054
W-16888404: change parameter description parsing and bump model by @arielmirra in #2055

Full Changelog: 5.6.2...5.6.3

@looseale

Bug fixing

Fixed bug in avro validation related with inner union schemas with default values

What's Changed

Release 5.6.1 to develop by @looseale in #2046
W-16772136 - Minor changes in Avro ADR due last library changes (naming) by @looseale in #2047
W-16837997 - Avro union fixes by @looseale in #2048
Publish 5.6.2 by @looseale in #2049

Full Changelog: 5.6.1...5.6.2

AVRO Support in AMF

We're happy to announce we've added support for AVRO Schema 1.9.0 in AMF:

as a standalone document
inside Async APIs

an AVRO Schema has the following properties:

It's defined in a .json or .avsc file
It doesn't have a special key that indicates it's an AVRO Schema, nor it's version (like JSON Schema does with it's $schema)

Where we support and DON'T support AVRO Schemas

We Support AVRO Schemas (inline or inside a $ref):

as a standalone document or file
- we encourage users to use the .avsc file type to indicate that's an avro file
- must use the specific AvroConfiguration
inside a message payload in an AsyncAPI
- the key schemaFormat MUST be declared and specify it's an AVRO payload
an AvroSchemaDocument can only be parsed with the specific AvroConfiguration

We don't support AVRO Schemas:

inside components --> schemas in an AsyncAPI
- because we can't determine if it's an AVRO Schema or any other schema

Known AVRO Validation limitations

We're using the Apache official libraries for JVM and JS. The validation libraries differ in interfaces and implementations, and each has some known constraints:

JVM avro validation constraints

validation per se is not supported, we try to parse an avro schema and throw parsing results if there are any
- this means it's difficult to have location of where the error is thrown, we may give an approximate location from our end post-validation
when a validation is thrown, the rest of the file is not being searched for more validations
- this is particularly important in large avro schemas, where many errors can be found but only one is shown

Both JVM & JS validation constraints

"default" values are not being validated when the type is bytes, map, or array
the validator treats as invalid an empty array as the default value for arrays ("default": []) even though the Avro Schema Specification has some examples with it
if an avro record has a field that is a union that includes the root record itself (recursive reference) we fail to validate it correctly because we treat that shape as an unresolved/undefined shape. This only applies to the field inside the record, when validating the complete record the library validates correctly. In the future we'll try to ignore the cases that we are now failing and/or show a warning instead

More information here

for more information on how to use AVRO in amf check the adrs/0014-avro-support.md file.

Full Changelog: 5.5.4...5.6.1

AVRO Support in AMF

We're happy to announce we've added support for AVRO Schema 1.9.0 in AMF:

as a standalone document
inside Async APIs

an AVRO Schema has the following properties:

It's defined in a .json or .avsc file
It doesn't have a special key that indicates it's an AVRO Schema, nor it's version (like JSON Schema does with it's $schema)

Where we support and DON'T support AVRO Schemas

We Support AVRO Schemas (inline or inside a $ref):

as a standalone document or file
- we encourage users to use the .avsc file type to indicate that's an avro file
- must use the specific AvroConfiguration
inside a message payload in an AsyncAPI
- the key schemaFormat MUST be declared and specify it's an AVRO payload
an AvroSchemaDocument can only be parsed with the specific AvroConfiguration

We don't support AVRO Schemas:

inside components --> schemas in an AsyncAPI
- because we can't determine if it's an AVRO Schema or any other schema

Known AVRO Validation limitations

We're using the Apache official libraries for JVM and JS. The validation libraries differ in interfaces and implementations, and each has some known constraints:

JVM avro validation constraints

validation per se is not supported, we try to parse an avro schema and throw parsing results if there are any
- this means it's difficult to have location of where the error is thrown, we may give an approximate location from our end post-validation
when a validation is thrown, the rest of the file is not being searched for more validations
- this is particularly important in large avro schemas, where many errors can be found but only one is shown

Both JVM & JS validation constraints

"default" values are not being validated when the type is bytes, map, or array
the validator treats as invalid an empty array as the default value for arrays ("default": []) even though the Avro Schema Specification has some examples with it
if an avro record has a field that is a union that includes the root record itself (recursive reference) we fail to validate it correctly because we treat that shape as an unresolved/undefined shape. This only applies to the field inside the record, when validating the complete record the library validates correctly. In the future we'll try to ignore the cases that we are now failing and/or show a warning instead

More information here

for more information on how to use AVRO in amf check the adrs/0014-avro-support.md file.

Full Changelog: 5.5.4...5.6.0

@looseale

AVRO transformation and rendering support (ALPHA)

In the previous release we added support for AVRO parsing, which means creating an AvroConfiguration with an AvroParsePlugin and with that create a BaseUnitClient that has the parse() method that parses an AVRO Schema and returns an AvroSchemaDocument.

This release, we've added support for the transform() and render() methods, letting the user resolve the model and export it to .jsonld or to .json again.

To do this, we've created the AvroRenderPlugin and added it to the AvroConfiguration as well as the necessary transformation pipelines. Now the AVRO Configuration is much more complete and looks like this:

object AvroConfiguration extends APIConfigurationBuilder {
  def Avro(): AMFConfiguration = {
    common()
      .withPlugins(List(AvroParsePlugin, AvroRenderPlugin))
      .withTransformationPipelines(
        List(
          AvroSchemaTransformationPipeline(),
          AvroSchemaEditingPipeline(),
          AvroSchemaCachePipeline()
        )
      )
  }
}

We're expecting to continue this growth and launch AVRO Validation in the next release, will keep you updated.

For more information about transformation and rendering check the following documentation:

What's Changed

W-16124325 -Avro interfaces by @looseale in #2015
release/5.5.3 to develop by @looseale in #2018
W-16006973: add AVRO ADR by @arielmirra in #2019
W-16006973: add avro-schema annotation on collections (items/values) parsing by @arielmirra in #2020
W-16006973: add avro annotations by @arielmirra in #2021
W-16246407 - Fix handle of method string kind by @looseale in #2022
W-15633231: add AVRO basic resolution pipelines by @arielmirra in #2023
W-15633198: add AVRO emission by @arielmirra in #2024
W-15633239 - AvroSchemaDocument ref handle by @looseale in #2025
W-15633231: add avro transformation pipelines and tests by @arielmirra in #2026
W-16346159: fix parsing avro record field without a type defined by @arielmirra in #2027
W-16346159: AVRO fixes by @arielmirra in #2028

Full Changelog: 5.5.3...5.5.4

@damianpedra

AVRO Support (ALPHA)

AVRO Schema Support in Async

Added support to parsing of AVRO Schemas in Async 2.x payload definitions (inlined or referencing to an external file).
This support is currently limited, only for parsing, not for validation or emission.

AVRO Schema Fragment

Added support to parsing of AVRO Schemas as a fragment using the new AvroConfiguration, returning an AvroSchemaDocument. This is only for AVRO Schema files parsed using this specific configuration, files referenced from an Async 2.x API will be processed as ExternalFragment as usual.
This support is currently limited, only for parsing, not for validation or emission.

What's Changed

W-15770660 Publish 5.5.2 dev by @damianpedra in #2004
W-15847817. Minor performance improvements by @jisoldi in #2001
W-15847817. Golden files regenerated after some order changes in annotations. by @jisoldi in #2005
W-15904614 - Some Lexical fixes by @looseale in #2008
W-15633176: add initial AVRO Schema parsing by @arielmirra in #2009
W-15633176: fix record fields parsing by @arielmirra in #2011
W-15633213: Add TCK for avro schemas by @damianpedra in #2013
W-16124325 - Avro interfaces by @looseale in #2014
W-16124325 - Publish 5.5.3 by @looseale in #2016
W-16124325 - Publish 5.5.3 by @looseale in #2017

Full Changelog: 5.5.2...5.5.3

Releases: aml-org/amf

5.7.2

AMF 5.7.2 Release Notes

What's New: Limited OpenAPI 3.1 Support

Root Model

New Fields in OAS 3.1 objects

Validation Improvements & Changes

Limitations

Full OAS 3.1 API

What's Changed

Contributors

Uh oh!

5.7.1

What's Changed

Contributors

Uh oh!

5.7.0

Java 21 support

Dependency Changes

Behavior Changes

What's Changed

Contributors

Uh oh!

5.6.4

What's Changed

New Contributors

Contributors

Uh oh!

5.6.3

What's Changed

Contributors

Uh oh!

5.6.2

Bug fixing

What's Changed

Contributors

Uh oh!

What's new in AMF 5.6.1

AVRO Support in AMF

Where we support and DON'T support AVRO Schemas

Known AVRO Validation limitations

JVM avro validation constraints

Both JVM & JS validation constraints

More information here

Uh oh!

What's new in AMF 5.6.0

AVRO Support in AMF

Where we support and DON'T support AVRO Schemas

Known AVRO Validation limitations

JVM avro validation constraints

Both JVM & JS validation constraints

More information here

Uh oh!

5.5.4

AVRO transformation and rendering support (ALPHA)

What's Changed

Contributors

Uh oh!

5.5.3

AVRO Support (ALPHA)

AVRO Schema Support in Async

AVRO Schema Fragment

What's Changed

Contributors

Uh oh!