Skip to content

EPIC: Implement ocm download resource and generic resource downloader binding #523

@jakobmoellerdev

Description

@jakobmoellerdev

Description
What is the goal of this epic?

The goal of this epic is to provide a parallel implementation to ocm download resource:

ocm download resource --help
ocm download resources — Download Resources Of A Component Version

Synopsis:
  ocm download resources [<options>]  <component> {<name> { <key>=<value> }}

Aliases:
  resources, resource, res, r

Options:
      --check-verified              enable verification store
  -c, --constraints constraints     version constraint
  -d, --download-handlers           use download handler if possible
      --downloader <name>=<value>   artifact downloader (<name>[:<artifact type>[:<media type>[:<priority>]]]=<JSON target config>) (default [])
  -x, --executable                  download executable for local platform
  -h, --help                        help for resources
      --latest                      restrict component versions to latest
      --lookup stringArray          repository name or spec for closure lookup fallback
  -O, --outfile string              output file or directory
  -r, --recursive                   follow component reference nesting
      --repo string                 repository name or spec
  -t, --type stringArray            resource type filter
      --verified string             file used to remember verifications for downloads (default "~/.ocm/verified")
      --verify                      verify downloads

User Story

For this we will need to pass through not only our downloader constraints but also provide a binding library that is able to work with our plugin systems to include various download formats.

The important downloaders we need to work with are:

  • Local Blob Downloaders
  • OCI / CTF OCI Artifact Downloaders

For the interface of a resource downloader we might want to reuse our existing OCI based interface, but we can experiment a bit during implementation:

type ResourceDownloader interface {
  GetCredentialConsumerIdentity(ctx context.Context, resource *constructor.Resource) (identity runtime.Identity, err error)
  DownloadResource(ctx context.Context, resource *constructor.Resource, credentials map[string]string) (data *blob.ReadOnlyBlob, err error)
}

This can be seamlessly integrated with our credential system thanks to the consumer identity.

To add new downloaders, we must merely implement this interface with different implementations based on their access type (e.g. for helm access we can register a helm download.

Note that the returned download data MAY be a specific target format. As such the blob cannot be just written to the filesystem, but must also be processable when written to a filesystem.

For this, we can make use of an additional interface AccessMethod, which existed in OCM before. This access method can be feeded with a blob.ReadOnlyBlob and then be used to either write the data to a filesystem or access it in code.

type OpenOptions struct {
  Resource *runtime.Resource // if we come from a descriptor, we can feed the method with additional information from the resource description
}

// Per AccessMethod implementation (library specific)
func Open(ctx context.Context, blob blob.ReadOnlyBlob, opts OpenOptions) (AccessMethod, error)

type AccessMethod interface {
   Download(ctx context.Context, fs fs.FS) (result *AccessDownloadResult, err error)
}

type AccessDownloadResult struct {
   WrittenFiles []string
}

we can easily integrate access methods and downloaders by providing registry implementations for them (see the plugin system there and coding a provider interface)

Scope
List all deliverables that are part of this epic. The Epic is considered DONE if all of the below mentioned deliverables are available.

  • Define a generic Resource Download Interface
  • Define a generic AccessMethod Interface, that has custom methods as well as an implementation that is able to convert the blob.ReadOnlyBlob from the download into a custom library implementation. On top of that, every access method SHOULD implement an additional ResourceWriter BlobFilesystemWriter interface that can write blob content to a filesystem. These writer implementations can be registered by media type of the blob and access type of the resource.
  • Implement and pass through the implementations for OCI Artifacts, generating OCI Image Layouts
  • Prepare new stories for other implementations, such as files and directories
  • Tests
  • End user documentation
  • Informed Community

Out of Scope

  • Implementing every single access type from the start
  • Staying 100% compatible with downloader configuration types (the concepts are differing by a lot and we may not stay compatible e2e)

Metadata

Metadata

Labels

area/ipceiImportant Project of Common European Interestkind/epicLarge multi-story topic

Type

No fields configured for Epic.

Projects

Status
🍺 Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions