[transfer] update imagestore interface to support multiple references #7964

dmcgowan · 2023-01-13T23:05:42Z

Updated image store to allow for a cleaner definition of import names and to allow the API to support multiple export names.

knight42 · 2023-01-14T04:47:45Z

api/types/transfer/imagestore.proto

+
+// ImageReference is used to create or find a reference for an image
+message ImageReference {
+	string Name = 1;


nit:

Suggested change

string Name = 1;

string name = 1;

knight42 · 2023-01-14T09:19:28Z

pkg/transfer/image/imagestore.go

+	return imgs, nil
 }

 func (is *Store) Get(ctx context.Context, store images.Store) (images.Image, error) {


How could we get extra image references from the image store?

I have that in a separate commit so far since still working on it, thinking the interface would be Lookup(ctx context.Context, store image.Store) ([]image.Image, error). I was thinking how to handle the prefix references on lookup, which could be interesting to use. Non prefix references should be straight forward.

AkihiroSuda · 2023-01-20T10:59:39Z

   default: === FAIL: . TestTransferImport/TagOnlyRef (0.08s)
    default: ==================
    default: WARNING: DATA RACE
    default: Read at 0x00c000edce10 by goroutine 2707:
    default:   github.com/containerd/containerd/integration/client.TestTransferImport.func1()
    default:       /vagrant/integration/client/import_test.go:561 +0x9fa
    default:   testing.tRunner()
    default:       /usr/local/go/src/testing/testing.go:1446 +0x216
    default:   testing.(*T).Run.func1()
    default:       /usr/local/go/src/testing/testing.go:1493 +0x47
    default: 
    default: Previous write at 0x00c000edce10 by goroutine 2710:
    default:   github.com/containerd/containerd/integration/client.(*imagesProgress).Progress()
    default:       /vagrant/integration/client/import_test.go:594 +0x128
    default:   github.com/containerd/containerd/integration/client.(*imagesProgress).Progress-fm()
    default:       <autogenerated>:1 +0xbe
    default:   github.com/containerd/containerd/pkg/transfer/proxy.(*proxyTransferrer).Transfer.func1()
    default:       /vagrant/pkg/transfer/proxy/transfer.go:76 +0x49b
    default: 
    default: Goroutine 2707 (running) created at:
    default:   testing.(*T).Run()
    default:       /usr/local/go/src/testing/testing.go:1493 +0x75d
    default:   github.com/containerd/containerd/integration/client.TestTransferImport()
    default:       /vagrant/integration/client/import_test.go:527 +0x171a
    default:   testing.tRunner()
    default:       /usr/local/go/src/testing/testing.go:1446 +0x216
    default:   testing.(*T).Run.func1()
    default:       /usr/local/go/src/testing/testing.go:1493 +0x47
    default: 
    default: Goroutine 2710 (running) created at:
    default:   github.com/containerd/containerd/pkg/transfer/proxy.(*proxyTransferrer).Transfer()
    default:       /vagrant/pkg/transfer/proxy/transfer.go:61 +0x388
    default:   github.com/containerd/containerd.(*Client).Transfer()
    default:       /vagrant/transfer.go:35 +0x250
    default:   github.com/containerd/containerd/integration/client.TestTransferImport.func1()
    default:       /vagrant/integration/client/import_test.go:552 +0x875
    default:   testing.tRunner()
    default:       /usr/local/go/src/testing/testing.go:1446 +0x216
    default:   testing.(*T).Run.func1()
    default:       /usr/local/go/src/testing/testing.go:1493 +0x47
    default: ==================
    default: ==================
    default: WARNING: DATA RACE
    default: Read at 0x00c000d1a020 by goroutine 2707:
    default:   github.com/containerd/containerd/integration/client.TestTransferImport.func1()
    default:       /vagrant/integration/client/import_test.go:565 +0xbe4
    default:   testing.tRunner()
    default:       /usr/local/go/src/testing/testing.go:1446 +0x216
    default:   testing.(*T).Run.func1()
    default:       /usr/local/go/src/testing/testing.go:1493 +0x47
    default: 
    default: Previous write at 0x00c000d1a020 by goroutine 2710:
    default:   github.com/containerd/containerd/integration/client.(*imagesProgress).Progress()
    default:       /vagrant/integration/client/import_test.go:594 +0xe4
    default:   github.com/containerd/containerd/integration/client.(*imagesProgress).Progress-fm()
    default:       <autogenerated>:1 +0xbe
    default:   github.com/containerd/containerd/pkg/transfer/proxy.(*proxyTransferrer).Transfer.func1()
    default:       /vagrant/pkg/transfer/proxy/transfer.go:76 +0x49b
    default: 
    default: Goroutine 2707 (running) created at:
    default:   testing.(*T).Run()
    default:       /usr/local/go/src/testing/testing.go:1493 +0x75d
    default:   github.com/containerd/containerd/integration/client.TestTransferImport()
    default:       /vagrant/integration/client/import_test.go:527 +0x171a
    default:   testing.tRunner()
    default:       /usr/local/go/src/testing/testing.go:1446 +0x216
    default:   testing.(*T).Run.func1()
    default:       /usr/local/go/src/testing/testing.go:1493 +0x47
    default: 
    default: Goroutine 2710 (running) created at:
    default:   github.com/containerd/containerd/pkg/transfer/proxy.(*proxyTransferrer).Transfer()
    default:       /vagrant/pkg/transfer/proxy/transfer.go:61 +0x388
    default:   github.com/containerd/containerd.(*Client).Transfer()
    default:       /vagrant/transfer.go:35 +0x250
    default:   github.com/containerd/containerd/integration/client.TestTransferImport.func1()
    default:       /vagrant/integration/client/import_test.go:552 +0x875
    default:   testing.tRunner()
    default:       /usr/local/go/src/testing/testing.go:1446 +0x216
    default:   testing.(*T).Run.func1()
    default:       /usr/local/go/src/testing/testing.go:1493 +0x47
    default: ==================
    default:     testing.go:1319: race detected during execution of test
    default:     --- FAIL: TestTransferImport/TagOnlyRef (0.08s)
    default: 
    default: === FAIL: . TestTransferImport/TagOnlyOverwriteDigestRefs (0.01s)
    default:     import_test.go:555: rpc error: code = NotFound desc = not found
    default:     --- FAIL: TestTransferImport/TagOnlyOverwriteDigestRefs (0.01s)
    default: 
    default: === FAIL: . TestTransferImport (0.68s)
    default:     testing.go:1319: race detected during execution of test

https://cirrus-ci.com/task/6699598622752768?logs=integration#L121

pkg/transfer/image/imagestore_test.go

dmcgowan · 2023-01-23T22:49:08Z

/retest

dmcgowan · 2023-01-25T18:36:23Z

/retest

pkg/transfer/image/imagestore.go

pkg/transfer/transfer.go

cpuguy83 · 2023-02-10T22:44:20Z

integration/client/import_test.go

+		// Name is the name of the test
+		Name string
+
+		// Images is the names of the images to creat


mikebrow · 2023-02-13T20:18:34Z

api/types/transfer/imagestore.proto

+	// name reference from the prefix.
+	// Only used if IsPrefix is true.
+	bool skip_named_digest = 5;
+}


nit nl missing

Signed-off-by: Derek McGowan <derek@mcg.dev>

dmcgowan force-pushed the transfer-image-store-references branch from 737524b to 9ad5d67 Compare January 13, 2023 23:14

dmcgowan changed the title ~~Update imagestore interface to support multiple references~~ [transfer] update imagestore interface to support multiple references Jan 13, 2023

dmcgowan force-pushed the transfer-image-store-references branch 2 times, most recently from 88cc5a6 to 923f008 Compare January 14, 2023 00:40

knight42 reviewed Jan 14, 2023

View reviewed changes

dmcgowan force-pushed the transfer-image-store-references branch from 923f008 to 1449ecd Compare January 18, 2023 19:12

dmcgowan force-pushed the transfer-image-store-references branch from 1449ecd to 923b556 Compare January 20, 2023 19:54

AkihiroSuda reviewed Jan 22, 2023

View reviewed changes

pkg/transfer/image/imagestore_test.go Show resolved Hide resolved

dmcgowan force-pushed the transfer-image-store-references branch from 923b556 to 2d39197 Compare January 23, 2023 22:08

mxpv reviewed Jan 25, 2023

View reviewed changes

pkg/transfer/image/imagestore.go Outdated Show resolved Hide resolved

dmcgowan force-pushed the transfer-image-store-references branch from 2d39197 to 2632d87 Compare January 25, 2023 19:22

dmcgowan added this to the 1.7 milestone Jan 25, 2023

estesp reviewed Feb 7, 2023

View reviewed changes

pkg/transfer/transfer.go Outdated Show resolved Hide resolved

dmcgowan force-pushed the transfer-image-store-references branch 2 times, most recently from 747bbbe to 60940b2 Compare February 7, 2023 21:55

mxpv approved these changes Feb 9, 2023

View reviewed changes

cpuguy83 approved these changes Feb 10, 2023

View reviewed changes

dmcgowan force-pushed the transfer-image-store-references branch from 60940b2 to 3e0bac9 Compare February 10, 2023 23:24

mikebrow reviewed Feb 13, 2023

View reviewed changes

Update imagestore interface to support multiple references

081601f

Signed-off-by: Derek McGowan <derek@mcg.dev>

dmcgowan force-pushed the transfer-image-store-references branch from 3e0bac9 to 081601f Compare February 13, 2023 21:59

AkihiroSuda approved these changes Feb 14, 2023

View reviewed changes

AkihiroSuda merged commit 4e2eb8b into containerd:main Feb 14, 2023

kiashok mentioned this pull request Jul 19, 2023

Invoke stable ABI compliant platform check in windows matcher #8778

Merged

dmcgowan deleted the transfer-image-store-references branch April 20, 2024 00:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[transfer] update imagestore interface to support multiple references #7964

[transfer] update imagestore interface to support multiple references #7964

Uh oh!

dmcgowan commented Jan 13, 2023

Uh oh!

knight42 Jan 14, 2023

Uh oh!

knight42 Jan 14, 2023

Uh oh!

dmcgowan Jan 16, 2023

Uh oh!

AkihiroSuda commented Jan 20, 2023

Uh oh!

Uh oh!

dmcgowan commented Jan 23, 2023

Uh oh!

dmcgowan commented Jan 25, 2023

Uh oh!

Uh oh!

Uh oh!

cpuguy83 Feb 10, 2023

Uh oh!

mikebrow Feb 13, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

[transfer] update imagestore interface to support multiple references #7964

[transfer] update imagestore interface to support multiple references #7964

Uh oh!

Conversation

dmcgowan commented Jan 13, 2023

Uh oh!

knight42 Jan 14, 2023

Choose a reason for hiding this comment

Uh oh!

knight42 Jan 14, 2023

Choose a reason for hiding this comment

Uh oh!

dmcgowan Jan 16, 2023

Choose a reason for hiding this comment

Uh oh!

AkihiroSuda commented Jan 20, 2023

Uh oh!

Uh oh!

dmcgowan commented Jan 23, 2023

Uh oh!

dmcgowan commented Jan 25, 2023

Uh oh!

Uh oh!

Uh oh!

cpuguy83 Feb 10, 2023

Choose a reason for hiding this comment

Uh oh!

mikebrow Feb 13, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants