-
Notifications
You must be signed in to change notification settings - Fork 1.5k
storage: support startOffset query during object listing #2556
Description
Is your feature request related to a problem? Please describe.
If we have a large number of files stored in the bucket and we only want to list objects starting from a certain prefix, listing objects starting from a specific starting point (under a lexicographical order) would be desirable. This feature is supported in the JSON API via startOffset. However, I could not find a way to set this startOffset value under the go sdk.
Describe the solution you'd like
Support for startOffset when listing objects should be simple. This can be done by:
- Add
StartOffsetfield tostorage.Querystruct - Request changes to internal storage client api to add
func (c *BucketsListCall) StartOffset(startOffset string)to modify the url query parameter for startOffset - Modify
func (it *ObjectIterator) fetch(pageSize int, pageToken string)to incorporate startOffset parameter if it is being set in the Query, ie.req.StartOffset(it.query.StartOffset)
To use this feature, I should be able to set the StartOffset field in storage.Query to some string value, and expect the internal listObjects call to respect the startOffset and list objects starting from startOffset.