[SVCS-268][SVCS-376] Add readonly provider for Bitbucket#198
Merged
felliott merged 8 commits intoCenterForOpenScience:developfrom Jul 3, 2017
Merged
[SVCS-268][SVCS-376] Add readonly provider for Bitbucket#198felliott merged 8 commits intoCenterForOpenScience:developfrom
felliott merged 8 commits intoCenterForOpenScience:developfrom
Conversation
* Some providers are read-only. If a writing method like upload or delete is called on such a provider, it should throw a ReadOnlyProviderError, which will return a 501 Not Implemented.
* No count limit was being passed in the calls to replace() in
core.util.ZipStreamGenerator. The replace was intended to strip
the base folder from its childrens' paths. Instead it could mutate
the intended file names if the base path string appeared multiple
times. Eg. Try to zip a folder called `foo` that contains a
subfolder called `subfoo/` and a file `subfoo/bar.txt`. You will
end up with a top-level file called `subbar.txt`, because
`'foo/subfoo/bar.txt'.replace('foo/', '')` will strip both
instances of `foo/`.
422176f to
6dc0042
Compare
* Passing 'foo/bar/' to bitbucket's url-building methods results in
the slash being percent-encoded ('foo%2Fbar'). path_tuple() will
return the path as a tuple, without the leading empty root
string: ('foo', 'bar', ) The url builders will properly encode
this as 'foo/bar'.
* For recursive operations like DAZ, self._parent_dir is not guaranteed to be the path's parent (though it should be an ancestor). Test whether that assumption holds true, and if not, fetch the parent dir listing.
* Bitbucket allows all characters except / in filenames. Bitbucket
API urls have the following requirements for character encoding:
* must encode: \#%?
* optional encode: [{]}|"<>~`!@ $^&*()+=,;:'
* encoding not used: ._-
Support for the percent sign in file names is spotty right now, b/c
the version of furl we use to build urls is buggy wrt %:
````
In [12]: furl.furl('http://foo.com/m%o/').url
Out[12]: 'http://foo.com/m%o/'
In [13]: furl.furl('http://foo.com/m%25o/').url
Out[13]: 'http://foo.com/m%o/'
In [14]: furl.furl('http://foo.com/m%2525o/').url
Out[14]: 'http://foo.com/m%o/'
In [32]: furl.furl('http://foo.com/a%23%3F%5Ce/?bar=baz').url
Out[32]: 'http://foo.com/a%23%3F%5Ce/?bar=baz'
In [33]: furl.furl('http://foo.com/a%23%3F%5Ce%25/?bar=baz').url
Out[33]: 'http://foo.com/a#?\\e%/?bar=baz'
````
6dc0042 to
5d5e28d
Compare
felliott
added a commit
that referenced
this pull request
Jul 3, 2017
* Add readonly support for Bitbucket to WaterButler. At the time this was being developed, Bitbucket did not have a write API, but that has since changed[1]. Write support will be added in a future ticket. This addon will not be useable until the corresponding OSF PR (CenterForOpenScience/osf.io#7409) is merged. Tickets: [SVCS-238][SVCS-266][SVCS-268][SVCS-376] Closes: #198 [1] https://bitbucket.org/site/master/issues/8672/is-there-any-api-to-make-a-commit-bb-9954
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Add support for a read-only bitbucket provider to WaterButler.
Feature complete, has passed QA Demo. Still needs code cleanup, tests, documentation.
Changes
to-come, expect a force-push
Side effects
None expected.
Ticket
[#SVCS-268] - JIRA
[#SVCS-376] - JIRA
Entangled with OSF PR: CenterForOpenScience/osf.io#6954