Skip to content

[SVCS-268][SVCS-376] Add readonly provider for Bitbucket#198

Merged
felliott merged 8 commits intoCenterForOpenScience:developfrom
felliott:feature/bitbucket
Jul 3, 2017
Merged

[SVCS-268][SVCS-376] Add readonly provider for Bitbucket#198
felliott merged 8 commits intoCenterForOpenScience:developfrom
felliott:feature/bitbucket

Conversation

@felliott
Copy link
Member

@felliott felliott commented Mar 10, 2017

Purpose

Add support for a read-only bitbucket provider to WaterButler.

Feature complete, has passed QA Demo. Still needs code cleanup, tests, documentation.

Changes

to-come, expect a force-push

Side effects

None expected.

Ticket

[#SVCS-268] - JIRA
[#SVCS-376] - JIRA

Entangled with OSF PR: CenterForOpenScience/osf.io#6954

felliott added 2 commits June 23, 2017 09:34
 * Some providers are read-only.  If a writing method like upload or
   delete is called on such a provider, it should throw a
   ReadOnlyProviderError, which will return a 501 Not Implemented.
 * No count limit was being passed in the calls to replace() in
   core.util.ZipStreamGenerator.  The replace was intended to strip
   the base folder from its childrens' paths.  Instead it could mutate
   the intended file names if the base path string appeared multiple
   times.  Eg. Try to zip a folder called `foo` that contains a
   subfolder called `subfoo/` and a file `subfoo/bar.txt`.  You will
   end up with a top-level file called `subbar.txt`, because
   `'foo/subfoo/bar.txt'.replace('foo/', '')` will strip both
   instances of `foo/`.
@felliott felliott force-pushed the feature/bitbucket branch from 422176f to 6dc0042 Compare June 23, 2017 13:37
@coveralls
Copy link

Coverage Status

Coverage increased (+0.9%) to 76.678% when pulling 6dc0042 on felliott:feature/bitbucket into 425a329 on CenterForOpenScience:develop.

felliott added 6 commits June 23, 2017 10:43
 * Passing 'foo/bar/' to bitbucket's url-building methods results in
   the slash being percent-encoded ('foo%2Fbar'). path_tuple() will
   return the path as a tuple, without the leading empty root
   string: ('foo', 'bar', )  The url builders will properly encode
   this as 'foo/bar'.
 * For recursive operations like DAZ, self._parent_dir is not
   guaranteed to be the path's parent (though it should be an
   ancestor).  Test whether that assumption holds true, and if not,
   fetch the parent dir listing.
 * Bitbucket allows all characters except / in filenames. Bitbucket
   API urls have the following requirements for character encoding:

   * must encode: \#%?
   * optional encode: [{]}|"<>~`!@ $^&*()+=,;:'
   * encoding not used: ._-

   Support for the percent sign in file names is spotty right now, b/c
   the version of furl we use to build urls is buggy wrt %:

   ````
   In [12]: furl.furl('http://foo.com/m%o/').url
   Out[12]: 'http://foo.com/m%o/'

   In [13]: furl.furl('http://foo.com/m%25o/').url
   Out[13]: 'http://foo.com/m%o/'

   In [14]: furl.furl('http://foo.com/m%2525o/').url
   Out[14]: 'http://foo.com/m%o/'

   In [32]: furl.furl('http://foo.com/a%23%3F%5Ce/?bar=baz').url
   Out[32]: 'http://foo.com/a%23%3F%5Ce/?bar=baz'

   In [33]: furl.furl('http://foo.com/a%23%3F%5Ce%25/?bar=baz').url
   Out[33]: 'http://foo.com/a#?\\e%/?bar=baz'
   ````
@felliott felliott force-pushed the feature/bitbucket branch from 6dc0042 to 5d5e28d Compare June 23, 2017 14:44
@felliott felliott changed the title [WIP] [SVCS-268] Add readonly provider for Bitbucket [SVCS-268][SVCS-376] Add readonly provider for Bitbucket Jun 23, 2017
@coveralls
Copy link

Coverage Status

Coverage increased (+1.0%) to 76.682% when pulling 5d5e28d on felliott:feature/bitbucket into 425a329 on CenterForOpenScience:develop.

@felliott felliott merged commit 5d5e28d into CenterForOpenScience:develop Jul 3, 2017
felliott added a commit that referenced this pull request Jul 3, 2017
 * Add readonly support for Bitbucket to WaterButler.  At the time
   this was being developed, Bitbucket did not have a write API, but
   that has since changed[1].  Write support will be added in a future
   ticket. This addon will not be useable until the corresponding OSF
   PR (CenterForOpenScience/osf.io#7409) is merged.

   Tickets: [SVCS-238][SVCS-266][SVCS-268][SVCS-376]
   Closes: #198

   [1] https://bitbucket.org/site/master/issues/8672/is-there-any-api-to-make-a-commit-bb-9954
@felliott felliott deleted the feature/bitbucket branch July 9, 2017 22:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants