👍

These projects are vendored for a reasons that I discuss publicly on a regular basis.

Also, my code decisions are not be influenced by the random decisions of Linux distros :)

Although, the work of said distros is much appreciated :)

@kennethreitz Any write-ups/blogs you can point to on the reasons?

Not that I know of — mostly in Q/A form.

The tl;dr is:

• I treat requests as a product. I want to be in 100% control of all aspects of the user experience.
• I believe these dependencies are implementation details. If there's something wrong with one of them, I can change it arbitrarily. I can test new patches before they are merged. I can remove them entirely.
• I want my users to be free to not use packaging if they don't won't to. They can just grab the tarball and go.
• Requests itself is therefore vendorable. This is particularly useful for system-level applications because Python does not allow for multiple versions of dependencies to be installed. If all the CLI tools you use written in Python used different versions of requests, and you had to use site-packages, that'd be a bad situation.

I could produce a patch that addressed the third of those points very effectively. The others only partially. The strategy would be that requests.packages would check for the existence of bundled versions of urllib3, charade, etc. If those were available, it would implement requests.packages.$foo via the bundled versions. If they weren't available, it would implement them via the system packages. Distros could then simply rm the bundled versions in their packages in order to use system packages.

The tarball downloaded from pypi (since it included the bundled versions of the libraries) would continue to use the bundled copies.

Would that be an acceptable patch? Or should I not waste my time working on something like that?

I hope this doesn't come across as abrasive (it's not meant to be!) — but, Requests has been through all of these cycles already. We supported exactly what you're proposing for a long time. It was a waste of a time, and a big headache for me. The less branching and indeterminacy the codebase has, the better.

Believe it or not, we also removed the bundled versions altogether for a good year! So many wasted lines in requirements.txt files around the world :)

This is the only solution that I'm happy with. As a matter of fact, I'm extremely happy with the current design :) 🍰

At the end of the day, Requests is valuable purely because of its opinions — and this is one of its strongest ones. It's the little things that matter the most — the most subjective things. This is the type of thing that I find we teach ourselves to trivialize — as it's the source of endless bikeshed.

Just simply make it a dependency and the world will go about it's way.

The architectural decisions of this project are a personal decision. I am simply choosing to make my own decision.

Open source is all about freedom — and if one of the founding forces of the open source movement, the linux distros, are encouraging developers to contort their works of art to acclimate to their packaging ideologies — I'm a little concerned for the state of things :)

No worries. The distros are going to unbundle the bundled code anyway -- I was just seeing if you'd take a patch to make that easier on them. Since you won't it saves me some coding time at the expense of the individual maintainers' times maintaining their patches to implement the unbundling. :-)

Thank you for requests.

Standard questions about bundling dependencies:

• How far behind upstream are the bundled versions?
• How frequently are checks for updated versions made?
  • (<3 pip-review from https://github.com/nvie/pip-tools)
• What is the process for reviewing upstream changes?
• What is the process for reporting/notifying re: updated versions of upstream packages?
• If there are urgent patches upstream, how long does it take for those to propagate downstream?
  • OS packaging
  • Vendor-bundles
  • End-user upgrades

...

• Why don't all packages bundle their dependencies for convenience [despite community standards]?

Everything is at my discretion, as it is all an implementation detail. None of any of this should matter to any user.

Also, it's worth noting as I did on the superfluous issue created before this PR that this has been discussed several times before. No amount of discussion will sway us.

@westurner
You compiled a nice set of questions there :) Statements from https://github.com/kennethreitz/requests/pull/1812#issuecomment-30871660 neither answer nor invalidate any of these questions to be clear. However that's in theory. In practice, most of the time, bundling just works and is better than all existing alternatives. Sharing dependencies, while sound in theory, is very difficult to do right and that's why people chose alternatives which in practice just work better. I have a strong feeling that sharing dependencies, which brings the need to manage them, once a must due to scarce computer resources (I suppose), nowadays creates more problems than it solves.
I also do not understand why packagers waste their time unbundling bundled packages. Do they also reverse monkey patching :) ? I mean if some piece of software chooses to bundle something then it does it for a reason. It's not like authors just don't know any better.
Returning to your questions; while there are no policies set, from which one could arrive at clear answers you expected, general answer is that contributors make whatever they are able and willing to do to make complaining users (more) happy. You know, because in life, people need working software more than official policies.

How many outdated bundled/embedded copies of OpenSSL could there be? Are
there standard mechanisms for ensuring that security updates reach end
users?

Here's this: https://news.ycombinator.com/item?id=7385150

Wes Turner
On Mar 19, 2014 2:38 PM, "Piotr Dobrogost" notifications@github.com wrote:

@westurner https://github.com/westurner
You compiled a nice set of questions there :) Statements from #1812
(comment)https://github.com/kennethreitz/requests/pull/1812#issuecomment-30871660neither answer nor invalidate any of these questions to be clear. However
that's in theory. In practice, most of the time, bundling just works and is
better than all existing alternatives. Sharing dependencies, while sound in
theory, is very difficult to do right and that's why people chose
alternatives which in practice just work better. I have a strong feeling
that sharing dependencies, which brings the need to manage them, once a
must due to scarce computer resources (I suppose), nowadays creates more
problems than it solves.
I also do not understand why packagers waste their time unbundling bundled
packages. Do they also reverse monkey patching :) ? I mean if some piece of
software chooses to bundle something then it does it for a reason. It's not
like authors just don't know any better.
Returning to your questions; while there are no policies set, from which
one could arrive at clear answers you expected, general answer is that
contributors make whatever they are able and willing to do to make
complaining users (more) happy. You know, because in life, people need
working software more than official policies.

—
Reply to this email directly or view it on GitHubhttps://github.com/kennethreitz/requests/pull/1812#issuecomment-38097168
.

From the Python programmer's perspective, embedded copies of OpenSSL are less dangerous than using the standard library's SSL module, which is a broken insecure mess is any version of Python less than 3.3.

• http://www.cvedetails.com/vulnerability-list/vendor_id-10210/product_id-18230/Python-Python.html
• http://www.cvedetails.com/vulnerability-list/vendor_id-217/Openssl.html

@westurner while I appreciate your concern, your efforts are for naught. Allow me to relay to you what I recently learned, security is not the foremost concern of requests (not @Lukasa's or my opinion). Its API is the first concern. Trying to support your point by making valid points about security will do nothing. Since little is likely to change, I'm unsubscribing from this issue and would encourage others who value their time to do the same.

I share abadgers' interest in making it easier to devendorize, even as i sympathize with upstream's strong desire to vendorize known good versions of packages. It's a tough bit of tension between upstream's requirements and a distro's very real requirements. I just devendorized pip, and with good upstream discipline, grep, and a decent editor it wasn't too hard. The pip developers are going to make it easier (as piotr-dobrogost's reference implies), but as long as we can identify and easily hack the import lines, I'm okay with it.

In practice, most of the time, bundling just works and is better than all existing alternatives.

Having said that I agree with @abadger and I see making is easy to use non-bundled dependencies as something maintainers of every software should do.

@kennethreitz totally agree with the decisions made here. If anyone has ever tried to ship Python software to be standalone you'll know why it's done like this.

@whalesalad exxxactly :)