This seems to cause the exact opposite error on other servers.

Can you give an example?

On Feb 5, 2014, at 2:28 PM, Tom Prince notifications@github.com wrote:

This seems to cause the exact opposite error on other servers.

—
Reply to this email directly or view it on GitHub.

pip install -U --force-reinstall --no-index --find-links http://data.hybridcluster.net/pip-1435 ujson==1.33

Exception:
Traceback (most recent call last):
  File "/home/tomprince/dev/b00ef9c48c7082c6/lib/python2.7/site-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/home/tomprince/dev/b00ef9c48c7082c6/lib/python2.7/site-packages/pip/commands/install.py", line 274, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/home/tomprince/dev/b00ef9c48c7082c6/lib/python2.7/site-packages/pip/req.py", line 1173, in prepare_files
    self.unpack_url(url, location, self.is_download)
  File "/home/tomprince/dev/b00ef9c48c7082c6/lib/python2.7/site-packages/pip/req.py", line 1320, in unpack_url
    retval = unpack_http_url(link, location, self.download_cache, self.download_dir, self.session)
  File "/home/tomprince/dev/b00ef9c48c7082c6/lib/python2.7/site-packages/pip/download.py", line 587, in unpack_http_url
    unpack_file(temp_location, location, content_type, link)
  File "/home/tomprince/dev/b00ef9c48c7082c6/lib/python2.7/site-packages/pip/util.py", line 621, in unpack_file
    unzip_file(filename, location, flatten=not filename.endswith(('.pybundle', '.whl')))
  File "/home/tomprince/dev/b00ef9c48c7082c6/lib/python2.7/site-packages/pip/util.py", line 491, in unzip_file
    zip = zipfile.ZipFile(zipfp)
  File "/usr/lib/python2.7/zipfile.py", line 766, in __init__
    self._RealGetContents()
  File "/usr/lib/python2.7/zipfile.py", line 807, in _RealGetContents
    raise BadZipfile, "File is not a zip file"
BadZipfile: File is not a zip file

Also:

$ pip install -d . --no-index --find-links http://data.hybridcluster.net/pip-1435 ujson==1.33
$ file usjon.zip
ujson-1.33.zip: gzip compressed data, from Unix
$ zcat ujson-1.33.zip | file -
/dev/stdin: Zip archive data, at least v2.0 to extract
$ curl http://data.hybridcluster.net/pip-1435/ujson-1.33.zip | file -
/dev/stdin: Zip archive data, at least v2.0 to extract

So, the reason this happens is your server is gzip encoding a zip file and we no longer decode that by default because some servers if you tell them to serve a .tar.gz file they serve an application/tar file with Content-Encoding: gzip. This probably previously worked because prior to this we didn't send the header to accept gzip encoding.

Strictly speaking we should be decoding Content-Encoding I suppose. We'll have to figure out how to solve this better.

I also put a setuptools tar.gz there, and it gets double compressed and so pip chokes on it as well.

Probably the right answer is to revert this, and make pip able to tell if we have a plain tarfile or a tar.gz even though the filename will be .tar.gz. Historically we've just used the filename for this.

Grr, that's not completely right either, because it was breaking checksums too. Probably we're going to have to pick which one to support.

This is a (I think) somewhat vanilla apache server.

I would err on the side of not supporting silly server written by people that don't understand what HTTP headers are supposed to mean.

No your server is doing it correctly. Well gziping a already compressed file is pointless, but it's not wrong. We switched to doing it incorrectly to fix things for one group, and that broke things for another group.

Although I may have a different/better answer. If we can get requests to stop sending the accept header for gzip compression then I think the .tar.gz case this originally was trying to fix will still work and your server should work fine because it won't be compressing on the fly. I need to see if we can do that with requests and if so if it actually does that.

Yeah. I just turned gzip compression off in the directory we are actually using, for .zip files.

Yeah, also getting double compressed tar.gz with --find-links