I don't think this test is run normally due to pytest.importorskip('httpretty'). See https://travis-ci.org/dask/dask/jobs/462914574#L8003

FWIW, looking at the Travis build history and comparing the last build to pass with the first build to fail from the test_s3 error, it looks like the only boto/boto3/botocore/moto version discrepancy is:

botocore==1.12.57 on passing test (https://travis-ci.org/dask/dask/jobs/462914574#L1243-L1245)
botocore==1.12.58 on failing test (https://travis-ci.org/dask/dask/jobs/463454018#L1253-L1255)

@jrbourbeau : I've tried a few combinations now, don't know what the problem is.
Internet suggests that enabling sudo on travis, or installing google-compute-engine (??) would solve this, but I don't see why. Passes locally with some of the version combinations I tried.

Also maybe BOTO_CONFIG=null ?

That was unexpected... now moto is not mocking at all, hm.

Checking in here. It looks like this is causing CI to fail.

Is it sufficient to pin to an older version, or have things broken more fundamentally than that?

Looks like my attempt with /dev/null was no more successful. @martindurant do you have any thoughts on how to resolve this?

The NoCredentialsError error was reported in getmoto/moto#1924. Manually setting AWS_SECRET_ACCESS_KEY and AWS_ACCESS_KEY_ID, to anything, seem to fix the issue.

There's an existing PR in pandas (pandas-dev/pandas#23731) to temporarily set those environment variables during testing (e.g. set "AWS_ACCESS_KEY_ID"="foobar_key"). Do we want to do something similar here as a fix?

Seems fine to me. I'm generally willing to defer judgement to the pandas devs on most things :)

cc @TomAugspurger

Seems fine to me. I haven't kept up with the boto issue over on the pandas side.

I switched back to installing moto with pip (conda-forge have v1.1.1, while PyPI has version 1.3.7) and added an ensure_safe_environment_variables context manager to allow for temporarily setting AWS_* environment variables.

This got rid of the NoCredentialsError we were seeing before. However there's a FileNotFoundError for the dask-data/nyc-taxi s3 data (see https://travis-ci.org/dask/dask/jobs/466839004#L1729). I can look into this more tomorrow. Any thoughts/comments here are appreciated.

That file does exist:

In [1]: import s3fs

In [2]: s3fs.S3FileSystem(anon=True).open('dask-data/nyc-taxi/2015/yellow_tripdata_2015-01.csv')
Out[2]: <S3File dask-data/nyc-taxi/2015/yellow_tripdata_2015-01.csv>

I suspect that we're using moto in that test rather than connect to real S3. @martindurant any thoughts on how to address this?

I updated the failing test to mock the S3 request. Some potentially controversial things

1. mocking in the first place
2. including a largish fixture in the tests

That s3_with_yellow_tripdata fixture takes about 1.5s to setup on my machine. I wouldn't recommend using the fixture for multiple tests. We can't use a session scope, since we use the s3 fixture which has a function scope, and some tests seem to rely on the state being reset between tests.

I'm not really sure about the orc failures...

@jcrist is there way to write pyarrow tables to ORC files? From the docs it looks like just reading is supported.

Looking more closely at https://travis-ci.org/dask/dask/jobs/468460525#L1735, it seems like the HTTP request is being intercepted by responses, a mocking library for requests (used by moto). I'll see if I can find an un-stopped mock.

I was able to reproduce the ORC failure locally with pytest dask/bytes/tests/test_s3.py dask/dataframe/io/tests/test_orc.py.

873c434 fixes it for me locally by using moto's context managers. It seems we were failing to un-mock something previously.

FYI, I'll probably merge this on green, since it's failing master's CI. Happy to do followups if anyone sees things they'd like changed.

Thanks Tom and Martin!!