This is a minimal integration test to make sure an aritafct can be logged using the mlflow artifacts service.

Why are these steps required? If there's a way to avoid them, that would be nice in order to reduce complexity for users.

This is mainly for development purposes. We can simplify the workflow by preparing two Docker files (Dockerfile and Dockerfile.dev) and switching them with variable substitution.

docker-compose.yml would look like:

# docker-compose.yml
  tracking-server:
    build:
      context: .
      # Defaults to `Dockerfile` if `DOCKERFILE` environment variable doesn't exist.
      dockerfile: "${DOCKERFILE:-Dockerfile}"
    depends_on:
      - postgres
      - artifacts-server

README.md would look like:

This directory contains a set of files for demonstrating the MLflow Artifacts service.

## Run the example

```sh
# Build services
docker-compose build

# Launch tracking and artifacts servers
docker-compose up -d

# Run `run.py` that uploads, downloads, and list artifacts
docker-compose run -v ${PWD}/run.py:/app/run.py client python run.py
```

## Explore the logging results:

```sh
# Make sure both tracking and artifacts servers are running
docker-compose ps
```

- MLflow UI is available at http://localhost:5000 to explore tracking results.
- MinIO Console is available at http://localhost:9001 to explore logged artifacts. The login username and password are `user` and `password`.

## Reset tracking and artifacts servers

```
docker-compose down --volumes --remove-orphans
```

👇👇👇

## Development

```
pip wheel --no-deps --wheel-dir dist ../..
DOCKERFILE=Dockerfile.dev docker-compose build
docker-compose run -v ${PWD}/run.py:/app/run.py client python run.py
```

What do you think about this approach?

Dockerfile and Dockerfile.dev would look like this.

Dockerfile:

FROM python:3.6

WORKDIR /app

RUN pip install psycopg2 boto3
RUN pip install mlflow

Dockerfile.dev:

FROM python:3.6

WORKDIR /app

RUN pip install psycopg2 boto3
RUN pip install mlflow
COPY dist ./dist
RUN pip install dist/mlflow-*.whl

Love it! Can we also clarify that this only applies when the tracking server is configured to stream artifacts and the experiment artifact root is http or mlflow-artifacts?

In general, we should figure out how to tell users about the http and mlflow-artifacts URIs for --default-artifact-root as well.

Love it! Can we also clarify that this only applies when the tracking server is configured to stream artifacts and the experiment artifact root is http or mlflow-artifacts?

Sure!

In general, we should figure out how to tell users about the http and mlflow-artifacts URIs for --default-artifact-root as well.

Can we new sections for HTTP and mlflow-artifacts schemes in https://www.mlflow.org/docs/latest/tracking.html#artifact-stores?

In general, we should figure out how to tell users about the http and mlflow-artifacts URIs for --default-artifact-root as well.

That's a good idea. We should also make it easy for users to learn about these through the mlflow server CLI in a follow-up PR.

Is there any timeout on the stream read operation? Else, we may get stuck here forever.

• request.stream is an instance of gunicorn.http.body.Body.
• gunicorn.http.body.Body.read doesn't seem to have an option related to timeout:

Source: https://github.com/benoitc/gunicorn/blob/ff58e0c6da83d5520916bc4cc109a529258d76e1/gunicorn/http/body.py#L202

    def read(self, size=None):
        size = self.getsize(size)
        ...

Got it. This probably isn't too big a deal because we set gunicorn worker timeouts. Worst case, we may hang here until the worker times out, which is fine :).

Is there a timeout we should specify here? Else, we may get stuck here

Yes, what about 10 minutes?. I'm checking the behavior of timeout when stream=True.

Found a related discussion:

psf/requests#1803 (comment)

Can we set timeout=10 (seconds)?

Can we add a docstring to this proto file explaining which use cases it serves?

Sure!

Can we explain what the artifacts service does and provide a small example command that users could run manually to serve artifacts via mlflow server in the README?

Updated the README!

Got it. This probably isn't too big a deal because we set gunicorn worker timeouts. Worst case, we may hang here until the worker times out, which is fine :).