Unpin alembic by harupy · Pull Request #5249 · mlflow/mlflow

harupy · 2022-01-11T05:49:41Z

What changes are proposed in this pull request?

Unpin alembic and fix migration scripts affected by this change.

Closes #5245, #4810, #4215

How is this patch tested?

Existing tests

Does this PR change the documentation?

No. You can skip the rest of this section.
Yes. Make sure the changed pages / sections render correctly by following the steps below.

Check the status of the ci/circleci: build_doc check. If it's successful, proceed to the
next step, otherwise fix it.
Click Details on the right to open the job page of CircleCI.
Click the Artifacts tab.
Click docs/build/html/index.html.
Find the changed pages / sections and make sure they render correctly.

Release Notes

Is this a user-facing change?

No. You can skip the rest of this section.
Yes. Give a description of this change to be included in the release notes for MLflow users.

(Details in 1-2 sentences. You can just refer to another PR with a description if this PR is part of a larger change.)

What component(s), interfaces, languages, and integrations does this PR affect?

Components

Interface

area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
area/windows: Windows support

Language

language/r: R APIs and clients
language/java: Java APIs and clients
language/new: Proposals for new client languages

Integrations

integrations/azure: Azure and Azure ML integrations
integrations/sagemaker: SageMaker integrations
integrations/databricks: Databricks integrations

How should the PR be classified in the release notes? Choose one:

rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
rn/feature - A new user-facing feature worth mentioning in the release notes
rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
rn/documentation - A user-facing documentation change worth mentioning in the release notes

WeichenXu123 · 2022-01-11T08:33:29Z

    # outside of the batch operation context.
    try:
-        op.drop_constraint(constraint_name="status", table_name="runs", type_="check")
+        # op.drop_constraint(constraint_name="status", table_name="runs", type_="check")


Why drop this ?

It's just temporarily commented out to see how the table definition looks like without this line.

How the runs table definition looks like for each database:

sqlite:

CREATE TABLE runs ( run_uuid VARCHAR(32) NOT NULL, name VARCHAR(250), source_type VARCHAR(20), source_name VARCHAR(500), entry_point_name VARCHAR(50), user_id VARCHAR(256), status VARCHAR(9), start_time BIGINT, end_time BIGINT, source_version VARCHAR(50), lifecycle_stage VARCHAR(20), artifact_uri VARCHAR(200), experiment_id INTEGER, CONSTRAINT run_pk PRIMARY KEY (run_uuid), FOREIGN KEY(experiment_id) REFERENCES experiments (experiment_id), CONSTRAINT runs_lifecycle_stage CHECK (lifecycle_stage IN ('active', 'deleted')), CONSTRAINT source_type CHECK (source_type IN ('NOTEBOOK', 'JOB', 'LOCAL', 'UNKNOWN', 'PROJECT')), -- 👇 Unnamed check constraint, expression looks correct -- the reason it's unnamed is probably because we don't specify `name` -- when constructing `Enum`: -- https://github.com/mlflow/mlflow/pull/5249/files#diff-3492d101d4bd194139919dcac84b713b0ee4526b79d32e45c44db3655f95e838R46 CHECK (status IN ('SCHEDULED', 'FAILED', 'FINISHED', 'RUNNING', 'KILLED')) )

postgres:

CREATE TABLE runs ( run_uuid VARCHAR(32) NOT NULL, name VARCHAR(250), source_type VARCHAR(20), source_name VARCHAR(500), entry_point_name VARCHAR(50), user_id VARCHAR(256), status VARCHAR(9), start_time BIGINT, end_time BIGINT, source_version VARCHAR(50), lifecycle_stage VARCHAR(20), artifact_uri VARCHAR(200), experiment_id INTEGER, CONSTRAINT run_pk PRIMARY KEY (run_uuid), CONSTRAINT runs_experiment_id_fkey FOREIGN KEY(experiment_id) REFERENCES experiments (experiment_id), CONSTRAINT source_type CHECK ((source_type)::text = ANY ((ARRAY['NOTEBOOK'::character varying, 'JOB'::character varying, 'LOCAL'::character varying, 'UNKNOWN'::character varying, 'PROJECT'::character varying])::text[])), CONSTRAINT runs_lifecycle_stage CHECK ((lifecycle_stage)::text = ANY ((ARRAY['active'::character varying, 'deleted'::character varying])::text[])), -- 👇 Named check constraint, expression looks correct CONSTRAINT runs_status_check CHECK ((status)::text = ANY ((ARRAY['SCHEDULED'::character varying, 'FAILED'::character varying, 'FINISHED'::character varying, 'RUNNING'::character varying, 'KILLED'::character varying])::text[])) )

mysql:

CREATE TABLE runs ( run_uuid VARCHAR(32) NOT NULL, name VARCHAR(250), source_type VARCHAR(20), source_name VARCHAR(500), entry_point_name VARCHAR(50), user_id VARCHAR(256), status VARCHAR(9), start_time BIGINT, end_time BIGINT, source_version VARCHAR(50), lifecycle_stage VARCHAR(20), artifact_uri VARCHAR(200), experiment_id INTEGER, PRIMARY KEY (run_uuid), CONSTRAINT runs_ibfk_1 FOREIGN KEY(experiment_id) REFERENCES experiments (experiment_id), -- 👇 Duplicate CONSTRAINT runs_chk_1 CHECK ((`status` in (_utf8mb4'SCHEDULED',_utf8mb4'FAILED',_utf8mb4'FINISHED',_utf8mb4'RUNNING',_utf8mb4'KILLED'))), CONSTRAINT runs_lifecycle_stage CHECK ((`lifecycle_stage` in (_utf8mb4'active',_utf8mb4'deleted'))), CONSTRAINT source_type CHECK ((`source_type` in (_utf8mb4'NOTEBOOK',_utf8mb4'JOB',_utf8mb4'LOCAL',_utf8mb4'UNKNOWN',_utf8mb4'PROJECT'))), -- 👇 Duplicate CONSTRAINT status CHECK ((`status` in (_utf8mb4'SCHEDULED',_utf8mb4'FAILED',_utf8mb4'FINISHED',_utf8mb4'RUNNING'))) )

Looks like we need this drop_constraint operation for mysql.

Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

dbczumar

@harupy Looks good, just need to address the check constraint issue for MySQL. Thanks for doing this!

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

harupy · 2022-01-12T07:18:27Z

Confirmed the new migration script works for both old and new versions of alembic:

harupy · 2022-01-12T07:28:34Z

@WeichenXu123 @dbczumar Could you take another look again and approve if everything looks ok?

harupy · 2022-01-12T08:52:52Z

-        docker-compose run mlflow-sqlite python run_checks.py --schema-output schemas/sqlite.sql
-        docker-compose run mlflow-postgres python run_checks.py --schema-output schemas/postgres.sql
-        docker-compose run mlflow-mysql python run_checks.py --schema-output schemas/mysql.sql
-        docker-compose run mlflow-mssql ./init-mssql-db.sh
-        docker-compose run mlflow-mssql python run_checks.py --schema-output schemas/mssql.sql
-        docker-compose down --rmi all --volumes --remove-orphans
+        docker-compose run mlflow-sqlite
+        docker-compose run mlflow-postgres
+        docker-compose run mlflow-mysql
+        docker-compose run mlflow-mssql
+        docker-compose down --volumes --remove-orphans --rmi all


Cleaned up this step by adding commands in docker-compose.yml.

harupy · 2022-01-12T09:07:53Z

-COPY dist ./dist
-
-RUN pip install dist/*.whl
 RUN pip install psycopg2 pymysql mysqlclient
+COPY dist ./dist
+RUN pip install dist/mlflow-*.whl


For rebuilding the image a bit faster.

harupy · 2022-01-12T09:13:35Z

      MYSQL_DATABASE: mlflowdb
      MYSQL_USER: mlflowuser
      MYSQL_PASSWORD: mlflowpassword
+    command: mysqld --default-authentication-plugin=mysql_native_password


In MySQL >= 8.0.4, this command is required to log in using a password.

harupy · 2022-01-12T12:14:11Z

+    # Ensure the following migration scripts work correctly:
+    # - cfd24bdc0731_update_run_status_constraint_with_killed.py
+    # - 0a8213491aaa_drop_duplicate_killed_constraint.py
+    client = mlflow.tracking.MlflowClient()
+    client.set_terminated(run_id=run.info.run_id, status="KILLED")


Added a check to ensure the updated migration scripts work properly.

harupy · 2022-01-12T12:15:18Z

-        cd tests/db
+        ./build_wheel.sh
        docker-compose pull
+        docker image ls | grep -E '(REPOSITORY|postgres|mysql|mssql)'


Show database image versions for debugging.

harupy · 2022-01-12T13:04:03Z

 """
 CORE_REQUIREMENTS = SKINNY_REQUIREMENTS + [
-    "alembic<=1.4.1",
+    "alembic",


dbczumar

LGTM!

harupy mentioned this pull request Jan 11, 2022

Print table schema in DB initialization test #5248

Merged

29 tasks

github-actions Bot added the rn/none List under Small Changes in Changelogs. label Jan 11, 2022

WeichenXu123 reviewed Jan 11, 2022

View reviewed changes

harupy added 7 commits January 11, 2022 18:53

unpin

e7e9c7d

Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

pin alembic

74550f2

Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

unpin

18d4d96

Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

update migration file

e751253

Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

handle error

a104bb0

Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

test

53ff3ca

Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

remove try catch

842381b

Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

harupy force-pushed the unpin-alembic branch from f1a4943 to 842381b Compare January 11, 2022 09:53

harupy mentioned this pull request Jan 11, 2022

Allow alembic>=1.7.5 #5245

Closed

29 tasks

harupy added 7 commits January 11, 2022 20:19

pin alembic

a9816b6

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

do not pin mysql version

cf5df44

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

upgrade mysql

9154b76

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

list images

305236c

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

docker image ls

bc40149

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

add check for killed status

79e0105

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

grep

2c22ed7

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

dbczumar reviewed Jan 12, 2022

View reviewed changes

harupy added 5 commits January 12, 2022 14:01

fix

9437139

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

specify commands

c95094c

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

try old alembic

de28923

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

nit

aa3c502

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

unpin

85b74e0

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

harupy requested review from WeichenXu123 and dbczumar January 12, 2022 07:19

harupy commented Jan 12, 2022

View reviewed changes

dbczumar approved these changes Jan 13, 2022

View reviewed changes

harupy merged commit d44567f into mlflow:master Jan 13, 2022

harupy deleted the unpin-alembic branch January 13, 2022 01:18

dbczumar mentioned this pull request Dec 6, 2022

[SETUP-BUG] Pinned requirement of alembic version getting old #4215

Closed

Conversation

harupy commented Jan 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes are proposed in this pull request?

How is this patch tested?

Does this PR change the documentation?

Release Notes

Is this a user-facing change?

What component(s), interfaces, languages, and integrations does this PR affect?

How should the PR be classified in the release notes? Choose one:

Uh oh!

WeichenXu123 Jan 11, 2022

Choose a reason for hiding this comment

Uh oh!

harupy Jan 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

harupy Jan 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

harupy Jan 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dbczumar left a comment

Choose a reason for hiding this comment

Uh oh!

harupy commented Jan 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

harupy commented Jan 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

harupy Jan 12, 2022

Choose a reason for hiding this comment

Uh oh!

harupy Jan 12, 2022

Choose a reason for hiding this comment

Uh oh!

harupy Jan 12, 2022

Choose a reason for hiding this comment

Uh oh!

harupy Jan 12, 2022

Choose a reason for hiding this comment

Uh oh!

harupy Jan 12, 2022

Choose a reason for hiding this comment

Uh oh!

harupy Jan 12, 2022

Choose a reason for hiding this comment

Uh oh!

dbczumar left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

harupy commented Jan 11, 2022 •

edited

Loading

harupy Jan 11, 2022 •

edited

Loading

harupy Jan 11, 2022 •

edited

Loading

harupy Jan 11, 2022 •

edited

Loading

harupy commented Jan 12, 2022 •

edited

Loading

harupy commented Jan 12, 2022 •

edited

Loading