Automatic-er database initialization.#4887
Merged
martenson merged 1 commit intogalaxyproject:devfrom Nov 1, 2017
Merged
Conversation
Member
Author
|
Hopefully something like this allows us to fix failing integration tests on the job handler and collection state PRs that work when you run them alone. |
nsoranzo
reviewed
Oct 27, 2017
lib/galaxy/model/mapping.py
Outdated
| from sqlalchemy.orm.collections import attribute_mapped_collection | ||
| from sqlalchemy.types import BigInteger | ||
|
|
||
| from sqlalchemy_utils import database_exists, create_database |
- Use sqlalchemy-utils to create postgres / mysql database if they don't exist the way it happens automatically for sqlite. - Add a new configuration parameter called database_template that allows creating postgres databases from an existing database using a TEMPLATE statement. - Use the database template parameter to give each integration test case class a clean postgres database.
38dc0fb to
9b90999
Compare
Member
Author
|
@nsoranzo Thanks I fixed that and another linting problem. |
jmchilton
added a commit
to jmchilton/galaxy
that referenced
this pull request
Oct 30, 2017
- Bake Selenium (chrome, chrome-wrapper, xvfb, and Selenium server) into Docker test image. (1) - Correct usage of common_startup.sh to fetch dev wheels (don't use pip directly). - Fix/enhance the test wrapper file to not set GALAXY_CONFIG_OVERRIDE_DATABASE_CONNECATION - this prevents integration database templating (galaxyproject#4887) from working with the Docker image. - Update target Dockerfile. 1) The Docker compose setup for Selenium tests currently used by Jenkins is nice in that it uses official Selenium and Postgres Docker images - but it is problematic in that it takes more resources than the other single container Docker jobs, it doesn't cleanup as nicely, it doesn't have a pre-migrated database or pre-installed dependencies so it takes longer to run, and isn't easily exposed to developers via run_tests.sh. A more pragmatic Jenkins job based on this work should be runnable for every PR - the current variant simply takes too long and requires too many resources.
This was referenced Nov 1, 2017
jmchilton
added a commit
to jmchilton/planemo
that referenced
this pull request
Feb 23, 2018
Add a new profile database type option "postgres_docker". The existing "postgres" type option expects a local database to be accessible via the "psql" command on Planemo's PATH - probably few people have postgres running directly on their development machine and fewer still have it configured to allow passwordless entry. This newer variant is a much simpler interface - it starts a named container for the official Docker postgres image and uses that to manage the profile. This should make using profiles easier for most developers. This new modality does have some shortcomings - it should probably be redone to persist the database somewhere -or- start and stop the same container (currently when the container is stopped the data is gone). This also updates the workflow test that encounters Galaxy locks to pickup the environment variable PLANEMO_TEST_WORKFLOW_RUN_PROFILE and PLANEMO_TEST_WORKFLOW_RUN_PROFILE_DATABASE_TYPE and use these to create and use a profile during testing if set as well as setting that variable in Travis to leverage the postgres service already configured for other tests. This more less fixes what I had in mind with galaxyproject#772 but it would be nice to leverage galaxyproject/galaxy#4887 for greater test isolation.
jmchilton
added a commit
to jmchilton/planemo
that referenced
this pull request
Feb 23, 2018
Add a new profile database type option "postgres_docker". The existing "postgres" type option expects a local database to be accessible via the "psql" command on Planemo's PATH - probably few people have postgres running directly on their development machine and fewer still have it configured to allow passwordless entry. This newer variant is a much simpler interface - it starts a named container for the official Docker postgres image and uses that to manage the profile. This should make using profiles easier for most developers. This new modality does have some shortcomings - it should probably be redone to persist the database somewhere -or- start and stop the same container (currently when the container is stopped the data is gone). This also updates the workflow test that encounters Galaxy locks to pickup the environment variable PLANEMO_TEST_WORKFLOW_RUN_PROFILE and PLANEMO_TEST_WORKFLOW_RUN_PROFILE_DATABASE_TYPE and use these to create and use a profile during testing if set as well as setting that variable in Travis to leverage the postgres service already configured for other tests. This more less fixes what I had in mind with galaxyproject#772 but it would be nice to leverage galaxyproject/galaxy#4887 for greater test isolation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
database_templatethat allows creating postgres databases from an existing database using a TEMPLATE statement.