Skip to content

PostgreSQL IntegrityError causes group result failure if group is first task on workers #4653

New issue
New issue
Closed
Closed
PostgreSQL IntegrityError causes group result failure if group is first task on workers#4653
Labels
Component: SQLAlchemy Database Results BackendStatus: Needs Verification ✘Status: Not a Bug
@VictorLeP

Description

@VictorLeP
VictorLeP
opened on Apr 10, 2018

When the first task sent by a client on a fresh celery setup is a group task (or possibly any set of tasks processed concurrently by multiple workers), and PostgreSQL is used as the result backend, the workers fail with an IntegrityError due to a duplicate key.

Steps to reproduce

  1. Send a group task to multiple workers as the first task in a new setup
  2. Retrieve the group result

Expected behavior

The workers are able to store their result at the PostgreSQL backend, and the group result is correctly retrieved by the client.

Actual behavior

The _store_result operation fails on the workers with an IntegrityError. The operation is retried three times, but sometimes this is not sufficient and the group task fails with the IntegrityError.

Based on the traceback, my suspicion is that as multiple workers are concurrently processing a task and ultimately want to store their result at the backend, they are concurrently setting up sessions with the database:

celery/celery/backends/database/__init__.py

Lines 95 to 105 in 02c977d

def ResultSession(self, session_manager=SessionManager()):
return session_manager.session_factory(
dburi=self.url,
short_lived_sessions=self.short_lived_sessions,
**self.engine_options)
@retry
def _store_result(self, task_id, result, state,
traceback=None, max_retries=3, **kwargs):
"""Store return value and state of an executed task."""
session = self.ResultSession()

As it is the first task the workers process, this entails preparing the models and creating the necessary tables with create_all (using SQLAlchemy):

celery/celery/backends/database/session.py

Lines 53 to 60 in 02c977d

def prepare_models(self, engine):
if not self.prepared:
ResultModelBase.metadata.create_all(engine)
self.prepared = True
def session_factory(self, dburi, **kwargs):
engine, session = self.create_session(dburi, **kwargs)
self.prepare_models(engine)

It seems PostgreSQL doesn't handle multiple sessions creating the same table very well (1,2,3), so a duplicate key in an index for those tables cause an IntegrityError.

#4267 could be an issue with a similar cause.

Traceback

[2018-04-10 13:46:40,433: WARNING/ForkPoolWorker-1] Failed operation _store_result.  Retrying 2 more times.
Traceback (most recent call last):
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
DETAIL:  Key (typname, typnamespace)=(celery_taskmeta, 2200) already exists.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
    return fun(*args, **kwargs)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 105, in _store_result
    session = self.ResultSession()
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
    **self.engine_options)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
    self.prepare_models(engine)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/session.py", line 54, in prepare_models
    ResultModelBase.metadata.create_all(engine)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4004, in create_all
    tables=tables)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1940, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1549, in _run_visitor
    **kwargs).traverse_single(element)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
    return meth(obj, **kw)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 757, in visit_metadata
    _is_metadata_operation=True)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
    return meth(obj, **kw)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 791, in visit_table
    include_foreign_key_constraints=include_foreign_key_constraints
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1009, in _execute_ddl
    compiled
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
    exc_info
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
DETAIL:  Key (typname, typnamespace)=(celery_taskmeta, 2200) already exists.
 [SQL: '\nCREATE TABLE celery_taskmeta (\n\tid INTEGER NOT NULL, \n\ttask_id VARCHAR(155), \n\tstatus VARCHAR(50), \n\tresult BYTEA, \n\tdate_done TIMESTAMP WITHOUT TIME ZONE, \n\ttraceback TEXT, \n\tPRIMARY KEY (id), \n\tUNIQUE (task_id)\n)\n\n'] (Background on this error at: http://sqlalche.me/e/gkpj)
[2018-04-10 13:46:40,575: WARNING/ForkPoolWorker-1] Failed operation _store_result.  Retrying 1 more times.
Traceback (most recent call last):
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
DETAIL:  Key (typname, typnamespace)=(taskset_id_sequence, 2200) already exists.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
    return fun(*args, **kwargs)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 105, in _store_result
    session = self.ResultSession()
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
    **self.engine_options)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
    self.prepare_models(engine)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/session.py", line 54, in prepare_models
    ResultModelBase.metadata.create_all(engine)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4004, in create_all
    tables=tables)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1940, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1549, in _run_visitor
    **kwargs).traverse_single(element)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
    return meth(obj, **kw)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 757, in visit_metadata
    _is_metadata_operation=True)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
    return meth(obj, **kw)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 782, in visit_table
    self.traverse_single(column.default)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
    return meth(obj, **kw)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 820, in visit_sequence
    self.connection.execute(CreateSequence(sequence))
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1009, in _execute_ddl
    compiled
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
    exc_info
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
DETAIL:  Key (typname, typnamespace)=(taskset_id_sequence, 2200) already exists.
 [SQL: 'CREATE SEQUENCE taskset_id_sequence'] (Background on this error at: http://sqlalche.me/e/gkpj)
[2018-04-10 13:46:40,670: WARNING/ForkPoolWorker-1] Failed operation _store_result.  Retrying 0 more times.
Traceback (most recent call last):
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
DETAIL:  Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
    return fun(*args, **kwargs)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 105, in _store_result
    session = self.ResultSession()
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
    **self.engine_options)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
    self.prepare_models(engine)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/session.py", line 54, in prepare_models
    ResultModelBase.metadata.create_all(engine)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4004, in create_all
    tables=tables)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1940, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1549, in _run_visitor
    **kwargs).traverse_single(element)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
    return meth(obj, **kw)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 757, in visit_metadata
    _is_metadata_operation=True)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
    return meth(obj, **kw)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 791, in visit_table
    include_foreign_key_constraints=include_foreign_key_constraints
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1009, in _execute_ddl
    compiled
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
    exc_info
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
DETAIL:  Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
 [SQL: '\nCREATE TABLE celery_tasksetmeta (\n\tid INTEGER NOT NULL, \n\ttaskset_id VARCHAR(155), \n\tresult BYTEA, \n\tdate_done TIMESTAMP WITHOUT TIME ZONE, \n\tPRIMARY KEY (id), \n\tUNIQUE (taskset_id)\n)\n\n'] (Background on this error at: http://sqlalche.me/e/gkpj)
[2018-04-10 13:46:40,675: WARNING/ForkPoolWorker-1] /home/ubuntu/venv/lib/python3.6/site-packages/celery/app/trace.py:549: RuntimeWarning: Exception raised outside body: IntegrityError('(psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"\nDETAIL:  Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.\n',):
Traceback (most recent call last):
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
DETAIL:  Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/app/trace.py", line 434, in trace_task
    uuid, retval, task_request, publish_result,
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/base.py", line 152, in mark_as_done
    self.store_result(task_id, result, state, request=request)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/base.py", line 309, in store_result
    request=request, **kwargs)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
    return fun(*args, **kwargs)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 105, in _store_result
    session = self.ResultSession()
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
    **self.engine_options)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
    self.prepare_models(engine)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/session.py", line 54, in prepare_models
    ResultModelBase.metadata.create_all(engine)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4004, in create_all
    tables=tables)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1940, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1549, in _run_visitor
    **kwargs).traverse_single(element)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
    return meth(obj, **kw)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 757, in visit_metadata
    _is_metadata_operation=True)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
    return meth(obj, **kw)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 791, in visit_table
    include_foreign_key_constraints=include_foreign_key_constraints
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1009, in _execute_ddl
    compiled
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
    exc_info
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
DETAIL:  Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
 [SQL: '\nCREATE TABLE celery_tasksetmeta (\n\tid INTEGER NOT NULL, \n\ttaskset_id VARCHAR(155), \n\tresult BYTEA, \n\tdate_done TIMESTAMP WITHOUT TIME ZONE, \n\tPRIMARY KEY (id), \n\tUNIQUE (taskset_id)\n)\n\n'] (Background on this error at: http://sqlalche.me/e/gkpj)

  exc, exc_info.traceback)))
[2018-04-10 13:46:40,712: ERROR/ForkPoolWorker-1] Task setup[cfe2557a-f9a1-4406-86d0-1499166cc1a6] raised unexpected: IntegrityError('(psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"\nDETAIL:  Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.\n',)
Traceback (most recent call last):
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
DETAIL:  Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/app/trace.py", line 434, in trace_task
    uuid, retval, task_request, publish_result,
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/base.py", line 152, in mark_as_done
    self.store_result(task_id, result, state, request=request)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/base.py", line 309, in store_result
    request=request, **kwargs)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
    return fun(*args, **kwargs)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 105, in _store_result
    session = self.ResultSession()
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
    **self.engine_options)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
    self.prepare_models(engine)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/celery/backends/database/session.py", line 54, in prepare_models
    ResultModelBase.metadata.create_all(engine)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4004, in create_all
    tables=tables)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1940, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1549, in _run_visitor
    **kwargs).traverse_single(element)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
    return meth(obj, **kw)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 757, in visit_metadata
    _is_metadata_operation=True)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
    return meth(obj, **kw)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 791, in visit_table
    include_foreign_key_constraints=include_foreign_key_constraints
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1009, in _execute_ddl
    compiled
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
    exc_info
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
DETAIL:  Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
 [SQL: '\nCREATE TABLE celery_tasksetmeta (\n\tid INTEGER NOT NULL, \n\ttaskset_id VARCHAR(155), \n\tresult BYTEA, \n\tdate_done TIMESTAMP WITHOUT TIME ZONE, \n\tPRIMARY KEY (id), \n\tUNIQUE (taskset_id)\n)\n\n'] (Background on this error at: http://sqlalche.me/e/gkpj)

Report

software -> celery:4.2.0rc2 (windowlicker) kombu:4.1.0 py:3.6.3
            billiard:3.5.0.3 py-amqp:2.2.2
platform -> system:Linux arch:64bit, ELF imp:CPython
loader   -> celery.loaders.app.AppLoader
settings -> transport:amqp results:db+postgresql://****

SQLAlchemy version 1.2.6

Metadata

Metadata

Assignees

No one assigned

    Labels

    Component: SQLAlchemy Database Results BackendStatus: Needs Verification ✘Status: Not a Bug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions