just realized that ci is currently not installing dev dependencies, so the django-test-migrations module isn’t found. Locally, with dev requirements installed, the tests pass fine. We might need to update the CI to include dev requirements when running tests or some other alternative..

I think I fixed the CI issue, but your tests are failing for some reason. Any idea why?

Suggestion from LLM:

This is a PostgreSQL-specific issue with django-test-migrations. The first test (TestRemoveHardcodedHoneypots) creates an IOC and adds it to a GeneralHoneypot via the M2M relation. That M2M operation creates deferred constraint triggers. When the second test's setUp calls migrator.apply_initial_migration, it tries to DROP/TRUNCATE the greedybear_ioc table, and PostgreSQL refuses because those triggers are still pending. The fix is to force PostgreSQL to resolve deferred constraints before migrator.reset() runs. In the MigrationTestCase.tearDown, execute SET CONSTRAINTS ALL IMMEDIATE before the reset:

def tearDown(self):
      from django.db import connection
      with connection.cursor() as cursor:
          cursor.execute("SET CONSTRAINTS ALL IMMEDIATE")
      self.migrator.reset()
      super().tearDown()

Suggestion from LLM:

This is a PostgreSQL-specific issue with django-test-migrations. The first test (TestRemoveHardcodedHoneypots) creates an IOC and adds it to a GeneralHoneypot via the M2M relation. That M2M operation creates deferred constraint triggers. When the second test's setUp calls migrator.apply_initial_migration, it tries to DROP/TRUNCATE the greedybear_ioc table, and PostgreSQL refuses because those triggers are still pending. The fix is to force PostgreSQL to resolve deferred constraints before migrator.reset() runs. In the MigrationTestCase.tearDown, execute SET CONSTRAINTS ALL IMMEDIATE before the reset:

def tearDown(self):
      from django.db import connection
      with connection.cursor() as cursor:
          cursor.execute("SET CONSTRAINTS ALL IMMEDIATE")
      self.migrator.reset()
      super().tearDown()

Thanks for the suggestion! I will apply the following snippet in tearDown and push the changes. Hopefully, this resolves the CI issue.

Locally, with dev requirements installed, the tests pass fine.

Now the CI also installs the dev requirements but your tests still don't pass. I tried them locally on my machine and they also fail. Do you know why? Do they still pass on your machine?

Locally, with dev requirements installed, the tests pass fine.

Now the CI also installs the dev requirements but your tests still don't pass. I tried them locally on my machine and they also fail. Do you know why? Do they still pass on your machine?

I'm actively working on this and trying to solve it. I discovered that the issue seems to be related to test isolation in the DB level. When I run only the migration test file in isolation (e.g., python manage.py test tests.test_migrations), it passes successfully. I initially thought it would work in the full test suite as well, so I mentioned it passed, but in reality, it fails when running the entire suite due to some DB state pollution or pending constraint/trigger events from other tests. I'm investigating further to identify the exact cause, and if I find a fix or more details, I'll update you right away.

while doing some investigation on this bug what I found is postgresql has a feature called "deferred constraints" where foreign key checks can be postponed until the end of a transaction. Django creates many-to-many (M2M) relationship tables with these deferred constraints by default.

proof of deferred constraint in db:

At First, i thought that error was a migration base class itself but, later found something interesting.

During normal tests :

• Tests create IOC objects and use M2M relationships: ioc.general_honeypot.add(honeypot).
• PostgreSQL says "I'll check this foreign key constraint later" and queues it as a "pending trigger event".
• The test completes, but PostgreSQL still has these pending checks in its queue.

When migration test came:

• The django-test-migrations library needs to DROP or TRUNCATE tables to test migrations from scratch

• Postgresql says: "I can't drop/truncate greedybear_ioc table because I still have pending trigger events (deferred constraint checks) waiting"

• ERROR: cannot DROP TABLE "greedybear_ioc" because it has pending trigger events

• django-test-migrations assumes it can flush all tables at any point in the test, but Postgres blocks flushes when deferred triggers exist.

What proves our code is not problem:

• When we run only migration tests, no other tests have created pending trigger events
  The table can be dropped clean and test case passes.

This is my first time dealing with this kind of PostgreSQL behavior, so I may be wrong, but my understanding is that things should ideally work with the normal Django test setup without needing manual SQL or cursor-based fixes.

In this case, making the migration test base class more complex with DB-specific queries feels heavy and harder to maintain.

As of now, running the migration tests in isolation seems like the cleanest solution .
I’d really like to hear your thoughts on this and whether you think i am missing something important here.

Hey @drona-gyawali ! Thanks for your investigation.

I’d really like to hear your thoughts on this and whether you think i am missing something important here.

You're not missing anything that is obvious to me at least. What are the options that we have now, in you opinion?

Hey @drona-gyawali ! Thanks for your investigation.

I’d really like to hear your thoughts on this and whether you think i am missing something important here.

You're not missing anything that is obvious to me at least. What are the options that we have now, in you opinion?

Thank you very much for taking the time to review this PR.

I think that the case we have in our project isn’t solvable purely at the programmatic level. Interfering with the schema or forcing PostgreSQL to ignore deferred constraints can lead to bigger problems and break other tests.

My simple and maintainable solution is to isolate migration tests using tags:

• We can mark migration tests with @tag("migration").

• We can configure a custom test_runner so that:

  • When a developer runs python manage.py test, normal tests run (e.g., 423 test cases) excluding migration tests.
  • If a developer wants to run migration tests, they can run python manage.py test --tag=migration.

This approach keeps the developer experience clean, avoids DB conflicts, and ensures that future schema changes won’t accidentally break normal tests.

Additionally, this separation makes it easier to configure CI pipelines, because normal tests and migration tests can be run independently, keeping CI fast and reliable while still allowing migration tests to run when needed.

As of now, this seems like the practical and safe solution given db deferred constraints behavior. and I really want to hear your thoughts on this approach.

Thanks for the explanation! :) The problem I have with your approach is that the migration tests won't run by default, reducing their value as regression guards. But if you are convinced that there is no better option, feel free to implement it that way.

The problem I have with your approach is that the migration tests won't run by default, reducing their value as regression guards.

I completely agree with your point. For now, we can run a separate CI job for these test cases as a regression guard if that work. I assure you, In the future, if I find a reliable solution, workaround, or any relevant update, I’ll implement it or notify you :)