Skip to content

Fix DatabaseDefault crash in _update_days_seen. Closes #908#910

Merged
regulartim merged 4 commits intointelowlproject:developfrom
Sanchit2662:fix/908-databasedefault-days-seen
Mar 3, 2026
Merged

Fix DatabaseDefault crash in _update_days_seen. Closes #908#910
regulartim merged 4 commits intointelowlproject:developfrom
Sanchit2662:fix/908-databasedefault-days-seen

Conversation

@Sanchit2662
Copy link
Copy Markdown
Contributor

@Sanchit2662 Sanchit2662 commented Mar 1, 2026

Summary

This PR fixes the 'DatabaseDefault' object has no attribute 'date' error that crashes the extraction pipeline during Cowrie (and potentially
other honeypot) processing. The issue was introduced after #887 switched IOC.last_seen from default=datetime.now to db_default=Now().

With db_default, unsaved IOC instances no longer get a real datetime in memory — they get a DatabaseDefault sentinel instead. When
_update_days_seen() tries to call .date() on that, it blows up.

I traced this to three code paths where last_seen never gets explicitly set:

  • cowrie.py — payload hostname IOC creation
  • cowrie.py — download URL IOC creation
  • utils.py — iocs_from_hits() when hits lack timestamp fields (this one affects all honeypot strategies, not just Cowrie)

Fix

Rather than patching each of the three creation sites individually, I went with a chokepoint approach — every IOC flows through
_update_days_seen() in ioc_processor.py regardless of which strategy created it, so guarding there covers all current and future cases in one
place.

  • In _update_days_seen(), I added an isinstance check before calling .date(). If last_seen isn't a real datetime, it falls back to
    datetime.now().
  • In _merge_iocs(), I added a guard to skip overwriting existing.last_seen when the incoming IOC carries a DatabaseDefault. Without this, a
    DatabaseDefault from a new unsaved IOC would silently corrupt a perfectly valid timestamp on an existing DB record.

Closes #908
Based on the PR details provided, here are the checked items:

Formalities

  • I have read and understood the rules about how to Contribute to this project.
  • I chose an appropriate title for the pull request in the form: <feature name>. Closes #999
  • My branch is based on develop.
  • The pull request is for the branch develop.
  • I have reviewed and verified any LLM-generated code included in this PR.

Docs and tests

  • I documented my code changes with docstrings and/or comments.
  • I have checked if my changes affect user-facing behavior that is described in the docs. If so, I also created a pull request in the docs repository.
  • Linter (Ruff) gave 0 errors. If you have correctly installed pre-commit, it does these checks and adjustments on your behalf.
  • I have added tests for the feature/bug I solved.
  • All the tests gave 0 errors.

GUI changes

  • I have provided a screenshot of the result in the PR.
  • I have created new frontend tests for the new component or updated existing ones.

Review process

  • We encourage you to create a draft PR first, even when your changes are incomplete. This way you refine your code while we can track your progress and actively review and help.
  • If you think your draft PR is ready to be reviewed by the maintainers, click the corresponding button. Your draft PR will become a real PR.
  • If your changes decrease the overall tests coverage (you will know after the Codecov CI job is done), you should add the required tests to fix the problem.
  • Every time you make changes to the PR and you think the work is done, you should explicitly ask for a review. After receiving a "change request", address the feedback and click "request re-review" next to the reviewer's profile picture at the top right.

…t#908

Signed-off-by: Sanchit2662 <sanchit2662@gmail.com>
@regulartim
Copy link
Copy Markdown
Collaborator

Hey @Sanchit2662 ! Where did the rest of the checklist from the PR template go?

@Sanchit2662
Copy link
Copy Markdown
Contributor Author

Hey @regulartim, sorry about that I accidentally removed part of the template while editing the description. I've added it back now. Let me know if anything else is missing!

Copy link
Copy Markdown
Collaborator

@regulartim regulartim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After looking at this I see a minor problem with the current logic (not because of your approach, it was there before): we are not using information that we could have used. You can assume that @timestamp is always present in any hit coming from Elasticsearch. This is ensured by our extraction logic. So we can set last_seen explicitly in the two IOC() constructors in cowrie.py. This should also fix the issue AND give us more precise information about the time when the hit happend. What do you think?

@Sanchit2662
Copy link
Copy Markdown
Contributor Author

Sanchit2662 commented Mar 2, 2026

That's a great point! Setting last_seen (and first_seen) from hit["timestamp"] in the two Cowrie constructors makes more sense , we'd get the actual event time instead of losing that information with a datetime.now() fallback. I'll update both IOC constructors in cowrie.py to pull from timestamp. I'll also keep the defensive isinstance check in _update_days_seen() as a safety net, in case any future code path introduces the same problem.

Signed-off-by: Sanchit2662 <sanchit2662@gmail.com>
@Sanchit2662
Copy link
Copy Markdown
Contributor Author

Hey @regulartim, I've pushed the update.

Signed-off-by: Sanchit2662 <sanchit2662@gmail.com>
Copy link
Copy Markdown
Collaborator

@regulartim regulartim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I think this is the right way to fix it! Thanks for your work. After addressing my comment, this is ready to merge.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I first thought we should leave your added guardrails here, but now I feel like removing them. If at this point last_seen is not a datetime object, something clearly went wrong and we should know about it. I think the code should crash here to unmask bugs.

Sorry, bu I would prefer if you completely remove your changes to this file.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@regulartim Done! Removed the guardrails from ioc_processor.py entirely.

Signed-off-by: Sanchit2662 <sanchit2662@gmail.com>
Copy link
Copy Markdown
Collaborator

@regulartim regulartim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! :)

@regulartim regulartim merged commit c733f58 into intelowlproject:develop Mar 3, 2026
4 checks passed
cclts pushed a commit to cclts/GreedyBear that referenced this pull request Mar 11, 2026
…t#908 (intelowlproject#910)

* Fix DatabaseDefault crash in _update_days_seen. Closes intelowlproject#908

Signed-off-by: Sanchit2662 <sanchit2662@gmail.com>

* Set explicit timestamps in Cowrie IOC constructors

Signed-off-by: Sanchit2662 <sanchit2662@gmail.com>

* Add missing @timestamp to Cowrie test hits

Signed-off-by: Sanchit2662 <sanchit2662@gmail.com>

* Remove defensive guards from ioc_processor.py

Signed-off-by: Sanchit2662 <sanchit2662@gmail.com>

---------

Signed-off-by: Sanchit2662 <sanchit2662@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants