Fix DatabaseDefault crash in _update_days_seen. Closes #908#910
Conversation
…t#908 Signed-off-by: Sanchit2662 <sanchit2662@gmail.com>
|
Hey @Sanchit2662 ! Where did the rest of the checklist from the PR template go? |
|
Hey @regulartim, sorry about that I accidentally removed part of the template while editing the description. I've added it back now. Let me know if anything else is missing! |
regulartim
left a comment
There was a problem hiding this comment.
After looking at this I see a minor problem with the current logic (not because of your approach, it was there before): we are not using information that we could have used. You can assume that @timestamp is always present in any hit coming from Elasticsearch. This is ensured by our extraction logic. So we can set last_seen explicitly in the two IOC() constructors in cowrie.py. This should also fix the issue AND give us more precise information about the time when the hit happend. What do you think?
|
That's a great point! Setting last_seen (and first_seen) from hit["timestamp"] in the two Cowrie constructors makes more sense , we'd get the actual event time instead of losing that information with a datetime.now() fallback. I'll update both IOC constructors in cowrie.py to pull from timestamp. I'll also keep the defensive isinstance check in _update_days_seen() as a safety net, in case any future code path introduces the same problem. |
Signed-off-by: Sanchit2662 <sanchit2662@gmail.com>
|
Hey @regulartim, I've pushed the update. |
Signed-off-by: Sanchit2662 <sanchit2662@gmail.com>
regulartim
left a comment
There was a problem hiding this comment.
Nice, I think this is the right way to fix it! Thanks for your work. After addressing my comment, this is ready to merge.
There was a problem hiding this comment.
I first thought we should leave your added guardrails here, but now I feel like removing them. If at this point last_seen is not a datetime object, something clearly went wrong and we should know about it. I think the code should crash here to unmask bugs.
Sorry, bu I would prefer if you completely remove your changes to this file.
There was a problem hiding this comment.
@regulartim Done! Removed the guardrails from ioc_processor.py entirely.
Signed-off-by: Sanchit2662 <sanchit2662@gmail.com>
…t#908 (intelowlproject#910) * Fix DatabaseDefault crash in _update_days_seen. Closes intelowlproject#908 Signed-off-by: Sanchit2662 <sanchit2662@gmail.com> * Set explicit timestamps in Cowrie IOC constructors Signed-off-by: Sanchit2662 <sanchit2662@gmail.com> * Add missing @timestamp to Cowrie test hits Signed-off-by: Sanchit2662 <sanchit2662@gmail.com> * Remove defensive guards from ioc_processor.py Signed-off-by: Sanchit2662 <sanchit2662@gmail.com> --------- Signed-off-by: Sanchit2662 <sanchit2662@gmail.com>
Summary
This PR fixes the 'DatabaseDefault' object has no attribute 'date' error that crashes the extraction pipeline during Cowrie (and potentially
other honeypot) processing. The issue was introduced after #887 switched IOC.last_seen from default=datetime.now to db_default=Now().
With db_default, unsaved IOC instances no longer get a real datetime in memory — they get a DatabaseDefault sentinel instead. When
_update_days_seen() tries to call .date() on that, it blows up.
I traced this to three code paths where last_seen never gets explicitly set:
Fix
Rather than patching each of the three creation sites individually, I went with a chokepoint approach — every IOC flows through
_update_days_seen() in ioc_processor.py regardless of which strategy created it, so guarding there covers all current and future cases in one
place.
datetime.now().
DatabaseDefault from a new unsaved IOC would silently corrupt a perfectly valid timestamp on an existing DB record.
Closes #908
Based on the PR details provided, here are the checked items:
Formalities
<feature name>. Closes #999develop.develop.Docs and tests
GUI changes
Review process