Tests(Cronjobs): Adding tests for MonitorLogs and MonitorHoneyPots.#669
Tests(Cronjobs): Adding tests for MonitorLogs and MonitorHoneyPots.#669regulartim merged 5 commits intointelowlproject:developfrom
Conversation
|
Hey @regulartim ! thank you for pointing me in the right direction in the previous PR. I think this is in a nice shape for a review. |
| info_calls = [call[0][0] for call in cronjob.log.info.call_args_list] | ||
| warning_calls = [call[0][0] for call in cronjob.log.warning.call_args_list] | ||
|
|
||
| self.assertTrue(any("logs available" in msg for msg in info_calls)) | ||
| self.assertTrue(any("no logs available" in msg for msg in warning_calls)) |
There was a problem hiding this comment.
Wait, you are testing the output of cronjob.log here, but that is a MagicMock right? So this will always succeed! Or am I on the wrong track here?
There was a problem hiding this comment.
I used MagicMock to make a mock object of cronjob.log so we dont actually log something, apart from that the behavior of it depends only on what is being logged by the upper methods that call it, so if your concern was did i mock and pass the log manually, so no that is actually not happening from my understanding. I feel this class can be improved a lot and the tests can be more clear with what they are doing. Thanks for the feedback!
There was a problem hiding this comment.
Ah, ok. I think I understand. I think I was confused because I expected that it is tested which honeypot logged what. You verify that some info and warning were logged, but not that Log4pot got the info and Cowrie got the warning. Because this is what happens, right?
There was a problem hiding this comment.
@regulartim i think i understand, as we know the name of the honeypot we can actually log the name of it in the assertion, the latest commit actually asserts equal with the number directly, but logging the name is also doable, might make the tests brittle if we do it but also enhances the code from readability point of view because we are clear just by reading whats happening. let me know what you prefer i would assert it accordingly.
| def stat_side_effect(): | ||
| mock_stat_result = MagicMock() | ||
| if stat_side_effect.call_count == 0: | ||
| mock_stat_result.st_mtime = recent_time | ||
| else: | ||
| mock_stat_result.st_mtime = old_time | ||
| stat_side_effect.call_count += 1 | ||
| return mock_stat_result |
There was a problem hiding this comment.
I don't understand what this does. Could you please explain?
There was a problem hiding this comment.
Yes! So if you refer line https://github.com/intelowlproject/GreedyBear/blob/develop/greedybear/cronjobs/monitor_logs.py#L27 we are checking the logs for 4 different pots, so stat_result which contains the time is called four times, so this function returns recent time for the first call and old time for rest. Hence later we assert that the send_message method was only called once with greedy bear as it was the only pot with latest log time.......this can be decoupled into two test functions surely but as already iterated by you that we prefer to follow the test format present, i took reference from test https://github.com/intelowlproject/GreedyBear/blob/develop/tests/greedybear/cronjobs/test_firehol.py where multiple test cases were written in side effect.
There was a problem hiding this comment.
I would prefer multiple functions and each method is responsible for testing a specific behavior.
There was a problem hiding this comment.
I am sorry, but I still find that function very confusing, although I do understand now why it is here and what it is doing. Would something like this be equivalent?
mock_stat.side_effect = [
MagicMock(st_mtime=recent_time),
MagicMock(st_mtime=old_time),
MagicMock(st_mtime=old_time),
MagicMock(st_mtime=old_time),
]I think this is more explicit and easier to read and understand. What do you think?
| @patch("greedybear.cronjobs.monitor_logs.Path.stat") | ||
| @patch("greedybear.cronjobs.monitor_logs.Path.exists") | ||
| @patch("greedybear.cronjobs.monitor_logs.send_message") | ||
| def test_logs_recent_activity(self, mock_send, mock_exists, mock_stat): |
There was a problem hiding this comment.
Could you also write a test, where the log was not populated and nothing was sent?
There was a problem hiding this comment.
yes! @regulartim would you prefer the general test design where we write multiple functions and each method is responsible for testing a specific behavior or would you prefer it to how it is currently and all the tests coupled in one single function using side effects?
There was a problem hiding this comment.
I would prefer multiple functions and each method is responsible for testing a specific behavior.
|
First of all, thanks for your work. But I think there is a lot to be done here. I commented on some parts of the code, but also in the bigger picture your code is not ready in my opinion. Here are a few tips for you:
|
️✅ There are no secrets present in this pull request anymore.If these secrets were true positive and are still valid, we highly recommend you to revoke them. 🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request. |
|
hey @regulartim, i have made multiple test cases to reflect the specific behavior that is being tested, also improved the naming. let me know if any further clarification or changes are required. Thank you again for the through feedback! |
regulartim
left a comment
There was a problem hiding this comment.
Good job! We're getting close! :)
| def stat_side_effect(): | ||
| mock_stat_result = MagicMock() | ||
| if stat_side_effect.call_count == 0: | ||
| mock_stat_result.st_mtime = recent_time | ||
| else: | ||
| mock_stat_result.st_mtime = old_time | ||
| stat_side_effect.call_count += 1 | ||
| return mock_stat_result |
There was a problem hiding this comment.
I am sorry, but I still find that function very confusing, although I do understand now why it is here and what it is doing. Would something like this be equivalent?
mock_stat.side_effect = [
MagicMock(st_mtime=recent_time),
MagicMock(st_mtime=old_time),
MagicMock(st_mtime=old_time),
MagicMock(st_mtime=old_time),
]I think this is more explicit and easier to read and understand. What do you think?
|
@regulartim i have made the necessary changes :D, sorry for the delay |
|
@regulartim I went through the test file of firehol.py, i believe the tests in that can be decoupled and more edges can be tested also that file no longer strongly follow the format we have established in these test files like seperation of concerns, would you like if i make an issue highlighting some concerns, if you feel its worth working on i would like to take that up. |
Would you please open a separate issue on that? |
No problem! :) |
|
For future PRs: when I am asking questions in a review, I would really like to discuss the code with you. Sometimes I'm wrong or I am interested in your rationale. |
|
yes! @regulartim i will open a new issue shortly. yes i would take care in future PRs, thanks again for the thorough review! |
Fixes: #653
Description
This PR adds tests for monitor_logs.py and monitor_honeypots.py.
Related issues
#653
Type of change
Please delete options that are not relevant.
Checklist
develop.Black,Flake,Isort) gave 0 errors. If you have correctly installed pre-commit, it does these checks and adjustments on your behalf.Important Rules