Skip to content

Fix output manager copying tmp file#304

Closed
ericzinnikas wants to merge 1 commit into
google:masterfrom
ericzinnikas:fix-tmp
Closed

Fix output manager copying tmp file#304
ericzinnikas wants to merge 1 commit into
google:masterfrom
ericzinnikas:fix-tmp

Conversation

@ericzinnikas

Copy link
Copy Markdown
Contributor

While the plaso file is put into the tmp directory it is not copied to the original output folder after task execution. This fix handles that (and updates the local evidence path to the output directory). I'm not sure if there is a nicer way to determine if we used a tmp dir or not, instead of trying in every instance...

This is probably fine even for local runs, so evidence ends up in the expected place right? I'm not sure about side-effects for GCS since I'm not entirely clear on how that file copying happens.

Prior to fix:

[INFO] Running plaso as [log2timeline.py --status_view none --hashers all --partition all --vss_stores all --logfile /evidence/output/1542396446-b3580c49c1444ee59fca5e82da78fc66-PlasoTask/b3580c49c1444ee59fca5e82da78fc66.log \
/tmp/1542396446-b3580c49c1444ee59fca5e82da78fc66-PlasoTask/b3580c49c1444ee59fca5e82da78fc66.plaso /Users/ericwz/tmp/usbcopy/osdfc.dd.raw]
<snip ... >
[INFO] Running psort as [psort.py --status_view none --logfile /evidence/output/1542399748-0cfbfd52b6e74242b1b1ddc355a7ae77-PsortTask/0cfbfd52b6e74242b1b1ddc355a7ae77.log -w /evidence/output/1542399748-0cfbfd52b6e74242b1b1ddc355a7ae77-PsortTask/0cfbfd52b6e74242b1b1ddc355a7ae77.csv \
/tmp/1542396446-b3580c49c1444ee59fca5e82da78fc66-PlasoTask/b3580c49c1444ee59fca5e82da78fc66.plaso]

After fix:

[INFO] Running plaso as [log2timeline.py --status_view none --hashers all --partition all --vss_stores all --logfile /evidence/output/1542407550-9961049d072a4f648007ca64be90c3fb-PlasoTask/9961049d072a4f648007ca64be90c3fb.log \
/tmp/1542407550-9961049d072a4f648007ca64be90c3fb-PlasoTask/9961049d072a4f648007ca64be90c3fb.plaso /Users/ericwz/SCHARDT.dd]
<snip ... >
[INFO] Running psort as [psort.py --status_view none --logfile /evidence/output/1542407794-250d20304cc14727b1b793420b383a57-PsortTask/250d20304cc14727b1b793420b383a57.log -w /evidence/output/1542407794-250d20304cc14727b1b793420b383a57-PsortTask/250d20304cc14727b1b793420b383a57.csv \
/evidence/output/1542407550-9961049d072a4f648007ca64be90c3fb-PlasoTask/9961049d072a4f648007ca64be90c3fb.plaso]

* Ensure evidence is copied / paths fixed
result.log(msg)
log.warning(msg)
else:
new_path, _ = self.output_manager.save_local_file(

@aarontp aarontp Dec 5, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hrm... not sure what the best solution is here. Since this returns the last path that was saved, if GCS was enabled, this could potentially get the GCS path instead of the local path. I'll have to take a closer look at this tomorrow, but we might need to refactor save_local_evidence() a bit more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another byproduct of this is that it would potentially save files twice. I think I have a fix by updating save_evidence() and save_local_evidence() that I'll send you shortly.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, See #319 as a possible fix for this issue. FWIW I verified it by making sure that the Plaso run was writing to /tmp and the Psort run was operating on the evidence in the permanent output directory.

@aarontp

aarontp commented Dec 12, 2018

Copy link
Copy Markdown
Member

AFAIK, #319 should fix this issue. LMK if you think there is something that you think still needs to be addressed here, and if not we can probably close it. Sorry for clobbering this PR, but GitHub isn't very good at multi-line suggestions yet, I and I needed to change a different file as well.

@ericzinnikas

Copy link
Copy Markdown
Contributor Author

Taking comments to #319

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants