Add unpack processing option

[eskawl]

###Changelog:

Add unpack to config

• Config takes a new value unpack under source
• Files under the is_archive class would be unpacked and recon would perfrom analysis on the unpacked files aswell
• Refer issue: Add unpack as a processor #1

[jondot]

Thanks! this look like a good implementation.
I see now that I have a few open questions:

• Unpacking: to which folder?
• Deciding if to unpack: although we can "blindly" run decompress on all files, I should probably add this to decompress as an option to allow for selective decompress. This will allow to unpack before any processing happens, in the walking of the directory stage.

I will take a look at this myself, and come back here for conclusions, thanks!

[eskawl]

Unpacking: to which folder?

We unpack to the current folder. I think that would be a sensible choice.

I should probably add this to decompress as an option to allow for selective decompress.

Unpacking would only happen when there files are marked under is_archive class. Do you want add further filters?

[jondot]

@eskawl I think we should unpack to a folder similar to the name of the file:

file.zip
file.zip/
   one.txt
   two.txt

The reason is to prevent someone from crafting a special zip file which will override the existing files to hide something.
And the other reason is human mistake, doing the same effect.

For is_archive I think unpacking is a special case, where it is unrelated to fields on a file -- so unpack anything you can unpack first, and then run processing on files. I added can_decompress in decompress v0.2.0 which is now available.

[eskawl]

unpack to a folder similar to the name of the file

I think it is not possible to have as directory with the same name of an existing file. See this Stackoverflow discussion.

Perhaps, we can add a suffix to the folder name to prevent accidental ovewrites.

so unpack anything you can unpack first

Yes, we can do this, but I think this would result in the user's file system getting bloated by unnecessary unpacks, I think we should add some kind of a filter here.