@night 1803 access accessdata active directory admissibility ads aduc aim aix ajax alissa torres amcache analysis anjp anssi answer key antiforensics apfs appcompat appcompatflags applocker april fools argparse arman gungor arsenal artifact extractor attachments attacker tools austin automating automation awards aws azure azuread back to basics backstage base16 best finds beta bias bitcoin bitlocker blackbag blackberry enterprise server blackhat blacklight blade blanche lagny book book review brute force bsides bulk extractor c2 carved carving case ccdc cd burning ceic cfp challenge champlain chat logs Christmas Christmas eve chrome cit client info cloud forensics command line computer forensics computername conference schedule consulting contest cool tools. tips copy and paste coreanalytics cortana court approved credentials cryptocurrency ctf cti summit cut and paste cyberbox Daily Blog dbir deep freeze defcon defender ata deviceclasses dfa dfir dfir automation dfir exposed dfir in 120 seconds dfir indepth dfir review dfir summit dfir wizard dfrws dfvfs dingo stole my baby directories directory dirty file system disablelastaccess discount download dropbox dvd burning e01 elastic search elcomsoft elevated email recovery email searching emdmgmt Encyclopedia Forensica enfuse eric huber es eshandler esxi evalexperience event log event logs evidence execution exfat ext3 ext4 extended mapi external drives f-response factory access mode false positive fat fde firefox for408 for498 for500 for526 for668 forenisc toolkit forensic 4cast forensic lunch forensic soundness forensic tips fraud free fsutil ftk ftk 2 full disk encryption future gcfe gcp github go bag golden ticket google gsuite guardduty gui hackthebox hal pomeranz hashlib hfs honeypot honeypots how does it work how i use it how to howto IE10 imaging incident response indepth information theft infosec pro guide intern internetusername Interview ios ip theft iphone ir itunes encrypted backups jailbreak jeddah jessica hyde joe sylve journals json jump lists kali kape kevin stokes kibana knowledgec korman labs lance mueller last access last logon leanpub libtsk libvshadow linux linux-3g live systems lnk files log analysis log2timeline login logs london love notes lznt1 mac mac_apt macmini magnet magnet user summit mathias fuchs md viewer memorial day memory forensics metaspike mft mftecmd mhn microsoft milestones mimikatz missing features mlocate mobile devices mojave mount mtp multiboot usb mus mus 2019 mus2019 nccdc netanalysis netbios netflow new book new years eve new years resolutions nominations nosql notifications ntfs ntfsdisablelastaccessupdate nuc nw3c objectid offensive forensics office office 2016 office 365 oleg skilkin osx outlook outlook web access owa packetsled paladin path specification pdf perl persistence pfic plists posix powerforensics powerpoint powershell prefetch psexec py2exe pyewf pyinstaller python pytsk rallysecurity raw images rdp re-c re-creation testing reader project recipes recon recursive hashing recycle bin redteam regipy registry registry explorer registry recon regripper remote research reverse engineering rhel rootless runas sample images san diego SANS sans dfir summit saturday Saturday reading sbe sccm scrap files search server 2008 server 2008 r2 server 2012 server 2019 setmace setupapi sha1 shadowkit shadows shell items shellbags shimcache silv3rhorn skull canyon skype slow down smb solution solution saturday sop speed sponsors sqlite srum ssd stage 1 stories storport sunday funday swgde syscache system t2 takeout telemetry temporary files test kitchen thanksgiving threat intel timeline times timestamps timestomp timezone tool tool testing training transaction logs triage triforce truecrypt tsk tun naung tutorial typed paths typedpaths uac unc understanding unicorn unified logs unread usb usb detective usbstor user assist userassist usnjrnl validation vhd video video blog videopost vlive vmug vmware volatility vote vss web2.0 webcast webinar webmail weekend reading what are you missing what did they take what don't we know What I wish I knew whitfield windows windows 10 windows 2008 windows 7 windows forensics windows server winfe winfe lite wmi write head xboot xfs xways yarp yogesh zimmerman zone.identifier

Tool Testing: File Carvers as seen on the Forensic Lunch 3/11/16

Greetings Readers,

This post documents the findings we discussed on the Forensic Lunch 3/11/16 episode where we discussed how different file carvers worked (watch it here).I asked James to write up the results of our test and provide links to our test images.

On Friday’s (3/11/16) Forensic Lunch we discussed the results of testing several different file carvers. Using a copy of a Windows 7 VHD we keep on hand for testing purposes we decided to find out how well each of the tools recovered some known data. The VHD contained 10 pdfs, 8 Word docs, 10 prefetch, 7 sqlite DB’s and 10 event logs were copied to the My Documents directory in the VM then shift-deleted in groups of 5.  In addition Office was installed and a Word Document was created and then 10 versions made bringing the total of Word Documents deleted to 18.  Since some programs that do file carving are file system aware a copy of the VHD was made were the MFT was overwritten with nulls using a hex editor.  The carvers tested were X-Ways, Blade, Bulk Extractor, and Blacklight. (Note if you would like to see a different carver tested put it in the comments below)

X-Ways and Blacklight are both tools that are file system aware. Meaning that when an image was carved these tools would traverse the MFT and display deleted files it could identify in the directory they had been deleted from if they were still being referenced and their contents were not overwritten.  This means that the tools would not include these recovered files within their ranges of unallocated space to be carved as they know the contents of those files is still valid.

In testing, Blacklight carved and validated more pdf’s and docx’s, while X-Ways provided support for carved prefetch and event logs where Blacklight currently does not have an option for these file types.  The version of the same image with no MFT provided similar numbers of documents carved for X-Ways and Blacklight with Blacklight giving fewer false positives on xlsx and pptx files as they have added in a validation step on carved data.

Blade and Bulk Extractor do not check for a file system and just start scanning through data.  Of all the carvers Blade by far carved the most documents.  By default Blade does not have definitions for Prefetch or event logs.  Additionally Blade does not identify what type of Office document and makes a copy of what it finds as a .pptx, .docx, and .xlsx.  This may just be a bug in the version of Blade we are using, we need to try the newly released version.

Bulk Extractor does not look for 2007 or greater office documents (docx, pptx, etc..)  specifically, they are included within the option for zip files.  On this test set it found 24,000 zip archives.  The word documents we deleted would be in here, but would require a second step to locate them. Possibly with a tool like exiftool to identify those zip files with Office metadata. Bulk Extractor did not find any pdf’s from its pdf signature but was able to find the same number of prefetch files as X-Ways.  It also found and carved a similar number of SQLite DB’s as X-Ways and Blacklight did when they were given the image with a destroyed MFT.

We also ran the carvers against the CFREDs L0 Documents test set.  This test set had 7 documents.  X-Ways and Blacklight had not trouble and got all 7 files correct.  Blade got 6 of the 7, missing one docx in the docx folder.  Bulk Extractor put what it found in the zip file listing and missed the pdfs.

In terms of speed Bulk Extractor was the fastest on our test set, finishing its run in 2 minutes and 1 second against a 50gb image.  Next was X-Ways with and without the MFT at 2:13 and 2:25.  Blacklight with the MFT ran in 8:02.  Blade ran in 9:10 and finally Blacklight without an MFT took 16:54.  Blacklight could not use the 12.5 GB compressed version of the 50GB VHD so we converted it into a full 50 GB raw image .  The CFREDs set is so small all the carvers got through it in a few seconds.

Overall X-Ways out of the box supported all the file types we were interested in.  For situations where default options are not enough Blade tends to be the most flexible for writing new definitions, allowing you to write detailed definitions.  Bulk Extractor’s filters were not as well suited for this test, but it provides options that are of use in IR scenarios and it is the only one of the four available for free.  Blacklight gave fewer false positives in the no MFT set and was the only tool we are aware of that we tested that will attempt to test the results of the carver for valid data.

Want to do your own testing? Want to verify our results? You can! click here for the test set with an MFT and result spreadsheet and here for the test set where the MFT has been nuked. The CFREDs images can be founds here: http://www.cfreds.nist.gov/FileCarving/index.html

Post a Comment

[blogger][disqus][facebook][spotim]

Author Name

Contact Form

Name

Email *

Message *

Powered by Blogger.