Tool Testing: File Carvers as seen on the Forensic Lunch 3/11/16
Greetings Readers,
This post documents the findings we discussed on the Forensic Lunch 3/11/16 episode where we discussed how different file carvers worked (watch it here).I asked James to write up the results of our test and provide links to our test images.
On Friday’s (3/11/16) Forensic Lunch we discussed the results of testing several different file carvers. Using a copy of a Windows 7 VHD we keep on hand for testing purposes we decided to find out how well each of the tools recovered some known data. The VHD contained 10 pdfs, 8 Word docs, 10 prefetch, 7 sqlite DB’s and 10 event logs were copied to the My Documents directory in the VM then shift-deleted in groups of 5. In addition Office was installed and a Word Document was created and then 10 versions made bringing the total of Word Documents deleted to 18. Since some programs that do file carving are file system aware a copy of the VHD was made were the MFT was overwritten with nulls using a hex editor. The carvers tested were X-Ways, Blade, Bulk Extractor, and Blacklight. (Note if you would like to see a different carver tested put it in the comments below)
X-Ways and Blacklight are both tools that are file system aware. Meaning that when an image was carved these tools would traverse the MFT and display deleted files it could identify in the directory they had been deleted from if they were still being referenced and their contents were not overwritten. This means that the tools would not include these recovered files within their ranges of unallocated space to be carved as they know the contents of those files is still valid.
In testing, Blacklight carved and validated more pdf’s and docx’s, while X-Ways provided support for carved prefetch and event logs where Blacklight currently does not have an option for these file types. The version of the same image with no MFT provided similar numbers of documents carved for X-Ways and Blacklight with Blacklight giving fewer false positives on xlsx and pptx files as they have added in a validation step on carved data.
Blade and Bulk Extractor do not check for a file system and just start scanning through data. Of all the carvers Blade by far carved the most documents. By default Blade does not have definitions for Prefetch or event logs. Additionally Blade does not identify what type of Office document and makes a copy of what it finds as a .pptx, .docx, and .xlsx. This may just be a bug in the version of Blade we are using, we need to try the newly released version.
Bulk Extractor does not look for 2007 or greater office documents (docx, pptx, etc..) specifically, they are included within the option for zip files. On this test set it found 24,000 zip archives. The word documents we deleted would be in here, but would require a second step to locate them. Possibly with a tool like exiftool to identify those zip files with Office metadata. Bulk Extractor did not find any pdf’s from its pdf signature but was able to find the same number of prefetch files as X-Ways. It also found and carved a similar number of SQLite DB’s as X-Ways and Blacklight did when they were given the image with a destroyed MFT.
We also ran the carvers against the CFREDs L0 Documents test set. This test set had 7 documents. X-Ways and Blacklight had not trouble and got all 7 files correct. Blade got 6 of the 7, missing one docx in the docx folder. Bulk Extractor put what it found in the zip file listing and missed the pdfs.
In terms of speed Bulk Extractor was the fastest on our test set, finishing its run in 2 minutes and 1 second against a 50gb image. Next was X-Ways with and without the MFT at 2:13 and 2:25. Blacklight with the MFT ran in 8:02. Blade ran in 9:10 and finally Blacklight without an MFT took 16:54. Blacklight could not use the 12.5 GB compressed version of the 50GB VHD so we converted it into a full 50 GB raw image . The CFREDs set is so small all the carvers got through it in a few seconds.
Overall X-Ways out of the box supported all the file types we were interested in. For situations where default options are not enough Blade tends to be the most flexible for writing new definitions, allowing you to write detailed definitions. Bulk Extractor’s filters were not as well suited for this test, but it provides options that are of use in IR scenarios and it is the only one of the four available for free. Blacklight gave fewer false positives in the no MFT set and was the only tool we are aware of that we tested that will attempt to test the results of the carver for valid data.
Want to do your own testing? Want to verify our results? You can! click here for the test set with an MFT and result spreadsheet and here for the test set where the MFT has been nuked. The CFREDs images can be founds here: http://www.cfreds.nist.gov/FileCarving/index.html
This post documents the findings we discussed on the Forensic Lunch 3/11/16 episode where we discussed how different file carvers worked (watch it here).I asked James to write up the results of our test and provide links to our test images.
On Friday’s (3/11/16) Forensic Lunch we discussed the results of testing several different file carvers. Using a copy of a Windows 7 VHD we keep on hand for testing purposes we decided to find out how well each of the tools recovered some known data. The VHD contained 10 pdfs, 8 Word docs, 10 prefetch, 7 sqlite DB’s and 10 event logs were copied to the My Documents directory in the VM then shift-deleted in groups of 5. In addition Office was installed and a Word Document was created and then 10 versions made bringing the total of Word Documents deleted to 18. Since some programs that do file carving are file system aware a copy of the VHD was made were the MFT was overwritten with nulls using a hex editor. The carvers tested were X-Ways, Blade, Bulk Extractor, and Blacklight. (Note if you would like to see a different carver tested put it in the comments below)
X-Ways and Blacklight are both tools that are file system aware. Meaning that when an image was carved these tools would traverse the MFT and display deleted files it could identify in the directory they had been deleted from if they were still being referenced and their contents were not overwritten. This means that the tools would not include these recovered files within their ranges of unallocated space to be carved as they know the contents of those files is still valid.
In testing, Blacklight carved and validated more pdf’s and docx’s, while X-Ways provided support for carved prefetch and event logs where Blacklight currently does not have an option for these file types. The version of the same image with no MFT provided similar numbers of documents carved for X-Ways and Blacklight with Blacklight giving fewer false positives on xlsx and pptx files as they have added in a validation step on carved data.
Blade and Bulk Extractor do not check for a file system and just start scanning through data. Of all the carvers Blade by far carved the most documents. By default Blade does not have definitions for Prefetch or event logs. Additionally Blade does not identify what type of Office document and makes a copy of what it finds as a .pptx, .docx, and .xlsx. This may just be a bug in the version of Blade we are using, we need to try the newly released version.
Bulk Extractor does not look for 2007 or greater office documents (docx, pptx, etc..) specifically, they are included within the option for zip files. On this test set it found 24,000 zip archives. The word documents we deleted would be in here, but would require a second step to locate them. Possibly with a tool like exiftool to identify those zip files with Office metadata. Bulk Extractor did not find any pdf’s from its pdf signature but was able to find the same number of prefetch files as X-Ways. It also found and carved a similar number of SQLite DB’s as X-Ways and Blacklight did when they were given the image with a destroyed MFT.
We also ran the carvers against the CFREDs L0 Documents test set. This test set had 7 documents. X-Ways and Blacklight had not trouble and got all 7 files correct. Blade got 6 of the 7, missing one docx in the docx folder. Bulk Extractor put what it found in the zip file listing and missed the pdfs.
In terms of speed Bulk Extractor was the fastest on our test set, finishing its run in 2 minutes and 1 second against a 50gb image. Next was X-Ways with and without the MFT at 2:13 and 2:25. Blacklight with the MFT ran in 8:02. Blade ran in 9:10 and finally Blacklight without an MFT took 16:54. Blacklight could not use the 12.5 GB compressed version of the 50GB VHD so we converted it into a full 50 GB raw image . The CFREDs set is so small all the carvers got through it in a few seconds.
Overall X-Ways out of the box supported all the file types we were interested in. For situations where default options are not enough Blade tends to be the most flexible for writing new definitions, allowing you to write detailed definitions. Bulk Extractor’s filters were not as well suited for this test, but it provides options that are of use in IR scenarios and it is the only one of the four available for free. Blacklight gave fewer false positives in the no MFT set and was the only tool we are aware of that we tested that will attempt to test the results of the carver for valid data.
Want to do your own testing? Want to verify our results? You can! click here for the test set with an MFT and result spreadsheet and here for the test set where the MFT has been nuked. The CFREDs images can be founds here: http://www.cfreds.nist.gov/FileCarving/index.html
