Home Archives for 2016

Building your own travel sized virtual lab with ESXi and the Intel SkullCanyon NUC

Hello Reader,
It's been awhile and I know that, sorry for not writing sooner but to quote Ferris Bueller

"Life moves pretty fast. If you don't stop and look around once in a while, you could miss it."

So while I've worked on a variety of cases, projects and new artifacts to share I've neglected the blog. For those of you who have been watching/listening you know I've kept up the Forensic Lunch videocast/podcast but to be fair the Blog is my first child and I've left it idle for too long.

Speaking of the Forensic Lunch if you watched this episode:
https://www.youtube.com/watch?v=Ru8fLioIVlA

You would have seen me talk about building my own portable cloud for lab testing and research. People seem to have received this very well and I've thoroughly enjoyed using it! So to that end I thought I would detail out how I set this up in case you wanted to do the same.

Step 1. Make an account on vmware.com (https://my.vmware.com/web/vmware/registration)

Step 2. Using chrome, not sure why I had some errors in firefox but I did, go to this page to register for the free version of ESXi. (Note this is the free version of ESXi that will generate a license key for life, the other version will expire after 60 days )
https://my.vmware.com/en/group/vmware/evalcenter?p=free-esxi6

Step 3. Make a note of your license key as seen in the picture below, you'll want to copy and paste this and keep it as it won't show up as a license key associated with your MyVmware account

Step 4. Click to download the product named "ESXi ISO image (Includes VMware Tools)". You could also download the vsphere client at this point or you can grab it from a link emebdded within the ESXI homepage when you get it installed.

Step 5. After downloading the ISO you will need to put it onto some form of bootable media for it to install onto your Intel Skull Canyon NUC as it has no optical drive of its own. I choose to do this to a USB thumb drive. To do turn the ISO into a successfully booting USB drive I used rufus and you can to.

Step 5a. Download Rufus: https://rufus.akeo.ie/downloads/rufus-2.11.exe
Step 5b. Execute Rufus
Step 5c. Configure Rufus to look something like what I have below. Where Device is the USB thumb drive you have plugged in and under ISO image I've selected the ESXi iso file I downloaded and click start.

Step 6. With your ESXi media now on a bootable USB drive you are ready to move on to the Intel Skull Canyon NUC itself. Start by actually getting one! I got mine at Fry's Electronics, Microcenter also carries them and they both price match Amazon now. If you wanted to get it online I would recommend Amazon to do so and you can support a good charity while doing so by using smile.amazon.com. I support the Girl Scouts of Northeast Texas with my purchases.

Link to Intel Skull Canyon NUC:
https://smile.amazon.com/Intel-NUC-Kit-NUC6i7KYK-Mini/dp/B01DJ9XS52/ref=sr_1_1?ie=UTF8&qid=1474577754&sr=8-1&keywords=skull+canyon

The NUC comes with a processor, case, power supply and fans all built in or in the box. What you will need to provide is the RAM and storage.

Storage
I used the Samsung 950 Pro Series 512GB NVMe M.2 drive, the NUC can actually fit two of these but one has been enough so far for my initial testing.

Link to storage drive:
https://smile.amazon.com/Samsung-950-PRO-Internal-MZ-V5P512BW/dp/B01639694M/ref=pd_bxgy_147_img_2?ie=UTF8&psc=1&refRID=7N9JV1CX8FJQ4Y3JT858

RAM
For RAM I used Kingston HyperX with two 16GB sticks to get the full 32GB of RAM this unit is capable of.
Link to the RAM here:
https://smile.amazon.com/Kingston-Technology-2133MHz-HX421S13IBK2-32/dp/B01BNJL96A/ref=pd_sim_147_2?ie=UTF8&pd_rd_i=B01BNJL96A&pd_rd_r=7N9JV1CX8FJQ4Y3JT858&pd_rd_w=eFJsO&pd_rd_wg=HPiy3&psc=1&refRID=7N9JV1CX8FJQ4Y3JT858

You can use other storage and RAM of course, I used these because I wanted the speed of NVMe M.2 (2GB/sec reads and 1.5GB/sec writes) with all the memory I could get to feed the VMs that will be running on the NUC.

Step 7. Put the storage and RAM into the NUC, plug it in to the wall, attach a USB keyboard and mouse, attach a monitor and boot up to the Intel Visual Bios. You will need to disable the Thunderbolt controller on the NUC before installing ESXi, you can re-enable it after you are done installing ESXi.

To see what to click specifically in order to do this go here:
http://www.virten.net/2016/05/esxi-installation-on-nuc6i7kyk-fails-with-fatal-error-10-out-of-resources/

Step 8. Pop in the bootable USB drive and install ESXI.

You are now ready to start loading ISO's and VMs into your datastore and in the next blog post I'll show how to create an isolated virtual network to put them on.

Also Read: Tool Testing: File Carvers as seen on the Forensic Lunch 3/11/16

National CCDC Redteam Debrief by David Cowen - Hacking Exposed Computer Forensics Blog

Hello Reader,
The 11th year of the National Collegiate Cyber Defense Competition has ended, congratulations to the University of Central Florida for a their third consecutive win. I hope you make it back next year for another test of your schools program and ability to transfer knowledge to new generations of blue teams.

If you want to show your support for my efforts, there is an easy way to do that.

Vote for me for Digital Forensic Investigator of the Year here:

https://forensic4cast.com/forensic-4cast-awards/

However the team that won over the Redteam was the University of Tulsa who came with a sense of humor. Behold their Hat and Badges:

Also you have to check out the player cards they made here:
https://engineering.utulsa.edu/news/tu-cyber-security-expert-creates-trading-cards-collegiate-cyber-defense-competition/

Here is an my favorite:

You can download my Redteam debrief here:
https://drive.google.com/file/d/0B_mjsPB8uKOAcUQtOThUNUpTZ0k/view?usp=sharing

Also Read: Daily Blog #380: National CCDC 2016

Daily Blog #380: National CCDC 2016 by David Cowen - Hacking Exposed Computer Forensics Blog

Hello Reader,
I'm in San Antonio for the National Collegiate Cyber Defense Competition which starts at 10am CST 4/22/16. If you didn't know I lead the red team here at Nationals where the top 10 college teams in the country come and find out who does the best defending their network while completing business objectives.

I'm hoping to follow up this post with some videos and links to what happens tomorrow, in the mean time make sure to follow #CCDC or #NCCDC on twitter to watch some of our funny business in real time.

Also Read: Daily Blog #379

Automating DFIR with dfVFS Part 6 by David Cowen - Hacking Exposed Computer Forensics Blog

Hello Reader,

It's time to continue our series by iterating through all the partitions within a disk or image, instead of just hard coding the one. To start with you'll need another image, one that not only has more than one partition but also has shadow copies for us to interact with next.

You can download the image here:

https://mega.nz/#!L45SRYpR!yl8zDOi7J7koqeGnFEhYV-_75jkVtI2CTrr14PqofBw

If you want to show your support for my efforts, there is an easy way to do that.

Vote for me for Digital Forensic Investigator of the Year here: https://forensic4cast.com/forensic-4cast-awards/

First let's look at the code now:

import sys

import logging

from dfvfs.analyzer import analyzer

from dfvfs.lib import definitions

from dfvfs.path import factory as path_spec_factory

from dfvfs.volume import tsk_volume_system

from dfvfs.resolver import resolver

from dfvfs.lib import raw

source_path="Windows 7 Professional SP1 x86 Suspect.vhd"

path_spec = path_spec_factory.Factory.NewPathSpec(

definitions.TYPE_INDICATOR_OS, location=source_path)

type_indicators = analyzer.Analyzer.GetStorageMediaImageTypeIndicators(

path_spec)

if len(type_indicators) > 1:

raise RuntimeError((

u'Unsupported source: {0:s} found more than one storage media '

u'image types.').format(source_path))

if len(type_indicators) == 1:

path_spec = path_spec_factory.Factory.NewPathSpec(

type_indicators[0], parent=path_spec)

if not type_indicators:

# The RAW storage media image type cannot be detected based on

# a signature so we try to detect it based on common file naming

# schemas.

file_system = resolver.Resolver.OpenFileSystem(path_spec)

raw_path_spec = path_spec_factory.Factory.NewPathSpec(

definitions.TYPE_INDICATOR_RAW, parent=path_spec)

glob_results = raw.RawGlobPathSpec(file_system, raw_path_spec)

if glob_results:

path_spec = raw_path_spec

volume_path_spec = path_spec_factory.Factory.NewPathSpec(

definitions.TYPE_INDICATOR_TSK_PARTITION, location=u'/',

parent=path_spec)

volume_system = tsk_volume_system.TSKVolumeSystem()

volume_system.Open(volume_path_spec)

volume_identifiers = []

for volume in volume_system.volumes:

volume_identifier = getattr(volume, 'identifier', None)

if volume_identifier:

volume_identifiers.append(volume_identifier)

print(u'The following partitions were found:')

print(u'Identifier\tOffset\t\t\tSize')

for volume_identifier in sorted(volume_identifiers):

volume = volume_system.GetVolumeByIdentifier(volume_identifier)

if not volume:

raise RuntimeError(

u'Volume missing for identifier: {0:s}.'.format(volume_identifier))

volume_extent = volume.extents[0]

print(

u'{0:s}\t\t{1:d} (0x{1:08x})\t{2:d}'.format(

volume.identifier, volume_extent.offset, volume_extent.size))

volume_path_spec = path_spec_factory.Factory.NewPathSpec(

definitions.TYPE_INDICATOR_TSK_PARTITION, location=u'/'+volume_identifier,

parent=path_spec)

mft_path_spec = path_spec_factory.Factory.NewPathSpec(

definitions.TYPE_INDICATOR_TSK, location=u'/$MFT',

parent=volume_path_spec)

file_entry = resolver.Resolver.OpenFileEntry(mft_path_spec)

stat_object = file_entry.GetStat()

print(u'Inode: {0:d}'.format(stat_object.ino))

print(u'Inode: {0:s}'.format(file_entry.name))

outFile = volume_identifier+file_entry.name

extractFile = open(outFile,'wb')

file_object = file_entry.GetFileObject()

data = file_object.read(4096)

while data:

extractFile.write(data)

data = file_object.read(4096)

extractFile.close

file_object.close()

volume_path_spec=""

mft_path_spec=""

Believe it or not we didn't have to change much here to get it to go from looking at one partition and extracting the $MFT to extracting it from all the partitions. What we had to do was four things.

1. We moved our file extraction code over by one indent allowing it to execute as part of the for loop we first wrote to print out the list of partitions in an image. Remember that in Python we don't use braces to determine how the code will be executed, its all indentation that decides how the code logic will be read and followed.

2. Next we changed the location where our volume path specification object is set to from a hard coded /p1 to whatever volume identifier we are currently looking at in the for loop.

volume_path_spec = path_spec_factory.Factory.NewPathSpec(

definitions.TYPE_INDICATOR_TSK_PARTITION, location=u'/'+volume_identifier,

parent=path_spec)

You can see that the location variable is now set to u'/' being appended to the volume_identifier variable. This would be resolved to /p1, /p2, etc.. as many partitions as we have on the image.

3. Now that we are going to extracting this file from multiple partitions we don't want to overwrite the file we previously extracted so we need to make the file name unique. We do that by appending the partition number to the file name.

outFile = volume_identifier+file_entry.name

extractFile = open(outFile,'wb')

This results in a file named p1$MFT, p2$MFT, and so on. To accomplish this we make a new variable called outfile which is set to the partition number (volume_identifier) appended with the file name (file_entry.name). Then we pass that the open file handle argument we wrote before.

4. One last simple change.

volume_path_spec=""

mft_path_spec=""

We are setting our partition and file path spec objects back to null. Why? Because if not they are globally set and will just keep appending on to the prior object.

That will result in very funny errors.

That's it! No more code changes.

You can get the code from Github:

https://github.com/dlcowen/dfirwizard/blob/master/dfvfsWizardv4.py

In the next post we will be iterating through shadow copies!

This is a 6-part series and here's link to all the parts:

Automating DFIR with dfVFS Part 6

Automating DFIR with dfVFS Part 5

Automating DFIR with dfVFS Part 4

Automating DFIR with dfVFS Part 3

Automating DFIR with dfVFS Part 2

Automating DFIR with dfVFS Part 1

Automating DFIR with dfVFS part 5 by David Cowen - Hacking Exposed Computer Forensics Blog

Hello Reader,

Wondering where yesterdays post is? Well, there was no winner of last weekends Sunday Funday.
That's ok though because I am going to post the same challenge this Sunday so you have a whole week to figure it out!

-- Now back to our regularly scheduled series --

I use Komodo from Activestate as my IDE of choice when writing perl and python. I bring this up because one of the things I really like about it is the debugger it comes with that allows you to view all of the objects you have made and their current assignments. I was thinking about the layer cake example I crudely drew in ascii in a prior post when I realized I could show this much better from the Activestate Debugger.

So here is what the path spec object we made to access the $MFT in a VHD looks like.

I've underlined in red the important things to draw your attention to when you are trying to understand how that file path specification object we built can access the MFT and all the other layers involved.

So if you look you can see from the top down its:

TSK Type with a Location of /$MFT

With a parent of TSK Partition type with a location of /p1

With a parent of VHDI type

With a parent of OS type with a location of the full path to where the vhd I'm working with sits.

Let's look at the same object with with an e01 loaded.

Notice what I highlighted, the image type has changed from VHDI to EWF. Otherwise the object, its properties and the methods are the same.

Let's do this one more time to really reinforce this with a raw/dd image.

Everything else is the same, except for the type changing to RAW.

So no matter what type of image we are working with dfVFS allows us to build an object in layers that permits the code that follows not to have to worry about the code behind. It is normalizing all the different image types access libraries so we can prevent things like the work around we do in pytsk.

Tomorrow, more code!

This is a 6-part series and here's link to all the parts:

Automating DFIR with dfVFS Part 6
Automating DFIR with dfVFS Part 5
Automating DFIR with dfVFS Part 4
Automating DFIR with dfVFS Part 3
Automating DFIR with dfVFS Part 2
Automating DFIR with dfVFS Part 1

dfVFS Testing Script Challenge by David Cowen - Hacking Exposed Computer Forensics Blog

Hello Reader,
If you have been following the blow the last two weeks you would have seen its been all about dfVFS. Phil aka Random Access posted something I was thinking about on his blog, https://thisweekin4n6.wordpress.com, that I thought was worthy of a Sunday Funday challenge. In short Phil saw that I posted a video regarding how to verify dfVFS was installed correctly and there is a whole post just on installing it and mentioned that someone should automate this process. I agree Phil, and now I turn it over to you Reader, let's try out your scripting skills in this weeks Sunday Funday Challenge.

The Prize:
$200 Amazon Giftcard

The Rules:

You must post your answer before Monday 4/18/16 3PM CST (GMT -5)
The most complete answer wins
You are allowed to edit your answer after posting
If two answers are too similar for one to win, the one with the earlier posting time wins
Be specific and be thoughtful
Anonymous entries are allowed, please email them to dcowen@g-cpartners.com. Please state in your email if you would like to be anonymous or not if you win.
In order for an anonymous winner to receive a prize they must give their name to me, but i will not release it in a blog post

The Challenge:

Read the following blogpost: https://www.hecfblog.com/2015/12/how-to-install-dfvfs-on-windows-without.html and then write a script in your choice of scripting language that will pull down and install those packages for a user. Second the script should then run the dfVFS testing script shown in this video https://www.hecfblog.com/2016/04/daily-blog-375-video-blog-showing-how.html to validate the install.

Also Read:

Automating DFIR with dfVFS Part 6
Automating DFIR with dfVFS Part 5
Automating DFIR with dfVFS Part 4
Automating DFIR with dfVFS Part 3
Automating DFIR with dfVFS Part 2
Automating DFIR with dfVFS Part 1

Automating DFIR with dfVFS part 4 by David Cowen - Hacking Exposed Computer Forensics Blog

Hello Reader,
In our last entry in this series we took our partition listing script and added support for raw images. Now our simple script should be able to work with forensic images, virtual disks, raw images and live disks.

If you want to show your support for my efforts, there is an easy way to do that.

Vote for me for Digital Forensic Investigator of the Year here: https://forensic4cast.com/forensic-4cast-awards/

Now that we have that working let's actually get it to do something useful, like extract a file.

First let's look at the code now:

import sys
import logging

from dfvfs.analyzer import analyzer
from dfvfs.lib import definitions
from dfvfs.path import factory as path_spec_factory
from dfvfs.volume import tsk_volume_system
from dfvfs.resolver import resolver
from dfvfs.lib import raw

source_path="stage2.vhd"

path_spec = path_spec_factory.Factory.NewPathSpec(
          definitions.TYPE_INDICATOR_OS, location=source_path)

type_indicators = analyzer.Analyzer.GetStorageMediaImageTypeIndicators(
          path_spec)

if len(type_indicators) > 1:
  raise RuntimeError((
      u'Unsupported source: {0:s} found more than one storage media '
      u'image types.').format(source_path))

if len(type_indicators) == 1:
  path_spec = path_spec_factory.Factory.NewPathSpec(
      type_indicators[0], parent=path_spec)

if not type_indicators:
  # The RAW storage media image type cannot be detected based on
  # a signature so we try to detect it based on common file naming
  # schemas.
  file_system = resolver.Resolver.OpenFileSystem(path_spec)
  raw_path_spec = path_spec_factory.Factory.NewPathSpec(
      definitions.TYPE_INDICATOR_RAW, parent=path_spec)

  glob_results = raw.RawGlobPathSpec(file_system, raw_path_spec)
  if glob_results:
    path_spec = raw_path_spec

volume_path_spec = path_spec_factory.Factory.NewPathSpec(
        definitions.TYPE_INDICATOR_TSK_PARTITION, location=u'/',
        parent=path_spec)

volume_system = tsk_volume_system.TSKVolumeSystem()
volume_system.Open(volume_path_spec)

volume_identifiers = []
for volume in volume_system.volumes:
  volume_identifier = getattr(volume, 'identifier', None)
  if volume_identifier:
    volume_identifiers.append(volume_identifier)
 
print(u'The following partitions were found:')
print(u'Identifier\tOffset\t\t\tSize')

for volume_identifier in sorted(volume_identifiers):
  volume = volume_system.GetVolumeByIdentifier(volume_identifier)
  if not volume:
    raise RuntimeError(
        u'Volume missing for identifier: {0:s}.'.format(volume_identifier))

  volume_extent = volume.extents[0]
  print(
      u'{0:s}\t\t{1:d} (0x{1:08x})\t{2:d}'.format(
          volume.identifier, volume_extent.offset, volume_extent.size))

print(u'')

path_spec = path_spec_factory.Factory.NewPathSpec(
        definitions.TYPE_INDICATOR_TSK_PARTITION, location=u'/p1',
        parent=path_spec)

mft_path_spec = path_spec_factory.Factory.NewPathSpec(
        definitions.TYPE_INDICATOR_TSK, location=u'/$MFT',
        parent=path_spec)

file_entry = resolver.Resolver.OpenFileEntry(mft_path_spec)


stat_object = file_entry.GetStat()

print(u'Inode: {0:d}'.format(stat_object.ino))
print(u'Inode: {0:s}'.format(file_entry.name))
extractFile = open(file_entry.name,'wb')
file_object = file_entry.GetFileObject()

data = file_object.read(4096)
while data:
          extractFile.write(data)
          data = file_object.read(4096)

extractFile.close
file_object.close()

The first thing I changed was what image I'm working from back to stage2.vhd.

source_path="stage2.vhd"

At this point though you should be able to pass it any type of supported image.

Next after the code we first wrote to list out the partitions within an image we added a new path specification layer to make an object that points to the first partition within the image.

path_spec = path_spec_factory.Factory.NewPathSpec(
definitions.TYPE_INDICATOR_TSK_PARTITION, location=u'/p1',
parent=path_spec)

You can see we are using the type of TSK_PARTITION again because we know this is a partition but the location has changed from the prior type we made a parition path spec object. This is because our prior object pointed to the root of the image so we could iterate through the partitions and the new object is referencing just the 1st partition.

Next we make another path specification object that build on the partition type object.

mft_path_spec = path_spec_factory.Factory.NewPathSpec(
definitions.TYPE_INDICATOR_TSK, location=u'/$MFT',
parent=path_spec)

Here we are creating a TSK object and telling it that we want it to point to the file $MFT at the root of the file system. Notice we didn't have to tell it the kind of file system, offsets to where it begins or any other data. The resolver and analyzer helper classes within dfVFS will figure all out of that out for us, if it can. In tomorrows post we will put in some more conditional code to detect when it can in fact not do that for us.

So now that we have a path spec object was a reference to a file we want to work with let's get an object for that file.

file_entry = resolver.Resolver.OpenFileEntry(mft_path_spec)

The resolver helper class OpenFileEntry function takes the path spec object we made that points to the $MFT and if it can access it will return an object that references it.

Next we are going to gather some data about the file we are accessing.

stat_object = file_entry.GetStat()

First we used the GetStat function available from the file entry object to return information about the file into a new object called stat object. This is similar to running the stat command on a file.

Next we are going to print what I'm refering to below as the Inode number:

print(u'Inode: {0:d}'.format(stat_object.ino))

MFT's don't have Inodes this is actually the MFT record number but the concept is the same. We are calling the stat_object property ino to access the mft record number. You could also access the size of the file, dates associated and other data but this is a good starting place.

Next we want to print the name of the file we are accessing.

print(u'Inode: {0:s}'.format(file_entry.name))

The file_entry object property contains the name. This is much easier than with pyTsk where we had to walk a meta sub object property structure to get the file name out.

Now we need to open a file handle to where we want to put the MFT data out to

extractFile = open(file_entry.name,'wb')

Notice two things. One we are using the file_entry.name property directly in the open file handle call, this means our extracted file will have the same name as the file in the image. Two we are passing in the options wb which means that the file handle can be written to, and when it is written to should be treated as a binary file. This is important in Windows systems as when you write out binary data any new lines could be interpreted unless you pass in the binary mode flag.

Now we need to interact with not just the properties of the file in the image, but what data its actually storing

file_object = file_entry.GetFileObject()

We do that by calling the GetFileObject function from the file_entry object. This is giving us a file object just like extractFile that normal python functions can read from. The file handle is being stored in the variable file_object.

Now we need to read the data from the file in the image and then write it out to a file on the disk.

data = file_object.read(4096)
while data:
extractFile.write(data)
data = file_object.read(4096)

First we need to read from the file handle we opened to the image. We are going to do that for 4k of data and then enter a while loop. The while loop is saying as long as there is data being read from the read call to file_object to keep reading 4k chunks. When we reach the end of the file our data variable will contain a null return and the while loop will stop iterating.

While there is data the write function on the extractFile handle will write the data we read and then we will read the next 4k chunk and iterate through the loop again.

Lastly for good measure we are going to close the handle to both file within the image and the file we are writing to on our local disk.

extractFile.close
file_object.close()

And that's it!

In future posts we are going to access volume shadow copies, take command line options, iterate through multiple partitions and directories and add a GUI. Lot's to do but we will do it one piece at a time.

You can download this posts code here on GitHub:

https://github.com/dlcowen/dfirwizard/blob/master/dfvfsWizardv3.py

This is a 6-part series and here's link to all the parts:

Automating DFIR with dfVFS Part 6
Automating DFIR with dfVFS Part 5
Automating DFIR with dfVFS Part 4
Automating DFIR with dfVFS Part 3
Automating DFIR with dfVFS Part 2
Automating DFIR with dfVFS Part 1

Automating DFIR with dfVFS part 3 by David Cowen - Hacking Exposed Computer Forensics Blog

Hello Reader,
In our last post I expanded on the concept of path specification objects. Now let's expand the support of our dfVFS code to go beyond just forensic images and known virtual drives to live disks and raw images.

If you want to show your support for my efforts, there is an easy way to do that.

Vote for me for Digital Forensic Investigator of the Year here: https://forensic4cast.com/forensic-4cast-awards/

Why is this not supported with the same function call you ask? Live disks and raw images do not have any magic headers that dfVFS can parse and know what it is dealing with. So instead we need to add some conditional logic to help it know when to test if what we are working with is an image or a raw disk.

First as we did last time let's see what the code looks like now:

import sys
import logging

from dfvfs.analyzer import analyzer
from dfvfs.lib import definitions
from dfvfs.path import factory as path_spec_factory
from dfvfs.volume import tsk_volume_system
## Adding Resolver
from dfvfs.resolver import resolver
## Adding raw support
from dfvfs.lib import raw

source_path="dfr-16-ntfs.dd"

path_spec = path_spec_factory.Factory.NewPathSpec(
          definitions.TYPE_INDICATOR_OS, location=source_path)

type_indicators = analyzer.Analyzer.GetStorageMediaImageTypeIndicators(
          path_spec)

if len(type_indicators) > 1:
  raise RuntimeError((
      u'Unsupported source: {0:s} found more than one storage media '
      u'image types.').format(source_path))

if len(type_indicators) == 1:
  path_spec = path_spec_factory.Factory.NewPathSpec(
      type_indicators[0], parent=path_spec)

if not type_indicators:
  # The RAW storage media image type cannot be detected based on
  # a signature so we try to detect it based on common file naming
  # schemas.
  file_system = resolver.Resolver.OpenFileSystem(path_spec)
  raw_path_spec = path_spec_factory.Factory.NewPathSpec(
      definitions.TYPE_INDICATOR_RAW, parent=path_spec)

  glob_results = raw.RawGlobPathSpec(file_system, raw_path_spec)
  if glob_results:
    path_spec = raw_path_spec

volume_system_path_spec = path_spec_factory.Factory.NewPathSpec(
        definitions.TYPE_INDICATOR_TSK_PARTITION, location=u'/',
        parent=path_spec)

volume_system = tsk_volume_system.TSKVolumeSystem()
volume_system.Open(volume_system_path_spec)

volume_identifiers = []
for volume in volume_system.volumes:
  volume_identifier = getattr(volume, 'identifier', None)
  if volume_identifier:
    volume_identifiers.append(volume_identifier)
 
print(u'The following partitions were found:')
print(u'Identifier\tOffset\t\t\tSize')

for volume_identifier in sorted(volume_identifiers):
  volume = volume_system.GetVolumeByIdentifier(volume_identifier)
  if not volume:
    raise RuntimeError(
        u'Volume missing for identifier: {0:s}.'.format(volume_identifier))

  volume_extent = volume.extents[0]
  print(
      u'{0:s}\t\t{1:d} (0x{1:08x})\t{2:d}'.format(
          volume.identifier, volume_extent.offset, volume_extent.size))

print(u'')

The first thing is different is two more helper functions from dfVFS being imported:

## Adding Resolver
from dfvfs.resolver import resolver
## Adding raw support
from dfvfs.lib import raw

The first one, resolver, is a helper function that attempts to resolve path specification
objects to file system objects. You might remember that in pytsk the first thing we did
after getting a volume object was to get a file system object. Resolver is doing this for us.

The second is 'raw'. Raw is the class that supports raw images in dfVFS. It defines the
rawGlobalPathSpec function that creates a special path specification object for raw images.

Next we are changing what image we are working with to a raw image:

source_path="dfr-16-ntfs.dd"

We are now ready to deal with a raw image aka a dd image or live disk/partition.

First we are going to change the conditional logic around our type indicator helper function call.
In the first version of the script we knew the type of image we were dealing with so we didn't bother
testing what the type_indicator function returned. Now we could be dealing with multiple types of
images (forensic image, raw image, unkown types) so we need to put in some conditional testing to deal with it.

if len(type_indicators) > 1: 
  raise RuntimeError((
      u'Unsupported source: {0:s} found more than one storage media '
      u'image types.').format(source_path))

if len(type_indicators) == 1:
  path_spec = path_spec_factory.Factory.NewPathSpec(
      type_indicators[0], parent=path_spec)

The first check we do with what is returned into type_indicators is see is more than one type has
been identified. Currently dfVFS only supports one type of image within a single file. I'm not quite
sure when this would happen but its prudent to check for. If this condition were to occur we are calling the built in raise function to call a 'RunTimeError' printing a message to the user that we don't support this type of media.

The second check is what we saw in the first example, there is one known type of media stored within this image. You can tell we are checking for 1 type because we are calling the length function on the type_indicators list object and checking to see if the length is 1.We are going to use what is returned ([0] refers to first element returned in the list contained within type_indicators) and create our path_spec object for the image. One thing does change here and that is we are no longer giving what is returned from the NewPathSpec function into a new variable. Instead we are taking advantage of the layering described in the prior post to store the new object into the same variable name knowing that the prior object has been layered in with the parent being set to path_spec.

Only two more changes and our script is done. Next we need to check to see if there are no known media format stored in type_indicators. We do that by checking to see if nothing is stored inside of the variable type_indicators using the if not operator. This basically says if the type_indicator variable is null (nothing was returned from the function called to populate it) run the following code.

if not type_indicators:
  # The RAW storage media image type cannot be detected based on
  # a signature so we try to detect it based on common file naming
  # schemas.
  file_system = resolver.Resolver.OpenFileSystem(path_spec)
  raw_path_spec = path_spec_factory.Factory.NewPathSpec(
      definitions.TYPE_INDICATOR_RAW, parent=path_spec)

There are two things that code is going to do if there is no type returned, indicating this is possibly a raw image. The first is to call the resolver helper class function OpenFileSystem with the path_spec object we have made. If this is is successful that we are creating a new path specification object and manually setting the type of the object we are layering on to be TYPE_INDICATOR_RAW or a raw image.

Last change we make is taking that new raw image path specification and making it work with the other dfVFS functions that may not explicitly work with a raw image object. We do that be calling the raw function's RawGlobPathSpec function and passing it two objects. The first is the file system object we made in the section just above and the second is the raw_path_spec object we made. The RawGlobPathSpec object is then going to bundle those objects up and if it is successful it will return a new path specification object that the rest of the library will work with.

  glob_results = raw.RawGlobPathSpec(file_system, raw_path_spec)
  if glob_results:
    path_spec = raw_path_spec

We then test the glob_results variable to make sure something was stored within it, a sign it ran successfully. If there is in fact an object contained within it we assign it to our path_spec variable.

That's it!

After running the script this should be what you see:

The following partitions were found:
Identifier Offset Size
p1 65536 (0x00010000) 314572800

You can download the image I'm testing with here: http://www.cfreds.nist.gov/dfr-images/dfr-16-ntfs.dd.bz2

You can download the source code for this example from GitHub here: https://github.com/dlcowen/dfirwizard/blob/master/dfvfsWizardv2.py

Tomorrow we continue to add more functionality!

This is a 6-part series and here's link to all the parts:

Automating DFIR with dfVFS Part 6
Automating DFIR with dfVFS Part 5
Automating DFIR with dfVFS Part 4
Automating DFIR with dfVFS Part 3
Automating DFIR with dfVFS Part 2
Automating DFIR with dfVFS Part 1

Automating DFIR with dfVFS part 2 by David Cowen - Hacking Exposed Computer Forensics Blog

Hello Reader,
In this short post I want to get more into the idea of the path specification object we made in the prior part. If this post had a catch title it would be zen and the art of path specification.

If you want to show your support for my efforts, there is an easy way to do that.

Vote for me for Digital Forensic Investigator of the Year here: https://forensic4cast.com/forensic-4cast-awards/

In the prior post, part 1 of the series, we made three path specification objects. I described path specification objects as the corner stone in understand dfVFS which I believe to be true. What I didn't point out is that the path specification objects in that first example code where building on top of themselves like a layer cake.

Let's take a look at the three objects we created again.

path_spec = path_spec_factory.Factory.NewPathSpec(
          definitions.TYPE_INDICATOR_OS, location=source_path)

source_path_spec = path_spec_factory.Factory.NewPathSpec(
            type_indicators[0], parent=path_spec)


volume_system_path_spec = path_spec_factory.Factory.NewPathSpec(
        definitions.TYPE_INDICATOR_TSK_PARTITION, location=u'/',
        parent=source_path_spec)

If you were to look carefully you would notice there are a couple of differences between the calls to the NewPathSpec function. the calls to the NewPathSpec function.

1. The type of path specification we are making is changing. We start with a operating system
file, then an image (which is being set by the return of our indicator query) and lastly
we are working with a partition.
2. Two of our path specifications declare a location, one does not
3. Most importantly, source_path_spec and volume_System_path_spec have parents. Those parents
are the path specification objects created prior.

So if you were to look at it like one single object with multiple layers it would look
something like this.

---------------------------
|  OS File Path Spec       |
---------------------------
|  TSK Image Type Path Spec |
----------------------------
|  TSK Partition Path Spec  |
----------------------------

The lowest layer in the object can reference the upper layers. This is why we don't just create one path specification object. Instead we are initializing each layer of the object one call at a time as we determine the type of image, directory, archive, etc.. we are working with to allow our path specification object to reflect the data we are trying to get dfVFS to work with.

Depending on what part of the dfVFS framework you are working with will determine how many of these layers need to exist prior to calling that function with your fully developed path specification object.

As we go father into the series I will show you how to interact with the files stored in the partitions we listed in part 1. Doing that will create yet another layer to our object, the file system layer. This is very similar to how we built our objects in pyTSK.

If you want to read how Metz explains Path Specification objects you can read about them here:

https://github.com/log2timeline/dfvfs/wiki/Internals

Tomorrow I will explain how we access raw images and then Thursday we will extract a file from an image.

This is a 6-part series and here's link to all the parts:

Automating DFIR with dfVFS Part 6
Automating DFIR with dfVFS Part 5
Automating DFIR with dfVFS Part 4
Automating DFIR with dfVFS Part 3
Automating DFIR with dfVFS Part 2
Automating DFIR with dfVFS Part 1

Subscribe to: Posts ( Atom )

Building your own travel sized virtual lab with ESXi and the Intel SkullCanyon NUC

Daily Blog #381 National CCDC Redteam Debrief

Daily Blog #380: National CCDC 2016

Daily Blog #379: Automating DFIR with dfVFS Part 6

Daily Blog #378: Automating DFIR with dfVFS part 5

Daily Blog #377: Sunday Funday 4/17/16 - dfVFS Testing Script Challenge

Daily Blog #376: Saturday Reading 4/16/16

Daily Blog #375: Video Blog showing How to verify and test your dfVFS install

Daily Blog #374: Automating DFIR with dfVFS part 4

Daily Blog #373: Automating DFIR with dfVFS part 3

Daily Blog #372: Automating DFIR with dfVFS part 2

Top Posts/Right Now