Sunday, May 10, 2015

Automating DFIR - How to series on programming libtsk with python Part 12

Hello Reader,
      How has a month passed since this last entry? To those reading this I do intend to continue this series for the foreseeable future so don't give up! In this part we will talk about accessing non NTFS partitions and the file system support available to you in pytsk. 

Now before we continue a reminder, don't start on this post! We've come a long way to get to this point and you should start at part 1 if you haven't already!

Part 1 - Accessing an image and printing the partition table
Part 2 - Extracting a file from an image
Part 3  - Extracting a file from a live system
Part 4 - Turning a python script into a windows executable
Part 5 - Auto escalating your python script to administrator
Part 6 - Accessing an E01 image and extracting files
Part 7 - Taking in command line options with argparse to specify an image
Part 8 - Hashing a file stored in a forensic image
Part 9 - Recursively hashing all the files in an image
Part 10 - Recursively searching for files and extracting them from an image
Part 11 - Recursively searching for files and extracting them from a live system 

Following this post the series continues

Part 13 - Accessing Volume Shadow Copies 

In this series so far we've focused on NTFS because that's where most of us spend our time investigating. However, the world is not Windows alone and luckily for us the sleuthkit libraries that pytsk binds to is way ahead of us. Here is the full list of file systems that the sleuthkit library supports:

  • NTFS 
  • FAT12 
  • FAT16 
  • FAT32 
  • exFAT
  • UFS1 (FreeBSD, OpenBSD, BSDI ...)
  • UFS1b (Solaris - has no type)
  • UFS2 - FreeBSD, NetBSD.
  • Ext2 
  • Ext3 
  • SWAP 
  • RAW 
  • ISO9660 
  • HFS 
  • Ext4 
  • YAFFS2 


                                Now that is what the current version of the sleuthkit supports, pytsk3 however is compiled against an older version. I've tested the following file systems to have worked with pytsk3
                                • NTFS
                                • FAT12
                                • FAT16
                                • FAT32
                                • EXT2
                                • EXT3
                                • EXT4
                                • HFS
                                Based on my testing I know that ExFAT does not appear to be supported in this binding and the testing of reader Hans-Peter Merkel I know that YAFFS2 is also not currently supported. I'm sure in the future when the binding is updated these file systems will then come into scope. 

                                So now that we know what we can expect to work let's change our code from part 10 to work against any supported file system, and allow it to work with multiple image types to boot. 

                                What you will need:
                                I'm using a couple of sample images from the CFReDS project for this part as they have a different partition for different variants of the same file system type. For instance the ext sample image we will use has ext2, ext3, and ext4 partitions all on one small image. Pretty handy! 

                                The first thing we will need to change is how we are opening our images to support multiple image types. We are going to be working with raw and e01 images all the time and keeping two separate programs to work with each seems dumb. Let's change our code to allow us to specify which kind of image we are working with and in the future we may automate that as well!

                                We need to add a new required command line option where we specify the type of image we are going to be working with:

                                argparser.add_argument(        '-t', '--type',
                                        dest='imagetype',
                                        action="store",
                                        type=str,
                                        default=False,
                                        required=True,
                                        help='Specify image type e01 or raw'
                                    )



                                We are defining a new flag (-t or --type) to pass in the type of image we are dealing with. We are then storing our input into the variable imagetype and making this now required. 

                                Now we need to test our input and call the proper Image Info class to deal with it. First let's deal with the e01 format. We are going to move all the pyewf specific code into this if block:
                                if (args.imagetype == "e01"):  filenames = pyewf.glob(args.imagefile)  ewf_handle = pyewf.handle()  ewf_handle.open(filenames)

                                Next we are going to define the code to work with raw images:

                                elif (args.imagetype == "raw"):    print "Raw Type"
                                    imagehandle = pytsk3.Img_Info(url=args.imagefile)



                                One last big change to make and all the rest of our code will work:
                                for partition in partitionTable:  print partition.addr, partition.desc, "%ss(%s)" % (partition.start, partition.start * 512), partition.len  try:        filesystemObject = pytsk3.FS_Info(imagehandle, offset=(partition.start*512))  except:          print "Partition has no supported file system"          continue  print "File System Type Dectected ",filesystemObject.info.ftype

                                We are moving our FS_Info call to open the file system into a try/except block so that if the partition type is not supported our program won't exit on error. Why do we have to test each partition? Because 1. We can't trust the partition description to always tell us what file system is in it, Windows 8 for instance changed them, and the only method tsk makes available to us to determine the file system is within the FS_Info Class. So we will see if pytsk supports opening any partition we find and if it does not we will print an error to the user. It we do support the file system type then we will print the type detected and the rest of our code will work with no changes needed!

                                Le's see what this looks like on each of the example images I linked at the beginning of the post.

                                FAT:
                                E:\development>python dfirwizard-v11.py -i dfr-01-fat.dd -t rawRaw Type0 Primary Table (#0) 0s(0) 1Partition has no supported file system1 Unallocated 0s(0) 128Partition has no supported file system2 DOS FAT12 (0x01) 128s(65536) 16384File System Type Dectected  TSK_FS_TYPE_FAT16Directory: /3 DOS FAT16 (0x06) 16512s(8454144) 65536File System Type Dectected  TSK_FS_TYPE_FAT16Directory: /4 Win95 FAT32 (0x0b) 82048s(42008576) 131072File System Type Dectected  TSK_FS_TYPE_FAT32Directory: /5 Unallocated 213120s(109117440) 1884033Partition has no supported file system

                                Ext:
                                E:\development>python dfirwizard-v11.py -i dfr-01-ext.dd -t raw
                                Raw Type
                                0 Primary Table (#0) 0s(0) 1
                                Partition has no supported file system
                                1 Unallocated 0s(0) 61
                                Partition has no supported file system
                                2 Linux (0x83) 61s(31232) 651175
                                File System Type Dectected  TSK_FS_TYPE_EXT2
                                Directory: /
                                Cannot retrieve type of Bellatrix.txt
                                3 Linux (0x83) 651236s(333432832) 651236
                                File System Type Dectected  TSK_FS_TYPE_EXT3
                                Directory: /
                                4 Linux (0x83) 1302472s(666865664) 651236
                                File System Type Dectected  TSK_FS_TYPE_EXT4
                                Directory: /
                                5 Unallocated 1953708s(1000298496) 143445
                                Partition has no supported file system

                                HFS:
                                E:\development>python dfirwizard-v11.py -i dfr-01-osx.dd -t raw
                                Raw Type
                                0 Safety Table 0s(0) 1
                                Partition has no supported file system
                                1 Unallocated 0s(0) 40
                                Partition has no supported file system
                                2 GPT Header 1s(512) 1
                                Partition has no supported file system
                                3 Partition Table 2s(1024) 32
                                Partition has no supported file system
                                4 osx 40s(20480) 524360
                                File System Type Dectected  TSK_FS_TYPE_HFS_DETECT
                                Directory: /
                                5 osxj 524400s(268492800) 524288
                                File System Type Dectected  TSK_FS_TYPE_HFS_DETECT
                                Directory: /
                                6 osxcj 1048688s(536928256) 524288
                                File System Type Dectected  TSK_FS_TYPE_HFS_DETECT
                                Directory: /
                                7 osxc 1572976s(805363712) 524144
                                File System Type Dectected  TSK_FS_TYPE_HFS_DETECT
                                Directory: /
                                8 Unallocated 2097120s(1073725440) 34
                                Partition has no supported file system

                                There you go! We can now search, hash and extract from most things that come our way. In the next post we are going back to Windows and dealing with Volume Shadow Copies!

                                Follow the github repo here: https://github.com/dlcowen/dfirwizard