Daily Blog #557: Changes in the NtfsDisableLastAccessUpdate key

Update 12/6/18: It turns out that my test system had a system volume smaller than 128gb in size meaning the last access dates were enabled (setting 2). According to @errno_faiil (Maxim Suhanov) if my system driver was larger than 128gb then the last access dates would be disabled (setting 3).

Want to know more? Watch this video: https://www.youtube.com/watch?v=yHG6MEH99Z0

Hello Reader,
        It looks like as of at least Windows 10 1803 a new change has come to an old registry key. The NtfsDisableLastAccessUpdate key found in 'SYSTEM\CurrentControlSet\Control\FileSystem' no longer is just a true/false 1/0 value. It now has four possible values stating how the access dates in NTFS were enabled or disabled.

Looking at my laptop's registry I can see the following value is currently set:

which leads to the question of... what does 80000002 mean? Luckily fsutil will translate the current value for us:

So the 8 appears to be some kind of upper bit masking while the 2 is the value set letting us know that NTFS Access updates are currently disabled by system policy.

Checking the set behavior command in fsutil shows us all the possible documented options:

As you can see we've moved from two possible states (on/off, true/false, 0/1) to four. The system is now tracking if the user or the system has enabled or disable last access dates in NTFS.

Why? I have no idea currently but it certainly does add more context to the decision. So all of you who have tools that interpret this value will need to update your tools!

Also Read: Daily Blog #556

Read More

Automating DFIR - How to series on programming libtsk with python Part 11

Hello Reader,
      I had a bit of a break thanks to a long overdue vacation but I'm back and the code I'll be talking about today has been up on the github repository for almost 3 weeks, so if you ever want to get ahead go there as I write the code before I try to explain it! Github repository is here: https://github.com/dlcowen/dfirwizard

Now before we continue a reminder, don't start on this post! We've come a long way to get to this point and you should start at part 1 if you haven't already!

Part 1 - Accessing an image and printing the partition table
Part 2 - Extracting a file from an image
Part 3  - Extracting a file from a live system
Part 4 - Turning a python script into a windows executable
Part 5 - Auto escalating your python script to administrator
Part 6 - Accessing an E01 image and extracting files
Part 7 - Taking in command line options with argparse to specify an image
Part 8 - Hashing a file stored in a forensic image
Part 9 - Recursively hashing all the files in an image
Part 10 - Recursively searching for files and extracting them from an image

Following this post the series continues:

Part 12 - Accessing different file systems
Part 13 - Accessing Volume Shadow Copies  

In this post we are going to augment the script on part 10 which went through an image and search/extracted files from all the NTFS partition in an image, and now we are going to do the same against all the NTFS partitions on a live system. You can obviously tweak this for any other file system but we will get to that in later posts in this series.

The first thing we need is a way to figure out what partitions exist on a live system in cross platform way so our future code can be tweaked to run anywhere. For this I choose the python library psutil which can provide a wealth of information about a system its running on, included information about available disks and partitions, you can read all about it here: https://pypi.python.org/pypi/psutil

To bring it into our program we need to call the import function again:

import psutil

and then because we are going to work against a live running system again we need our old buddy admin

import admin

which if you remember from part 5 will auto escalate our script to administrator just in case we forgot to run it as such.

We are going to strip out the functions we need to find all the parts of a forensic image and replace it with out code to test for administrative access:

if not admin.isUserAdmin():
  admin.runAsAdmin()
  sys.exit()

Next we replace the functions we called to get a partition table from a forensic image with a call to psutil to return a listing of paritions and iterate through them. The code looks like the following which I will explain:

partitionList = psutil.disk_partitions()
for partition in partitionList:
  imagehandle = pytsk3.Img_Info('\\\\.\\'+partition.device.strip("\\"))
  if 'NTFS' in partition.fstype:

So here instead of calling pytsk for a partition table we are calling psutil.disk_partitions which will return a list of partitions that are available to the local system. I much prefer this method than trying to iterate through all volume letters as we will get back just those partitions available as well as what file system they are seen as running as. Our list of active partitions will be stored in the varaible partitionList. Next we will iterate through the partitions using the for operator storing each partition returned into the partition variable. Next we are creating a pytsk3 Img_Info object for each partition returned but only continuing if psutil recognized the partition is NTFS.

The next thing we are changing is our try catch blog in our recursive directory function. Why? I found in my testing that live systems react much differently than forensic images in setting certain values in libtsk. So rather than using entryObject.info.meta.type to determine if I'm dealing with a regular file I am using entryObject.info.name.type which seem to always be set regardless if its a live system or a forensic image. I'm testing to see if I can capture the type of the file and it's size here as there are a lot of interesting special files that only appear at run time that will throw an error if you try to get their size. 

try:
        f_type = entryObject.info.name.type
        size = entryObject.info.meta.size
      except Exception as error:
          print "Cannot retrieve type or size of",entryObject.info.name.name
          print error.message
          continue

So in the above code I'm getting the type of file (lnk, regular, etc..) and it's size and if I can't I'm handling the error and printing out the error before continuing on. You will see errors, live systems are an interesting place to do forensics.

I am now going to make a change I alluded to earlier on in the series. We are going to buffer out reads and writes so we don't crash our of our program because we are trying to read a massive file into memory. This wasn't a problem in our first examples as we were working from small test images I made before, but now they we are dealing with real systems and real data we need to handle our data with care.

Our code looks as follows:

            BUFF_SIZE = 1024 * 1024
            offset=0
            md5hash = hashlib.md5()
            sha1hash = hashlib.sha1()
            if args.extract == True:
                  if not os.path.exists(outputPath):
                    os.makedirs(outputPath)
                  extractFile = open(outputPath+entryObject.info.name.name,'w')
            while offset < entryObject.info.meta.size:
                available_to_read = min(BUFF_SIZE, entryObject.info.meta.size - offset)
                filedata = entryObject.read_random(offset,available_to_read)
                md5hash.update(filedata)
                sha1hash.update(filedata)
                offset += len(filedata)
                if args.extract == True:
                  extractFile.write(filedata)

            if args.extract == True:
                extractFile.close

First we need to determine how much data we want to read or write at one time from a file. I've copied several other examples I've found and I'm setting that amount to 1 meg of data at a time by setting the variable BUFF_SIZE equal to 1024*1024 or one megabyte. Next we need to keep track of where we are in the file we are dealing with, we do that by creating a new variable called offset and setting the offset to 0 to start with.

You'll notice that we are creating our hash objects, directories and file handles before we read in any data. That is because we want to do all of these things one time prior to iterating through the contents of a file. If a file is a gigabyte in size then our function will be called 1,024 times and we just want one hash and one output file to be created.

Next we starting a while loop which will continue to execute until our offset is greater or equal to the size of our file, meaning we've read all the data within it. Now files are not guaranteed to be allocated in 1 meg chunks, so to deal with that we are going to take advantage of a python function called min. Min returns the smaller of to values presented which in our code is the size of the buffer compared to the remaining data left to read (the size of the file - our current offset). Whichever value is smaller will be stored in the variable available_to_read.

After we know how much data we want to read in this execution of our while loop we are going to read it as before from our entryObject passing in the offset to start from and how much data to read, storing the data read into the variable filedata. We are then calling the update function provided by our hashing objects. One of the nice things our the hashlibs provided by python is that if you provide additional data to an already instantiated object it will just continue to build the hash rather than having to read it all in at once.

Next we are incrementing our offset by adding to itself the length of data we just read so we will skip past it on the next while loop execution. Finally we write the data out to our output file if we elected to extract the files we are searching for.

I've added one last bit of code to help me catch any other weirdness that may seep through.

        else:
          print "This went wrong",entryObject.info.name.name,f_type

An else to look for any condition that does not match one of existing if statements.

That's it! You now have a super DFIR Wizard program that will go through all the active NTFS partitions on a running system and pull out and hash whatever files you want!

You can find the complete code here: https://github.com/dlcowen/dfirwizard/blob/master/dfirwizard-v10.py

In the next post we will talk about parsing partitions types other than NTFS and then go into volume shadow copy access!

Continue Reading: Automating DFIR - How to series on programming libtsk with python Part 10

Read More

Daily Blog #184: Artifacts from alternative file system drivers on NTFS Part 4

Hello Reader,
       In this series we've explored the POSIX namespace, how the ntfs-3g driver uses it, what default system files use it and the win32 api's interaction with it. Today let's focus on what additional artifacts exist soley within the MFT that in combination with the POSIX namespace let us identify absolutely that a non native NTFS driver wrote to the disk.

To accomplish a unique signature that reflects the actions that ntfs-3g takes when writing to a NTFS volume we need to examine three fields within a MFT file record. If you want to see this in a more interactive fashion watch last weeks Forensic Lunch where we walked through it.

1. Namespace

The Namespace can be one of 4 things that determines the encoding of the filename being stored there.The namespace as we discussed previously will be Posix or File Name Namespace 0. This on its own though does not identify a ntfs-3g written file as we've discussed in this series.

2.  LSN

The LSN or Logfile Sequence Number references the most recent change stored within the $logfile. The LSN in a native windows system writing to NTFS has full support for the $logfile and will populate this field to reflect the record entry made. The ntfs-3g driver only updates the restart area and does not populate the $logfile, because of this the LSN value will be 0 for all ntfs-3g written files. If you are looking at a pre vista system then the LSN and Namespace are the only two correlation points you have to identify ntfs-3g written files.

3. USN

The USN or Update Sequence Number references those entries written into the $USNJRNL:$J. We've talked about the USN many times in this blog and hopefully you are familiar with the basic functionality by now. In our testing we were expecting this value to be set to 0 just like the LSN but instead a 64 bit value will be assigned, we are still examining the source to determine the method use in the numbers duration but they do to seem to increase but can be duplicated. The USN values used are outside the range of valid USN Journals that we've seen. The USN number is also the offset into the USN Journal to where that last change has been recorded.

So there we go.
If you are looking at a Windows 2000/XP/2003 system than the Namespace and LSN are your points of analysis to determine if a file was written using the ntfs-3g driver.

If you are looking at a Windows Vista/7/2008 system than the Namespace, LSN and USN fields will determine if a file was written using the ntfs-3g driver.

We haven't tested windows 8 yet but will do so and write a blog to reflect when we have done so and solved what the value used in the USN field means.

Make Sure to Read: 

 Artifacts from alternative file system drivers on NTFS Part 3

 Artifacts from alternative file system drivers on NTFS Part 2

 Artifacts from alternative file system drivers on NTFS Part 1

Read More

Daily Blog #179: Artifacts from alternative file system drivers on NTFS Part 3

Hello Reader,
          In the two prior posts in this series we've examined the characteristics of a POSIX file name made by the linux ntfs-3g driver and the POSIX file names we should expect to see in a normal windows system. Today we are going to focus on the win32 api's that allow file creation to see which would allow a POSIX file name to be created in the first place.

There are three main functions exposed by the win32 api for file creation:

1. CreateFile

 This function is the main function for opening and creating a file on the disk or for accessing a device such as COM1 or a physical drive. Createfile has support for POSIX naming conventions by passing in the 'FILE_FLAG_POSIX_SEMANTICS' flag in the optional dwFlagsAndAttributes field when creating a file. What is interesting is that this flag when set does not actually create a POSIX namespace file name attribute.

 

2. CreateFile2

 This function appears to be related to Windows Store based win 8 apps that operate within sandboxed environments. There is no stated POSIX support which is interesting. This means I need to test to see what Win8 default files are POSIX.

 

3. CreateFileTransacted

CreateFileTransacted support the same methods as CreateFile, including POSIX, but creates a transactional NTFS stream that file resides in until the transaction is committed.  We are doing research into Transactional NTFS and plan to write more about this later. Interesting to note that this article begins with a warning about the possible deprecation of this functionality in the future.

 

So in my current testing I cannot find a win32 api that creates a POSIX filespace filename attribute. Here is my perl code for calling into the win32 api and createfile:

 

 #!/usr/bin/perl -w

use Win32API::File qw( :ALL );

my $hDisk= Win32API::File::CreateFile( "//./H:/\$PosixTesTingAgain", GENERIC_ALL(),
      FILE_SHARE_READ()|FILE_SHARE_WRITE(), [], CREATE_NEW(), FILE_FLAG_POSIX_SEMANTICS(), [] );
      

I've tried this with a couple variations on file name conventions to force a POSIX only compatible name, but then it just fails. I'm not done yet though and will continue trying to find a function that will allow this namespace to be attributed within windows. 

 

Why? It's important to understand whats possible so we can determine if a user program could ever create a POSIX file name. If we can't, that is a great evidence point in supporting whether a file was created by the linux ntfs-3g driver or windows.

Make Sure to Read: 

 Artifacts from alternative file system drivers on NTFS Part 4

 Artifacts from alternative file system drivers on NTFS Part 2

 Artifacts from alternative file system drivers on NTFS Part 1

Read More

Daily Blog #178: Artifacts from alternative file system drivers on NTFS Part 2

Hello Reader,
           Yesterday we went through the linux ntfs-3g driver's interaction with the MFT in NTFS. If you haven't read that you should as it explains why POSIX filespace's are the focus of today's post. Today I am going to compare the MFT my system and a test system to see how many POSIX file names are created by default so we can determine a set of rules to see if we can ascertain when a file was created by the linux ntfs-3g driver.

I parsed my MFT using mft2csv as we just added the filespace name support to v3 of anjp which we are polishing up for this months beta release. I like mft2csv and think its an easy tool to use when you just care about high detail MFT parsing. My system drive has 628,480 MFT records, its been in active use for over a year with the current install. Of those 628,480 have POSIX filespace records. So having a POSIX namespace in your filename alone does not indicate that the linux-3g driver was used in creating a file. Whats interesting here is that these POSIX filenames break down to some basic categories:

1. System files/directories like \boot and it's directories, $Recycle.bin and $Extend
2. Applications from OEMs like cygwin and hp
3. Shared libraries from visual studio, microsoft common libs and other programs
4. FTK Job work queues
5. The entire QT SDK
6. Windows directories and files
7. The user profile directories for default and public

Why is that interesting? Not a single file that I or program at my direction made in the last year as a POSIX filespace name is stored under my profile.  I want to test this on a non OEM windows install to see if the same number of POSIX filespace file names are created when you install from MS media directly.

So taking the logic one step forward, based on my limited sample set so far there should not be a user created file originally made by the native NTFS driver and using standard win32 system calls that results in a POSIX filespace file names. I am going to take this a step further tomorrow by finding which win32 calls can create POSIX filespace named files and testing this same theory against my virtual machines.

Make Sure to Read: 

 Artifacts from alternative file system drivers on NTFS Part 4

 Artifacts from alternative file system drivers on NTFS Part 3

 Artifacts from alternative file system drivers on NTFS Part 1

Read More

Daily Blog #177: Artifacts from alternative file system drivers on NTFS Part 1

Hello Reader,
        Often times when I talk to security professionals a kind of game arises where they try to come up with a scenario where they can perform an action on a system that we cannot detect in our analysis. Often times these question sessions lead to the idea that the hypothetical will simply mount his NTFS drive in Linux and perform his bad actions there to get around Windows logging and artifacts. 

I've talked before about the Linux driver's lack of support for the $logfile and $Usnjrnl leading to a lack of artifacts that can be correlated but we can now actually go one further.

The inherent limitation when basing the detection of a past event on the journals ($logfile and $USNJrnl) is that they have a finite time of existence before being written over. While the shadow copies will retain them for a period of time they will eventually expire and be overwritten as well. This finite lifespan and further research into MFT internals ( and a very nice tweet from Willi Ballenthin) lead me to an interesting documentation page from the linux3g project: http://inform.pucp.edu.pe/~inf232/Ntfs/ntfs_doc_v0.5/concepts/filename_namespace.html that listed all the available NTFS name spaces for FILENAME attributes.

This then lead to the mount.ntfs-3g man page, http://linux.die.net/man/8/mount.ntfs-3g,  that had the following statement:

"Windows Filename Compatibility

NTFS supports several filename namespaces: DOS, Win32 and POSIX. While the ntfs-3g driver handles all of them, it always creates new files in the POSIX namespace for maximum portability and interoperability reasons. This means that filenames are case sensitive and all characters are allowed except '/' and '\0'. This is perfectly legal on Windows, though some application may get confused. The option windows_names may be used to apply Windows restrictions to new file names."

 So I grabbed the CFReDS project deletion file testing image 11, you can download it here http://www.cfreds.nist.gov/dfr-images/dfr-11-ntfs.dd.bz2 and ran it through mft2csv which I know actually identifies which namespace a filename is set to. From prior testing with the $logfile and confirmation from NIST I knew that the files in this test where created in Linux and then deleted in Windows. This is what mft2csv see's from those files:

Tomorrow we'll talk about what other default files are created as POSIX by windows to prevent false positives and end this series Thursday talking about how user created file could be POSIX.

Make Sure to Read: 

 Artifacts from alternative file system drivers on NTFS Part 4

 Artifacts from alternative file system drivers on NTFS Part 3

 Artifacts from alternative file system drivers on NTFS Part 2

Read More

Daily Blog #75: Forensic Lunch 9/6/13! - Discussion with Eric Zimmerman, Phil Hagel, Lee Whitefield

Hello Reader,
                It's time again for the Forensic Lunch! Today join Eric Zimmerman, Phil Hagel, Lee Whitefield and in the G-C Studios Matt, Nicole, Rebecca and myself!

Topics include:
Forensic Image benchmarking
FUSE and NTFS3g
Network Forensics
HFS+ Journal Parsing
and more!

Also Read: Daily Blog #74

Read More

CEIC 2013 and the public beta of the NTFS TriForce

Greetings Reader!,

                              Thanks to all of you who came in person to my presentation at CEIC this morning, we had a mountain of information to show you and you kept up! We had a standing room only session and lots of great questions were asked.  I'm going to try google drive for all my hosting of session materials this time, I hope it works well!

We had a lot of fun today, we walked attendee's through data structure, four labs showing how to use the Triforce to solve four different forensic scenarios and how to use libvshadow in windows to expose shadow copies that you can extract the $MFT, $Logfile and $USNJRNL::$J from!

I'll be posting blog entries in the next two weeks giving walk throughs of each of the labs and more fun data for everyone to try out our new tool on.

Lastly, its time for the public beta of the TriForce. Please click on the link below to download it and get updated on new versions that we will be releasing as we get closer to a defined product.


Here is the link to the public beta signup:

https://docs.google.com/forms/d/1GzOMe-QHtB12ZnI4ZTjLA06DJP6ZScXngO42ZDGIpR0/viewform


Here are my slides from today:

https://docs.google.com/file/d/0B_mjsPB8uKOARWdvdGRsQlh6V1E/edit?usp=sharing

Here is a link to the labs from today:

https://docs.google.com/file/d/0B_mjsPB8uKOAekQwTmQwYWoyY3M/edit?usp=sharing

Here is a link to the demo video today:

http://www.youtube.com/watch?v=5it4EenSaok&feature=youtu.be&a

Here is a link to download the windows compiled version of libvshadow:

https://docs.google.com/file/d/0B_mjsPB8uKOAMTRrcVJWbC01YzQ/edit?usp=sharing

Also Read: NCCDC 2013 Lessons Learned

Read More