Daily Blog #740: USN V4 Data Ranges

Hello Reader,

In the prior post we talked aboutt the differences between USN v2, v3 and v4. I thought it would be good to a bit more into v4 and why it may be creating evcen more useful data for you. v4 stores 'data ranges' which means that it tells you what sectors on the disk where involved in a change. 

What does that mean for you?

1. If a file was deleted you have the data ranges it previously existed in, even if it was fragmented

2. If a file was changed you can see what blocks may have nay previous partial contents

What gets more interesting is that in the testing done it also appears that in some cases its not just the ranges of data that is being written to the USN Journal but some of the contents of those sectors as well. I'm going to be doing some testing to see if this limited to just resident files or if like EXT4 it will record the contents of all blocks changed!

Also Read: USN Versions (2, 3 and 4)

Read More

Daily Blog #739: USN Versions (2, 3 and 4)

 

 

Hello Reader,

Since I first looked at the USN Journal many years ago and Matthew and I released ANJP to parse it along with other data I've known there where multiple USN versions. I thought this post would be a good place to document what the differences are and when to expect to see them. 

 

USN Journal v2 – The Vista Era

• Introduced:
  With the arrival of Windows Vista, the USN Journal came into existence.
  
• What It Contains:
  At its core, v2 records fundamental details for every change on an NTFS volume: the unique file reference number, parent file reference, a monotonically increasing USN, timestamp, and a set of reason flags indicating why the change occurred.
• Default Status:
  On any NTFS volume running Windows Vista or later, v2 is created not at format but after a certain number of changes occur if windows search indexing is turned on. .
• Documentation: https://learn.microsoft.com/en-us/windows/win32/api/winioctl/ns-winioctl-usn_record_v2
  

USN Journal v3 – Refinements in Windows 8

• Introduced:
  Windows 8 ushered in USN Journal v3.
• What’s New:
  Building on v2’s foundation, v3 expanded the record structure to capture additional metadata. While the basic fields remain, v3 started to include more nuanced details about certain operations—particularly those around renames and changes to alternate data streams. The idea was simple: as our file operations got more sophisticated, our change records needed to do the same.
• Default Status:
  For systems running Windows 8 and later, v3 became the default journal version on NTFS volumes. Again the journal is a subsystem that was meant to assist with drive indexing, backup programs and other utilities that needed to know when things changed and why. If you just format a disk the USN journal will not appear until you have created data that requires tracking. 
• Documentation: https://learn.microsoft.com/en-us/windows/win32/api/winioctl/ns-winioctl-usn_record_v3
  

USN Journal v4 – The Windows 10 Evolution

• Introduced:
  With Windows 10 (in a series of updates that refined NTFS’s internal structures), Microsoft rolled out USN Journal v4.
• What’s New:
  v4 is less about a radical overhaul and more about fine-tuning. It includes extra fields to provide even more granular information about changes—covering aspects such as improved record consistency, additional flags for security-related modifications, and adjustments for better alignment with newer NTFS features. In short, v4 offers a more complete picture of file system activity while ensuring that the data is as robust and future-proof as possible.
• Default Status:
  According to the MSDN documentation V4 records are only read if you Range Tracking is turned on within tghe journal.  .Otherwise my Windows 10 and 11 systems return V3 records. 
• Documentation : https://learn.microsoft.com/en-us/windows/win32/api/winioctl/ns-winioctl-usn_record_v4
  

 

Also Read:  Arsenal Recon LevelDB Recon

Read More

Contents in Sparse Mirror may be Smaller than they Appear

By Matthew Seyer

As many of you know, David Cowen and I are huge fans of file system journals! This love also includes all change journals designed by operating systems such as FSEvents and the $UsnJrnl:$J. We have spent much of our Dev time writing tools to parse the journals. Needless to say, we have lots of experience with file system and change journals. Here is some USN Journal logic for you.

USN Journal Logic

First off it is important to know that the USN Journal file is a sparse file. MSDN explains what a sparse file is: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365564(v=vs.85).aspx. When the USN Journal (referred to as $J from here on out) is created it is created with a max size (the area of allocated data) and an allocation delta (the size in memory that stores records before it is committed to the $J on disk). This is seen here: https://technet.microsoft.com/en-us/library/cc788042(v=ws.11).aspx.

The issue is that many forensics tools cannot export a file out as a sparse file. Even if they could only a few file systems support them and I don’t even know if a sparse file on NTFS is the same as a sparse file on OSX. But this leads to a common problem. The forensic tool sees the $J as larger than it really is:

While this file is 20,380,061,592 bytes in size, the allocated portion of records is much smaller. Most forensic tools will export out the entire file with the unallocated data as 0x00. Which makes sense when you look at the MSDN Sparse File section (link above). When we extract this file out with FTK Imager we can verify with the windows command `fsutil sparse` to see that the exported file is not a sparse file (https://technet.microsoft.com/en-us/library/cc788025(v=ws.11).aspx):

 Trimming the $J

Once its exported out what’s a good way to find the start of the records? I like to use 010 Editor. I scroll towards the end of the file where there are still empty blocks (all 0x00s) then I search for 0x02 as I know I am looking for USN Record version 2:

Now if I want to export out just the record area I can start at the beginning of this found record and select to the end of the file and save the selection as a new file: 

The resulting file is 37,687,192 bytes in size and contains just the record portion of the file.

This is significantly smaller in size! Now, how do we go about this programmatically?

Automation

While other sparse files can have interspersed data, the $J sparse file keeps all of its data at the end of the file. This is because you can associate the Update Sequence Number in the record to the offset of the file! If you want to look at the structure of the USN record here it is: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365722(v=vs.85).aspx. Now I will note that I would go about this two different ways. One method for a file that has been extracted out from one tool and a different method for if it would be extracted out using the TSK lib. But for now, we will just look at the first scenario.

Because the records are located in the lasts blocks of the file, I would start from the end of the file and work our way backwards to find the first record, then write out just the records portion of the file. This saves a lot of time because you are not searching through potentially many gigs of zeros. You can find the code at https://github.com/devgc/UsnTrimmer. I have commented the code so that it is easy to understand what is happening.

Now lets use the tool:

We see that the usn.trim file is the same as the one we did manually but lets check the hash to make sure we have the same results as the manual extract:

So far I have verified this on SANS408 image system $J extract and some local $J files. But of course, make sure you use multiple techniques to verify. This was a quick proof of concept code.

Questions? Ask them in the comments below or send me a tweet at @forensic_matt

Also Read: Daily Blog #51: Understanding the artifacts USNJrnl

Read More

Forensic Lunch 4/3/15 - Devon Kerr - WMI and DFIR and Automating DFIR

Hello Reader,

We had another great Forensic Lunch!

Guests this week:
Devon Kerr talking about his work at Mandiant/Fireeye and his research into WMI for both IR and attacker usage.

• You can email Devon here: devon.kerr@mandiant.com 
• You can follow him on twitter here: https://twitter.com/_devonkerr_
• Get cool tools from the Mandiant github here: https://github.com/mandiant
• Watch Devon talk more about WMI and IR at the SANS DFIR Summit: http://dfir.to/1BvOw7G

Matthew and I going into the Automating DFIR series and our upcoming talk at CEIC

• We are on the CEIC agenda here:
• https://www.guidancesoftware.com/ceic/Pages/ceic-agenda-table.aspx
• and here is the latest post in the Automating DFIR Series:
•  https://www.hecfblog.com/2015/03/automating-dfir-how-to-series-on_22.html

You can watch the show on Youtube:  https://www.youtube.com/watch?v=y-xtRkwaP2g

or below!

Also Read: Automating DFIR - How to series on programming libtsk with python Part 11

Read More

SANS Webcast and PFIC Slides/Labs

Hello Reader,
        If you attended my session at PFIC hopefully you already took these labs with you, if not I'll be linking them down below. For those of who attended my SANS webcast today I hope you found it useful! Now you can try it yourself.

If you didn't attend either I'll explain what's contained within. I presented on how to do USN Journal Analysis using the free version of our tool Triforce ANJP to:

• Recover the names of wiped files
• Prove what was uploaded and downloaded from Dropbox
• Show what attachments were accessed from Outlook 2007 and greater

and more analysis tips. Hopefully you'll find it helpful!

Link to SANS webcast:
https://www.sans.org/webcasts/hands-on-usn-journal-analysis-99177

First here are the slides from today's webcast:
https://mega.co.nz/#!WgwhmKYb!JhwWvGLlug9T0yCU6dlR29S23fx0up2M_LL3Aml6q24

Link to download the sample evidence to do the labs from today's webcast:
https://mega.co.nz/#!3pwmDLzZ!IFUw9rBm2-0Kryu_ASBxKIcFnQSdCNQl7uRyG4DpHvQ

Download Triforce ANJP here:

Link N/A

Also Read: Looks like I need more copies Registry Recon

Read More