@night 1803 access accessdata active directory admissibility ads aduc aim aix ajax alissa torres amcache analysis anjp anssi answer key antiforensics apfs appcompat appcompatflags applocker april fools argparse arman gungor arsenal artifact extractor attachments attacker tools austin automating automation awards aws azure azuread back to basics backstage base16 best finds beta bias bitcoin bitlocker blackbag blackberry enterprise server blackhat blacklight blade blanche lagny book book review brute force bsides bulk extractor c2 carved carving case ccdc cd burning ceic cfp challenge champlain chat logs Christmas Christmas eve chrome cit client info cloud forensics command line computer forensics computername conference schedule consulting contest cool tools. tips copy and paste coreanalytics cortana court approved credentials cryptocurrency ctf cti summit cut and paste cyberbox Daily Blog dbir deep freeze defcon defender ata deviceclasses dfa dfir dfir automation dfir exposed dfir in 120 seconds dfir indepth dfir review dfir summit dfir wizard dfrws dfvfs dingo stole my baby directories directory dirty file system disablelastaccess discount download dropbox dvd burning e01 elastic search elcomsoft elevated email recovery email searching emdmgmt Encyclopedia Forensica enfuse eric huber es eshandler esxi evalexperience event log event logs evidence execution exfat ext3 ext4 extended mapi external drives f-response factory access mode false positive fat fde firefox for408 for498 for500 for526 for668 forenisc toolkit forensic 4cast forensic lunch forensic soundness forensic tips fraud free fsutil ftk ftk 2 full disk encryption future gcfe gcp github go bag golden ticket google gsuite guardduty gui hackthebox hal pomeranz hashlib hfs honeypot honeypots how does it work how i use it how to howto IE10 imaging incident response indepth information theft infosec pro guide intern internetusername Interview ios ip theft iphone ir itunes encrypted backups jailbreak jeddah jessica hyde joe sylve journals json jump lists kali kape kevin stokes kibana knowledgec korman labs lance mueller last access last logon leanpub libtsk libvshadow linux linux-3g live systems lnk files log analysis log2timeline login logs london love notes lznt1 mac mac_apt macmini magnet magnet user summit mathias fuchs md viewer memorial day memory forensics metaspike mft mftecmd mhn microsoft milestones mimikatz missing features mlocate mobile devices mojave mount mtp multiboot usb mus mus 2019 mus2019 nccdc netanalysis netbios netflow new book new years eve new years resolutions nominations nosql notifications ntfs ntfsdisablelastaccessupdate nuc nw3c objectid offensive forensics office office 2016 office 365 oleg skilkin osx outlook outlook web access owa packetsled paladin path specification pdf perl persistence pfic plists posix powerforensics powerpoint powershell prefetch psexec py2exe pyewf pyinstaller python pytsk rallysecurity raw images rdp re-c re-creation testing reader project recipes recon recursive hashing recycle bin redteam regipy registry registry explorer registry recon regripper remote research reverse engineering rhel rootless runas sample images san diego SANS sans dfir summit saturday Saturday reading sbe sccm scrap files search server 2008 server 2008 r2 server 2012 server 2019 setmace setupapi sha1 shadowkit shadows shell items shellbags shimcache silv3rhorn skull canyon skype slow down smb solution solution saturday sop speed sponsors sqlite srum ssd stage 1 stories storport sunday funday swgde syscache system t2 takeout telemetry temporary files test kitchen thanksgiving threat intel timeline times timestamps timestomp timezone tool tool testing training transaction logs triage triforce truecrypt tsk tun naung tutorial typed paths typedpaths uac unc understanding unicorn unified logs unread updates usb usb detective usbstor user assist userassist usnjrnl validation vhd video video blog videopost vlive vmug vmware volatility vote vss web2.0 webcast webinar webmail weekend reading what are you missing what did they take what don't we know What I wish I knew whitfield windows windows 10 windows 2008 windows 7 windows forensics windows server winfe winfe lite wmi write head xboot xfs xways yarp yogesh zimmerman zone.identifier

Happy new year, new post The NTFS Forensic Triforce

Feliz Nuevo Ano Reader!,
                                           Thanks for sticking with me and my erratic schedule through 2012. One of my resolutions for 2013 is to get better about regularly blogging and writing about new things we are seeing/doing. The new book is in copyedit at the moment, http://www.amazon.com/Computer-Forensics-Infosec-Guide-Beginners/dp/007174245X/ref=sr_1_21?ie=UTF8&qid=1357249339&sr=8-21&keywords=computer+forensics, not quite sure why they changed the title but it's supposed to be Computer Forensics, A beginner's guide. It's meant for those people who already in IT and moving into a DFIR role either within their company or on their own. I'm actually putting together a series of Youtube videos to go along with it, they'll be found here http://www.youtube.com/learnforensics and I'll be uploading some sample cases to work through that match the book. More on all of that when the book is released though!

Now for why you are (most likely) here, NTFS internal forensics. Over the past year or so if you've been reading or watching me speak you'll know that we've been focused on the $logfile. Since then we've expanded our research into other file systems (we have a working ext3/4 journal parser now that can recover deleted files names and re-associate them with their inodes/metadata) but we are always keeping an eye on NTFS to see what else we can do to expand our knowledge and capabilities. After re-evaluating the USN Journal thanks to Corey Harrell's blog (http://journeyintoir.blogspot.com/2013/01/re-introducing-usnjrnl.html) we've come to recognize that to get a bigger picture our view of previous file system activities can link up to form the NTFS Forensics TRIFORCE! It's dumb I know but it works, see the illustration below.


The $MFT, Master File Table, is always the primary indicator of what the current state of the file system is. If a defrag hasn't run, which windows 7 is very aggressive about and defaults to once a week now for auto defragging (mine is set for every Wednesday but I don't know if that's a default setting), then you can see the deleted/Inactive NTFS file/directory entries prior to the last defrag run. That's not enough for us as forensic investigators though, we need to know more about the prior states of the file system in order to perform our work. So how can we roll back time and see what happened before? A good answer in windows vista/7 systems for many has been shadow copies and shadow copies are amazing for the forensic investigator. What the MFT and the contents of the file system for a shadow copy show you though is the current state of the file system at the time it was capture, a snap shot of the file system. To know what actions took place between snap shots or before snap shots you have to look deeper. The two file system journals, and in this post we are just focusing on file system journals not forensic artifacts that are logging actions related to a specific user activity, that exist are the $logfile and the $USNJrnl.

The $logfile, the primary focus of our initial research, contains the before and afters or undo/redo for each change to the MFT. It contains the full entry of what is being changed (File records, $STDINFO blocks, etc...) and the metadata contained within them (with the exception of resident files on windows 7 journals whose contents appear to be nulled out). The $logfile is great for getting a very granualar level exactly what changes have occurred to a file system.

The $USNJrnl creates a summary of actions taken against a document from the time its opened to the time its closed. The $USNJrnl like the $logfile is circular being that overtime it will be overwritten but like the $logfile if you access the volume shadow copy using libvshadow you can get access to prior backups of it. The $USNJrnl keeps the name of the file being changed, the file id of the file being changed in the MFT, the Parent ID of the directory that contains the file in the MFT and the date the change occured as well as the USN which if the offset into the $USNJrnl where the data regarding the file begins.

Each of these data sources by themselves provide a wealth of information, but all are incomplete without each other. The MFT does not reflect past states, the $USNJrnl does not contain metadata and the $logfile does not always reference a file by name and file id. However, taken together they link up as seen in more detail below to create a view of historical actions like we've never been able to see before:
Ok so what does all that mean?
For any individual file we can determine the following:
     a. What changes have occurred to the file and when
     b. What metadata did the file have before and after each change (modification, creation, access, size, location)
     c. What was a file renamed to?
     d. What files previously existed in a directory?

Who is this useful too?
     1. Malware Analysts you can now see every file system change happening including anti-forensic attempts like time stamp alteration, deletion, renaming and overwriting
    2. Incident Responders you can do the same against attackers who are attempting to hide their activities and track what files they are accessing even if access dates are disabled
    3. Forensic Investigators  - so much more data regarding a suspects activities on the file system including the detection of spoliation
    4. Everyone!

What are the limitations?
   1. Both the $logfile and the $usnjrnl are circular with a max size, so there is a finite amount of data each will keep before it begins to overwrite themselves
   2. If you are using an operating system such as OSX/Linux using the ntfs 3G drivers for writing/accessing a NTFS volume they do not update the $logfile or $usnjrnl for their activities. It will update the $logfile to show that the file system was cleanly mounted though.
   3. Because the $USNJrnl keeps less data than the $logfile its like that the $usnjrnl will contain more historical data than the $logfile

What can you do to make this even better?
   1. If you analyze machines in your environment you can alter the sizes of the $usnjrnl and the $logfile so that they retain much more data.
   2. You can get a copy of our forthcoming NTFS-TRIFORCE parser which will take in data from all three sources to get a complete view of the file system.

More details you say? I'll do that in the next blog post (promise) it's taken too long to just write this one.

Post a Comment

  1. Awesome job on explaining the relationship between the $LogFile, $MFT, and $UsnJrnl files especially the linked diagram. Being able to see all that historical information will be very beneficial on pretty much ever type of case. Great work on this.

    ReplyDelete
  2. Thanks Corey, I hope to have a tool that combines all of this into log2timeline format soon.

    ReplyDelete
  3. David, thank you very much for sharing your work. I started to follow your blog searching the net for ntfs logfile information: let me say I'm looking forward to your next post and the triforce tool!

    ReplyDelete
  4. Triforce sounds cool. Great explanation.

    ReplyDelete

[blogger][disqus][facebook][spotim]

Author Name

Contact Form

Name

Email *

Message *

Powered by Blogger.