Home Archives for 2012

Happy Holidays - Research update

Happy Holidays Reader!,
As we get ready to actually take a couple days off for christsmas and get ready for the next year I wanted to give an update on our current file system journal research.

NTFS - Our NTFS Journal Parser has hit 1.0, we are still writing up a blog post to try to realy encapsulate what we want to say but you can get copy now by emailing me at dcowen@g-cpartners.com. If you sent a request prior and I missed it I apologize, just send it again.

EXT3/4 - We have a working EXT3 Journal parser now, and we can reassociate file names to deleted inodes. However, its not as straight forward as the NTFS Journal (as strange as that may sound) as the EXT3/4 Journal is a redo only journal meaning it does not store the ability to roll back a change just the ability to redo a change. We are able to recovery file names because when a change is recorded its not just the inode being changed thats written to the redo log, its the entire block that the inode is stored in! So we have alot of ancillary data that we have to parse through and then do a search through all those entries for directory entries pointing to the known deleted inode.

We've had to actually switch to a database backend for this to work as there is alot of data we have to make it through to get these changes. If you want in on the beta please email me dcowen@g-cpartners.com and we'll get you in the loop.

The only real change we've found between the ext3 and ext4 journals is a crc value inserted, but we'll look at it further as we move forward to make sure of that. I'm excited about ext4 as its the default file system for most android phones.

HFS+ - This is next on our list, we got a shiny new mac mini for testing on this and we are looking forward to it.

That's it, I hope you understand its been a busy year for so I haven't been able to write all the other cool stuff we want to share with you. In book news Computer Forensics, A Beginner's Guide is 1/3 the way through copy edit and the book should be out in the spring. I'm feeling pretty good about it and its aleady listed on the Amazon store (since its soooo delayed which is 100% my fault). I actually managed to snag www.learndfir.com for the book and I'll have tools, documents, images and this blog mirrored there. Also we are making tutorial videos for the cases in the book that will be put on our YouTube channel LearnForensics.

I hope to see you all at conferences this year, if you are looking for a speaker on advaned file system forensics please let me know!

Talk to you all in 2013!

Also Read: PFIC 2012 Slides & Bsides DFW

Hello Reader,
With another presentation done here are my slides from PFIC, where I again presented on Anti Anti Forensics. This is a similar presentation to the one I did at Bsides DFW but with more details on the actual structure of $logfile records and more information.

Slides can be found here: Slides

We are getting close to the official release of ANJP (Advanced NTFS Journal Parser) as we write up our official blog post to put up on the SANS blog. Until then, if you would like a copy of the version 1 free tool please email me dcowen@g-cpartners.com so I can get you going. Our goal is to get the community access to our research as quickly as possible!

I'm looking for conferences to spread the good word on journaled file system forensics for next year, so if you are looking for advanced content please let me know!

Also Read: Updates and DFIR Conferences

Updates and DFIR Conferences by David Cowen - Hacking Exposed Computer Forensics Blog

Hello Readers,
I know I've been silent, our workload and conferences have kept me quite busy. Updates for you:

Book News

Computer Forensics, A beginners guide is out to copy edit or will be soon. Looking at an early Q1 2013 release to bookstores. I've been working on this book for way to long but having a child while writing a book will do that.

Hacking Exposed, Computer Forensics Third Edition we just signed the contract for this. Look for a new edition in 2014 with a lot of new content and new sections. We really want to keep this series not only relevant but expand its scope from the US legal system to the world.

Conference News

I spoke at Derbycon this past weekend, but not on forensics. I spoke on running a successful red team, which is both my professional past as well as part of the work I do at the National Collegiate Cyber Defense Competition. People seemed to enjoy the content and here are my slides!

My derbycon slides with notes!

I'll be speaking next at BsidesDFW on November 3, 2012 on Anti Anti forensics. I won't be staying very long after as I have to catch a plane to Utah but I do plan to go to the movie screening the night before so hopefully I'll see you there!

My last planned presentation of the year is at Paraben's Forensic Innovations Conference so if you're going I hope to see you there. I'll be doing my Anti-Anti Forensics talk again but this time be doing a live demonstration of the updated tool we've been showing in the blog here. which leads me to my next update

NTFS $Logfile Parser

After a good response from our beta testers we are feeling confident in elimination of bugs in what we are getting ready to release as version 1.0. In addition we got some great fixes after testing our parser on the NIST CFReDS project's deleted file recovery test images. If you are looking to validate a new tool or test a current one the NIST CFReDS images are great and well documented as a control.

We've decided to call the parser ANJP, Advanced NTFS Journal Parser, to have a clear and distinct acronym from anything else. We plan to expand our research into Ext3 and HFS+ after this and will have AEJP and AHJP parsers released at a later date to expand what we believe is a vital piece of information missing from your examinations. There is a lot of research around Ext3/HFS+ regarding recovering deleted files from the journal, but we can't find much focus on mapping out file creations, time stamps changing or files being renamed. All things possibly unique to the interest of the DFIR community. Our plan is to expand out our research so you can take advantage of all the data available to you.

So what will be in version 1.0?

Identification of deleted files with full metadata, in our testing on the NIST CFReDS images we recovered all deleted file records with full metadata.

Identification of files being created with full metadata
Identification of files being renamed with metadata before and after the rename.
Log2timeline output

But Dave, what about all the other cool things you've mentioned?

There is much more we can determine from the NTFS $logfile, but we've realized that understanding it isn't as simple as just reading the csv it outputs. We don't want to release a tool that becomes a source for false positives and bad testimony so we are going to do follow the Viaforensics model (thanks for thinking this up guys!). We are going to be offering a one day training class that explains NTFS, the MFT and most importantly the $logfile. That class will explain how to parse the log, the event records, the redo/undo operation codes and how to stitch those together to find the information we provide in version 1.0.

Extending beyond that we will then explain how to take the Update Sequence Arrays, timestamp changes, file id/directory ids and tie them back into the MFT, recovering resident files, identifying the approximate number of external drives changes, determining how many systems an external drive was plugged into and be able to make good, reliable conclusions from them for use in your case work. At the end of the class you'll get a copy of the super duper version 1.0 that gives you way more information that you will be qualified to draw opinions from. There won't be a dongle or a license or any other such thing. If you decide to give a copy to someone we just hope they don't testify to its results without taking our class.

In the future as we continue our research we may be able to reduce the possibility for error in the additional evidence sources and when we will / as we do we will update the publicly released tool to include those. Until then we think everyone is best served by this model that gets the most reliable evidence in everyone's hands ASAP and giving those who want to go deeper a chance to.

I hope to have version 1.0 released in the next week or two and I'll be posting it here when I do.

If you are running a conference and want us to do the ANJP training at your event let us know, we want to get as many people as possible using this as possible! When you see what all we can determine from the $logfile we think you'll agree.

What conferences do you get the most from?

I am planning my 2013 conference schedule and I've asked twitter and I want to ask you the reader, what conferences do you get the most from? I'm planning on CEIC, PFIC and possibly blackhat but otherwise I want to hear your suggestions! Leave a comment and lets talk.

Also Read: Time to find a fancy hat, I'm speaking at Derbycon

I'm speaking at Derbycon by David Cowen - Hacking Exposed Computer Forensics Blog

Hello Reader,
I've been trying to find more conferences to speak at lately (If you are running a conference let me know) to let more people know about fun forensic artifacts. I've been selected to speak at Derbycon 2.0 but not for a forensic topic this time (though I did submit one). Instead the fine folks at Derbycon like my topic titled 'How to run a successful redteam'. If you've been following the blog you'd know that once a year I lead the national red team for the national collegiate cyber defense competition and have been doing so for 5 years now. We've learned alot on how to build and succeed as a competition red team and I thought it would be a good idea to share what we've learned.

So if you are going to at Derbycon and want to either:
a) have a beer with me and talk forensics or
b) find out how to be a lethal red team full of 'i love it when a plan comes together' moments

Then I'll see you there! Let me know you read the blog if you don't mind it's always nice to know someone is one the other side of the screen.

Also Read: Making Forensics Faster

Making Forensics Faster by David Cowen - Hacking Exposed Computer Forensics Blog

Hola Reader,
There are always bigger hard drives, at least for the near future, and more devices available to individuals every day. While the everyday work flow for an individual is drastically improved, hopefully, by obtaining this additional storage and processing power it only further compounds the amount of evidence a computer forensic examiner has to go through.

It's not uncommon now for a single person to have over a terabyte of data between their laptop, desktop, mobile phone, tablet, hosted email, file sharing and backup services. The question then becomes how do we keep up with the larger volumes of data without allowing each individual to take a month to process.

There are several options out there, depending on your choice of tools (encase, SIFT, FTK, Prodiscover, SMART, etc...) to really speed up the processing of the evidence so you can get full text indexes built and artifacts parsed. In most cases the answer comes down to:

1. Faster CPUs to process the evidence
2. More RAM to process the evidence
3. Distributing the workload across multiple systems (if your software permits this)
4. Faster storage to hold the evidence

In my lab we've approached this from all of these points.

1. Faster CPUs to process the evidence
We have multi cored, multi processor CPUs on our systems. What we've found is that typically, unless we are password cracking, that the I/O from the disks can't keep up with resources available. Meaning our CPUs are never maxed out. So the CPUs are not the bottleneck for getting more speed.

2. More RAM to process the evidence
We have machines with up to 49GBs of ram, under large loads we sometimes see usage into the 30GBs but again because of I/O from the disks more than enough is available. So the RAM is not the bottleneck for getting more speed.

3. Distributing the workload across multiple systems (if your software permits this)
We've tried using FTK's feature that allows multiple systems to distribute processing across them. The problem comes in that the evidence is still being stored and accessed from system. Again the issue comes up the I/O from the disks can't keep up with amount of requests being made from the additional systems now requesting data to process over the network. So distributed processing alone did not speed up processing.

4. Faster storage to hold the evidence
Here we find the greatest benefit. Initially we bought large direct attached storage RAIDs to store and process our evidence. While these systems get us I/O speeds around 100-200MBs depending on the type of disks it still wasn't enough speed to max out the CPUs and RAM. So we started looking around at other options. When you start looking at larger/faster storage systems things can get very expensive, very quickly.

If we had a very large budget we could have gone for a RAMSAN, http://www.ramsan.com/, and gotten a couple terabytes of storage at 4GBs a second read and write speeds. That kind of I/O would clearly max out most systems attached to it and easily keep up with the demands of a distribute processing system.

Unfortunately we don't have that large of a budget for storage, especially not with the amount of data that we working with. A RAMSAN 810 according to this article, start at $45,000 as of 8/23/11. So we look at the midline prosumer range of solid state storage as they are faster than similar SAS 15k drives but priced around the same in the most cases for larger sizes. Most prosumer SSD disks can go up to 300-400MBs a seconds and are connected via SATA meaning you could easily turn them into a small RAID and load your images there. However again the cost of doing this can quickly scale up depending on the amount of storage you need to hold the evidence.

Instead we have been implementing is a middle of the road approach. Instead of loading the entire evidence set into faster storage we purchased a PCI-E based SSD card from amazon with read speeds of 1.5GBs and write speeds of 1.2GBs it has a smaller amount of storage (we opted for 240gbs) but allows the most heavy part of the processing (dealing with all the data extracted from the forensic images) to be done on the fastest storage. To accomplish this in FTK we pointed the ADTemp directory to the PCI-E SSD card and our processing speeds improved dramatically. We were able to complete a full text index of a 149GB full forensic image in 1 hour. This test wasn't even optimal as the forensic image wasn't even copied onto faster RAID storage but instead was just attached via USB3.

We've since ordered PCI-E SSD cards for all of our evidence processing servers and will post most benchmarks as we move forward in our testing and processing. I would also like to expand our SSD storage to include the evidence storage media and database media but I'm doing this one step a time to find out what parts are getting me the largest increases in speed for the dollar.

Paul Henry (@phenrycissp on twitter) has already taken this a step further by putting all his pieces onto SSD SATA storage and integrating in the PCI-E SSD card for his temporary directory. His new issue is finding test images large enough to show meaningful results since its' processing so quickly So if you have a test image 100GBs or larger, please let him know!

Also Read: Back to Basics, CD and DVD basic forensics

Export LNK Contents & Export LNK metadata in FTK

Bonjour Reader!,
I know I have large gaps in my blog posts, its not for a lack of ideas but it is for a lack of time. With the economic recovery in full swing in the legal world we are very busy.

However, I still need to finish my new book and start getting back to blogging more regularly so please feel free to harass me on twitter @hecfblog if I don't write a post once a week.

In this short post I am going to point out a feature in FTK that has existed since 3.3 atleast that I never knew existed. The feature is called 'export lnk contents' in ftk 3.3 and 'export LNK metdata' in ftk 4.0 and it may be the one feature that I wish existed in FTK for the last 8 years of using it. When I've mentioned what this feature is and what it does to fellow examiners each of them has said the same two things:

1. "Woh! This going to save me so much time!"
2. "Why didn't they tell everyone this was here?!"

So in relation to point number 2, let me do that for them.

HEY EVERYONE, FTK will now export out all of the metadata of a lnk file and the contents of the parsed lnks to a file (from atleast 3.2-4.0)!

It can do this with one, some or all LNK files just highlight them, right click a lnk and the context menu will show the option! Suddenly all the manual copy and pasting into a spreadsheet or running other tools (like tzworks lslnk) are no longer necessary. This is especially great when it comes to carved LNK files that may not actually be valid and break many third party tools when they try to parse them.

What all does it export you say?
Keep reading!

Surely there is no way they snuck in a feature everyone wanted and didn't tell anyone?
I sure didn't see it!

It must be missing something right?
Not that I can see! It exports out into a tab seperated file:

* Shortcut File - Name of the LNK file

* Local Path - The path to the file the LNK file is pointing to

* Volume Type - The type of volume (Fixed, Removable, CDROM) of the volume being accessed

* Volume Label - The volume label for the volume being accessed

* Volume Serial Number - The VSN of the volume being accessed

* Network Path - If this was done over the network, the full UNC path to the file

* Short Name - The 8.3 name of the file

* File Size - Size of the file in bytes

* Creation time (UTC) - When the file the LNK file is pointing to was created

* Last write time (UTC) - When the file the LNK file is pointing to was modified

* Last access time (UTC) - When the file the LNK file is pointing to was accessed

* Directory - If file the LNK file is ponting to is a directory

* Compressed - If file the LNK file is ponting to is compressed

* Encrypted - If file the LNK file is ponting to is encrypted

* Read-only - If file the LNK file is ponting to is marked read only

* Hidden - If file the LNK file is ponting to is marked hidden

* system - If file the LNK file is ponting to is marked as a system file

* Archive - If file the LNK file is ponting to is marked as to be archived

* Sparse - If file the LNK file is ponting to is 'sparse'

* Offline - If file the LNK file is ponting to is offline

* Temporary - If file the LNK file is ponting to is a ntfs temporary file

* Reparse point - If file the LNK file is ponting to is extended directory information

* Relative Path - The relative path to the LNK file

* Program arguments - Any arguements stored for the execution of the program

* Working directory - Where the executable will default for reads/writes without a path

* Icon - What icon is associated with the executable if any

* Comment - This is an outlook feature, not sure why its included

* NetBIOS name - The network names of the system the LNK file was accessing

* MAC address - The MAC of the system the LNK file was accessing

So the next time you are working a case in FTK and you want to know what was being accessed from external drives (and you are checking shell bags and other artifacts seperately of course) then make a filter for all file with the extension 'LNK' and right click on one and export all of them to TSV. Import that TSV into excel, sort by Local Path and your done! This may be one the biggest time savers I've found in FTK in years and I now use it on every case.

Have you found a feature you love that everyone seems to miss? Leave it in the comments below.

Also Read: Back to Basics, CD and DVD basic forensics

Subscribe to: Posts ( Atom )

Happy Holidays - Research update

PFIC 2012 Slides & Bsides DFW

Updates and DFIR Conferences

Time to find a fancy hat, I'm speaking at Derbycon

Updates and status

CEIC 2012 - Anti Anti Forensics Materials

New Project, Tool testing

CEIC 2012 - Anti Anti Forensics

New Tools for Shadow Copies

NCCDC 2012 Wrap up post

See you at CEIC

Making Forensics Faster

The best feature you never knew existed - Export LNK Contents & Export LNK metadata in FTK

Top Posts/Right Now