The Most/Recent Articles

Showing posts with label dfvfs. Show all posts
Showing posts with label dfvfs. Show all posts

Daily Blog #667: Pancake Viewer Part 3 with Test Kitchen

Pancake Viewer Part 3 with Test Kitchen - By David Cowen

Hello Reader,
         Tonight we had another Test Kitchen and this one was much more successful than last nights.
Tonight we went into Matt's Pancake Viewer code, followed the functions, talked about how it worked and how it used DFVFS.

We ended the night with a demonstration of Pancake Viewer opening a E01 file and what underlying calls/functions were being made.

So if you are looking to follow along and tweak the code yourself or you just want to learn DFVFS/Python you can watch the video below:




This is a 4-part series and here are the links to each one:

Daily Blog #666: Pancake Viewer Part 2 with Test Kitchen

Pancake Viewer Part 2 with Test Kitchen by David Cowen

Hello Reader,
       Well it happened, we reached daily blog 666 and as you would expect ... it all went wrong. 

In order to show how to get DFVFS running I decided to do it in a Test Kitchen live stream that you can watch below, its 2 hours. Yes it took 2 hours and some whiskey to finally go from creating an Azure account, running a Windows 10 vm to getting DFVFS installed. 

Here is the TLDR:
1. Getting Azure up and starting a new Win 10 desktop was pretty painless and fairly cheap, we will find out how cheap at the end of the month
2. I used ninite to install what I needed but it installed Python 2.7 (boo) so I had to install 3.7
3. I attempted to follow the instructions on the DFVFS build page ... don't do that. 
4. Instead  install the Microsoft Visual Studio C++ Build tools:


5. Then use matthew's script:


which will install all of the msi's for the compiled libraries
6, Then install dfvfs, you can use pip for this too
pip install dfvfs


That should do it! 
If not watch the video below and skip around to find things that could help. Next post we will work out what errors exist in the current program, get it to run and then figure out what we want to extend first.


This is a 4-part series and here are the links to each one:

Daily Blog #575: Solution Saturday 12/22/18 - DFVFS Challenge Solution


DFVFS Challenge Solution by David Cowen - Hacking Exposed Computer Forensics Blog


Hello Reader,
I always love introducing new winners to the community and this week I get my wish. Please congratulate Bastien Lardy with his winning Python DFVFS submission!



The Challenge:
Write a python script using DFVFS that uses the source scanner function to enumerate partitions and shadow copies. It should then provide the ability to extract a file and provide its hash. What additional functionality you decide to add in from there will determine which answer is the most complete. If you need test images to code against consider the Defcon CTF images. 


The winning answer:
This is a python2 (I had issue with python3 and mediator...) script that reads an input disk image and searches (based on full path or regex filters), extracts or computes hash. If shadow copies exist, it will prompt a message whether to process those or not.


Also Read: Daily Blog #574

Daily Blog #569: Sunday Funday 12/16/18 - DFVFS Python Script Challenge

DFVFS Python Script Challenge by David Cowen - Hacking Exposed Computer Forensics Blog



Hello Reader,
             Last week I got you searching for DFVFS, this weeks let's see you program in DFVFS! We've done a lot of different challenges for the Sunday Funday series so why not continue to mix it up and see what you've learned.

Need some code examples? Look at yesterdays winning answer:
https://www.hecfblog.com/2018/12/daily-blog-568-solution-saturday-121518.html



The Prize:

$100 Amazon Giftcard

The Rules:

  1. You must post your answer before Friday 12/21/18 7PM CST (GMT -5)
  2. The most complete answer wins
  3. You are allowed to edit your answer after posting
  4. If two answers are too similar for one to win, the one with the earlier posting time wins
  5. Be specific and be thoughtful
  6. Anonymous entries are allowed, please email them to dcowen@g-cpartners.com. Please state in your email if you would like to be anonymous or not if you win.
  7. In order for an anonymous winner to receive a prize they must give their name to me, but i will not release it in a blog post


The Challenge:
Write a python script using DFVFS that uses the source scanner function to enumerate partitions and shadow copies. It should then provide the ability to extract a file and provide its hash. What additional functionality you decide to add in from there will determine which answer is the most complete. If you need test images to code against consider the Defcon CTF images. 


Also Read: Daily Blog #568

Daily Blog #568: Solution Saturday 12/15/18 - DFVFS Challenge Winner Announcement

DFVFS Challenge Winner Announcement by David Cowen - Hacking Exposed Computer Forensics Blog



Hello Reader,
This week I changed up the challenge and you stepped up to the task. This week the master of DFIR knowledge summarization used his skills to pull of a win by one project. Congratulations to Phill Moore (and his baby) for this weeks win!



The Challenge:

Find all the projects out there that are making use of DFVFS (https://github.com/log2timeline/dfvfs) with a description of what the code does with it. Let's see if we can find all the possible exemplar code bases out there to help others adopt this framework!

The Winning Answer:


dfVFS, or Digital Forensics Virtual File System, provides read-only access to
file-system objects from various storage media types and file formats.
The goal of dfVFS is to provide a generic interface for accessing file-system
objects, for which it uses several back-ends that provide the actual
implementation of the various storage media types, volume systems and file
systems.


Log2timeline is a tool designed to extract timestamps from various files found on a typical computer system(s) and aggregate them. Plaso documentation
A DFVFS backed viewer project with a WxPython GUI for viewing file systems and file system metadata. Forensic Lunch about it

Allows you to extract a file from forensic images, virtual disks, raw images and live disks, including from volume shadows. Blog posts:

An open source tool set built on dfVFS. Blog post about it
Evidence Fetcher (efetch) is a web-based file explorer, viewer, and analyzer. Efetch supports viewing hundreds of file types including office, registry, PST, image, and SQLite files. Efetch supports navigating RAW, E01, ZIP, GZ, TAR, VMDK, VHD, QCOW, and BZ2 files thanks to dfVFS.
Blog post about it

A tool for when you have a bunch of documents to figure out of. Gransk is an open source tool that aims to be a Swiss army knife of document processing and analysis. Its primary objective is to quikly provide users with insight to their documents during investigations. It includes a processing engine written in Python and a web interface. Under the hood it uses Apache Tika for content extraction, Elasticsearch for data indexing, and dfVFS to unpack disk images.

A Python implementation of VMPOP (Virtual Machine POPulation) framework. dfVFS is used to enable for Data Extraction features

ArtifactExtractor is a script that extracts common Windows artifacts from source images and VSCs.

AutoProvWindows
This program automatically finds provenance related to a file on an image
I’m not exactly sure what that means.

Technically doesn’t use DFVFS, but a long time ago Dave put out a Sunday Funday challenge to automate the installation so I wrote this script. It may not work any more, it worked at the time.

Also Read: Daily Blog #567

Daily Blog #562: Sunday Funday 12/9/18 - DFVFS Challenge

DFVFS Challenge by David Cowen - Hacking Exposed Computer Forensics Blog



Hello Reader,
        We've had a lot of different kinds of challenges to attract different people within the community to participate. This week I'm changing the challenge up again to open up who can participate this week in a test of your google and basic code reading skills.

The Prize:

$100 Amazon Giftcard

The Rules:

  1. You must post your answer before Friday 12/14/18 7PM CST (GMT -5)
  2. The most complete answer wins
  3. You are allowed to edit your answer after posting
  4. If two answers are too similar for one to win, the one with the earlier posting time wins
  5. Be specific and be thoughtful
  6. Anonymous entries are allowed, please email them to dcowen@g-cpartners.com. Please state in your email if you would like to be anonymous or not if you win.
  7. In order for an anonymous winner to receive a prize they must give their name to me, but i will not release it in a blog post


The Challenge:

Find all the projects out there that are making use of DFVFS (https://github.com/log2timeline/dfvfs) with a description of what the code does with it. Let's see if we can find all the possible exemplar code bases out there to help others adopt this framework!

Also Read: Daily Blog #561

Daily Blog #379: Automating DFIR with dfVFS Part 6

Automating DFIR with dfVFS Part 6 by David Cowen - Hacking Exposed Computer Forensics Blog



Hello Reader,
         It's time to continue our series by iterating through all the partitions within a disk or image, instead of just hard coding the one. To start with you'll need another image, one that not only has more than one partition but also has shadow copies for us to interact with next.

You can download the image here:


If you want to show your support for my efforts, there is an easy way to do that. 

Vote for me for Digital Forensic Investigator of the Year here: https://forensic4cast.com/forensic-4cast-awards/

First let's look at the code now:

import sys
import logging

from dfvfs.analyzer import analyzer
from dfvfs.lib import definitions
from dfvfs.path import factory as path_spec_factory
from dfvfs.volume import tsk_volume_system
from dfvfs.resolver import resolver
from dfvfs.lib import raw

source_path="Windows 7 Professional SP1 x86 Suspect.vhd"

path_spec = path_spec_factory.Factory.NewPathSpec(
          definitions.TYPE_INDICATOR_OS, location=source_path)

type_indicators = analyzer.Analyzer.GetStorageMediaImageTypeIndicators(
          path_spec)

if len(type_indicators) > 1:
  raise RuntimeError((
      u'Unsupported source: {0:s} found more than one storage media '
      u'image types.').format(source_path))

if len(type_indicators) == 1:
  path_spec = path_spec_factory.Factory.NewPathSpec(
      type_indicators[0], parent=path_spec)

if not type_indicators:
  # The RAW storage media image type cannot be detected based on
  # a signature so we try to detect it based on common file naming
  # schemas.
  file_system = resolver.Resolver.OpenFileSystem(path_spec)
  raw_path_spec = path_spec_factory.Factory.NewPathSpec(
      definitions.TYPE_INDICATOR_RAW, parent=path_spec)

  glob_results = raw.RawGlobPathSpec(file_system, raw_path_spec)
  if glob_results:
    path_spec = raw_path_spec

volume_path_spec = path_spec_factory.Factory.NewPathSpec(
        definitions.TYPE_INDICATOR_TSK_PARTITION, location=u'/',
        parent=path_spec)

volume_system = tsk_volume_system.TSKVolumeSystem()
volume_system.Open(volume_path_spec)

volume_identifiers = []
for volume in volume_system.volumes:
  volume_identifier = getattr(volume, 'identifier', None)
  if volume_identifier:
    volume_identifiers.append(volume_identifier)
 
print(u'The following partitions were found:')
print(u'Identifier\tOffset\t\t\tSize')

for volume_identifier in sorted(volume_identifiers):
  volume = volume_system.GetVolumeByIdentifier(volume_identifier)
  if not volume:
    raise RuntimeError(
        u'Volume missing for identifier: {0:s}.'.format(volume_identifier))

  volume_extent = volume.extents[0]
  print(
      u'{0:s}\t\t{1:d} (0x{1:08x})\t{2:d}'.format(
          volume.identifier, volume_extent.offset, volume_extent.size))

  volume_path_spec = path_spec_factory.Factory.NewPathSpec(
        definitions.TYPE_INDICATOR_TSK_PARTITION, location=u'/'+volume_identifier,
        parent=path_spec)

  mft_path_spec = path_spec_factory.Factory.NewPathSpec(
        definitions.TYPE_INDICATOR_TSK, location=u'/$MFT',
        parent=volume_path_spec)

  file_entry = resolver.Resolver.OpenFileEntry(mft_path_spec)


  stat_object = file_entry.GetStat()

  print(u'Inode: {0:d}'.format(stat_object.ino))
  print(u'Inode: {0:s}'.format(file_entry.name))
  outFile = volume_identifier+file_entry.name
  extractFile = open(outFile,'wb')
  file_object = file_entry.GetFileObject()

  data = file_object.read(4096)
  while data:
      extractFile.write(data)
      data = file_object.read(4096)

  extractFile.close
  file_object.close()
  volume_path_spec=""
  mft_path_spec=""

Believe it or not we didn't have to change much here to get it to go from looking at one partition and extracting the $MFT to extracting it from all the partitions. What we had to do was four things.

1. We moved our file extraction code over by one indent allowing it to execute as part of the for loop we first wrote to print out the list of partitions in an image. Remember that in Python we don't use braces to determine how the code will be executed, its all indentation that decides how the code logic will be read and followed.

2. Next we changed the location where our volume path specification object is set to from a hard coded /p1 to whatever volume identifier we are currently looking at in the for loop.

 volume_path_spec = path_spec_factory.Factory.NewPathSpec(
        definitions.TYPE_INDICATOR_TSK_PARTITION, location=u'/'+volume_identifier,
        parent=path_spec)

You can see that the location variable is now set to u'/' being appended to the volume_identifier variable. This would be resolved to /p1, /p2, etc.. as many partitions as we have on the image.

3. Now that we are going to extracting this file from multiple partitions we don't want to overwrite the file we previously extracted so we need to make the file name unique. We do that by appending the partition number to the file name.

  outFile = volume_identifier+file_entry.name
  extractFile = open(outFile,'wb')

This results in a file named p1$MFT, p2$MFT, and so on. To accomplish this we make a new variable called outfile which is set to the partition number (volume_identifier) appended with the file name (file_entry.name). Then we pass that the open file handle argument we wrote before.

4. One last simple change.

volume_path_spec=""
mft_path_spec=""

We are setting our partition and file path spec objects back to null. Why? Because if not they are globally set and will just keep appending on to the prior object. 

That will result in very funny errors.
That's it! No more code changes. 

You can get the code from Github: 


In the next post we will be iterating through shadow copies!