Daily Blog #805: Mount That Thing!

 

Hello Reader,

If you've ever done forensics on modern linux systems disk images you may have encountered the dread that comes with dealing with lots of LVMs (Logical Volume Management) which none of the commercial forensics tools seem to be able to fully handle, yes even Xways.  Well instead of being full of existential dread of having to export, reimport and handle all of these partitions you can take advantage of the command line kung fu of Hal Pomeranz to automate this process for you!

Hal wrote a tool called MTT or Mount That Thing which .. well it's mounts things! You provide it with the linux disk images and it takes care of finding, identifying and mounting all of the LVMs and partitions within it so the data is accessible.  

Overview of the Script

This script is designed to automate the following operations:

• Mounting disk images (E01 or raw)

• Handling LVM volumes

• Automatically identifying and mounting partitions

• Exporting mounted partitions into E01 format if desired

• Safely unmounting and cleaning up devices and volumes when finished

All mount operations are performed read-only, with noexec and other conservative options to preserve evidence integrity.

---

Key Features

Mounting Disk Images

• E01 support: If the image is in Expert Witness format, the script uses ewfmount to extract the raw image and proceed with analysis.

• Partition detection: For full disk images (e.g., MBR), it uses losetup -P to enumerate partitions and identify associated file systems.

• LVM support: Detects and activates volume groups, carefully handling potential naming collisions with already mounted LVM volumes.

• Filesystem recognition: Supports EXT2/3/4, XFS, BTRFS, and FAT file systems, with logic to apply the appropriate mount options for each.

• Root partition detection: Identifies the likely root partition via fstab or naming heuristics and mounts it first.

• Command logging: All mount operations are logged to a MOUNTING file within the target directory for reproducibility and audit trails.

Export to E01 Format

When invoked with the -E flag, the script will:

• Export each mounted partition using ewfacquire

• Segment the output if required via the -S option (e.g., for 2 GB chunks)

• Name exports based on their mount point or partition origin to maintain clear context

• Store exports and logs in an exported/ subdirectory of the target mount path

This is especially useful for archiving or handing off discrete pieces of evidence.

Safe and Comprehensive Unmounting

Using the -U flag, the script will:

• Unmount all associated filesystems

• Deactivate volume groups via vgchange -a n

• Detach all loopback devices with losetup -d

• Kill any ewfmount processes by unmounting their working directory

This ensures that the analyst can return the system to a clean state after analysis or re-run the script on a new image without residual device conflicts.

---

Usage Example

Mount and export an image:

./mount_image.sh -d /mnt/evidence -E -S 2147483648 image.E01

Unmount everything cleanly:

./mount_image.sh -U /mnt/evidence

Default behavior places mount artifacts under a mount/ directory, but this can be overridden with the -d flag.

Give it a shot! 

https://github.com/halpomeranz/dfis/blob/master/mtt.sh

Read More

Daily Blog #783: Automating rpm checks

 

Hello Reader,

I'm recreating my 24 year old perl scipt in bash to allow someone to validate all of the installed rpms on a system against both the local rpm DB and the repository it came from.  This should allow a certain since of comfort on if any core system packages have been manipulated.

 

#!/bin/bash

# Files to store results
VERIFIED="verified"
FAILURES="failures"
DEBUG="debug"

# Clean previous results
> "$VERIFIED"
> "$FAILURES"
> "$DEBUG"

# Iterate over installed RPM packages
for package in $(rpm -qa); do
  echo "Processing package: $package"

  # Find repository URL
  repo_url=$(dnf repoquery -q --location "$package" 2>/dev/null | head -n 1)

  if [[ -z "$repo_url" ]]; then
    echo "Repository URL not found for package: $package" | tee -a "$FAILURES"
    echo "$repo_url $package" | tee -a "$DEBUG"
    continue
  fi

  # Get local file hashes from RPM database
  rpm -ql --dump "$package" | while read -r line; do
    file_path=$(echo "$line" | awk '{print $1}')
    rpm_hash=$(echo "$line" | awk '{print $4}')

    # Skip directories and non-executable files
    if [[ ! -x "$file_path" ]]; then
       continue
    fi
    
    if [[ ! -f "$file_path" ]]; then
       continue
    fi

    if [[ -h "$file_path" ]]; then
       continue
    fi
    # Calculate local disk hash
    disk_hash=$(sha256sum "$file_path" 2>/dev/null | awk '{print $1}')

    if [[ "$disk_hash" != "$rpm_hash" ]]; then
      echo "Hash mismatch (Local RPM DB) - Package: $package, File: $file_path" | tee -a "$FAILURES"
      echo "$dish_hash $rpm_hash $package $file_path" | tee -a "$DEBUG"
      continue
    fi

    # Get repository RPM hash
    repo_hash=$(rpm -qp --dump "$repo_url" 2>/dev/null | grep " $file_path " | awk '{print $4}')

    if [[ -z "$repo_hash" ]]; then
      echo "File not found in repository RPM - Package: $package, File: $file_path" | tee -a "$FAILURES"
      echo "$repo_hash $repo_url $file_path" | tee -a "$DEBUG"
      continue
    fi

    if [[ "$disk_hash" == "$repo_hash" ]]; then
      echo "Verified - Package: $package, File: $file_path" >> "$VERIFIED"
    else
      echo "Hash mismatch (Repository) - Package: $package, File: $file_path" | tee -a "$FAILURES"
      echo "$disk_hash $repo_hash $package $file_path" | tee -a "$DEBUG"
    fi
  done
done

echo "Verification complete. Results are stored in '$VERIFIED' and '$FAILURES'."

Also Read: Validating linux packages other than rpms

 

Read More

Daily Blog #782: Validating linux packages other than rpms

 

Hello Reader,

      We've talked about validating rpms in several posts now but there are other package managers besides rpm. Let's talk about how we can do the same validation with other package managers.

 

1. Debian/Ubuntu (dpkg & debsums)

Install debsums if you haven't already:

sudo apt install debsums

Verify file hashes for a specific package:

sudo debsums -s <package-name>

Verify a specific file:

sudo debsums -s <package-name> | grep /path/to/file

Verify all installed packages:

sudo debsums -cs

---

2. Arch Linux (pacman)

Check integrity of a specific package:

pacman -Qkk <package-name>

Verify a single file:

pacman -Qkk <package-name> | grep /path/to/file

Verify all installed packages:

pacman -Qkk

---

3. openSUSE (rpm & zypper)

openSUSE uses RPM, so you can use standard RPM verification commands:

Check integrity of a file against the RPM database:

rpm -Vf /path/to/file

Verify all installed packages:

rpm -Va

---

4. Alpine Linux (apk)

Newer Alpine Linux versions (3.15+) include the apk audit command:

Verify integrity of a package:

apk audit <package-name>

Verify all installed packages:

apk audit

Also Read: Self validating linux executables

Read More

Daily Blog #781: Validating local linux hashes to their distros

 

Hello Reader,

In my previous blog post, I explained how to use the rpm tool to validate a file on disk against the local RPM metadata. But what if you suspect that an entire package—not just a single file—has been tampered with?

This is where we can leverage a powerful feature I mentioned earlier: extracting metadata directly from the Linux distribution’s repository. Since this remote repository should be unaffected by any local security incidents, it allows you to verify that packages like the core system utilities in this example remain unaltered by a potential threat actor.

---

Step 1: Identify the Repository URL

To fetch the official package version, you first need to determine the correct repository URL. You can do this using the dnf repoquery command:

dnf repoquery --location coreutils (you can specify any package name)

This will return a URL similar to:

https://ftp.redhat.com/pub/redhat/linux/enterprise/9/en/os/x86_64/Packages/coreutils-8.32-31.el9.x86_64.rpm

---

Step 2: Extract the Official Package Hash

Now that you have the package URL, you can use rpm to retrieve its metadata, including file hashes, without downloading the full package:

rpm -q --dump -p https://ftp.redhat.com/pub/redhat/linux/enterprise/9/en/os/x86_64/Packages/coreutils-8.32-31.el9.x86_64.rpm

---

Step 3: Compare Local vs. Repository Hashes

To ensure your package is untouched, compare:

1. The hash of the local file (e.g., /bin/ls)
2. The hash stored in the local RPM database
3. The hash from the official repository package

If all three hashes match, you can be highly confident that your package has not been altered.

Of course, this assumes there isn’t a worst-case scenario where the original distribution’s repository has been compromised—but let’s hope it never comes to that!

By following these steps, you can verify system integrity efficiently using native Linux tools. 

Also Read: Self Validating Linux Executables

Read More

Daily Blog #779: Sunday Funday 3/16/25

Hello Reader, 

We've been bouncing around topics a lot and I realized I haven't had a Linux challenge in quite some time.  This week let's see your work as you document all of the logs and artifacts left behind not just from SSH'ing into a linux system but also create a tunnel between the two systems.

The Prize:

$100 Amazon Giftcard

The Rules:

1. You must post your answer before Friday 3/21/25 7PM CST (GMT -6)
2. The most complete answer wins
3. You are allowed to edit your answer after posting
4. If two answers are too similar for one to win, the one with the earlier posting time wins
5. Be specific and be thoughtful
6. Anonymous entries are allowed, please email them to dlcowen@gmail.com. Please state in your email if you would like to be anonymous or not if you win.
7. In order for an anonymous winner to receive a prize they must give their name to me, but i will not release it in a blog post
8. AI assistance is welcomed but if a post is deemed to be entirely AI written it will not qualify for a prize. 

The Challenge:

What are all of the artifacts left behind on a Linux system (both server and client) when someone authenticates via SSH and creates a SSH Tunnel.

Also Read: Daily Blog #778: Solution Saturday 3/15/25

Read More

Daily Blog #662: Forensic Lunch 4/3/20 - Discussion on WinSCP, Linux Forensics Course, SANS DFIR, and More

Hello Reader,
   Today we had another episode of the Forensic Lunch!

On this episode:

• Mari Degrazia (@MariDegrazia) discussing her research into WinSCP and later movement, you can read more here: http://az4n6.blogspot.com/2020/02/detecting-laterial-movment-with-winscp.html
• Hal Pomeranz (@hal_pomeranz) talking about his new Linux Forensics course that you can download here: https://ia801406.us.archive.org/6/items/HalLinuxForensics/HalLinuxForens ics_archive.torrent
• Alex Levinson (@alexlevinson) Gave an update on the National Collegiate Cyber Defense Competition which as gone all virtual this year
• Matt Seyer (forensic_matt) talked about our upcoming SANS DFIR presentation and tools he's working on
• Sarah Edwards (@iamevltwin) gave colorful commentary and meaningful insights 

You can watch the show below:

Also Read: Daily Blog #661

Read More