MetadataMiner - Network Graphing of Microsoft Office Document Metadata

MetadataMiner is a tool that I wrote a few years ago to allow for bulk extraction of metadata from Microsoft Office documents. The metadata in Office documents is contained in the SummaryInformation and DocumentSummaryInformation structures that we have reviewed in previous posts. The SummaryInformation structure contains the following data fields:

• Title
• Subject
• Author
• Keywords
• Comments
• Template
• Last Saved By
• Revision Number
• Total Editing Time
• Last Print Date
• Creation Date
• Last Save Date
• Number of Pages
• Number of Words
• Number of Characters
• Thumbnail
• Creating Application
• Security

And the DocumentSummaryInformation structure contains the following data fields:

• Category
• Company
• Manager
• Hidden character count
• Line count
• Note count

MetadataMiner is a tool that can extract these fields from bulk documents. It records all of these field values into a database to allow for querying and searching. This can be useful for example, if you have a bunch of documents and want to quickly determine relationships within that batch of documents. 

Say for example you have one document of interest and want to find all of the other documents that were authored or "last saved by" that same individual. MetadataMiner lets you do this quickly. You can also generate a spreadsheet of all of the metadata fields for each document and then search, sort, and filter to your heart's content. If you want to build a timeline of document activity, you can sort on the "Creation Date" field and then you can see what documents were created at what time - in sequential order.

One of the new features that I have added in is the ability to visualize relationships between documents. When a document is created, the Author field and the "Last Saved By" field are set to the same value - that is, the username of the individual who first created the document. If a document is shared with another user, and that user edits the document, then that user's username is stored in the "Last Saved By" field. Thus there is a relationship there between the document's original author and the person with whom the document was shared.

MetadataMiner is available on GitHub at this link. It is written in Java so you will need a Java runtime environment, (I know, I know - security issues) but you don't have to enable the browser plugin. MetadataMiner is a standalone app so you don't need to run it in your browser thus you are less prone to the security vulnerabilities that go along with the Java browser plugin. To run MetadataMiner, simply download the JAR file from GitHub (or clone the project if you prefer via the following command:

git clone https://github.com/ipduffy/MetadataMiner

Then follow the instructions on GitHub to get MetadataMiner up and running and parsing your documents. I have run it on document collections that are fairly large with no issues so I'm pretty confident in sharing this tool with the forensics community. I am still semi-actively developing this project so if you have any feedback, bug reports, or feature requests, feel free to hit me up.

Microsoft Compund Document Internals (Part 5 - Rebuilding a Corrupted Document Header)

Recently I was contacted by an individual who had been infected with the CryptorBit virus. This virus is a ransomware variant that supposedly encrypts your files and then demands payment of a certain amount of money in order to unlock your files. The person who contacted me explained that rather than encrypting the entire contents of the file (like CryptoLocker), this particular bit of malware just encrypts the first 512 bytes of the file. This provides an interesting opportunity with respect to MS Office Compound Documents because the first 512 bytes are the Compound Document Header (described in detail at MSDN). The interesting thing here is that many of the values that are in the Compound Document Header are static values, and those that are variable can possibly be derived from the actual file contents. Since the file contents are not affected by CryptorBit, we should, in theory, be able to scan the streams, storages, and directory entries within the Compound Document file and recover the data necessary to rebuild the Document header. Challenge accepted.

Forensic Lunch 1/31/14

Thanks again to David Cowen and his team for the opportunity to present my work on the Compound Document File format. For those who were interested in the tools / scripts that were discussed on the show, here are some links:

Python scripts for parsing MS Compound Documents - I have not had a chance to download and test/evaluate these yet but I'm hoping I'll have some free time to do so soon.

Microsoft OffVis tool for parsing MS Compound Documents and detecting malware - direct download

Link to good article describing OffVis and what it does

MSDN documentation on the Microsoft Compound File Binary format

Microsoft Office Compound Document Internals (Part 4 - SummaryInformation)

In this segment of the series on Microsoft's Compound Document file format, I am going to discuss the extraction of information from the SummaryInformation data structure. You may have noticed the SummaryInformation references in the directory entries that we viewed in this previous post. The SummaryInformation structure is the internal data structure within Compound Document files that contains the metadata information - things such as the author's username, the username of the last person to have edited the document, date and time information for file creation, last save, and last print, and statistical information about the file. The SummaryInformation data structure is described here at MSDN. In this post, we will walk through our sample document and extract the document property metadata from the SummaryInformation structure.

Microsoft Office Compound Document Internals (Part 3 - FAT, DIFAT, and Mini FAT)

As I have mentioned in previous posts, the Microsoft Office Compound Document file uses an internal File Allocation Table (FAT) structure to keep track of allocated and unallocated sectors within the file. In addition to the FAT, there is also a Double Indirect File Allocation Table (DIFAT) which is used to keep track of file sectors used by the FAT. Additionally, the Compound Document file also uses a MiniFAT, which allocates storage in the Mini Stream, which will be the topic of another post. All of these structures are used to map the allocation status of each sector within the Compound Document file, and are used to recover sector chains - that is, sequences of sectors that contain the data for a particular stream and storage. In this post, we will discuss the FAT concept in general, and the implementation specifics of FAT within the Compound Document file.