Making it Rain on this Labor Day – Giving Back to the DFIR and Security Communities

In the spirit of our American holiday Labor Day, where normal people might be watching a parade, barbecuing, and shopping Labor Day deals - I’ve decided to forgo the crowds the mall and give back to the community, monetarily. (BBQ is still fair game.) 

Donating is of those things that can easily be put off, and put off, and put off. Tomorrow's holiday is a perfect day to celebrate the contributions of folks who work on research and software in my communities that I use all the time.

First up is Patrick Wardle’s Objective-See Tools and Blog. I’ve been reading his research and using (and recommending) his tools for years. His Patreon site is here. Uniquely, his Patreon donators can attend his new Objective by the Sea Mac security conference in Maui for free (as if you need more excuses)! I’m really stoked to be able to attend and speak there myself!

Next is a tool that I’ve come to love and use daily, DB Browser for SQLite. Doing mainly Mac and mobile forensics (yes, I do Android too! Shh, don’t tell anyone, it’ll ruin my Apple street cred), I use this seriously every single day. I appreciate that its multi-platform and really allows me to dig into SQLite databases. The tool has really improved greatly in the last few years. They recently introduced their Patreon

Back to blogs - I decided to throw some Dollarydoos towards my Aussie friend and insane blog aggregator Phill Moore. Phill takes the time each week read and comment on hundreds of blogs and organizes them to give DFIR folk a quick and easy way to catch up with what’s going on in the community on his thisweekin4n6.com blog. He also condenses some of this information in a monthly podcastHis Patreon for his podcast is here.

Finally, I will end on more tools – Jailbreaks! I use all sorts of jailbreaks for forensic research. In no way could I have done much of my current iOS research without the contributions of Meridian and LiberiOS jailbreaks for iOS 10 and 11. I depend on jailbreaks to provide me with full file system access, so I can dive into various databases and third-party app data that isn’t backed up with a normal iTunes backup. These can be forensic gold mines and without jailbreaks those of us in the forensic community likely wouldn’t have a chance to know or capture the data contained within them. Ben Sparks (Meridian) has a donate link on his page. Jonathan Levin (LiberiOS) advertises (upon successful jailbreak) to donating to the charity of your choice using the hashtag #LiberiOS. I chose Girls Who Code.

I encourage everyone who uses something (tool, research, whatever) and appreciates it to give something back – whether it be additional research, donations, a drink, or a simple ‘hey, this really helped me’ kind of message (you’d be surprised the warm fuzzies this can give to a frustrated researcher!). FWIW - I prefer a cool story involving my research/tools alongside a drink.

 
 

Knowledge is Power! Using the macOS/iOS knowledgeC.db Database to Determine Precise User and Application Usage

Having access to precise and granular user and application usage can be extremely useful in a forensic investigation, some of which are listed here. I find that pattern-of-life data is some of the most useful information on a device - it really does tell the story about a user and their devices. I've done fairly extensive research on this previously on iOS (much of which can be used for macOS as well) but have yet to really dive into this database.

  • Application Usage
  • Application Activities
  • Safari Browser History
  • Device Power Status
  • Lock Status (iOS Only)
  • Battery Usage (iOS Only)
  • App Installations (iOS Only)
  • Audio Status (iOS Only)

We can use some of these records to help answer a myriad of investigative questions or just about any type of investigation.

  • What applications did a particular user use? How often do they use this application? How long are they using this application for?
  • What websites did they visit? Are they doing research and putting this information into another application?
  • Where did they ask for directions? Are they driving distracted?
  • How often do they check their email? What led them to click on a specific email and infect themselves?
  • How often are they chatting? Who are they chatting with?

The knowledgeC.db database can be found on macOS and iOS devices. On Mac systems there will be a system context database located in the /private/var/db/CoreDuet/Knowledge directory, while a user context database is located in the user’s ~/Library/Application Support/Knowledge/ directory. *Update 08/7/18: The data in this article is specific to macOS 10.13 and iOS 11. Other versions may contain the same data but the schemas/contents may be slightly different. A note about iOS 10 database is at the end of this article.

On iOS there is only one main knowledgeC.db database located in /private/var/mobile/Library/CoreDuet/Knowledge/ that appears to merge the contents of the user and system context databases found on macOS. (Note: Others may exist for other applications, they are not covered here but follow a similar database schema.) It is worth noting that this database only appears to be available on a physical acquisitions and/or jailbroken iOS devices. I have not seen it in iTunes-style backups. 

The database has many tables which have many columns. This article will only go over three of these that I have found to be particularly interesting. I encourage you to look at your own data to discover other items of investigative value. Timestamps in this database use the Mac Epoch time (01/01/2001 00:00:00 UTC).

ZOBJECT – Contains potentially thousands of usage entries for approximately 4 weeks. I will use this table as my primary table of analysis and add on other tables as needed throughout this article. Other tables that ZOBJECT entries may reference are located in these tables:

  • ZSOURCE – Source of the ZOBJECT entries
  • ZSTRUCTUREDMETADATA – Additional metadata associated with ZOBJECT entries

Taking a look at ZOBJECT, you’ll see that entries appear to have a “type” associated in the ZSTREAMNAME column. Using the following tiny SQLite query, I can see what “types” of entries I’m dealing with.

SELECT
DISTINCT ZOBJECT.ZSTREAMNAME
FROM ZOBJECT
ORDER BY ZSTREAMNAME

The output of this on the system context knowledgeC.db database on macOS shows the following “types”:

  • "/activity/level"
  • "/app/activity"
  •  "/app/inFocus"
  • "/app/intents"
  • "/device/isPluggedIn"
  • "/display/isBacklit"
  • "/safari/history"

An iOS example shows the following “types”:

  • "/app/activity"
  • "/app/inFocus"
  • "/app/install"
  •  "/app/intents"
  • "/audio/outputRoute"
  • "/device/batteryPercentage"
  • "/device/isLocked"
  • "/device/isPluggedIn"
  • "/display/isBacklit"
  • "/display/orientation"
  • "/inferred/motion"
  • "/media/nowPlaying"
  • "/portrait/entity"
  • "/safari/history"
  • "/search/feedback"
  • "/user/isFirstBacklightOnAfterWakeup"
  • "/widgets/viewed"

 Application Usage

Let’s start with just the “/app/inFocus”. This “type” is available for macOS and iOS and will show us what application is being used at a given time. I will use the following SQL query for this.

SELECT
datetime(ZOBJECT.ZCREATIONDATE+978307200,'UNIXEPOCH', 'LOCALTIME') as "ENTRY CREATION", 
CASE ZOBJECT.ZSTARTDAYOFWEEK 
    WHEN "1" THEN "Sunday"
    WHEN "2" THEN "Monday"
    WHEN "3" THEN "Tuesday"
    WHEN "4" THEN "Wednesday"
    WHEN "5" THEN "Thursday"
    WHEN "6" THEN "Friday"
    WHEN "7" THEN "Saturday"
END "DAY OF WEEK",
ZOBJECT.ZSECONDSFROMGMT/3600 AS "GMT OFFSET",
datetime(ZOBJECT.ZSTARTDATE+978307200,'UNIXEPOCH', 'LOCALTIME') as "START", 
datetime(ZOBJECT.ZENDDATE+978307200,'UNIXEPOCH', 'LOCALTIME') as "END", 
(ZOBJECT.ZENDDATE-ZOBJECT.ZSTARTDATE) as "USAGE IN SECONDS",
ZOBJECT.ZSTREAMNAME, 
ZOBJECT.ZVALUESTRING
FROM ZOBJECT
WHERE ZSTREAMNAME IS "/app/inFocus" 
ORDER BY "START"

This query is only using data from the ZOBJECT table (no JOINS needed yet!) I’ve filtered out only entries that contain the “/app/inFocus” in the ZSTREAMNAME column using a WHERE statement. I sorted the output by the “START” timestamp using the ORDER BY statement.

For all timestamps in these queries, I have to add 978307200 to convert a Mac Epoch to a Unix Epoch value. SQLite has no idea what a Mac Epoch is – this is a handy conversion to use if you are doing any Mac/iOS forensic database analysis, you will see it very often. I’ve converted all timestamps to my local system time for sanity reasons (you can also choose to use UTC here if you are of the ‘UTC or GTFO’ persuasion). In my testing I have found the timestamps to be accurate.

I’ve extracted the following columns:

  • ZCREATIONDATE - When the entry was written to the database.
  • ZSTARTDAYOFWEEK – What day did this entry occur. I created a CASE statement here to make an analyst friendly output of the days of the week instead of numbers 1-7.
  • ZSECONDSFROMGMT – The GMT offset in seconds. I converted this to an GMT offset in hours by dividing by 3600. This is useful if you have a device that has travelled time zones in the last 4 weeks.
  • ZSTARTDATE and ZENDDATE – A start and end timestamp for the entry. The following “USAGE IN SECONDS” “column” is not a true database column. I’ve done the math between the START and END dates to quickly see usage times in seconds.
  • ZSTREAMNAME – The “type” of entry.
  • ZVALUESTRING – The bundle ID for the application.

The output example above shows a few items of interest.

  • No entries were recorded on Friday 08/03/2018. (I can in fact take a whole day off and not use my laptop – crazy right?!)
  • In these (very) few entries shown, I spend a decent amount of time using Google Chrome (com.google.Chrome).
  • The com.apple.loginwindow entry is tracking time between device logons. Since I didn’t log in on Friday, this “app usage” time is a quite long (~35 hours, divide by 3600 to get number of hours). During this time the laptop lid was closed and just sitting idle.
  • Other applications that I used during this period include:
    • Calendar (com.apple.iCal) – Very briefly, it was only a few seconds – perhaps I just switched to a desktop with the application open. I tend to flip between many desktops – each with a different app open on it.
    • Apple Mail (com.apple.mail)
    • Messages (com.apple.iChat)

Since some data can appear slightly different I will bounce between macOS and iOS examples. This example is from an iOS device.

The examples above is a tiny fraction of events that took place on my system. Once you take the entirety of the data available (4 weeks!) you can start to see patterns. 

  • When is the device in use?
  • What applications are used?
  • How applications used? How often, how long?

This data is great but there are of course caveats associated! This application usage is really only for GUI-based applications. If your users are always in the Terminal, you’ll see plenty of time in com.apple.Terminal. 

Another issue is user attribution. The first example came from the system level context database on macOS how can I associate this usage to a particular user? (More on this in a bit. The second example came from an iOS device. There is only one “user” for iOS devices.)

Application Activities

Using another stream “type” - “/app/activity”, I can add more context to what is happening on the device. Using the same query above, I’ll add a few more items. 

SELECT
datetime(ZOBJECT.ZCREATIONDATE+978307200,'UNIXEPOCH', 'LOCALTIME') as "ENTRY CREATION", 
ZOBJECT.ZSECONDSFROMGMT/3600 AS "GMT OFFSET",
CASE ZOBJECT.ZSTARTDAYOFWEEK 
    WHEN "1" THEN "Sunday"
    WHEN "2" THEN "Monday"
    WHEN "3" THEN "Tuesday"
    WHEN "4" THEN "Wednesday"
    WHEN "5" THEN "Thursday"
    WHEN "6" THEN "Friday"
    WHEN "7" THEN "Saturday"
END "DAY OF WEEK",
datetime(ZOBJECT.ZSTARTDATE+978307200,'UNIXEPOCH', 'LOCALTIME') as "START", 
datetime(ZOBJECT.ZENDDATE+978307200,'UNIXEPOCH', 'LOCALTIME') as "END", 
(ZOBJECT.ZENDDATE-ZOBJECT.ZSTARTDATE) as "USAGE IN SECONDS", 
ZOBJECT.ZSTREAMNAME, 
ZOBJECT.ZVALUESTRING,
ZSTRUCTUREDMETADATA.Z_DKAPPLICATIONACTIVITYMETADATAKEY__ACTIVITYTYPE AS "ACTIVITY TYPE",  
ZSTRUCTUREDMETADATA.Z_DKAPPLICATIONACTIVITYMETADATAKEY__TITLE as "TITLE", 
ZSTRUCTUREDMETADATA.Z_DKAPPLICATIONACTIVITYMETADATAKEY__USERACTIVITYREQUIREDSTRING as "ACTIVITY STRING",
datetime(ZSTRUCTUREDMETADATA.Z_DKAPPLICATIONACTIVITYMETADATAKEY__EXPIRATIONDATE+978307200,'UNIXEPOCH', 'LOCALTIME') as "EXPIRATION DATE"
FROM ZOBJECT
left join ZSTRUCTUREDMETADATA on ZOBJECT.ZSTRUCTUREDMETADATA = ZSTRUCTUREDMETADATA.Z_PK
WHERE ZSTREAMNAME is "/app/activity" or ZSTREAMNAME is "/app/inFocus"
ORDER BY "START"

I performed a LEFT JOIN on ZSTRUCTUREDMETADATA as the activity entries contain some extra metadata info (this is where I can get more context.) From this new table I’m extracting the following:

  • Z_DKAPPLICATIONACTIVITYMETADATAKEY__ACTIVITYTYPE – What is the user viewing or doing with the application.
  • Z_DKAPPLICATIONACTIVITYMETADATAKEY__TITLE – Name of the item being viewed or edited.
  • Z_DKAPPLICATIONACTIVITYMETADATAKEY__USERACTIVITYREQUIREDSTRING – A combination of the previous two columns above. You may choose to use this one, the previous two, or all three!
  • Z_DKAPPLICATIONACTIVITYMETADATAKEY__EXPIRATIONDATE – An “expiration” timestamp. I’m still researching this one – it always seems to be a month into the future.

It is worth noting the ZSTRUCTUREDMETADATA table has over 100 columns – it is worth it to peruse what data your apps are populating in this table. I’m only extracting a fraction of the column data. Customize these SQL queries as needed for your data, each device, each user will be different.

On the macOS system context example I can now see application activities. On my macOS example I’ve only seen the Apple Mail and Notes applications populate this data. (Note: I’ve cut off the first few columns to make it easier to view for this article.)

  • What email mailbox or message am I viewing?
  • What Note am I editing?

The same query can be used on iOS. However, one line needs to be removed as that column does not exist.

ZSTRUCTUREDMETADATA.Z_DKAPPLICATIONACTIVITYMETADATAKEY__USERACTIVITYREQUIREDSTRING as "ACTIVITY STRING",

In my iOS example I found you can get more context to Apple Maps data as shown below. (I also saw Apple News populated.) Now you can see what directions I was searching!

Another stream “type” - /app/intents can provide more context to application activity.

SELECT
datetime(ZOBJECT.ZCREATIONDATE+978307200,'UNIXEPOCH', 'LOCALTIME') as "ENTRY CREATION", 
CASE ZOBJECT.ZSTARTDAYOFWEEK 
    WHEN "1" THEN "Sunday"
    WHEN "2" THEN "Monday"
    WHEN "3" THEN "Tuesday"
    WHEN "4" THEN "Wednesday"
    WHEN "5" THEN "Thursday"
    WHEN "6" THEN "Friday"
    WHEN "7" THEN "Saturday"
END "DAY OF WEEK",
datetime(ZOBJECT.ZSTARTDATE+978307200,'UNIXEPOCH', 'LOCALTIME') as "START", 
datetime(ZOBJECT.ZENDDATE+978307200,'UNIXEPOCH', 'LOCALTIME') as "END", 
(ZOBJECT.ZENDDATE-ZOBJECT.ZSTARTDATE) as "USAGE IN SECONDS",
ZOBJECT.ZSTREAMNAME, 
ZOBJECT.ZVALUESTRING,
ZSTRUCTUREDMETADATA.Z_DKAPPLICATIONACTIVITYMETADATAKEY__ACTIVITYTYPE AS "ACTIVITY TYPE",  
ZSTRUCTUREDMETADATA.Z_DKAPPLICATIONACTIVITYMETADATAKEY__TITLE as "TITLE", 
ZSTRUCTUREDMETADATA.Z_DKAPPLICATIONACTIVITYMETADATAKEY__USERACTIVITYREQUIREDSTRING as "ACTIVITY STRING", 
datetime(ZSTRUCTUREDMETADATA.Z_DKAPPLICATIONACTIVITYMETADATAKEY__EXPIRATIONDATE+978307200,'UNIXEPOCH', 'LOCALTIME') as "EXPIRATION DATE",
ZSTRUCTUREDMETADATA.Z_DKINTENTMETADATAKEY__INTENTCLASS as "INTENT CLASS", 
ZSTRUCTUREDMETADATA.Z_DKINTENTMETADATAKEY__INTENTVERB as "INTENT VERB", 
ZSTRUCTUREDMETADATA.Z_DKINTENTMETADATAKEY__SERIALIZEDINTERACTION as "SERIALIZED INTERACTION",
ZSOURCE.ZBUNDLEID
FROM ZOBJECT
left join ZSTRUCTUREDMETADATA on ZOBJECT.ZSTRUCTUREDMETADATA = ZSTRUCTUREDMETADATA.Z_PK
left join ZSOURCE on ZOBJECT.ZSOURCE = ZSOURCE.Z_PK
WHERE ZSTREAMNAME is "/app/activity" or ZSTREAMNAME is "/app/inFocus" or ZSTREAMNAME is "/app/intents" 
ORDER BY "START"

Building on the same query as before, I’m now LEFT JOIN’ing another table ZSOURCE for the ZBUNDLEID column. I also added three more columns from the ZSTRUCTUREDMETADATA table.

  • Z_DKINTENTMETADATAKEY__INTENTCLASS – An activity type
  • Z_DKINTENTMETADATAKEY__INTENTVERB – Activity verb
  • Z_DKINTENTMETADATAKEY__SERIALIZEDINTERACTION – Binary plist with extra metadata information.

On my macOS system context example I have more context on when Messages are sent, or calls are synced. In my research, I have found that the messages sent timestamps are accurate however the call syncing is misleading. This isn’t likely when the calls are being made but when they are synced to the macOS system. A further look at the BLOB data can provide the appropriate timestamps. (Note: I’ve removed the /app/activity columns to make this data easier to view.)

The BLOB data can be very important but very easy to miss – it depends on how your SQLite viewer of choice shows it to you. I usually use SQLite Database Browser which will show the word “BLOB” for this type of data. I can double-click on this cell and review the data in various formats (Text, Binary, and Image). I use the Binary option for a quick review of embedded binary plists. I can extract this plist and view it in something a tad more analyst friendly like XCode or plutil.

Once extracted I can review the data. I used plutil in this example. It is a serialized NSKeyArchiver plist, with an embedded binary plist (see the hex at beginning 0x62706c69 73743030 = “bplist00”) because why the heck not, yay for binary plist inception! After extracting the hex (again) and saving it as another binary plist I can open it and view its metadata. 

It’s another NSKeyedArchiver plist! For more information on how to parse this type of plist see my article here. It will be worth it, this will provide user context (ie: who is chatting with whom, when, and using which accounts!).

What else do we get on iOS? How about Maps specific data! Was someone driving distracted using many apps? (FWIW I wasn’t, I was a passenger in a car.) The BLOB data for these entries contains geo location based metadata details.

…or some Spotlight interactions?

Safari History

For some reason Safari browsing activity is specifically tracked under its own “type” – /safari/history. Add ‘or ZSTREAMNAME is "/safari/history"’ to the end of the WHERE clause in the previous query to get this information. In this case, the ZVALUESTRING is not a bundle ID but a URL that was visited using Safari. This is another method you can use to attempt to tie a specific user account to application usage. Correlate this data to user Safari history and caches. Similar data will be shown on iOS however the bundle ID will be com.apple.mobilesafari.

Fantastic you might say! Now I have multiple places to look for Safari history entries. Oh, but wait – if the user clears their browsing history, these entries get removed too. Damn….and before you ask – Private Mode browsing doesn’t show up either.

User Correlation

If you’ve read this far, I’ve been focused on the knowledgeC.db database for iOS and the system level context database. Attributing this to specific users may be potentially difficult. On iOS there is really only one “user” but on macOS there could be many. You should also take a look at the user’s knowledgeC.db database to provide context.

It may be possible to correlate entries in each database to determine which user is doing what. Some of the data contains more specific information such as the email or phone number whom they are chatting with or the address searched for in the Maps application. 

This database may also have synced data from associated devices. The items that show a specific iOS version in the OSBUILD column and a GUID in the ZDEVICEID column came from my iPhone X. This includes third-party applications (not shown).

The stream “types” for the ZOBJECT table are /portrait/entity and
/portrait/topic. Items of type “portrait/entity” are fairly self-explanatory, however the “/portrait/topic” are unknown at this time. Each of these start with “Q” in the ZVALUESTRING column. 

The strings in the NAME column are extracted from the email, chat, webpage, app that is being used. Looking at my own data I see everything from names, addresses, contact information, to terms on webpages I’ve viewed, messages I’ve sent, or emails I’ve written.

Still Reading?

I’m impressed! There are many stream “types” that I didn’t get a chance to cover such as the device status and iOS specific entries. I hope to have a Part II out sometime soon!

*Update 08/7/18: The databases presented in this article are for iOS 11 and macOS 10.13. Thanks to a reader, I was informed that the iOS 10 database schema is a bit different but most of the data appears to be the same. the ZEVENT number needs to be paired with the primary key in the ZOBJECT table and the BLOB data in ZSTRUCTUREDMETADATA table is stored in the ZMETADATA column. Thanks to "PN" for the info!

Presentation - #DFIRFIT or BUST: A Forensic Exploration of iOS Health Data (SANS DFIR Summit)

At the SANS DFIR Summit in Austin this year I had the pleasure of presenting with Heather Mahalik on iOS Health Data. We get into data acquisition, database contents, patten of life analysis, workout metadata, locational data, forensic data recreations, and finally tool support (or lack thereof).

Video of it will be out eventually, my Resources page will be updated when that happens, however slides are available below:

Find the presentation here!

Finally we had limited edition custom shirts made with our awesome #DFIRFIT logo by our famous DFIR Photoshopper in Residence Brian Moran. We hope that more of these shirts are made available, keep an eye on Twitter!

As always, the SANS DFIR Summit is a great time and a absolutely fantastic conference. I hope to see you all next year!

 

 

 

Presentation Slides & Demo Videos - Getting Saucy with APFS

I just had the honor of presenting at one of my favorite BSides Conference BsidesNOLA on the State of the new Apple File System (APFS). Sadly, I didn't have the time to go through the demos but I have uploaded them to YouTube and the slides have been uploaded to my Github, as promised.

Slides (PDF)

Mounting an APFS disk in MacOS using xmount

Mounting an APFS disk in SANS SIFT (Linux) using ewfmount

Enjoy! I hope ya'll find it useful! 

Ok Internet, Lets Test this APFS Plaintext Password Bug Properly

There has been some confusion (myself included) on what is vulnerable to this bug and what isn't. Some folks can replicate and some cannot - so I think its high time to test this properly to see what versions and scenarios are affected by this bug.

The table below is sorted by macOS version - I'm just testing 10.13.x here. (If you happen to get it to leak the password on 10.12.x please let me know.) I'm also organizing by disk formatting scenario since I believe this is where most of the confusion comes from.

APFS encrypted volumes can be created on the disk level as well as the volume level and it truly seems to make a difference. Please also test if you find (or don't find) the results in the Unified logs and/or the install.log or neither (and god forbid any other locations you might come across!). I'm also consistently using the "Erase" button versus the "Partition" button.

If I am missing a certain scenario that you think should be added please let me know. If you disagree with the current findings - also let me know (I will also expect screenshots/videos from you to be sure we're on the same page.) I have tested 10.3.3 (host system) and 10.13.4 (VM) but would love someone to sanity check me - I've been wrong in the past!) I would have done more testing on different versions but have a limited set of systems as I'm traveling right now.

I will update the spreadsheet below as necessary.

For previous related blogs see here and here.

YAY TEAMWORK!

pay attention to me cat GIF-source.gif