This week I had the privilege of presenting an updated version of my "Analysis and Correlation of Mac Logs" talk at the CTIN Conference.
The updated slides have been uploaded to my presentation area on GitHub.
Note: Some content from the older presentation is still valid, however due to timing I had to cut out some slides, therefore both presentations are still available.
I have added some output options to the script – CSV and KML.
See a related post here - "Parsing iOS Frequent Locations"
The script can now be called with a ‘-output’ argument with the following options:
Note: The verbose script output is still available from standard output.
python dump_freq_locs.py –output e <StateModel#.plist>
Updated script (v1.1) is available in GitHub
Example of the CSV output in Excel
Example of the KML output in Google Earth
Example of the KML output in Google ‘My Maps’
In my iOS Frequent Locations blog post, I mentioned that the locations are stored in a ‘less than analyst friendly’ format. These plist files are in a binary plist format – no different than other binary plists except it is hard to put context to their structure. Rather than directly associated key/value pairs, the keys and values are stored in a seemingly random way.
I noticed that this format is being used more often, especially in iOS device files. The binary plist format is not going away anytime soon! Expect to see them on iOS and OS X systems.
Generally speaking, this format is used to allow developers easier access to their stored data. They can also be used for backward/forward compatibility with their applications. As far as forensic analysis is concerned, think of them as data stored in a serialized tree-based format. We need to start at the base of the tree and work from there to manually put context to these values. The following links describe the NSKeyedArchiver format in greater detail.
As an analyst if you ever open a plist file and are presented the following plist structure, you can be sure there will be groans, expletives, and general signs of discontent coming from the analyst. Analyzing this file will not be as easy as just popping it into your favorite plist viewer and reading keys and values like a “normal” binary plist file.
I was recently reviewing new and changed artifacts on an El Capitan (10.11) system when I came across a drastic change in the “Recent Items” for each user. The file com.apple.recentitems.plist no longer existed in the user’s /Preferences directory. In previous versions, the plist file looked like the example below. Keys and values have context and I can quickly see the last ten applications the user accessed.
In 10.11 this data now uses the NSKeyedArchiver format. The location of this artifact has also changed. These files are now located in ~/Library/Application Support/com.apple.sharedfilelist/ directory.
The file extension has also changed from ‘.plist’ to ‘.sfl’. However, they are still binary plists as shown in the screenshot below - note the file header ‘bplist00’.
Because of the file extension change, just trying to open them in the Xcode plist viewer is challenging. We end up with the following view. (Note: Not all NSKeyedArchiver binary plists will have a custom file extension, some do have the expected ‘.plist’ extension. These should open up as expected in Xcode.)
We could rename all these ‘.sfl’ files as ‘.plist’, open them in Xcode, and review. You will see something like the following screenshot. Notice the keys named $version, $objects, $archiver (with the value of NSKeyedArchiver), and $top. These are the telltale signs you got yourself an NSKeyedArchiver binary plist on your hands. Lucky you!
If we expand the plist a bit more, we can see items of potential interest. We can see application names, file paths, binary blobs - all sorts of good stuff! Unfortunately, there is no context to any of this. It is not guaranteed that keys in sequential order are related in any way.
Xcode isn’t even showing you everything here, a better tool to review these types of files is the command line utility plutil. (Note: Other plist viewers may show you this information without the need to use other utilities, I’m looking at you Blacklight – excellent work!)
In the terminal we will use the command below to dump these files making them easier to analyze. The ‘-p’ flag to plutil prints to standard output in a JSON format. I have also redirected them to an output file rather than standard output so I can review the file easier.
plutil –p <plist_file> > <output_file_here>
The partial output below shows what was missing in the Xcode viewer. Notice the additional information in the ‘1’ key from the Xcode example above. We now can view the ‘NS.keys’ and ‘NS.objects’ along with their values and class types. This is exactly what we need to put context to this data.
Here is a copy of the example output I am using in this article if you would like to follow along.
The root of these files is the ‘root’ key in $top. Always go here first to find the start of the tree. In the example below the ‘root’ key has a value of ‘1’. This means we start at key ‘1’. Time to scroll to the top of the output file!
Looking at key ‘1’ under ‘$objects’, we see it has two sub keys, ‘NS.keys’ and ‘NS.objects’.
The Keys and Objects are directly related, in the example below, key with value 2 is associated with object value 5, key 3 with object 6, and key 4 with object 10.
Each ‘value’ number is a key within the plist, so for this section we will be looking at keys 2, 3, 4, 5, 6, and 10.
There is also a $class (value ‘9’) listed here – these values describe the type of data contained. In this case we are looking at a ‘NSDictionary’ data type. In many instances, depending on the plist file, you will find proprietary data types. The StateModel files used in the iOS Frequent Locations makes heavy use of these.
Lets look at keys 2, 3, 4. ‘NS.keys’ usually tell us what the key name is:
Now let’s look at their associated objects, in the 5, 6, and 10 keys. This is usually where the data is stored for each key name above.
Ok, so we are looking at version 1 of something. We need to keep digging into these nested values to figure what what that something is!
Let’s look into key 3, object 6. We see this has a $class value of 9 (NSDictionary). Enumerating this data gives us a key of 7, with an associated object of 8.
Looks like we have a max file count of 10 for LSSharedFileList recents. This is expected, 10 recent items have been the max amount for years!
Let’s go back and look at key 4, value 10. This is listed as “items”. The 10 key holds ten objects (0-9) with the following values 11, 21, 27, 33, 39, 45, 51, 57, 63, 69. All these have to be enumerated themselves. Let look at the first “item” key 11.
“Item” key 11 has the following data:
We could now go through and enumerate the rest of the “items”, however, I will leave that task to the reader for practice!
You can see that these files can be highly nested with many different data types. Going through these files can be a very manual process which is why I expect to write many scripts parsing these files out when I get the time (or am frustrated enough) to do so.
Each one of these NSKeyedArchiver files is its own special file, no two will be alike which makes parsing these even more difficult. For the scripters out there – I have become very fond of ccl_bplist.py to help in this process.
UPDATE (1/2/15) - Additional blog article and python implementation for SFL files by @mikeymikey
The Artifact:
The phrase “Location, Location, Location” has special meaning for those looking for real estate but can also mean everything to a forensicator looking for locational data. One of the most useful (or creepy if its yours) artifacts on an iOS device is the “Frequent Locations”. iOS will store, for a lengthy amount of time, locations a user’s device has been. This feature came out in iOS 7 and is used to record a pattern of activity for the user. Most often, a user will notice a message like one in the screenshot below - this is the 'routined' process in action on iOS devices. My phone “knew” I was likely headed to Arlington at this specific moment and ‘helped me’ by showing the traffic times. (Way to be creepy, iPhone!)
Users can review and clear their frequent location histories as shown in the next screenshot.
Settings -> Privacy -> Location Services -> System Services -> Frequent Locations
Got an iPhone? Play along! Take a look at your frequent locations.
Don’t have locations? Location Services must be enabled. iOS Location Services must be enabled and more specifically the option ‘Frequent Locations’ under ‘System Services’ must be turned on. There are many services that may be configured under ‘Location Services’ this is only one setting. The ‘Frequent Locations’ are enabled by default.
Locations are clustered by general areas. In the example above you can see I live in the Northern Virginia area and visit many cities in and around the area. These clusters also include the number of sub-locations and general timestamps. You can select one of these areas and get a more detailed street-level overview. This level shows that there are a number of ‘visits’ for each location, and again - a general day-based timestamp.
If we keep selecting down, we can get specific visit and timing information. Each location can have multiple visits. (Note the ‘N Nelson St.' example with 49 visits above.) The example below shows one visit on December 13th at around 6:30 – 6:45pm, however the visit radius (shown in blue) can of course be very specific or more general depending on GPS availability. (I believe this was a visit to Whole Foods.)
Going back to the top - I recently traveled to San Francisco to teach my SANS FOR518 course and have locations from there in the broader ‘United States’ cluster history. Since I traveled outside of my normal Northern Virginia area, iOS made a new “cluster” of locations to include those in the San Francisco Area but also some in the NoVA area as well! I’m sure there is an algorithm somewhere to determine this – but sometimes it just doesn’t make sense how it is organized.
The Script:
I wanted to study these artifacts in more detail as they have obvious benefit to the forensics community. These ‘frequent locations’ are stored in two binary plists in the following location:
/private/var/mobile/Library/Caches/com.apple.routined/
(Note: These files are protected and are only accessible in a physical dump or by physical (jailbroken) access to a device. You will not find these files in an iCloud/iTunes backup or a logical dump by forensic tools.)
I was elated to find them in a common file format, however the binary plist format used is the NSKeyedArchiver format which is less than human analyst friendly. (More on these types of binary plists in a future blog article.)
I needed to write a script to truly begin to understand how and why these entries get created as well as put the plist data into context. Enter, the script.
“Big caveat here. I’m the first to tell anyone that I’m not a programmer. In fact – I’ve been avoiding it like the plague for the longest time (I had a not-so-fun experience with it in school – mind you I loved my scripting course.) I have come to realize that as a forensic analyst some times you just have to suck it up and script something to make your life easier - this is one of those times.
I decided to learn and script in Python since there were many forensic-based tools and packages that already use Python – it seems pretty industry standard. I’ve also noticed that I keep saying to myself (and my students) that this would be so much better or easier if there was a tool to do this or to do that! Sometimes you just gotta do it yourself if you want it done! So I will be trying to contribute as needed to the Mac forensics community – I foresee this being the first of many scripts.”
The dump_freq_locs.py script takes in the StateModel#.archive files (where # is 1 or 2).
The script also has two dependencies, hexdump.py and ccl_bplist.py. These files can be installed or just simply placed in the same directory you are running the dump_freq_locs.py script from. (Installation on OS X 10.11 systems are limited thanks to SIP.)
The script parses the StateModel files and dumps the information to standard output so you will likely want to redirect to a file for later analysis.
$ python dump_freq_locs.py StateModel1.archive > SM1_parsed.txt
The script output contains three sections some of which I can describe here, but as stated above – I don’t know the significance of some of these items….yet. Parsing is the first step toward understanding.
This script was tested of a variety of different devices including iPhones and iPads on different iOS’s from iOS 7-9. As usual the format of these StateModel files has changed slightly from version to version. If you see I’m missing certain information from the parsed output please let me know! On that same note, if I’m not doing something Pythonically correct or weird – let me know too! I’m new to this Python/scripting thing.
I plan on adding future capabilities to this script, like CSV and KML outputs. If you have specific output format needs – let me know, I’ll take them into consideration!
Enjoy!
I was lucky enough to be able to teach at the SANS DFIR Summit in Prague this October as well as attend the Summit. I presented a SANS 360 presentation (it was more like a SANS 600!) on OS X Spotlight Queries.
Find it here.
Other Summit presentations are available here under the heading 'DFIR Prague Summit 2015'
