Empty Is Not Absent: Silent Failure Modes in Open-Source Mobile Forensic Parsers on iOS 26 and Android 16

1 Introduction

Open-source mobile forensic suites [11] ship hundreds of artifact modules, each written against the on-device layout seen when the module was contributed. Operating systems often change that layout with a release: a table is dropped, a key moves, a date field takes a value the language runtime cannot hold, a store is split by user. The symptom can be an empty table rather than an error message. An empty table is also the correct result for a feature the device’s user never used, so the two cases look alike to the examiner.

A motivating failure. On the Android 16 image of our sweep, the artifact that reads the Samsung media database reported none of the 213 image rows that the database held (Listing 1). The release removed a location table but kept a view over it, so the artifact’s address join fails; the database error is written to the log, and the report shows an empty table. On four Android 16 images the artifact returned 0, 0, 0 and 3 rows where the fixed artifact returns 5,523, 215, 4,582 and 1,450 (Section 5.3). Nothing in the report distinguishes this from a phone with no images.

Artifact    Samsung media database (Android 16)
Report      0 rows; no error shown
Log         database error: location view
            cannot be queried
Raw store   image table holds rows
            (213 on the sweep image)
After fix   image rows kept when the view
            cannot be queried

The problem: silent failure. We call a run a silent failure when it completes, the report shows a plausible table, often an empty one, and nothing in the report says that the store held more. The problem this report addresses is narrower than tool reliability in general: in which forms the artifact parsers of two open-source suites fail silently on current iOS 26 and Android 16 images, and which check exposes each form. We found little that measures how a parser suite behaves on a current release, artifact by artifact (Section 2).

Approach. We ran every artifact of both suites on current images, classified each outcome, and traced the zero-row outcomes to the raw stores. We ask four research questions.

• RQ1 (Outcomes). On a current image, how many artifacts produce rows, and why do the others not?
• RQ2 (Defects). What share of the zero-row and failed outcomes are drift defects rather than genuine absence, and what kinds of defect are they?
• RQ3 (Detection). Which cheap checks reveal such defects, and which kinds does the outcome classification itself miss?
• RQ4 (Store assumptions). When parsers are written for system stores that no open tool read, do the stores hold what their names and schemas suggest?

Two case studies carry RQ3 and RQ4 beyond the sweep. The first concerns a failure that produces rows and is therefore invisible to an outcome sweep: a module that resolves a path pattern to its first match can return one Android user’s database and silently skip another’s (Section 6). The second concerns parsers for five system stores of Samsung Android 16 (One UI 8) devices that no open tool read (Section 7).

Contributions. This report makes four contributions.

• A measurement of the outcome of every artifact of an iOS suite and an Android suite on current images, with a four-class outcome taxonomy (Sections 3.2 and 5.1). The counts are measurements; the taxonomy is a simple device and not a method claim.
• A typology of the defects isolated by attributing zero-row outcomes to the raw stores: four drift defects in existing artifacts with three behaviours and one defect in new code, each matched to the check that exposed it (Sections 3.1, 5.3 and 5.4) [7], [8].
• A measured case of silent partial loss from first-match path resolution on four Android 16 images, a doubling test that detects it, and a crude static count of how many modules use a first-element pattern (Section 6) [9].
• Case studies of five system stores (Smart Switch, Health Connect, Bluetooth file transfers, blocked numbers with Android Auto cars, and the telephony network and SIM logs) with row counts, schemas, and three assumptions that did not hold (Section 7) [1]–[5].

Scope. All results describe the images listed in Section 4.1. They are observations of current releases, not measurements of what an update changed, and the defect counts are a lower bound; Section 9 states the limits. Section 2 positions the work, Section 3 defines the failure classes and the method, Section 4 the setup, Section 5 the sweep results, Sections 6 and 7 the case studies, and Section 8 what the results mean for reading an empty or plausible table.

2 Background and Related Work

This section describes the suites under test and the mechanisms that shift a layout under a parser, then places the work among studies of forensic tool validation, testing without an oracle, and other tools and reports.

2.1 The suites under test

ALEAPP (Android) and iLEAPP (iOS) are open-source Python suites in which each artifact is a function that locates files in an extraction by path pattern, reads them, and returns rows for a report [11]. Both accept an extraction as a zip, tar or directory. A pattern that matches no file is a normal outcome: most artifacts target apps or features a given phone does not have. This is why an empty result carries little information on its own.

2.2 How layouts shift under a parser

Three mechanisms recur in this report. First, a release changes or removes a store, so a query that used to succeed finds a missing table or key. Second, a stored value falls outside what the parser’s language can represent; Python’s datetime accepts years 1 to 9999 and raises ValueError otherwise [19]. Third, the file a pattern matches is not the file intended. SQLite keeps a write-ahead log and a shared-memory index beside a database as separate files [20], so a wildcard that ends in the database name also matches these sidecars, and a different copy of the database may exist for another Android user. Android keeps app data per user and states that no user can access another’s app data [10], which is why a second user, such as a work or secure-folder profile, holds its own copy of the same database name.

2.3 Validation of forensic tools

Horsman argues that digital forensic tools need systematic testing and that their reliability cannot be assumed [16]. Garfinkel names growing evidence volume and the risk of tools becoming obsolete among the field’s challenges [14]. Both motivate the stance taken here: the output of a parser is a claim about a store, and it needs a check made by other means. Neither measures how a parser suite behaves on a current release, artifact by artifact.

2.4 Testing without an oracle

Metamorphic testing checks a property that must hold between the outputs of related inputs when no oracle for a single output exists [12]. Forensic parsing is such a setting: there is no list of the rows an artifact should return. Our doubling test for second users (Section 3.5) is a small instance of a metamorphic relation.

2.5 Other tools and reports of the same symptom

Other publicly available tools read related inputs, for example the Mobile Verification Toolkit [18], and a commercial suite announced an import of Samsung Smart Switch backups [13], which we did not test. An external report of the same symptom class is a public issue stating that a unified-log parser partially failed on iOS 27 log archives [17]; we cite its existence and did not study it.

Position of this work. To our knowledge there is no published per-artifact outcome sweep of these two suites on current releases; we searched informally and did not run a systematic literature search, so this is a statement about what we found, not a proof of novelty. The work differs from tool validation as usually practised in its unit of analysis: it records the outcome of every artifact of a suite, attributes the zeros to the stores, and studies a failure that produces rows.

3 Failure Model and Method

This section defines the failure behaviours we look for and the checks that expose them, then describes how outcomes were classified and attributed.

3.1 Failure behaviours

A parser can understate a store in several ways, and they differ in what the report shows. Table 1 lists the five behaviours this study met or could not examine, how each looks in the report, and the signal that exposed it. A loud failure raises and aborts the whole artifact. A silent zero returns nothing without raising in the report. A noisy but benign failure changes the log and not the rows. Silent partial loss returns rows, but only from one of several stores that should be read. A moved path no longer matches any file. The threat is not an attacker but ordinary change: new releases, unusual values, and per-user copies of stores.

3.2 Outcome taxonomy

Each artifact was run once per image with the suite’s own command-line tool in file-system (zip) mode and put into one class: rows (at least one row returned), no file (no path pattern matched), matched, no rows (a file matched and zero rows came back), or error (the artifact raised). On the iOS images a fifth group of six artifacts reads device properties rather than files and does not fit the first four; we count it separately. For the Android image only the first and third classes were tallied; the remainder is reported as one group because the working notes did not split it.

3.3 Attribution of zeros

For the matched, no rows and error classes we compared the artifact with the raw store: the row count of the table or the presence of the key the artifact reads (Figure 1). An empty table or an unused feature is recorded as expected. A table that holds rows the artifact did not return, a key present under another name, or an exception on a stored value is recorded as a defect, reproduced with a synthetic file, fixed, and covered by a test. The no file class was not examined further.

3.4 Independent recount

Row counts on the real images were re-derived with separate code (plain SQL, plistlib, zipfile, JSON readers) and compared with the artifact output before each fix was proposed. We call these independent counts. A claim that failed this recount was withdrawn (Section 6.3).

3.5 Doubling test for second users

For each affected artifact we built a tree from real files in which the primary user’s databases also appear at a second user’s path and at a decoy package path. If the artifact reads one database per user, the row count must exactly double; if it takes the first match, it stays the same. The suite’s storage-view test mode applies the same idea to synthetic cases, with two legs: a duplicate view of one file must not change the count, and a second user’s copy must double it. This is a metamorphic relation [12], not a proof of correctness: it detects a dropped second user, not a wrong one.

3.6 Static pattern count

On the Android suite's checkout at the sweep base we counted artifact module files that call get_file_path or index files_found at a fixed position. This is a pattern count, not an audit: some single-file uses are correct. It gives an upper-bound scale for how many modules could share the first-match behaviour, and nothing more.

3.7 Cross-image agreement

For the two iOS sweeps we computed, per artifact, whether the outcome class agrees between the images. The images differ in device, user and OS version, so disagreement is confounded; the figure is a noise floor for any differential check, not a drift measurement.

4 Experimental Setup

This section lists the evidence, the tool versions and the reference measures against which the output was checked.

4.1 Evidence

All images are full file system extractions and were opened read-only; Table 2 lists them by use. Only counts, schemas and structure are reported, never content. Device models, identities, usage and ownership are not reported, and we did not establish that images used for different purposes come from different devices.

4.2 Tools and versions

The iOS sweep used the iOS suite's main branch at the time of the work (1,272 artifacts in 486 module files). The Android sweep and the static count used the Android suite's main branch at the sweep base (1,335 artifacts; the static count covered 529 artifact module files). Our test records cite a full run of 1,333 modules for the same suite on a slightly different base; we use 1,335 because it is the figure in the sweep notes. The sum of the two set sizes, 2,607, is the number of artifact parsers studied. It is not the number of artifact runs: the iOS set was run twice and the Android set once, which makes 3,879 runs. Each fix was checked by continuous integration where it ran and otherwise by the equivalent checks run locally (Artifact Availability).

4.3 Reference measures

The data stored on the device are the reference. We compare each artifact’s output with three measures taken from the store by other means: the raw store itself (table row counts and key presence), the independent counts, and for second users the exact-doubling relation. Where none of these applies, as for the no file class, outcomes are left unattributed.

5 Results

Most artifacts found no file, about 18 percent of the iOS set matched a file and returned nothing, and the drift defects among those outcomes each needed a different check to find. Table 3 answers each research question; the subsections and the two case studies give the evidence.

5.1 Outcome of the sweeps (RQ1)

On both iPhones most artifacts found no file, which is the normal case: a phone does not hold every app a suite supports (Table 4, Figure 2). The class that needs a person is the next one. About 18 percent of the iOS set matched a file and returned nothing, and this is where drift lives, because a defect and an unused feature produce the same outcome there. The two iPhones differ in device, user and release, so the difference between their counts is confounded (Section 5.2).

The Android counts are less certain than the iOS counts. A full Android run in an earlier session filled the working disk (37 GB of output) and was aborted. The counts in Table 4 come from the sweep in a later session, during which the disk also became full and the output was deleted afterwards. The notes give the tallies but do not state that the run completed, so the Android counts may describe a partial pass.

5.2 Agreement between the two iPhone sweeps

The two iOS images agree on the outcome class of most artifacts, and their disagreements are not evidence of drift (Table 5). The same class occurred on both images for 1,007 of 1,272 artifacts (79.2 percent); 265 (20.8 percent) differ. Of the 322 artifacts that were in the matched-no-rows or error class on at least one image, we attribute two to drift (Section 5.3). Disagreement is dominated by device usage and by the one-release difference between the images, and a differential check would have to separate these.

5.3 Drift defects (RQ2)

Checking matched-no-rows and error outcomes against the stores isolated the five defects in Table 6. Four are drift defects in existing artifacts and were found in the sweeps; a fifth was found while validating a new parser on real files. They span the first three behaviours of Table 1. Calendar Events failed loudly: one creation date outside Python’s year range aborted the whole artifact. The location-services app list and the media database failed silently. The Waze artifact was noisy but benign, because the database it misread had an empty places table.

Defects are a small fraction of the zero-row outcomes, and that fraction is a lower bound. Of the 322 iOS artifacts with a matched-no-rows or error outcome on at least one image, two were drift defects (Calendar Events and the app list) and one changed only log output. On the Android image, one of the 143 matched-no-rows artifacts was a defect that we fixed. The working reports state that the remaining iOS cases were checked against table row counts and found to be empty tables or unused features, with the knowledgeC battery, plug, do-not-disturb and location streams simply absent on iOS 26; they do not record the check per artifact, and for Android they say only that many of the 143 are legitimately empty. We therefore report the defect counts as a lower bound on what the sweep found and not as a rate.

One negative result belongs here. The Aggregate Dictionary store exists on 2 of 5 iOS 26 phones but holds only a stale seven-day window from 2022 and 2024, so no module was written for it.

5.4 Which check exposes which defect (RQ3)

Each defect was found by a different check, and no check found them all. A retrospective reading of Table 6 gives the following. The outcome class alone would have flagged Calendar Events (an error) and the media database (matched, no rows), but it would not have separated either from the empty cases; that needed the raw count. The app list was matched-no-rows on both iOS images and showed no difference between them, so a cross-image comparison would not have found it either; only the key inspection did. The Waze effect appeared in the log comparison of fixed and unfixed code, not in any outcome class. The second-user loss produced rows in every case and lies outside the taxonomy; the doubling test found it, and it was cheap because a storage-view test mode already existed (Sections 3.5 and 6). Of the five behaviours in Table 1, only the loud failure is visible to someone reading the report.

6 Case Study: Hidden Second Users

The outcome classes of Section 3.2 cannot see a parser that returns rows from the wrong database. This case study measures one such silent partial loss on Samsung Android 16 devices and the test that exposes it.

6.1 The failure

Four artifacts, the app-update database (frosting), the calendar, the settings search index and the file-manager recent-files list, took the first matching database. On a device with a secure-folder user, whose data lives under user 150, the second user’s smaller copy can sort first. We changed each artifact to read one database per Android user and to add an Android User column [9].

6.2 Measured loss

On the three devices with a second user, the unfixed artifact returned only the second user’s app-update rows, and none of the primary user’s (Table 7, Figure 3). After the change the app-update artifact returns 606, 870, 1,198 and 932 rows on the four devices. The unfixed artifact returned the secure-folder copy on the three two-user devices: 1,207 of the 3,000 rows on those devices, all of them the second user’s; none of the primary user’s 1,793 rows was returned. The unfixed artifacts returned no calendar events and no settings-search rows on three of the four devices. Independent counts on each user’s database agree with the new output.

The Android Auto artifacts had the same first-match pattern. The doubling test on a tree built from a real database gives 16 rows from 8 for the allowed-cars artifact, and 1,122 from 561 for the new blocked-numbers artifact [1]. We found no real image on which a second user’s Android Auto database existed, so that case is demonstrated on constructed data only.

6.3 How far the pattern reaches

The recount caught one error of our own. An earlier version of our test record gave 677 primary-user rows for the file-manager recent-files artifact; the independent recount found that figure on none of the four images, and it was replaced by the counts in Table 7.

The doubling test is not a test of the whole suite. The first-element pattern appears in 67 of the 529 artifact module files at the sweep base (26 call get_file_path, 42 index files_found at a fixed position, with overlap). Some of these use a single system path and are correct, so 67 is a bound on scale and not on defects; the remaining modules with the pattern were not run against a two-user image. In a new WhatsApp Business parser [6], for which no real Business data were available, the doubling test returned the single count for the five artifacts that have data, and nine second-user legs are recorded as known failures. The defect lies in the suite’s existing WhatsApp code, which reads only the first message store, and was not fixed here.

7 Case Study: Five Under-Parsed Android System Stores

Writing parsers for stores that no open tool read (Table 8) exposed a second source of silent failure: assumptions about a store that were wrong, and stores that exist but hold nothing for the examiner. All counts in this section were re-derived with separate code.

7.1 Stores that exist but hold little

Smart Switch. The transfer app keeps one folder of JSON files per transfer session and a backup-media index. On the four images the app held 2, 0, 1 and 1 sessions; one image held the app but no history folder. Several keys present in older session files are absent from newer ones, so columns such as the backup result and message counts are empty on one image.

Health Connect. The platform store shares health and fitness data between apps [15]. On all four images every record table was empty (store versions 23, 20, 26 and 26); only the application, preference and first-grant tables hold rows, so the ten record artifacts are validated on synthetic data only.

Bluetooth file transfers and Android Auto rejected cars. The schema is present and empty: a 16-column table on all four images, and the rejected-cars table with 24 columns and no rows anywhere.

Blocked numbers. The table holds rows on three of the four images. On the first device the identifiers run from 1 to 126 without gaps; on the third and fourth they reach 612 and 960 over 561 and 958 rows, so identifiers are missing (51 and 2), which we do not interpret.

7.2 A misleading name: the telephony network log

A table named trafficstats suggests per-SIM data usage. It is not that (Table 9). What the table holds is a lookup log (type, response time, response code) stamped with serving-cell data.

The SIM event log (123 to 428 rows on five Android 16 images) carries no ICCID or IMSI. Its subscription identifiers match the subscription table on 5 of 5 devices, and its time is device-local with no zone stored. Six damaged variants (truncated, zeroed first page, header only, empty, random bytes) of four real files returned no rows and no exception. The parsers for this store are described in [5].

8 Discussion

This section draws the practical consequences of the results and states what is new.

8.1 Implications for tool authors and examiners

Three practices follow from what we saw; each is a suggestion and not a proven remedy. First, an artifact could report how many files its pattern matched, so that 0 and 3 are distinguishable from 1; this would have exposed the first-match behaviour at no cost. Second, tests for a new artifact could include the second-tenant doubling leg, which an existing storage-view test mode already supports (Section 3.5). Third, an examiner who reads an empty result could ask whether the artifact matched a file at all. The case studies add a fourth caution: a store’s name can mislead, and a store can be present and empty, so the column meanings of a new parser are worth checking on real data before its output is read as evidence.

8.2 What is new

The sweep counts and the typology are new observations, as far as we found. The outcome taxonomy and the table-count comparison are ordinary engineering practice. The doubling test applies an established testing idea [12] to a known harness; in our view the measured silent partial loss from first-match path resolution, and how cheaply that test finds it, is the most transferable result. The case studies are a mixture: the telephony correction is a result; the empty-store entries are documentation of absence.

9 Threats to Validity

Internal validity. Counts were produced by the parsers under study and re-derived with separate code for the fixed artifacts and stores; the sweep outcome classes themselves were not independently recounted. One person attributed the zeros, and the per-artifact attribution was not recorded, so the defect counts are lower bounds. The no file class (603 and 732 on iOS) was not examined: a path that moved in a new release looks identical to an app that was never installed, so drift of that kind is invisible here, and the true number of defects is probably larger.

Construct validity. A zero is not a defect and a row is not a user action; we did not observe the actions that produced these records. The Android tallies split only two classes, and the disk filled during the Android work, so completion of a clean full pass is not demonstrated. The artifact counts differ slightly between sources (1,335 against 1,333 modules for the Android suite) because they come from different bases.

External validity. The study uses three sweep images, four store images and one vendor family per platform. Usage is unknown, and unused features look like failures. There is no Pixel image, no iOS 27 file system image and no iOS 16 or 17 image in this study, and the Samsung secure-folder behaviour may not match other vendors’ work profiles.

Conclusion validity. No statistical inference is made. The doubling test detects a dropped second user and cannot detect a wrong one. The static count of 67 modules is a pattern count and not an audit, and the iOS suite was not counted. Drift is inferred from single images: no pair of releases of the same device was compared, so the study does not measure the effect of an operating system update, only the state of current releases. We rate confidence as moderate for the counts of the fixed artifacts and stores, which were re-derived with separate code, and low for prevalence: three sweep images and four store images are not a sample.

Limitations. The study describes the two suites only; other tools, such as the Mobile Verification Toolkit [18], were not run on these images. The results do not show that a zero-row outcome is a defect, that other versions or vendors behave the same, or how many examiner conclusions an unreported silent zero would change; we did not measure the last.

10 Conclusion and Future Work

An empty table from the parsers of the two suites examined is not, on its own, evidence of absence. Running 2,607 artifact parsers across three current images showed that most outcomes are absence, that about 18 percent of iOS artifacts match a file and return nothing, and that a small number of those are drift defects of three behaviours. All five defects, four in existing artifacts and one in new code, were reproduced with a synthetic file, fixed and covered by a test. A first-match path pattern is a separate class that returns rows from the wrong user; on four Android 16 images it hid all 1,793 of the primary user’s app-update rows on three, and a doubling test finds it. The case studies show that a store’s name can mislead and that a store can be present and empty. A count of matched files per artifact and a second-user test are cheap enough for any parser suite to adopt, open-source suites included.

Future work, in order of value: (1) a clean second Android sweep with the unmatched class split, and a sweep of both suites on a pair of releases of one device to measure drift as a difference; (2) tracing a sample of the no file class to find moved paths; (3) a module-wide second-user audit with the doubling test on every module that uses a first-element pattern; (4) a recorded per-artifact attribution so that a defect rate can be stated; (5) a Pixel image and an iOS 27 file system image; (6) confirming the column meanings of the stores in Table 8 against data whose underlying actions (for example a file transfer or a health sample) are documented.