Lab Technical Report LAB-TR-0033
Synthetic Android Phones That Match Measured Ones Feature by Feature but Not Across Stores: A Calibrated Generator and Benchmark
Lab Technical Report, preprint, not peer reviewed
29 October 2026
Abstract
Synthetic phones are the only phones whose history is fully known, yet we found no held-out test of whether their stores look like those of phones in use. We built a generator of Android device histories whose parameters are summaries of 21 per-device features measured on 14 phones (device age, time from first boot to setup, records dated before first boot, identifier counters, retention horizons), with three dependence groups and a ledger of planted events. We validated it by device: leave-one-out calibration, 200 split halves with a real-against-real noise floor, and transfer between releases. A documentation-only generator was rejected on 19 of 21 features, the calibrated one on none, and 16 of 21 split-half distances lay within the floor. Dependence inside the modelled groups was reproduced; dependence across groups was not: message volume and the share of messages dated before first boot correlate at 0.77 in the phones and 0.01 in the synthetic devices. On 60 generated devices, reading an empty result as “no” gave an unsound absence for 53% of date-bound questions; a rule-based solver, 2.5%. We release the generator, the profile, 48 sample devices and the scoring code.
CCS Concepts • Applied computing → System forensics; • Computing methodologies → Modeling and simulation; • General and reference → Evaluation.
Keywords synthetic data, mobile forensics, Android, calibration, benchmark, held-out validation, identifier counters, retention horizons
ACM Reference FormatDaniel Addai, Sanders Larweh Tetteh, Selorm Adjabeng, and Kingsley Albert Egyir. 2026. Synthetic Android Phones That Match Measured Ones Feature by Feature but Not Across Stores: A Calibrated Generator and Benchmark. Lab Technical Report LAB-TR-0033, version 1.0. SectraTEK Lab, Prudential Associates. Preprint, not peer reviewed. https://lab.sectratek.com/research/reports/lab-tr-0033/
1 Introduction
Real phones can rarely be shared, and on a real phone nobody knows what happened when. Synthetic phones can be shared and come with a record of every event planted in them, so they are the usual material for testing tools, training examiners and checking methods [13], [14]. Whether they are good material depends on whether their stores look like the stores of phones in use. That is rarely measured: published generators argue realism from the way they produce data [13], compare one generated image with one made by hand [8], or compare artifacts with human-made ones [9].
A motivating example. A rule that decides whether a date lies before a phone’s own life began is tested on generated phones that are set up at first boot, as a generator built from documentation would make them. It scores perfectly. On the 14 measured phones, setup came a median of 98 days after the first boot record [4], and the generated data contain no record dated before first boot. The documentation-only dataset of this study has 0 restored records among its 60 devices; a dataset generated from the measured summaries has 5,407. The rule never meets the case it is meant for.
The problem. We need a generator whose parameters come from measured phones, a validation that says on which properties its output matches phones it has not seen and on which it does not, and a scoring protocol that others can apply to their own tools. We ask four questions.
- RQ1 (Construction). What generator, with parameters that can be published, reproduces the measured aggregates and produces files and a ledger of planted events?
- RQ2 (Calibration). Do synthetic devices match held-out measured devices feature by feature, by a stated distance, and better than a generator built from documentation alone?
- RQ3 (Limits). Which realism properties are reproduced and which are not?
- RQ4 (Use). Does a benchmark of recovery tasks on generated devices tell a naive reader from a rule-based one, and what changes when the data are generated from documentation only?
Table 1 gives the answers.
| RQ | Answer | Evidence |
|---|---|---|
| RQ1 Construction | A profile of 21 five-number summaries, three dependence factors and 13 stated scenario parameters drives a specification-level generator that expands to SQLite files and a ledger. Features read back from the files equal those computed from the specification on all 200 generated devices (stores of more than 20 live rows). | Sections 3.1 to 3.3, 4.1; Tables 3, 5, 6; Figure 1 |
| RQ2 Calibration | Documentation-only generator rejected on 19 of 21 features by leave-one-out calibration; the calibrated generators on none. Split-half distance: 16 of 21 features within the real-against-real floor (documentation only: 4 of 21). | Sections 4.2, 4.3; Tables 7, 8; Figures 2, 3 |
| RQ3 What is and is not reproduced | Reproduced: single features, zero masses, and the correlation inside the restore group (split-half difference 0.29 against a floor of 0.37; 0.54 without dependence). Not reproduced: correlations across groups (5 of 12 pairs differ by more than 0.35), release-specific differences, and anything at record level that was not measured. | Sections 4.4 to 4.6; Tables 9 to 11 |
| RQ4 Benchmark | Four recovery tasks with a ledger discriminate a naive reader from a rule-based one. On 60 calibrated devices reading an empty result as "no" gives an unsound absence on 53.3% of questions, a rule-based reader on 2.5%. A benchmark generated from documentation only has no restored records and under-states the unsound share (35.1%). | Section 4.7; Table 12; Figure 4 |
Contributions. This report makes four contributions.
- A generator of Android device histories in which every parameter is a published summary or a stated scenario setting, with record-level expansion to SQLite files and a ledger of planted events (Sections 3.1 to 3.3).
- A validation by device, with leave-one-out calibration, split halves against a real-against-real floor, and release transfer, and a list of realism properties that are and are not reproduced (Sections 3.4, 4.2 to 4.6).
- Four recovery tasks with a scoring protocol and two baseline solvers, and the finding that a benchmark generated from documentation alone hides the cases that matter (Sections 3.5, 4.7).
- The package: code, the profile, 48 sample devices and scoring code (Artifact Availability).
Scope. The calibration concerns 14 Android full file system extractions of the Lab’s research corpus, 12 of them Samsung, Android 10 to 16. The generator makes Android phones with message, call and media stores, install records, device-life traces, retention horizons and wall-clock steps. It makes no record content, no iOS device and no bug report. Section 6 states the limits.
2 Background and Related Work
2.1 Generators of synthetic forensic data
Generators drive a machine, or write into one, to produce a story. Yannikos and Winter model the creation, change and deletion of data on a disk [24]. TraceGen emulates user actions on a Windows machine through automation interfaces and writes a log of the actions it executed [9], [13]; ForTrace produces disk, memory and network traces and a report of the generated artifacts [12], [13]. For Android, FADE injects files and database entries into an emulator through the debug bridge [7], [13]; AutoPoD-Mobile builds case-like scenarios for mobile devices [18] and was extended with language-model agents that draft the storyline [20]; Demmel and colleagues replay community playbooks that drive the graphical interface of an emulator and keep a file of the executed actions [8]. Re-imagen uses language-model personas to produce coherent background activity on disk images [22].
2.2 Datasets, realism and what is known about phone data
Standardised corpora were proposed early [10], and surveys show that most research datasets are generated for the study and few are released [17]. For mobile devices, a structured search found 31 public corpora, most with too few traces to resemble an ordinary phone image [16]. Knowledge of what real phones hold is thin: an interview study with six law-enforcement professionals describes typical contents and states that knowledge of data distributions in real smartphones is limited [15]. On realism, Göbel and colleagues argue that asking how realistic synthetic data are is the wrong question and that the useful one is how a synthetic set differs from a similar real one; their list of differences is qualitative [13]. Voigt and colleagues propose a metrics-based assessment of disk images and define realism as indistinguishability for a verifier restricted to chosen properties; their evaluation compares property values of synthetic and real images side by side, without a distance measure or test [23]. A later guide sets out principles for building datasets [14]. Nemetz and colleagues built a corpus of SQLite files that exercise corner cases of the file format to benchmark tools [19]; its files are crafted, not calibrated.
2.3 Measurements this study builds on
The parameters of the profile come from the Lab’s earlier measurements: the start of a device’s own life and the records that predate it [4], identifier counters and the rows they imply [6], retention horizons [1], clock behaviour [2], and the settings of SQLite databases [3]. The date-bound questions of the benchmark are those of an earlier report on answerability [5].
2.4 Position of this work
On 29 October 2026 we queried the arXiv, Zenodo, OpenAlex and Crossref interfaces with 12 queries (arXiv with 8 more); Semantic Scholar answered 4 of 24 attempts and dblp none, because of rate limits and a bot check. We read the abstracts of the closest 15 works, 7 of them in full (Section 7). We found no generator whose parameters are fitted to distributions measured on phones and then tested on phones held out of the fit; the nearest source of distributions is an interview study [15]. We do not claim that none exists: the publisher search pages of the ACM Digital Library, DFRWS and Forensic Science International: Digital Investigation were not reachable by script, and about half of the full texts were not accessible. What is new here is the calibration and its held-out validation, and the list of properties that fail. The generator’s file writer, ledger and scoring are engineering. The probability integral transform [11], the Kolmogorov–Smirnov distance and the energy distance [21] are standard tools used as such. We compare no tools.
| Work | Target | How data are produced | Record of what was generated | How realism is examined |
|---|---|---|---|---|
| Yannikos and Winter [24] | Disk images | Model of actions that create, change and delete data | Not stated in the abstract | Not stated |
| TraceGen [9] | Windows machine | Scripted emulation of user actions through automation interfaces | Log of executed actions | Artifacts of emulated actions compared one by one with human-made ones |
| ForTrace [12] | Disk, memory, network | Simulated interaction; extends an earlier framework | Report of the synthesis steps | Evaluated on scenarios |
| FADE [7] | Android emulator | Files and database entries injected with a debug bridge | Not stated [13] | Recognised by a forensic tool [13] |
| AutoPoD-Mobile [18, 20] | Android devices | Case-like scenarios; model drafted by language-model agents | Not stated in the abstracts | Proof of concept; validation of generated data (abstract) |
| Demmel et al. [8] | Android emulator | Community playbooks that drive the graphical interface | File of executed actions | One generated image compared with one manual image |
| Re-imagen [22] | Disk images | Language-model personas and background activity | Not stated in the abstract | Coherence, argued |
| Voigt et al. [23] | Disk images | Assessment, not a generator | Not applicable | Realism as indistinguishability; property values compared side by side |
| This report | Android phones | Per-feature summaries and dependence fitted to 14 measured phones; scenario parameters stated | Ledger of planted events | Held-out measured phones: calibration, distance, dependence |
3 Method
3.1 Features and profile
A feature is one number measured on one phone. We use 21: age, gap, volumes of messages, calls, media rows and apps, three pre-boot shares and a restored-app share, two between-boot-and-setup shares, three unused-identifier shares, the counter-set flag, tombstone cover, and four retention horizons. Each is defined as in the report that measured it: the first boot record W and the earliest trace of user setup, as in [4], and the age as the time from W to acquisition; shares of records dated before W or between W and setup [4]; the share of issued identifiers without a live row, after removing a counter set near 109, and the tombstones per unused media identifier [6]; horizons as the time from the oldest retained record to acquisition on devices whose clock passed the rule of [1]. Features that a phone lacks, such as the notification history, are absent for that phone.
A profile holds, per feature, the number of phones that have it, the share of exact zeros, and the minimum, quartiles and maximum of the non-zero values; for the counter-set flag, the share of phones. Three groups of features share a dependence factor: the four restore features (R), the age and the four volumes (V), and the three unused-identifier shares (D). Within a group, a normal copula with a single loading draws correlated probabilities, which are mapped through the piecewise-linear quantile function of each feature; the loading ρ is set from the mean pairwise Spearman correlation r of the group as ρ = 2 sin(πr/6), kept between 0 and 0.95. Features outside the groups are independent. Table 5 gives the calibrated profile. The released profile rounds every summary value to two significant digits, because a minimum or a maximum is the value of one phone. The validation below used the unrounded summaries inside the Lab; the benchmark datasets and the samples use the released profile.
3.2 Generator
The generator has two levels (Figure 1). At the specification level, a device is a set of integers: acquisition time, first boot time W, the earliest trace of setup, counts of live and removed rows per table, a counter-set flag, times of the setup traces, and horizons capped by the device age. At the record level, a specification is expanded in time order. Native records are placed between the earliest trace of setup and acquisition; a restore batch is written at one moment shortly after setup, dated before W (plus a further 5.2% of that number dated between W and setup); rows to be removed are created and later deleted, one by one or, in a clean-up scenario, in a single sweep; the media counter is set to 109 between two kept rows when the flag is set; identifiers follow the SQLite rule for the table kind (AUTOINCREMENT never reuses an identifier, a plain rowid table takes the largest live one plus one); and a wall-clock step shifts the recorded dates of later native records. The device is written as one SQLite file with the auto-vacuum, secure-delete and write-ahead-log settings drawn at the rates of [3]; deleted rows are inserted and then deleted so that the file shows what those settings leave. The ledger lists every record with its true time, recorded time, restored or removed status, every counter set and every application-use event, whether stored or expired. Table 3 lists the settings that no phone fixes.
| Parameter | Value used | Basis |
|---|---|---|
| Age of restored records before first boot | 180 to 2,000 d, uniform | assumption |
| Restored rows dated after first boot | +5.2% of the pre-boot count | [4]: 2,318 on 44,650 |
| Restore delay after the first trace of setup | 0 to 1 d, uniform | assumption |
| Planted setup before its earliest trace | 0 to 1 d, uniform | assumption |
| Counter movement that is not a removal | 0 (stress: 0.3) | not identifiable |
| Clean-up sweep of media | probability 0 (stress: 0.5) | none seen [6] |
| Devices with a wall-clock step | 4 of 7 | [2], bug reports |
| Wall-clock step size | 1 min to 3 d, uniform | assumption |
| Databases with freed cells zeroed | 95% | [3]: 93 to 98% |
| Databases in WAL mode; with pending frames | 52%; 40% | [3]: 49 to 56%; 34 to 47% |
| Message table without AUTOINCREMENT | 2 of 13 phones | [6] |
| Reason mix of other apps (setup, user, unknown) | 4.7, 57.3, 38.0% | [4]: 46, 564, 374 of 984 apps |
| Other setup traces within a day of the earliest | 36 to 86% within a day, by trace; the others log-uniform up to 66 to 1,900 d | earlier runner of [4], not printed there; rounded |
Several behaviours are not measurable on the phones and are set by scenario parameters (Table 3). The main one is the split of counter movement between removals and other mechanisms, which a phone cannot show because it has no reference ledger; the calibrated profile attributes all of it to removals and a stress scenario attributes 30%.
3.3 Features measured from generated devices
The same feature functions are applied to a specification (count level) and to the tables read back from the written files. The count-level function lets the validation generate thousands of devices at full scale without writing records. A round-trip test checks that the two give the same values (Section 4.1). Samples are written at a scale of 0.01 of the measured volumes.
3.4 Validation
The unit is the device. We compare feature distributions by three distances, each stated here once. The Kolmogorov–Smirnov distance KS is the largest difference between two empirical distribution functions. The 1-Wasserstein distance W1 is the area between them, in the units of the feature after transformation: log10(1+x) for counts and days, the share itself for shares and flags. For an energy statistic over the four restore features we use the unbiased form of [21] on devices that have all four.
Leave-one-out calibration. For each phone, the generator is fitted on the other phones, thousands of synthetic devices are drawn, and the held-out value of each feature is mapped through the synthetic distribution (randomised where there are ties). If the generator is calibrated, these probability integral transform values are uniform [11]; we test that with the Kolmogorov–Smirnov statistic against Uniform(0,1) at the 5% level, per feature, without correction. The run is repeated with three seeds.
Split halves. The phones are split at random into two halves of 7, the generator is fitted on one and compared with the other, 200 times. The floor is the same distance between the two real halves. A generator whose distance does not exceed the floor is as close to the held-out half as the other half is.
Release transfer. The generator is fitted on the phones of one release group (Android 16: 9; Android 10 to 14: 5) and compared with the other group.
Dependence. Spearman correlations within the restore group are compared between phones and synthetic devices; pairs from different groups are listed in a separate table.
Three generators are compared. D0 uses only what public documentation states (the retention limits of the Android source, as listed in [1]) and round values a test designer might pick: setup at first boot, no restored records, a range of removal shares from 0 to 0.3, no counter edits. D0 is a deliberately naive baseline, not a competing calibrated generator: it has no restored records and no setup gap, so it is rejected by design on the seven features that describe them (the gap and six shares). C0 is fitted with independent features. C1 is fitted with the three dependence factors.
3.5 Benchmark
A dataset is a set of generated devices with their ledgers and a list of date-bound questions per device. Half of the questions of a family name a day on which an event of that family occurred; the others are drawn uniformly from 60 days before W to acquisition. The ledger state of a question is PRESENT if a live native record of the family lies on that day, UNOBSERVABLE if the day precedes W, lies beyond the usage-statistics horizon (application use), or holds only removed events, and ABSENT otherwise. Table 4 lists the four tasks. Two baseline solvers are included: a naive one that takes dates, gaps and empty results at face value, and a rule-based one that sets W from the first boot record and U from the earliest trace, labels a row restored if it is dated before U or more than 12 hours before the highest earlier-identifier date [4], bounds removals from the counter after rebasing a counter set [6], and answers UNOBSERVABLE for days before W, beyond the horizon, or where the table shows removal [5]. They are baselines, not recommendations.
| Task | Question | Score |
|---|---|---|
| T1 Boundary | When was the first boot (W) and when was the phone set up (U)? | Absolute error in days; share within one day |
| T2 Labels | Is each live message, call and media row restored (carried to the phone) or created on it? | Precision and recall for "restored" |
| T3 Removals | Give a closed interval for the number of rows ever created and since removed, per table. | Share of intervals that contain the ledger value |
| T4 Questions | For a family (message, call, media, app use) and a day: PRESENT, ABSENT or UNOBSERVABLE? | Confusion with the ledger state; share of unsound "absent" |
3.6 Evidence and setup
The 14 phones are all the Android full file system extractions of the Lab’s research corpus that the earlier reports [4], [6] measured: 12 Samsung and 2 from other vendors; Android 16 (9) and 10 to 14 (5). Their per-device features were taken from the stored reports of the earlier runners; this study opened no extraction. One runner script (Python standard library only) extracts the features, fits the profiles, runs the validation, the round-trip test and the benchmark, and writes a report of aggregates from which every number here was taken. All random draws use fixed seeds.
4 Results
4.1 The generator and its round trip (RQ1)
The package has 1,627 lines of Python (code, tests and runners). The full runner (three leave-one-out runs, 200 split halves, the round-trip test and three benchmark datasets of 60 devices) took 207 seconds on one machine. Table 6 gives the round-trip test: all 21 features computed from the specification equal those read from the written files on all 200 devices, except for media tables of at most 20 live rows, where an examiner cannot tell from the identifiers which rows after a counter set were removed (81 device-feature pairs set aside). Table 5 gives the profile fitted on all 14 phones.
| Feature | n | Zeros | Min | Median | Max |
|---|---|---|---|---|---|
| Age (d) | 14 | 0 | 130 | 530 | 3,200 |
| Boot to setup (d) | 14 | 14% | 26 | 120 | 560 |
| Messages | 13 | 0 | 53 | 2,800 | 100,000 |
| Calls | 13 | 0 | 47 | 1,600 | 2,500 |
| Media rows | 14 | 0 | 40 | 1,100 | 58,000 |
| Apps | 14 | 0 | 15 | 80 | 230 |
| Msgs pre-boot (%) | 13 | 46% | 0.86 | 84 | 100 |
| Calls pre-boot (%) | 13 | 62% | 19 | 75 | 100 |
| Media pre-boot (%) | 14 | 29% | 4.4 | 31 | 84 |
| Apps restored (%) | 14 | 36% | 1.6 | 28 | 51 |
| Msgs boot-setup (%) | 13 | 54% | 0.25 | 3.4 | 16 |
| Calls boot-setup (%) | 13 | 62% | 0.16 | 24 | 72 |
| Media ids unused (%) | 14 | 0% | 3.2 | 64 | 97 |
| Msg ids unused (%) | 9 | 0% | 0.027 | 14 | 90 |
| Call ids unused (%) | 12 | 0% | 6 | 51 | 100 |
| Counter set 109 | 14 | 50% of phones | |||
| Tombstone cover (%) | 13 | 0% | 0.81 | 39 | 99 |
| Usage horizon (d) | 13 | 0 | 1.2 | 9.8 | 10 |
| DropBox horizon (d) | 13 | 0 | 0.04 | 2.4 | 3.1 |
| Netstats horizon (d) | 13 | 0 | 17 | 94 | 980 |
| Notif. horizon (d) | 9 | 0 | 0.037 | 1 | 1.5 |
| Check | Result |
|---|---|
| Devices generated and read back (scale 0.05) | 200 |
| Features compared per device | 21 |
| Features that differ between specification, tables and SQLite files | 0 of 21 |
| (device, feature) pairs set aside: media table with at most 20 live rows, where the counter jump cannot be read reliably | 81 |
4.2 Leave-one-out calibration (RQ2)
The documentation-only generator was rejected on 19 of 21 features in each of three runs; the margins-only and the dependence generators on 0 (Table 7, Figure 2). D0 was not rejected on device age, non-system apps. The mean KS statistic was 0.55 for D0 and 0.18 and 0.17 for C0 and C1. With 9 to 14 values per feature the test rejects only when the statistic exceeds 0.35 (14 values) to 0.43 (9 values), so it detects gross miscalibration, not small shifts. With a Bonferroni correction for the 21 tests (level 0.0024, exact Kolmogorov distribution), D0 is rejected on 14, 15 and 13 features in the three runs and C0 and C1 on none. D0 is rejected on 12 of the 14 features that its design does not fix at zero.
| Generator | Features tested | Rejected (3 runs) | Mean KS |
|---|---|---|---|
| Documentation only (D0) | 21 | 19 / 19 / 19 | 0.55 |
| Calibrated margins (C0) | 21 | 0 / 0 / 0 | 0.18 |
| Calibrated margins and dependence (C1) | 21 | 0 / 0 / 0 | 0.17 |
4.3 Split halves and the floor (RQ2)
Table 8 and Figure 3 give the split-half distances. The mean KS distance of the calibrated generators to the held-out half was 0.387, against 0.554 for D0 and a floor of 0.400; 16 of 21 features lay within the floor for C1 (D0: 4). The 1-Wasserstein distance gave the same order. On the energy statistic over the restore features, the median was 0.062 for C1, 0.086 for C0 and 0.425 for D0, against -0.040 for two real halves, which is zero in expectation: C1 is about 7 times and C0 about 5 times closer than D0, but not at the floor. The features whose median KS for C1 exceeded the floor were messages, calls, media counter set near 109, horizon: daily usage statistics, horizon: network statistics, by 0.01 to 0.06. Per feature, C1 was no farther from the held-out half than the floor in 43 to 68% of the 200 splits (D0: 1 to 96%). With 7 phones per half the line between “within” and “outside” is not sharp.
| Generator | Mean KS | Mean W1 | Median KS within floor |
|---|---|---|---|
| Documentation only (D0) | 0.554 | 0.329 | 4 of 21 |
| Calibrated margins (C0) | 0.386 | 0.201 | 16 of 21 |
| Calibrated margins and dependence (C1) | 0.387 | 0.200 | 16 of 21 |
| Real half against real half (floor) | 0.400 | 0.203 | – |
4.4 Transfer between releases (RQ3)
Fitting on one release group and testing on the other (Table 9) gave a mean KS of 0.45 and 0.45 for C1, the same as comparing the groups’ own devices (0.44) and below D0 (0.59 and 0.56). The generator carries no release effect, and with 5 and 9 phones the data cannot show one.
| Fit on, test on | nf | D0 | C0 | C1 | Own devices |
|---|---|---|---|---|---|
| fit A16, test A10-14 (n 9/5) | 19 | 0.59 | 0.45 | 0.45 | 0.44 |
| fit A10-14, test A16 (n 5/9) | 21 | 0.56 | 0.45 | 0.45 | 0.44 |
4.5 Dependence inside the modelled groups (RQ3)
Among the four restore features the phones show correlations of 0.38 to 0.73 (Table 10). Without dependence (C0) the mean absolute difference to the phones was 0.53; with it (C1), 0.11. In split halves, the median difference between a half and the generator was 0.29 for C1 and 0.54 for C0, against 0.37 between two real halves. The full-sample comparison is in-sample, because the loading was fitted on these phones; the split-half comparison is not.
| Pair | Phones | C0 | C1 |
|---|---|---|---|
| pre-boot messages with pre-boot calls | 0.38 | 0.02 | 0.42 |
| pre-boot messages with pre-boot media | 0.72 | -0.01 | 0.48 |
| pre-boot messages with restored apps | 0.49 | 0.02 | 0.48 |
| pre-boot calls with pre-boot media | 0.73 | 0.01 | 0.44 |
| pre-boot calls with restored apps | 0.39 | 0.01 | 0.44 |
| pre-boot media with restored apps | 0.49 | -0.01 | 0.50 |
4.6 What is not reproduced (RQ3)
Correlations across groups are not reproduced. In the phones, a larger message store goes with a larger share of messages dated before first boot (Spearman 0.77, 13 phones; synthetic 0.01), a longer time from first boot to setup with a smaller share (-0.62; 0.01), an older phone with more unused media identifiers (0.61; -0.03), and more tombstones with fewer unused identifiers (-0.64; 0.00). 5 of the 12 pairs of Table 11 differ by more than 0.35. With 13 or 14 phones a single correlation is uncertain by about 0.3, so we do not say which of these relations are real; we say that the generator would not show them if they are.
| Pair | n | Phones | Synthetic | Difference |
|---|---|---|---|---|
| boot-to-setup time and pre-boot messages | 13 | -0.62 | 0.01 | 0.63 |
| boot-to-setup time and restored apps | 14 | -0.19 | -0.00 | 0.19 |
| device age and pre-boot media | 14 | -0.31 | -0.01 | 0.30 |
| device age and unused media ids | 14 | 0.61 | -0.03 | 0.64 |
| media volume and unused media ids | 14 | -0.27 | -0.05 | 0.21 |
| message volume and pre-boot messages | 13 | 0.77 | 0.01 | 0.76 |
| call volume and pre-boot calls | 13 | 0.09 | -0.00 | 0.09 |
| app count and restored apps | 14 | 0.46 | -0.01 | 0.47 |
| counter set and unused media ids | 14 | 0.16 | -0.00 | 0.16 |
| usage horizon and device age | 13 | -0.03 | 0.01 | 0.05 |
| boot-to-setup time and device age | 14 | 0.37 | 0.04 | 0.32 |
| tombstone cover and unused media ids | 13 | -0.64 | 0.00 | 0.64 |
Other properties are outside what was measured and so are not reproduced or not tested: the order and timing of records inside a store (native records are uniform in time, with no diurnal or weekly rhythm); the content of any record; applications other than the three stores; iOS devices; bug reports; and any property of phones outside the 14 measured ones.
4.7 Benchmark scores (RQ4)
Table 12 and Figure 4 give the scores. On the calibrated dataset, the naive reader labelled no restored row (recall 0 of 5,407), placed W a median 966 days from the first boot record, and answered ABSENT where the ledger says present or unobservable for 53.3% of all 6,000 questions (77% of the 4,130 questions with one of those two ledger states). The rule-based reader found W exactly, U within a median of 0.40 days (set by the planted latency of 0 to 1 day, Table 3), labelled restored rows with precision 0.966 and recall 1.000 (restored rows are dated before the first trace of setup by construction, as 99.998% of creator-marked messages were [4]), answered ABSENT wrongly for 2.5% of all questions, and answered UNOBSERVABLE although the ledger says absent for 20.8%. Its removal intervals contained the ledger value in 94.8% of 172 cases; the 9 that did not were all media tables with a counter set, where rows removed after the set are not visible in the identifiers (9 of 30 such tables, 0 of 142 others).
On the documentation-only dataset there were 0 restored records, so T2 cannot be scored; the naive reader’s unsound share was 35.1%, against 53.3% on the calibrated one; every interval of the rule-based reader was valid (100%), because no counter was set. The stress scenario (30% of counter movement not a removal, clean-up sweeps in half of the devices) lowered the validity of intervals for the naive reader to 30.4% and left the rule-based reader at 97.1%.
| Dataset, solver | W err | U err | T2 P / R | T3 valid | T4 as ledger | Unsound absent | Withheld |
|---|---|---|---|---|---|---|---|
| calibrated, naive | 965.9 | 1147.2 | – / 0.000 | 48.3 | 46.0 | 53.3 | 0.0 |
| calibrated, rules | 0.00 | 0.40 | 0.966 / 1.000 | 94.8 | 68.9 | 2.5 | 20.8 |
| documentation only, naive | 3.03 | 3.03 | – / – | 86.7 | 64.6 | 35.1 | 0.0 |
| documentation only, rules | 0.00 | 0.00 | – / – | 100.0 | 77.1 | 0.0 | 22.7 |
| stress, naive | 741.0 | 832.4 | – / 0.000 | 30.4 | 49.3 | 50.1 | 0.0 |
| stress, rules | 0.00 | 0.48 | 0.913 / 1.000 | 97.1 | 70.3 | 2.8 | 19.7 |
5 Discussion
What the validation licenses. On 16 of 21 features, synthetic devices from the calibrated profile are as close to held-out measured phones as the two halves of the measured phones are to each other. With 7 phones against 7, or 13 against 1, the comparison excludes gross differences, which D0 shows, and it cannot exclude small ones. It also follows in part from the construction: a generator that resamples the summaries of its fit set will be calibrated on a held-out device drawn from the same population. The informative results are the contrast with D0 and the properties that fail.
What fails is relational. The calibrated generator matches most features and misses the relations between stores. The failure matters for the uses in Section 1: a tool that combines stores, such as one that reasons from the boundary and the counters together, meets in the generated data combinations that the measured phones do not show and misses combinations that they do. The scenario parameters of Table 3 and the profile can be changed to build such cases deliberately, but they then stop being calibrated.
Why the benchmark needs measured parameters. The same two readers score differently on documentation-only and on calibrated data: the unsound share of the naive reader is 35% against 53%, restored-record labelling cannot be scored at all, and the failure of the removal interval on seeded counters does not appear. A benchmark that omits the measured conditions overstates how well a method does.
Use. The generator suits regression tests, teaching, method comparison under stated scenarios, and sensitivity analysis of rules. It does not estimate how often anything happens on real phones, and a score on generated devices is not a score on real ones. Refitting the profile on new measurements is a single command.
6 Threats to Validity and Limitations
Internal. The features are defined by the same earlier runners that produced them, so the generator is validated against those definitions, not against an independent reading of the phones. The leave-one-out calibration passes partly by construction (Section 5). The round-trip test shows that the implementation is consistent, not that the data are realistic. D0 is one deliberately naive baseline: its rejection shows that the features cannot be guessed from documentation, and says nothing about other uncalibrated generators. The 21 leave-one-out tests are reported without correction in Table 7; the correction changes the D0 counts, not the conclusion (Section 4.2). Of the parameters of Table 3, 3 are assumptions and two are scenario switches that no phone fixes; the median error of the setup time (T1) is a function of one of them.
External. 14 phones, 12 of them Samsung, 9 on Android 16, all from one research corpus that is not a random sample of phones. Five-number summaries discard the shape of each distribution between the quartiles, and one phone, whose first boot record looks like a clock default, sets the maxima of the age and the gap. Release effects and vendor effects could not be tested. Parameters that rest on bug reports (wall-clock steps, 4 of 7) come from a different class of evidence.
Construct. “Realistic” here means that the distribution of a stated feature matches; nothing is claimed about record content or the plausibility of a story. The ledger is exact for generated devices and says nothing about what happened on real ones. The two baseline solvers are simple readers chosen to show that the tasks discriminate; no tool is compared.
Limitations. No iOS devices, bug reports, record content, application-specific databases, diurnal rhythm, or cause attribution for counter movement; sample datasets are written at scale 0.01, where tables are small and counters hard to rebase.
7 Conclusion
A generator whose parameters are summaries of 14 measured phones reproduces 16 of 21 single features of held-out phones as well as the phones reproduce each other, and a documentation-only generator reproduces 4. It does not reproduce relations across stores, so it should be used for what it is calibrated for and its scenario parameters changed deliberately otherwise. On the generated devices, reading an empty result as “no” was unsound for 53% of questions, a rule-based reader for 2.5%. Future work: more phones and vendors so that cross-store relations can be fitted and tested, iOS, and bug reports.
Literature search. Twelve queries on 29 October 2026 to the arXiv, Zenodo, OpenAlex and Crossref interfaces (arXiv with 8 more; Semantic Scholar and dblp largely unavailable); abstracts of the closest 15 works read, among them full text of [8], [9], [12], [13], [15], [16] and [23]; the others by abstract or Crossref record, as tagged in the reference list. The search pages of the ACM Digital Library, DFRWS and Forensic Science International: Digital Investigation were not reachable by script, so those venues were searched only through the aggregators.
Artifact Availability
The generator, the profiles, the sample datasets (devices, ledgers, questions), the baseline solvers and the scoring code will be released as one package in the Lab's public artifacts repository, with no release date set (Python standard library only; code under the MIT licence, profiles, data and text under CC BY 4.0). The package is not public yet, and the sentences that follow describe it as planned. The package reproduces the round-trip test, the benchmark tables and the parameter tables from the synthetic data. It cannot reproduce the validation tables, which need per-phone feature rows of the Lab’s research corpus; those rows and the runner that reads the Lab’s earlier reports are kept by the Lab. The package includes the validation code and a synthetic features file so that others can run the protocol on their own measurements. The profile to be released is rounded to two significant digits; the benchmark tables use it, and the validation tables use the unrounded summaries. Pending write-ahead-log files of the samples differ between runs in their salt and checksum bytes; every other file is reproduced byte for byte from the fixed seeds. The extractions cannot be released.
Data Handling Statement
The evidence for this study is drawn from the Lab’s research corpus of mobile device extractions and was read only, together with synthetic data, namely every generated device. The study used per-device aggregate counts, shares and durations from the stored reports of earlier runners and opened no extraction itself. The report gives aggregate counts and device classes. It contains no identifiers, personal data, message or file content, or case information. The profile releases summaries of features over the 14 phones, rounded to two significant digits, not per-phone rows. The sample datasets contain invented identifiers only.
Declaration of AI Assistance
The authors conceived the research questions and directed the work. The generator, the measurement code, the data checks and the first draft of this report were produced with AI assistance (Anthropic Claude). The authors reviewed the results and the text and are responsible for the content.
References
- Daniel Addai, Sanders Larweh Tetteh, Selorm Adjabeng, and Kingsley Albert Egyir. 2026. Retention Horizons of Volatile Stores in iOS and Android Extractions. Lab Technical Report LAB-TR-0021, version 1.0. SectraTEK Lab, Prudential Associates. https://lab.sectratek.com/research/reports/lab-tr-0021/
- Daniel Addai, Selorm Adjabeng, Kingsley Albert Egyir, and Sanders Larweh Tetteh. 2026. Most Ordinary Phones Trip a Naive Clock-Anomaly Check: Base Rates of Clock Anomalies in Unselected Mobile Extractions. Lab Technical Report LAB-TR-0022, version 1.0. SectraTEK Lab, Prudential Associates. https://lab.sectratek.com/research/reports/lab-tr-0022/
- Daniel Addai, Kingsley Albert Egyir, Sanders Larweh Tetteh, and Selorm Adjabeng. 2026. Zeroed Cells, Kept Pages, Pending Logs: What SQLite Leaves of a Deleted Row on 47 iOS and Android Devices. Lab Technical Report LAB-TR-0023, version 1.0. SectraTEK Lab, Prudential Associates. https://lab.sectratek.com/research/reports/lab-tr-0023/
- Daniel Addai, Selorm Adjabeng, Kingsley Albert Egyir, and Sanders Larweh Tetteh. 2026. Device-Life Boundary Indicators and the Classification of Restored Versus Native Records on Android. Lab Technical Report LAB-TR-0028, version 1.0. SectraTEK Lab, Prudential Associates. https://lab.sectratek.com/research/reports/lab-tr-0028/
- Daniel Addai, Kingsley Albert Egyir, Sanders Larweh Tetteh, and Selorm Adjabeng. 2026. Measuring the Answerability of Date-Bound Questions from Mobile Device Extractions. Lab Technical Report LAB-TR-0029, version 1.0. SectraTEK Lab, Prudential Associates. https://lab.sectratek.com/research/reports/lab-tr-0029/
- Daniel Addai, Sanders Larweh Tetteh, Selorm Adjabeng, and Kingsley Albert Egyir. 2026. Upper Bounds on Record Loss from Identifier Counters in Android Media and Message Stores. Lab Technical Report LAB-TR-0030, version 1.0. SectraTEK Lab, Prudential Associates. https://lab.sectratek.com/research/reports/lab-tr-0030/
- Alberto A. Ceballos Delgado, William B. Glisson, George Grispos, and Kim-Kwang Raymond Choo. 2022. FADE: A forensic image generator for Android device education. WIREs Forensic Science 4, 2 (2022), e1432. https://doi.org/10.1002/wfs2.1432 (Crossref record checked; content taken from [13]; not re-verified. Crossref gives 2021 as the online year.)
- Markus Demmel, Thomas Göbel, Patrik Gonçalves, and Harald Baier. 2024. Data Synthesis Is Going Mobile—On Community-Driven Dataset Generation for Android Devices. Digital Threats: Research and Practice 5, 3 (2024), 1–19. https://doi.org/10.1145/3688807 (Full text read, in the accepted-manuscript form hosted by the authors’ institution.)
- Xiaoyu Du, Christopher Hargreaves, John Sheppard, and Mark Scanlon. 2021. TraceGen: User activity emulation for digital forensic test image generation. Forensic Science International: Digital Investigation 38 (2021), 301133. https://doi.org/10.1016/j.fsidi.2021.301133 (Full text read.)
- Simson Garfinkel, Paul Farrell, Vassil Roussev, and George Dinolt. 2009. Bringing science to digital forensics with standardized forensic corpora. Digital Investigation 6 (2009), S2–S11. https://doi.org/10.1016/j.diin.2009.06.016 (Crossref record checked; not read.)
- Tilmann Gneiting, Fadoua Balabdaoui, and Adrian E. Raftery. 2007. Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society: Series B 69, 2 (2007), 243–268. https://doi.org/10.1111/j.1467-9868.2007.00587.x (Abstract read.)
- Thomas Göbel, Stephan Maltan, Jan Türr, Harald Baier, and Florian Mann. 2022. ForTrace—A holistic forensic data set synthesis framework. Forensic Science International: Digital Investigation 40 (2022), 301344. https://doi.org/10.1016/j.fsidi.2022.301344 (Full text read.)
- Thomas Göbel, Harald Baier, and Frank Breitinger. 2023. Data for digital forensics: Why a discussion on “How realistic is synthetic data” is dispensable. Digital Threats: Research and Practice 4, 3 (2023), 1–18. https://doi.org/10.1145/3609863 (Full text read.)
- Thomas Göbel, Frank Breitinger, and Harald Baier. 2025. Optimising data set creation in the cybersecurity landscape with a special focus on digital forensics: Principles, characteristics, and use cases. Forensic Science International: Digital Investigation 52 (2025), 301882. https://doi.org/10.1016/j.fsidi.2025.301882 (Abstract read; full text not accessible.)
- Patrik Goncalves, Andreas Attenberger, and Harald Baier. 2022. Smartphone data distributions and requirements for realistic mobile device forensic corpora. In Advances in Digital Forensics XVIII, IFIP AICT 653, 47–63. https://doi.org/10.1007/978-3-031-10078-9_3 (Full text read, author-hosted copy.)
- Patrik Gonçalves, Klara Dološ, Michelle Stebner, Andreas Attenberger, and Harald Baier. 2022. Revisiting the dataset gap problem—On availability, assessment and perspective of mobile forensic corpora. Forensic Science International: Digital Investigation 43 (2022), 301439. https://doi.org/10.1016/j.fsidi.2022.301439 (Full text read.)
- Cinthya Grajeda, Frank Breitinger, and Ibrahim Baggili. 2017. Availability of datasets for digital forensics—And what is missing. Digital Investigation 22 (2017), S94–S105. https://doi.org/10.1016/j.diin.2017.06.004 (Abstract read.)
- Margaux Michel, Dirk Pawlaszczyk, and Ralf Zimmermann. 2022. AutoPoD-Mobile—Semi-automated data population using case-like scenarios for training and validation in mobile forensics. Forensic Sciences 2, 2 (2022), 302–320. https://doi.org/10.3390/forensicsci2020023 (Abstract read; full text not accessible.)
- Sebastian Nemetz, Sven Schmitt, and Felix Freiling. 2018. A standardized corpus for SQLite database forensics. Digital Investigation 24 (2018), S121–S130. https://doi.org/10.1016/j.diin.2018.01.015 (Abstract read.)
- Dirk Pawlaszczyk, Philipp Engler, Ronny Bodach, Christian Hummert, Margaux Michel, and Ralf Zimmermann. 2025. AI-driven dataset creation in mobile forensics using LLM-based storyboards. Forensic Science International: Digital Investigation 55 (2025), 302002. https://doi.org/10.1016/j.fsidi.2025.302002 (Abstract read; full text not accessible.)
- Gábor J. Székely and Maria L. Rizzo. 2013. Energy statistics: A class of statistics based on distances. Journal of Statistical Planning and Inference 143, 8 (2013), 1249–1272. https://doi.org/10.1016/j.jspi.2013.03.018 (Crossref record checked; not read.)
- Lena L. Voigt, Felix Freiling, and Christopher J. Hargreaves. 2024. Re-imagen: Generating coherent background activity in synthetic scenario-based forensic datasets using large language models. Forensic Science International: Digital Investigation 50 (2024), 301805. https://doi.org/10.1016/j.fsidi.2024.301805 (Abstract read; full text not accessible.)
- Lena L. Voigt, Felix Freiling, and Christopher Hargreaves. 2025. A metrics-based look at disk images: Insights and applications. Forensic Science International: Digital Investigation 52 (2025), 301874. https://doi.org/10.1016/j.fsidi.2025.301874 (Full text read.)
- York Yannikos and Christian Winter. 2013. Model-based generation of synthetic disk images for digital forensic tool testing. In 2013 International Conference on Availability, Reliability and Security (ARES), 498–505. https://doi.org/10.1109/ARES.2013.65 (Abstract read.)