Reading Android bug reports in a single pass

Summary

• Before this change, a bug report zip given to ALEAPP was staged, but nothing read the main text inside it. Only the existing tombstone and ANR artifacts matched anything in the zip.
• The change adds a reader for that text, scripts/dumpstate_reader.py, and four artifacts in the category Android Bug Report: Device Facts, Sections, Installed Packages and Package Times.
• The reader reads the text once, line by line, and never changes the staged file. A zip member is streamed, not extracted.
• The Sections table lists every section the text opens, with its line range, the duration the platform printed, whether the platform cut the dump at its time limit, and which artifact presents it. A section no artifact reads says "not read".
• Package Times keeps every time as printed and adds a Time Note column that marks unset times and values that the phone image shares across many packages.
• The tables present data only: no severity, no finding, no interpretation. The pull request was merged on 26 October 2026.

The issue

An examiner receives an Android bug report: the zip a phone writes when it is asked for a diagnostic snapshot. Its main member is one long text, the dumpstate file, with the build properties, the package manager's list of installed apps and a section for every system service. In the five research reports behind this work the text had between 343 and 656 sections, and in the pull request's performance check one text was 214 MB.

Before this change an examiner who ran ALEAPP on such a zip got no rows about any of it. The zip was staged member by member, but only the tombstone and ANR artifacts matched anything in it. An empty result looked like a report with nothing in it (Figure 1).

Two further problems appear once the text is read. The first is time. The package manager prints install and update times as wall-clock text, and for an app that sits on the read-only system partitions the value is read from the phone image, not from anybody installing anything. One value can therefore be printed for hundreds of unrelated packages. In the five research reports, 62 to 91 percent of the package time rows carried such a placeholder or image-wide value (Figure 3). A timeline built from them would put hundreds of events at one moment that belongs to the firmware.

The second is silent partial reading. The platform stops a slow service dump at a time limit, so a list can be short because the dump was cut, not because the phone held few apps. In one of the five reports, the Samsung A-series phone, the package dump ends with the platform's 10 second timeout marker, and 572 packages were read from it. A table that does not say so would read as complete.

The essence

The change adds one reader and four tables. The reader finds the bug report text, confirms it by its == dumpstate: banner, passes over it once and hands each section's lines to the parser registered for that section. The tables present the result as the platform printed it, with the line number of every row, and the Sections table says which parts of the text no table reads.

Two rules keep the tables honest. A time is converted to UTC only with the zone that the report itself records in persist.sys.timezone; where that property is missing, the UTC column is empty and the printed text is kept. And 1970-01-01 00:00:00 and 1969-12-31 23:59:59, the platform's "no time" values, get no UTC value at all.

Our methodology

The format was learned from the structure of real bug reports and checked against the platform sources cited in LAB-TR-0001. The text opens with a banner and header lines, then repeats two kinds of block: a command section that starts with a line like ------ TITLE (command) ------ and a service section that starts with DUMP OF SERVICE name:. A section ends at its duration line when the platform prints one, and otherwise at the next header. Inside a service, the platform can print a DUMP TIMEOUT marker when it stopped the dump at its limit.

The reader therefore does not trust a file name. It confirms the banner in the first 8,192 bytes before it reads anything, and a file that matches the name patterns but not the banner is left out and logged. It counts lines on the line-feed byte only, so a lone carriage return or form feed inside a line cannot move the numbers away from what a text editor shows. Figure 2 follows the path from file to report columns.

The fixtures are synthetic. bugreport_fixture.py builds them from the structure of real reports with invented values, with package names under com.example.*. Eight constructed layouts run through each of the four artifacts give 32 new test cases, and 47 new unit tests were added. With them the repository's case run went from 171 to 203 passes.

The real reports were used to check the reader, never as fixtures. Each row count was compared with a count made from the raw text with plain regular expressions, and with the rows of an independent parser written for another tool, field by field. For Package Times the comparison included the factory-stamp marks, and the same rows were marked in both implementations (Figure 3). The repository's own output checker found only uniform or empty columns, such as Package Source on some builds, and the artifact notes now name each of them.

Damaged and unusual input was tried on variants of one real text: empty, banner only, cut at half and inside the package list (105 complete packages kept), CRLF line endings, stray carriage returns, form feeds and NUL bytes, Latin-1, a byte order mark, UTF-16 text (not read, by design), a 150 MB single line, and the text repeated twelve times (twelve times the package rows, 241 MB peak). None raised an error. The Lab's validation of the same reader (LAB-TR-0001) found, with a test of this kind, that one damaged line inside the package dump had cut 284 packages to 15 without any error; a section heading must now end with a colon.

Speed and memory were measured on the whole run. The four artifacts took 12 seconds on a text of 163 MB and 214 MB, including the load of the tool's 1,360 modules, with a 230 to 241 MB peak working set; the text itself is read in 1 to 3 seconds. A privacy scan of every added file (zips and nested zips opened), every commit message and the branch name against the identifiers of the five reports found no hit. That scan was later shown to be incomplete: three numbers copied into fixtures were found afterwards and replaced (see the post on ALEAPP #1809).

The value

For an examiner who holds a bug report and little else, this turns a text of hundreds of megabytes into tables that can be sorted, filtered and exported, with each row tied to a line number and a file. Device Facts answers which build, patch level and time zone the report records. Installed Packages answers which packages the package manager listed, with version, installer fields and per-user state as printed. Sections answers a question that is easy to forget to ask: what did the tool not look at?

Package Times is where the most care went. A row with no note shows the value the dump prints for that package. A row with a note keeps its time, so a timeline can still be filtered on the column, and the note says why the time should not be read as an install time (Figure 4).

The tables are the package manager's state at the moment of capture. An app removed before then has no row, and an app removed for one user can still show a row with installed=false for that user. A time or an installer field does not show how an app was installed, or that it was ever used.

The factory-stamp thresholds (20 packages, 2 seconds and 30 seconds) were chosen on these five reports. They are a rule of this reader, not a statement of the platform, and a factory value shared by fewer than 20 packages is not marked. Android 11 to 14, other vendors and a bug report zip found inside a full file system extraction were not examined on real data. Every real report was in a UTC+0 zone, so offsets and daylight-saving edges are tested on constructed text only.

Where this sits in the Lab's research

This change is the foundation of Lab Technical Report LAB-TR-0001, "Making Silent Failures Visible: A Single-Pass Section Reader for Android Bug Reports". The report describes the reader's design (the section index, timeout detection, bounded memory), its validation on seven research bug reports, and the seven reader defects that validation found. It counts 158 artifacts in 22 groups on the same reader; this pull request is the first of the seven that released them. LAB-TR-0001 also reports that, on the five reports compared, the stamp sets of the reader and of a second parser were equal.

Cite: Addai, D. (2026). Making Silent Failures Visible: A Single-Pass Section Reader for Android Bug Reports. Lab Technical Report LAB-TR-0001, version 1.1. Zenodo. doi:10.5281/zenodo.23719508. Read it on this site at LAB-TR-0001.

The public article Android bug reports, category by category (LAB-0002, early research) describes what the sections of a bug report hold and which questions they can answer.

Sources

Every number and claim in this post, and where it comes from. Numbers that could not be sourced were left out.

Related posts

• Usage, settings and boot history from bug reports. Five more artifacts on the same reader.
• Bug report artifact groups and invented fixture values. Categories per question, device identifiers, and the fixture numbers replaced.