Skip to content

fitsverify: fix wrtserr reading one row past its message array - #166

Open
cruzzil wants to merge 8 commits into
heasarc:developfrom
cruzzil:fix/fitsverify-wrtserr-stack-overread
Open

cruzzil wants to merge 8 commits into
heasarc:developfrom
cruzzil:fix/fitsverify-wrtserr-stack-overread

Conversation

@cruzzil

@cruzzil cruzzil commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

A7 from an audit of the fitsverify sources (INSPECTION there). Confirmed real, and
reachable: CFITSIO's error stack holds up to 25 messages (errmsgsiz in
fitscore.c), so a broken enough file can hand wrtserr more than the 20 rows
it has.

The bug

utilities/fvrf_misc.c:wrtserr():

char tmp[20][80];
...
while(nstack < 20) {
    tmp[nstack][0] = '\0';
    i = fits_read_errmsg(tmp[nstack]);
    if(!i && tmp[nstack][0]=='\0') break;
    nstack++;
}
...
for(i=0; i<=nstack; i++) fprintf(out,errfmt,tmp[i]);

The inclusive print bound is deliberate — when the read loop breaks,
tmp[nstack] is the empty string it just cleared, and printing it emits the
trailing blank line the report format expects. But when the stack holds 20+
messages the loop exits on its own condition with nstack == 20, and the print
loop reads tmp[20], 80 bytes past the array.

Second, fits_read_errmsg writes up to FLEN_ERRMSG (81) bytes into rows that
are only 80 wide, so a maximum-length message overruns into the next row and
truncates the message stored there.

The fix

char tmp[21][FLEN_ERRMSG] fixes both, plus one line to terminate the row the
print loop ends on — without it the array access is in bounds but the contents
are still uninitialised when the read loop stopped on its own count, so the
report would print stack garbage instead of the blank line.

Test

tests/test_fitsverify.c already #includes fvrf_misc.c, so wrtserr() can
be called directly: with 2, 25, and one maximum-length message on the stack.
Before each call the test leaves a known 'Z' pattern on the stack where tmp
will sit, so the out-of-bounds read is deterministic rather than dependent on
what happened to be there — without the fix the report ends in a row of Zs
and the test fails. The maximum-length case checks that the message after it is
still intact.

Merge order

A stacked set of eight fitsverify fixes on top of a build-system base,
all targeting develop. Merge in this order:

  1. Make the checked-in Makefile.in reproducible from Makefile.am #170 - Makefile.in reproducible from Makefile.am (build system, no code change)
  2. fitsverify: fix NULL dereference in get_cmp on a complex value with no comma #160 - get_cmp NULL dereference
  3. fitsverify: do not divide by a zero TFORM substring width #161 - TFORM zero substring width
  4. fitsverify: bound the bit column justification report #162 - bit column report overflow
  5. fitsverify: report the real card number in init_hdu diagnostics #163 - uninitialised keyword index
  6. fitsverify: keep test_agap's column template inside the row #164 - test_agap column template
  7. fitsverify: stop scanning table data after a failed read #165 - test_agap short read
  8. fitsverify: fix wrtserr reading one row past its message array #166 - wrtserr message array <- this PR
  9. fitsverify: bound the naxes[] accesses on a binary table #167 - naxes[] bounds

#170 is split out so that the Makefile.in regeneration is reviewable on
its own; the eight fixes share tests/test_fitsverify.c, which is why they
are stacked on each other rather than independent. Merged in this order
there are no conflicts. Until the ones before it are merged this PR's diff
also shows their commits; it shrinks to just its own change as they go in.

Provenance

Found by an agent, fixed by an agent, reviewed by a human. The defect list came
out of automated analysis, the reproducer, the fix and the test were written by
an agent, and a human reviewed the change on the fork
(cruzzil#16) before it was sent here.

cruzzil and others added 8 commits August 10, 2026 18:45
Running automake over this tree reproduces Makefile.in byte for byte, with one
exception:

    DISTCHECK_CONFIGURE_FLAGS = --enable-reentrant

That line is in the generated Makefile.in but has never been in Makefile.am,
so it can only have been added to the generated file by hand.  Any
regeneration silently drops it, which takes --enable-reentrant out of the
configure run that make distcheck performs - so the reentrant build quietly
stops being covered.

Move it to Makefile.am, where automake emits it in the same place it already
occupies.  Makefile.in is unchanged by this commit, and that is the point:
regenerating with automake 1.18.1, the version that produced the current file,
is now a no-op, so later changes to Makefile.am can be regenerated without
losing anything.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…o comma

get_cmp() only assigns pr_end and pi_beg inside the `if(!set_comm && *p ==
',')` branch.  A complex-looking value with no comma - e.g. a card reading
`CPLX    = (1 2)` - records NO_COMMA and then falls through to

    *pr_end = '\0';

which writes through a NULL pointer, and to

    while(isspace((int)*pi_beg) && *pi_beg != '\0')  pi_beg++;

which reads through a pointer that was never initialised at all.  fitsverify
dies with SIGSEGV on any file containing such a card.

There is no imaginary part to analyse when there is no comma, so skip the
real/imaginary breakdown entirely and let the NO_COMMA error already recorded
be the reported diagnostic.  pi_beg is also initialised at its declaration.

fitsverify now reports

    *** Error:   keyword #4, CPLX : Complex value "(1 2" misses ",".

and exits 1 instead of dumping core.

This adds tests/test_fitsverify.c, which the rest of the fitsverify fixes
build on: it #includes the fvrf_*.c sources that can be driven directly and
stubs the few symbols utilities/ftverify.c would otherwise provide.  The first
tests cover the crashing values along with well formed complex values, values
with a missing closing paren, and values with too many commas.

Makefile.in is regenerated with automake 1.18.1 for the new test program.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The local-convention "rAw" check in test_bin_ext() guards the substring width
with isdigit(), which admits '0':

    width = (int)strtol(p,NULL,10);
    if(repeat%width != 0)  {

A binary table column declared as

    TFORM1  = '10A0    '

therefore evaluates 10 % 0 and fitsverify dies with SIGFPE.

A zero substring width is itself an invalid TFORM, so report it as an error
and move on to the next column rather than computing the multiple.  The file
is now reported instead of crashing the verifier:

    *** Error:   TFORM 10A0 of column 1: the substring width must not be zero.

The check sits in the middle of test_bin_ext(), so a whole file is the only
way to reach it: tests/test_fitsverify.c gains the machinery to build one and
run the fitsverify binary over it, and a test that walks a set of A format
TFORMs, including the zero width ones, failing if the process is killed by a
signal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The "X vector ... is not left justified" report appends five characters per
byte of the column to errmes, which is a 256 byte buffer:

    for (l = 1; l<= repeat[i]; l++) {
       sprintf(comm, "0x%02x ", (unsigned char) data[k*repeat[i]+l]);
       strcat(errmes,comm);
    }

repeat[i] here is the width of the column in bytes, and nothing bounds it.  A
table with

    NAXIS1  =                  500
    TFORM1  = '3996X   '

whose fill bits are set writes 2500 bytes into the 256 byte buffer; glibc's
fortify check turns that into SIGABRT.

Track the used length and stop appending before the buffer is full, leaving
room for an elision marker and the trailing text.  Reports that already fitted
are unchanged; wider ones now end

    ... 0xff 0xff ... is not left justified.

The new test covers both a narrow bit column, whose report fits, and a wide
one: it fails if the process is killed by a signal, or if the report lists
more elements than errmes can hold - which is what catches the overflow on
builds without _FORTIFY_SOURCE, where it is silent corruption rather than an
abort.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
init_hdu() parses the first cards of every HDU into

    FitsKey tmpkey;

an automatic struct whose kindex member is never assigned.  It is then read
and printed, directly at the "is not allowed" and "is duplicated" messages and
indirectly through check_log()/check_str()/check_int(), which all print
pkey->kindex.  The result is a garbage keyword index in user facing output:

    *** Error:   Keyword #1418221847, XTENSION: "1234" is not a string.

Zero the struct at its declaration and set kindex to the card number that is
already being passed to fits_parse_card at each of the three call sites: 1 for
the first card of the HDU, 1+j in the NAXISn loop, m+1 for END.  The same file
now reports

    *** Error:   Keyword #1, XTENSION: "1234" is not a string.

Healthy files are unaffected: the field is only ever printed on an error path.

The new test runs the fitsverify binary over an extension whose XTENSION value
is not a string and requires the reported index to be the card's own position.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_agap() builds a NAXIS1 long template marking which bytes of a row belong
to a data column, driven by the TBCOLn and TFORMn keywords:

    temp = (int*)malloc(rowlen * sizeof(int));
    ...
    for (t = tbcol; t < tbcol+width; t++) temp[t-1]=1;

Nothing bounds that write against rowlen.  CFITSIO does range check TBCOLn
against the row length, but ffainit skips those checks entirely when the row
length is zero, and NAXIS1 = 0 is accepted.  An ASCII table with

    NAXIS1  =                    0
    TBCOL1  =            100000000
    TFORM1  = 'A20     '

therefore writes 100000000 ints outside a zero sized allocation; fitsverify
dies with SIGSEGV, and with a smaller TBCOL1 it silently corrupts the heap.
t-1 also writes below the buffer when TBCOL1 is 0.

Clamp the template writes to the row that was allocated for them.  Three
related problems in the same loop are fixed with it:

  - width and tbcol were uninitialised locals;
  - status was never reset inside the loop, so after any failed CFITSIO call
    every later call short circuited and left the previous column's width and
    start in place - each column now starts from a clean status and is skipped
    if any of its keywords cannot be read;
  - the zeroing loop counter was an int while rowlen is a LONGLONG.

keyname is also sized FLEN_KEYWORD rather than 9, so building "TFORM%d" can't
overrun it.

Output on well formed files is unchanged (verified byte for byte against
testprog.fit).  The new test runs the fitsverify binary over the zero row
length cases and fails if it is killed by a signal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_agap() checks the status of fits_read_tblbytes() and reports the failure,
then falls through and scans the buffer anyway.  On a file truncated inside
the table data that means reporting on rows which are not in the file: the
counts come from whatever the buffer happened to hold.

The read error is real and CFITSIO reports it correctly - the buffer layer is
not at fault here - so the fix belongs in fitsverify: stop the loop rather
than scan rows that were never read.

On a 500 row ASCII table whose first row holds ten non-ASCII bytes, truncated
so that the data cannot be read:

    before: *** Error:   tried to move past end of file
            *** Error:   row 1 contains non-ASCII characters.
            *** Error:   This ASCII table contains 20 non-ASCII-text characters
    after:  *** Error:   tried to move past end of file

Note the count of 20 for a file that contains ten such bytes.  The same file
untruncated still reports the ten, so nothing legitimate is silenced.

The new test checks both halves: the whole file must report its bad bytes and
the exact count, the truncated one must not report on data it could not read.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
wrtserr() collects the CFITSIO error stack into

    char tmp[20][80];

then prints it with

    for(i=0; i<=nstack; i++) fprintf(out,errfmt,tmp[i]);

The inclusive bound is deliberate: when the read loop breaks, tmp[nstack] is
the empty string it just cleared, and printing it emits the trailing blank
line the report expects.  But when the stack holds 20 or more messages the
loop exits on its own condition with nstack == 20, and the print loop then
reads tmp[20] - 80 bytes past the array.  CFITSIO's stack holds up to 25
messages (errmsgsiz), so this is reachable from a badly broken file.

The rows are also one byte too narrow: fits_read_errmsg writes up to
FLEN_ERRMSG (81) bytes, so a maximum length message overruns into the next
row and truncates whatever was there.

Declare tmp as [21][FLEN_ERRMSG], which fixes both, and terminate the row the
print loop ends on so that it is an empty string in the case where the read
loop stopped on its own count rather than on an empty message.

The new test calls wrtserr() directly with 2, 25 and one maximum length
message on the stack, after leaving a known pattern on the stack where tmp
will sit, and checks the report holds the messages and nothing else.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@cruzzil
cruzzil force-pushed the fix/fitsverify-wrtserr-stack-overread branch from b8cfc9d to fb90e63 Compare August 10, 2026 10:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant