Search before asking
Paimon version
master, 142f823
Compute Engine
Engine independent, format tables.
Minimal reproduce step
Found by code inspection rather than from a failing job. Make a format writer throw an unchecked exception out of close(); several format writers wrap IO failures in RuntimeException or UncheckedIOException.
What doesn't meet your expectations?
FormatTableSingleFileWriter.close() routes only IOException to the cleanup path:
try {
if (writer != null) {
writer.close();
writer = null;
}
if (out != null) {
out.flush();
outputBytes = out.getPos();
committer = ((TwoPhaseOutputStream) out).closeForCommit();
out = null;
}
} catch (IOException e) {
LOG.warn("Exception occurs when closing file {}. Cleaning up.", path, e);
abort();
throw e;
} finally {
closed = true;
}
An unchecked throw out of writer.close() skips abort() entirely, so out is left open and the data staged for the two phase commit is never discarded. The finally block then sets closed = true, so a later close() returns at the top of the method and nothing else will clean up.
Note that abort() itself is written defensively, catching Throwable around both the closeForCommit and the discard calls. The problem is only that it never runs.
Anything else?
Fix shape: widen the catch around the cleanup decision to Throwable, keeping the existing rethrow behaviour for IOException.
Reported alongside two stream leak reports found in the same pass while reviewing #8962.
Are you willing to submit a PR?
Search before asking
Paimon version
master, 142f823
Compute Engine
Engine independent, format tables.
Minimal reproduce step
Found by code inspection rather than from a failing job. Make a format writer throw an unchecked exception out of
close(); several format writers wrap IO failures inRuntimeExceptionorUncheckedIOException.What doesn't meet your expectations?
FormatTableSingleFileWriter.close()routes onlyIOExceptionto the cleanup path:An unchecked throw out of
writer.close()skipsabort()entirely, sooutis left open and the data staged for the two phase commit is never discarded. Thefinallyblock then setsclosed = true, so a laterclose()returns at the top of the method and nothing else will clean up.Note that
abort()itself is written defensively, catchingThrowablearound both thecloseForCommitand thediscardcalls. The problem is only that it never runs.Anything else?
Fix shape: widen the catch around the cleanup decision to
Throwable, keeping the existing rethrow behaviour forIOException.Reported alongside two stream leak reports found in the same pass while reviewing #8962.
Are you willing to submit a PR?