Skip to content

Stop the integration-test build from recompiling the root modules - #94

Merged
alexander-yevsyukov merged 10 commits into
masterfrom
deterministic-coverage
Oct 19, 2026
Merged

alexander-yevsyukov merged 10 commits into
masterfrom
deterministic-coverage

Conversation

@alexander-yevsyukov

Copy link
Copy Markdown
Contributor

Problem

codecov/project (threshold 0.05%) failed pull requests at random. Between CI runs of
the same tree, Codecov credited some api files with full coverage or with none:
FieldTypeExts.kt, FilePatterns.kt, TypeExts.kt, DescriptorExts.kt,
value/Options.kt, MessageTypeDependencies.kt, ValueConverter.kt — up to 182 hits.
api's own FilePatternsSpec and OptionsSpec lost their hits too, so this was not
about cross-module attribution.

Root cause

tests/settings.gradle.kts made the integration-test build a composite of the root one
(includeBuild("..")). So:

  1. integrationTest runs the tests build after every root test task and, in practice,
    before koverXmlReport. Gradle substitutes the compiler-* artifacts with the root
    projects and runs :compiler:api:launchSpineCompiler, kspKotlin, compileKotlin,
    and jar from there, in the root's api/build.
  2. These tasks are not up to date: the Validation Gradle plugin writes
    JavaValidationRenderer.pb.json into the settingsDirectory input of every
    LaunchSpineCompiler in a doFirst action, after Gradle has fingerprinted the inputs.
    (The same is why CI's dokkaGenerate step recompiles :api.)
  3. The rerun regenerates the sources. The Validation code generator names the fields of
    compiled regular expressions with SecureRandomString, so the sources really differ
    (TypeUrlPrefixPattern_C0Kx… vs TypeUrlPrefixPattern_YxGa…). Kotlin's incremental
    compiler then recompiles the hand-written files that use them. The class files differ
    only in their constant-pool layout, but that changes their JaCoCo class IDs.
  4. koverXmlReport analyzes the rewritten class files, finds no execution data with
    matching IDs, and reports those classes as uncovered.

Reproduced locally with a clean build of this branch before the fix. The exec files of
all test tasks matched 249/249 executed api classes as compiled before
integrationTest, but 5 of them (FieldTypes, FilePatterns, Types,
DescriptorExtsKt, OptionValue) mismatched the classes the report read. The report had
FieldTypeExts.kt at 0/73 lines and FilePatterns.kt at 0/9.

Fix

The tests build no longer includes the root build. It consumes what localPublish
(already a dependency of integrationTest) puts into Maven Local:

  • pluginManagement resolves the io.spine.compiler plugin from Maven Local at the
    version under development (in-place-consumer applies it via plugins {}).
  • A beforeProject hook declares Maven Local the exclusive source of
    io.spine.tools:compiler-* at the version under development, in both the build-script
    and project repositories. Repository order would not do it: standardToSpineSdk() lists
    Maven Local last, and the tests root script adds the repositories to the subprojects
    before their own scripts run. With the rule, a remote repository that already has the
    same version never serves it. If localPublish did not run, the build fails instead.
    Verified: with the version under test absent from Maven Local but present in the
    registry, resolution fails; other compiler-* versions still resolve remotely.
  • tests/README.md explains this and how to run the tests build on its own.

After the fix, both a cached and an uncached (--no-build-cache) clean build:

  • run no root tasks from the nested build;
  • leave the api class files and generated sources unchanged across integrationTest;
  • match 249/249 executed api classes;
  • produce the same Kover report, file by file: FieldTypeExts.kt 73/73, FilePatterns.kt
    7/9, TypeExts.kt 17/19, DescriptorExts.kt 45/46, value/Options.kt 42/42,
    MessageTypeDependencies.kt 30/30, ValueConverter.kt 11/15.

The uncached run had freshly randomized generated sources (8 files differed from the cached
run), yet the coverage was the same.

Follow-ups (in validation)

  • Generate deterministic identifiers instead of SecureRandomString (it also defeats the
    build cache for every consumer).
  • Write the renderer settings before the inputs of LaunchSpineCompiler are fingerprinted.

Version: 2.0.0-SNAPSHOT.077 (first .076; re-bumped because #90 now claims .076).

🤖 Generated with Claude Code

alexander-yevsyukov and others added 8 commits October 16, 2026 22:24
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`tests/settings.gradle.kts` made the `tests` build a composite of the root
one. `integrationTest` runs it after every root `test` task and, usually,
before `koverXmlReport`, so Gradle reran `:api:launchSpineCompiler`,
`compileKotlin`, and `jar` of the root build in its own `api/build`. The
Validation plugin keeps `launchSpineCompiler` out of date (it adds its
settings file to the task's inputs in `doFirst`), and its generated code
uses random identifiers. The recompiled classes got new JaCoCo class IDs,
and the root report dropped their coverage at random.

The `tests` build now consumes what `localPublish` puts into Maven Local,
resolving the Compiler plugin and the `compiler-*` artifacts from there
first, so it no longer touches the root project directories.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The `tests` root script adds the standard repositories to its subprojects
before their own scripts run, so the Maven Local repository appended in
`beforeProject` came last for the project dependencies. An exclusive content
filter, scoped to the version under development, does not depend on the
repository order, and it fails the build if `localPublish` did not run
instead of falling back to a remote copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-bump for a version collision: PR #90 now claims `2.0.0-SNAPSHOT.076`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 16, 2026 22:12
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-16T22:15:51.298415Z 3aa5bd4 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Nested Gradle resolution and repeatable coverage uploads still need CI confirmation and final human review.

No open issues 🎉

What changed in this PR

Isolates integration tests from the root build to prevent recompilation from invalidating coverage data.

Changes:

  • Replaces composite-build resolution with locally published Compiler artifacts.
  • Documents standalone integration-test execution.
  • Updates the development version and dependency reports to .077.
File Description
version.gradle.kts Bumps the Compiler version to .077.
tests/​settings.gradle.kts Removes the root composite build and configures Maven Local resolution.
tests/​README.md Documents publication prerequisites and build isolation.
docs/​dependencies/​pom.xml Updates the project version.
docs/​dependencies/​dependencies.md Refreshes report versions and timestamps.
.agents/​tasks/​deterministic-coverage.md Records diagnosis, verification, and remaining CI confirmation.

Review effort: Balanced

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3aa5bd4446

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/settings.gradle.kts Resolved
Comment thread tests/settings.gradle.kts Resolved
alexander-yevsyukov and others added 2 commits October 16, 2026 23:17
# Conflicts:
#	docs/dependencies/dependencies.md
#	docs/dependencies/pom.xml
#	version.gradle.kts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 16, 2026 22:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The Gradle resolution changes warrant human confirmation that consecutive CI runs preserve stable coverage.

No open issues 🎉

Review effort: Balanced

@codecov

codecov Bot commented Oct 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.99%. Comparing base (79e573c) to head (3e168d9).

Additional details and impacted files
@@            Coverage Diff            @@
##             master      #94   +/-   ##
=========================================
  Coverage     76.99%   76.99%           
  Complexity      692      692           
=========================================
  Files           205      205           
  Lines          4021     4021           
  Branches        402      402           
=========================================
  Hits           3096     3096           
  Misses          796      796           
  Partials        129      129           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@alexander-yevsyukov
alexander-yevsyukov requested a balanced review from Copilot October 19, 2026 17:19
@alexander-yevsyukov alexander-yevsyukov moved this from 🏗 In progress to In Review in v2.0 Oct 19, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human review should confirm the reported end-to-end Gradle isolation and coverage results.

No open issues 🎉

Review effort: Balanced

@alexander-yevsyukov
alexander-yevsyukov merged commit 977762d into master Oct 19, 2026
12 checks passed
@alexander-yevsyukov
alexander-yevsyukov deleted the deterministic-coverage branch October 19, 2026 17:35
@github-project-automation github-project-automation Bot moved this from In Review to ✅ Done in v2.0 Oct 19, 2026
alexander-yevsyukov added a commit that referenced this pull request Oct 19, 2026
`2.0.0-SNAPSHOT.077` is taken by #94 on `master`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants