Support rpm-based and apk-based runners in job-level and workflow-level instrumentation by Copilot · Pull Request #3268 · plengauer/Thoth

Copilot · 2026-03-24T12:51:16Z

Instrumentation actions blindly assumed Debian runners—fetching .deb and calling apt-get—causing failures on RPM-based runners (type: dpkg: not found, then otelapi.sh: No such file or directory).

Shared install logic (`actions/instrument/shared/install.sh`)

Detect OS via dpkg / rpm / apk presence and set pkg_ext and install_package accordingly
install_package is the only OS-specific function, exported and accepting multiple arguments; ensure_installed is a single common function that uses it via xargs
Debian path preserves existing apt cache optimization
RPM path supports dnf → yum → zypper → rpm fallback chain
APK path uses apk add --allow-untrusted
Non-deb paths do plain fetch-and-install (no caching), per design intent

Callers (`workflow/main.sh`, `repository/main.sh`, `checksuite/main.sh`)

Call install.sh unconditionally (it's always one of the three package managers)

Job-level (`inject_and_init.sh`)

Gate all caching optimization (@actions/cache restore/save, fast deb install) behind type dpkg
Add RPM and APK branches for otelcol-contrib download and install

Tests

Add job-rpm test job using container: registry.access.redhat.com/ubi9:latest with a simple hello-world step
Add job-apk test job using container: alpine:latest with a simple hello-world step

Original prompt

This section details on the original issue you should resolve

<issue_title>Support rpm-based runners in job-level and workflow-level instrumentation</issue_title>
<issue_description>in the shared code handlign the install, we shouldnt blindly fetch the debian and install it, but check the OS and fetch the apk/rpm if the os fits. for these new ones, lets not do all the caching opimiztation, just plain fetching and installing. for that, limit the caching optimization for job-level to debians only. debians shall be identified by the presence of dpkg. rpm based on when rpm is present, and alpines in a similar way

2026-03-16T17:36:03.0568903Z Current runner version: '2.332.0'
2026-03-16T17:36:03.0575252Z Runner name: '***'
2026-03-16T17:36:03.0576238Z Runner group name: '***'
2026-03-16T17:36:03.0577170Z Machine name: '***'
2026-03-16T17:36:03.0581358Z ##[group]GITHUB_TOKEN Permissions
2026-03-16T17:36:03.0583880Z Actions: write
2026-03-16T17:36:03.0584380Z ArtifactMetadata: write
2026-03-16T17:36:03.0584905Z Attestations: write
2026-03-16T17:36:03.0585353Z Checks: write
2026-03-16T17:36:03.0585814Z Contents: write
2026-03-16T17:36:03.0586279Z Deployments: write
2026-03-16T17:36:03.0586789Z Discussions: write
2026-03-16T17:36:03.0587243Z Issues: write
2026-03-16T17:36:03.0587698Z Metadata: read
2026-03-16T17:36:03.0588126Z Models: read
2026-03-16T17:36:03.0588579Z Packages: write
2026-03-16T17:36:03.0589039Z Pages: write
2026-03-16T17:36:03.0589495Z PullRequests: write
2026-03-16T17:36:03.0590084Z RepositoryProjects: write
2026-03-16T17:36:03.0590671Z SecurityEvents: write
2026-03-16T17:36:03.0591412Z Statuses: write
2026-03-16T17:36:03.0591957Z ##[endgroup]
2026-03-16T17:36:03.0594291Z Secret source: Actions
2026-03-16T17:36:03.0595049Z Prepare workflow directory
2026-03-16T17:36:03.0899061Z Prepare all required actions
2026-03-16T17:36:03.0934839Z Getting action download info
2026-03-16T17:36:03.6405445Z Download action repository 'plengauer/Thoth@v5.43' (SHA:b9414156be4866abb8fb061ac000611b2196bbf2)
2026-03-16T17:36:05.7275904Z Complete job name: export
2026-03-16T17:36:05.7921715Z ##[group]Run plengauer/Thoth/actions/instrument/workflow@v5.43
2026-03-16T17:36:05.7923712Z with:
2026-03-16T17:36:05.7924623Z   self_monitoring: false
2026-03-16T17:36:05.7925641Z   debug: true
2026-03-16T17:36:05.7926876Z   github_token: ***
2026-03-16T17:36:05.7928112Z   repository: ENG-TD-Universe/java-maven-sdlc-event-gimbet2
2026-03-16T17:36:05.7929586Z   workflow_run_id: 23157386077
2026-03-16T17:36:05.7930684Z   workflow_run_attempt: 1
2026-03-16T17:36:05.7931752Z   self_monitoring_anonymize: true
2026-03-16T17:36:05.7933435Z env:
2026-03-16T17:36:05.7934441Z   OTEL_SERVICE_NAME: eng-td-universe-workflows
2026-03-16T17:36:05.7936319Z   OTEL_EXPORTER_OTLP_ENDPOINT: ***
2026-03-16T17:36:05.7939179Z   OTEL_EXPORTER_OTLP_HEADERS: ***
2026-03-16T17:36:05.7940392Z   OTEL_EXPORTER_OTLP_PROTOCOL: http/protobuf
2026-03-16T17:36:05.7941596Z ##[endgroup]
2026-03-16T17:36:05.9248070Z ##[group]Validate Configuration
2026-03-16T17:36:05.9250191Z ##[endgroup]
2026-03-16T17:36:05.9251887Z ##[group]Ensuring rate limit
2026-03-16T17:36:06.1833750Z ##[endgroup]
2026-03-16T17:36:06.1835465Z ##[group]Install Dependencies
2026-03-16T17:36:06.1837675Z /home/runner/_work/_actions/plengauer/Thoth/v5.43/actions/instrument/workflow/main.sh: line 16: type: dpkg: not found
2026-03-16T17:36:06.1840004Z No debian, assuming dependencies are preinstalled.
2026-03-16T17:36:06.1841875Z ##[endgroup]
2026-03-16T17:36:06.1843567Z ##[group]Resolving Workflow Run Attempt
2026-03-16T17:36:06.1976293Z ##[endgroup]
2026-03-16T17:36:06.1978197Z ##[group]Resolving Jobs, Steps, Logs, and Artifacts
2026-03-16T17:36:06.8997428Z Artifacts ... done
2026-03-16T17:36:07.0997106Z Jobs and Steps ... done
2026-03-16T17:36:07.2026426Z Logs ... done
2026-03-16T17:36:07.2051287Z ##[endgroup]
2026-03-16T17:36:07.5511532Z ##[group]Determine Trace ID
2026-03-16T17:36:07.5512151Z ##[endgroup]
2026-03-16T17:36:07.5512678Z ##[group]Export
2026-03-16T17:36:07.5529053Z /home/runner/_work/_actions/plengauer/Thoth/v5.43/actions/instrument/workflow/main.sh: line 96: otelapi.sh: No such file or directory
2026-03-16T17:36:07.5547717Z node:internal/process/promises:392
2026-03-16T17:36:07.5548357Z       new UnhandledPromiseRejection(reason);
2026-03-16T17:36:07.5548929Z       ^
2026-03-16T17:36:07.5549100Z
2026-03-16T17:36:07.5553377Z UnhandledPromiseRejection: This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason "1".
2026-03-16T17:36:07.5555139Z     at throwUnhandledRejectionsMode (node:internal/process/promises:392:7)
2026-03-16T17:36:07.5555806Z     at processPromiseRejections (node:internal/process/promises:475:17)
2026-03-16T17:36:07.5556601Z     at process.processTicksAndRejections (node:internal/process/task_queues:104:32) {
2026-03-...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

- Fixes plengauer/Thoth#3260

<!-- START COPILOT CODING AGENT TIPS -->
---

💬 Send tasks to Copilot coding agent from [Slack](https://gh.io/cca-slack-docs) and [Teams](https://gh.io/cca-teams-docs) to turn conversations into code. Copilot posts an update in your thread when it's finished.

…el instrumentation - Update shared install.sh to detect OS type (dpkg/rpm/apk) and install the correct package format with the appropriate package manager - Update workflow, repository, and checksuite callers to invoke install.sh for rpm and apk systems too (not just dpkg) - Limit caching optimization in job-level instrumentation to Debian only - Add rpm and apk support for otelcol-contrib installation in job-level - Add job-rpm test using Red Hat UBI9 container image Co-authored-by: plengauer <100447901+plengauer@users.noreply.github.com> Agent-Logs-Url: https://github.com/plengauer/Thoth/sessions/a8d32120-5c08-4eba-aa58-2e031cb93188

plengauer

@copilot pls address feedback

…all unconditionally, add job-apk test - Refactor install.sh: install_package is the only OS-specific function, accepts multiple args, is exported and used by ensure_installed - Remove conditional around install.sh calls in checksuite, repository, and workflow main.sh - call unconditionally - Add job-apk test using alpine:latest container Co-authored-by: plengauer <100447901+plengauer@users.noreply.github.com> Agent-Logs-Url: https://github.com/plengauer/Thoth/sessions/a024e1b9-d469-47ee-a982-de657b789fe7

Copilot · 2026-03-24T22:41:26Z

@copilot pls address feedback

All feedback addressed in 33e1574:

Extracted common code in install.sh — install_package is the only OS-specific function (accepts multiple args, exported), ensure_installed uses it
Callers (checksuite, repository, workflow) now call install.sh unconditionally
Added job-apk test using alpine:latest container

Add a conditional definition for sudo if not available.

plengauer · 2026-04-11T11:05:12Z

@copilot rebase and resolve conflicts

…ased-runners # Conflicts: # actions/instrument/shared/install.sh Co-authored-by: plengauer <100447901+plengauer@users.noreply.github.com>

Copilot · 2026-04-11T15:31:25Z

@copilot rebase and resolve conflicts

Merged latest main in 3dc14d5, resolving conflicts in install.sh to incorporate architecture-specific package naming while preserving multi-OS support.

plengauer · 2026-04-15T11:15:06Z

@copilot resolve the merge conflicts in this pull request

…ased-runners # Conflicts: # .github/workflows/observability_workflow.yml Co-authored-by: plengauer <100447901+plengauer@users.noreply.github.com>

Copilot · 2026-04-15T11:18:46Z

@copilot resolve the merge conflicts in this pull request

Resolved in ffefffb — merged latest main and resolved the conflict in observability_workflow.yml (added "Autofix" to the workflows allowlist from main).

plengauer · 2026-05-24T15:59:14Z

@copilot resolve the merge conflicts in this pull request

…ased-runners Resolved conflicts: - inject_and_init.sh: kept multi-OS (dpkg/rpm/apk) structure with updated arch mapping from main - Lock files (.github/aw/, .github/workflows/*.lock.yml): took main's versions (auto-generated)

Copilot · 2026-05-24T16:03:28Z

@copilot resolve the merge conflicts in this pull request

Resolved in 573e374 — merged latest main, resolving conflicts in inject_and_init.sh (incorporated updated arch mapping aarch64→arm64, le→el into all three OS branches) and took main's auto-generated lock files.

plengauer · 2026-08-06T17:57:49Z

@copilot this PR has merge conflicts with main. Could you rebase and resolve the conflicts?

plengauer · 2026-08-26T06:48:59Z

@copilot this PR has failing checks that need fixing:

ci / github / job-apk — the new alpine:latest test job. Fails at step 4, Run actions/download-artifact@v8.0.0, after roughly one second; step 5 Run ./actions/instrument/job is then skipped, so the instrumentation this PR is meant to test never actually runs. See https://github.com/plengauer/Thoth/actions/runs/26366057355/job/77613355808
ci / github / job-rpm — the new registry.access.redhat.com/ubi9:latest test job. Identical failure shape: step 4 Run actions/download-artifact@v8.0.0 fails after about a second, step 5 Run ./actions/instrument/job skipped. See https://github.com/plengauer/Thoth/actions/runs/26366057355/job/77613355819
ci / all fails only as the roll-up of those two, not on its own: https://github.com/plengauer/Thoth/actions/runs/26366057355/job/77621488868
copilot-setup-steps also failed on this same head commit: https://github.com/plengauer/Thoth/actions/runs/26366056365

Two caveats on the analysis, so you don't chase the wrong thing:

These runs are from 2026-05-24 and their Actions logs have already been purged (the logs API returns HTTP 410), so the exact error text is not recoverable. The failing step name and its ~1s duration are all the evidence that survives.
Because both new container jobs die in the artifact-download step — before actions/instrument/job is ever invoked — this most likely is not a bug in the new install.sh / inject_and_init.sh OS-detection logic. A sub-second actions/download-artifact failure inside a custom container usually means the container can't run the action's JS entrypoint (the runner-injected Node needs glibc, which alpine does not provide) or the artifact simply wasn't present for that job. Worth verifying before touching the install logic.

Suggested fix direction: give job-rpm and job-apk whatever their container needs for actions/download-artifact to run (e.g. install the required libs in the container, or use a glibc-based Alpine variant), or drop the artifact dependency from these two hello-world jobs if they don't actually need it. Either way, these two jobs need a fresh CI run to confirm, since nothing from May is diagnosable anymore.

(🤖 generated by Claude)

plengauer · 2026-09-11T16:03:00Z

@copilot here's what this PR needs before it's mergeable. The last two requests here (rebase on 2026-08-06, failing checks on 2026-08-26) never ran. The first session was cancelled, and the second died at start-up with Model "claude-opus-4.6" is not available (job). Nothing has been pushed since 573e374, so this is the complete, current plan:

Rebase onto main and resolve the conflicts. The branch is about 414 commits behind. The conflicts are in actions/instrument/shared/install.sh and actions/instrument/job/inject_and_init.sh. Important: the previous merge in 573e374 silently dropped several main changes from inject_and_init.sh. The branch no longer has any of these, although they were on main at the merge base (4e97ecc) and still are:
- the container / ubuntu-slim start-up guard (container_marker_file / cgroup_file, Temporarily disable job-level instrumentation on ubuntu-slim runners #3480)
- the ghs_[0-9]+_… JWT-style token redaction patterns in the collector config (5 occurrences)
- the gh_repo_properties → github.repository.property.* resource attributes, and the malformed-OTEL_RESOURCE_ATTRIBUTES fix (Prevent malformed OTEL_RESOURCE_ATTRIBUTES from GitHub job instrumentation #3431)
Please take main's version of these and layer this PR's rpm/apk changes on top, instead of reverting them. Keep in mind that the container guard exits early inside containers, and the new job-rpm / job-apk tests run inside containers. Make sure those tests actually exercise the install path, not just the early exit.
Fix ci / github / job-rpm and ci / github / job-apk (rpm, apk). Both fail at the actions/download-artifact step, and ci / all fails only as the roll-up. The logs have expired (HTTP 410), but the workflow itself shows the cause. .github/workflows/test_github.yml downloads packages_${{ inputs.ref }}, and no build job uploads an artifact with that name. build.yml uploads packages.deb.<ref>, packages.rpm.<ref> and packages.apk.<ref>. Use packages.rpm.${{ inputs.ref }} in job-rpm and packages.apk.${{ inputs.ref }} in job-apk, and pin actions/checkout / actions/download-artifact to the same SHAs the other jobs in that file use.
Make install.sh find the rpm/apk files the build actually produces. Otherwise the tests from item 2 will silently install from the release fallback instead of the PR's build. install.sh looks for opentelemetry-shell_${version}_${arch}.${pkg_ext} locally and for release assets ending in _${arch}.${pkg_ext}, with Debian-style arch (amd64/arm64). build.yml names the rpm opentelemetry-shell-<version>-1.<arch>.rpm (the <arch> values are noarch plus the directory names under agent.instrumentation.http). meta/apk/APKBUILD (pkgrel=0, arch=noarch) produces opentelemetry-shell-<version>-r0.apk. Please align the local-file and release-asset patterns for rpm and apk with those names.
copilot-setup-steps also failed on this head, in its printenv step. main has since replaced the *-action-*/*/*.sh glob loop with a find … || true loop, so taking main's version during the rebase should be enough. No PR-specific fix is needed.

After pushing, please confirm that job-rpm and job-apk go green and that their logs show the local rpm/apk being installed.

(🤖 generated by Claude)

Initial plan

0d921c2

Copilot AI assigned Copilot and plengauer Mar 24, 2026

Copilot started work on behalf of plengauer March 24, 2026 12:51 View session

Copilot AI changed the title ~~[WIP] Implement support for rpm-based runners in instrumentation~~ Support rpm-based and apk-based runners in job-level and workflow-level instrumentation Mar 24, 2026

Copilot AI requested a review from plengauer March 24, 2026 13:09

Copilot finished work on behalf of plengauer March 24, 2026 13:09

plengauer requested changes Mar 24, 2026

View reviewed changes

Comment thread .github/workflows/test_github.yml

Comment thread actions/instrument/checksuite/main.sh Outdated

Comment thread actions/instrument/repository/main.sh Outdated

Comment thread actions/instrument/shared/install.sh Outdated

Comment thread actions/instrument/workflow/main.sh Outdated

Copilot started work on behalf of plengauer March 24, 2026 22:37 View session

Copilot AI requested a review from plengauer March 24, 2026 22:42

Copilot finished work on behalf of plengauer March 24, 2026 22:42

plengauer added 7 commits March 25, 2026 22:02

Refactor caching and dependency installation logic

e17a893

Update install.sh

6db5bb3

Define sudo function conditionally if missing

14ca0bb

Add a conditional definition for sudo if not available.

Update install.sh

88ef062

Update inject_and_init.sh

937b559

Update install.sh

2edb10c

Update inject_and_init.sh

27c86b1

plengauer linked an issue Apr 3, 2026 that may be closed by this pull request

Support rpm-based runners in job-level and workflow-level instrumentation #3260

Open

Merge branch 'main' into copilot/support-rpm-based-runners

0f017a8

Copilot started work on behalf of plengauer April 11, 2026 11:05 View session

Copilot finished work on behalf of plengauer April 11, 2026 15:21

Merge remote-tracking branch 'origin/main' into copilot/support-rpm-b…

3dc14d5

…ased-runners # Conflicts: # actions/instrument/shared/install.sh Co-authored-by: plengauer <100447901+plengauer@users.noreply.github.com>

deploy opentelemetry

e8bb33a

Copilot started work on behalf of plengauer April 15, 2026 11:15 View session

Merge remote-tracking branch 'origin/main' into copilot/support-rpm-b…

ffefffb

…ased-runners # Conflicts: # .github/workflows/observability_workflow.yml Co-authored-by: plengauer <100447901+plengauer@users.noreply.github.com>

Copilot finished work on behalf of plengauer April 15, 2026 11:20

plengauer added 5 commits April 15, 2026 13:20

Recompile Agentic Workflows (#3368)

9df1d76

deploy opentelemetry

8f734cd

Merge branch 'main' into copilot/support-rpm-based-runners

8c3e3cc

Recompile Agentic Workflows (#3418)

3df74d9

deploy opentelemetry

cc68f14

Copilot started work on behalf of plengauer May 24, 2026 15:59 View session

Copilot finished work on behalf of plengauer May 24, 2026 16:03

plengauer removed a link to an issue Jul 1, 2026

Support rpm-based runners in job-level and workflow-level instrumentation #3260

Open

Copilot stopped work on behalf of plengauer due to an error August 6, 2026 18:00
The session was cancelled by the user.

Copilot started work on behalf of plengauer August 26, 2026 06:50 View session

Copilot stopped work on behalf of plengauer due to an error August 26, 2026 06:51
Request session.create failed with message: Model "claude-opus-4.6" is not available.

plengauer mentioned this pull request Sep 11, 2026

MacOS support #550

Open

Navigation Menu

Support rpm-based and apk-based runners in job-level and workflow-level instrumentation - #3268

Support rpm-based and apk-based runners in job-level and workflow-level instrumentation#3268
plengauer with Copilot wants to merge 20 commits into
mainfrom
copilot/support-rpm-based-runners

Copilot AI commented Mar 24, 2026 •

edited

Loading

Uh oh!

plengauer left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI commented Mar 24, 2026

Uh oh!

plengauer commented Apr 11, 2026

Uh oh!

Copilot AI commented Apr 11, 2026

Uh oh!

plengauer commented Apr 15, 2026

Uh oh!

Copilot AI commented Apr 15, 2026

Uh oh!

plengauer commented May 24, 2026

Uh oh!

Copilot AI commented May 24, 2026

Uh oh!

plengauer commented Aug 6, 2026

Uh oh!

plengauer commented Aug 26, 2026

Uh oh!

plengauer commented Sep 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Copilot AI commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Shared install logic (actions/instrument/shared/install.sh)

Callers (workflow/main.sh, repository/main.sh, checksuite/main.sh)

Job-level (inject_and_init.sh)

Tests

Uh oh!

plengauer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI commented Mar 24, 2026

Uh oh!

plengauer commented Apr 11, 2026

Uh oh!

Copilot AI commented Apr 11, 2026

Uh oh!

plengauer commented Apr 15, 2026

Uh oh!

Copilot AI commented Apr 15, 2026

Uh oh!

plengauer commented May 24, 2026

Uh oh!

Copilot AI commented May 24, 2026

Uh oh!

plengauer commented Aug 6, 2026

Uh oh!

plengauer commented Aug 26, 2026

Uh oh!

plengauer commented Sep 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Mar 24, 2026 •

edited

Loading

Shared install logic (`actions/instrument/shared/install.sh`)

Callers (`workflow/main.sh`, `repository/main.sh`, `checksuite/main.sh`)

Job-level (`inject_and_init.sh`)