fix(test): stop interface microservice tests leaking threads on failure

[EmilyRagan]

Problem

The Python 3.12 unit test job died with a segfault in this run:

##[error]Process completed with exit code 139
Fatal Python error: Segmentation fault
Current thread ...:
  File ".../traceback.py", line 368 in _get_code_position
  ...
  File ".../openc3/system/system.py", line 264 in add_target
  File ".../test/test_helper.py", line 80 in setup_system
  File ".../test/microservices/test_tsdb_microservice.py", line 38 in setUp

Cause

Two coupled problems in test/microservices/test_interface_microservice.py:

1. Fixed sleeps before state assertions. Tests did thread.start(), time.sleep(0.1), then asserted the interface state is CONNECTED. On a loaded runner 100 ms is not enough and the state is still ATTEMPTING. Three tests failed this way in that job: test_connect_handles_parameters, test_handles_a_clean_disconnect, test_run_does_not_write_status_after_cancel_thread_set.

2. Cleanup only on the success path. im.shutdown() and thread.join() were written after the assertions, so a failed assertion skipped them and leaked a live microservice into the rest of the session. The fault dump confirms it — five threads still running in interface_microservice.py:742 run, :944 disconnect, microservice.py:275 _status_thread and metric.py:119, long after those tests "finished".

Those leaked threads keep touching System while a later setup_system() does System.instance_obj = None and rebuilds. That made add_target's process_file raise, and formatting the resulting traceback segfaulted the interpreter.

Fix

Two helpers, both following the addCleanup pattern four tests in this file already used:

• start_microservice(im) — starts im.run and registers addCleanup(thread.join, 5) + addCleanup(im.shutdown) (LIFO: shutdown signals stop, then join waits), so cleanup happens even when an assertion fails.
• wait_for_state(state) / wait_for_output(stdout, text) — poll with a 5s timeout instead of sleeping a fixed 0.1–0.5s, then assert, so the failure message still shows what was captured.

Every test that constructs an InterfaceMicroservice now registers addCleanup(im.shutdown) before its assertions, including the four that never start a run thread but still create handler and metrics threads. shutdown() is idempotent (shutdown_complete guard), so tests that need to assert on post-shutdown state still call it inline.

Testing

• uv run pytest test/microservices/test_interface_microservice.py — 18 passed (4.9s, down from ~11s of fixed sleeps)
• uv run pytest test/microservices/ — 91 passed
• uv run ruff check / ruff format --check — clean
• Verified the leak directly with a scratch test that forces an assertion failure after starting the microservice:
  • old pattern: 4 threads left alive afterwards — ['MainThread', 'Thread-1 (update_thread_body)', 'Thread-2 (_status_thread)', 'Thread-3 (run)', 'Thread-4 (run)'] — and pytest hung at exit
  • new pattern: back to MainThread only

The segfault itself is load-dependent and did not reproduce locally (that ordering passed 3/3 before this change), so this fixes the cause rather than a reproducible crash.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.13%. Comparing base (b04eb16) to head (d46cde3).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3769      +/-   ##
==========================================
+ Coverage   79.09%   79.13%   +0.03%     
==========================================
  Files         894      894              
  Lines       66865    66865              
  Branches     2551     2599      +48     
==========================================
+ Hits        52890    52915      +25     
+ Misses      13313    13290      -23     
+ Partials      662      660       -2     

Flag	Coverage Δ
frontend	66.11% <ø> (+0.09%)	⬆️
python	79.30% <ø> (+0.03%)	⬆️
ruby-api	81.87% <ø> (+0.10%)	⬆️
ruby-backend	84.44% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[mcosgriff]

I have a fix for openc3/python/test/api/test_cmd_api.py::test_get_cmd_value_returns_command_values on my branch https://github.com/OpenC3/cosmos/pull/3767/changes#diff-5d5032335847f9c5d77d2185b7a3d3e853dfe1f5c031fc2a28717224ec8df64e