Skip to content

Fix broken pipe during test and gracefully exit the server#4701

Open
soulomoon wants to merge 90 commits intomasterfrom
1875-tests-randomly-fail-with-exception-fd111-hputbuf-resource-vanished-broken-pipe---test-option-j1-workaround
Open

Fix broken pipe during test and gracefully exit the server#4701
soulomoon wants to merge 90 commits intomasterfrom
1875-tests-randomly-fail-with-exception-fd111-hputbuf-resource-vanished-broken-pipe---test-option-j1-workaround

Conversation

@soulomoon
Copy link
Collaborator

gracefully exit the server

gracefully exit the server
@soulomoon soulomoon changed the title Fix broken pip during test Fix broken pipe during test Aug 22, 2025
@soulomoon
Copy link
Collaborator Author

Key idea:

  1. Wait for the refactor loop to finish in shutdown handler.
  2. After sending the shutdown response, stop sending further response to client. (This is done in lsp)

The worker thread checks this flag before dequeuing each job; if set, it exits immediately,
ensuring that no new work is started after shutdown is requested.
This mechanism is necessary because some downstream code may swallow async exceptions,
making 'cancel' unreliable for stopping the thread in all cases.
If 'cancel' does interrupt the thread (e.g., while blocked in STM or in a cooperative job),
the thread exits immediately and never checks the TMVar; in such cases, the stop flag is redundant.
@soulomoon soulomoon changed the title Fix broken pipe during test Fix broken pipe during test and gracefully exit the server Aug 26, 2025
Copy link
Collaborator

@fendor fendor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just some residual files that we likely should get rid of

…111-hputbuf-resource-vanished-broken-pipe---test-option-j1-workaround
…111-hputbuf-resource-vanished-broken-pipe---test-option-j1-workaround
@soulomoon
Copy link
Collaborator Author

soulomoon commented Mar 4, 2026

windows will still see some broken pipes, but it is not caused by the same gracefully exit issue.
For ubuntu pipeline, broken pipes are gone.
@fendor I've clean up the debug files. This patch is ready.

Copy link
Collaborator

@fendor fendor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks like a big improvement!

I have added a couple of comments, let me know what you think about it and whether I should address any of them myself.

Copy link
Collaborator

@fendor fendor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks like a big improvement!

I have added a couple of comments, let me know what you think about it and whether I should address any of them myself.

@soulomoon soulomoon requested a review from fendor March 8, 2026 09:21
@soulomoon
Copy link
Collaborator Author

@fendor Thanks for the review! I’ve addressed the comments and pushed the updates. Please take another look when you have time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: needs review This PR is ready for review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tests randomly fail with "Exception: fd:111: hPutBuf: resource vanished (Broken pipe)" (--test-option=-j1 workaround)

4 participants