feat(relay): Provide a flag to exit container on healthcheck failure

[aminvakil]

Another attempt of fixing getsentry/sentry#109002 after #5750.

docker-compose does not restart services if they become unhealthy for any reason whether it's web container not accessible and returning NXDOMAIN or any another reason, this PR adds a kill signal to PID 1 to healthcheck failures which can be enabled using kill-on-fail argument, effectively stopping container, and as restart_policy is unless-stopped in self-hosted, it should restart itself.

This may rises a problem of getting relay in a restart loop if it does not get healthy, which is fine IMO.

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

[aminvakil]

There is a failure on building it, I should add deps to it, will make ready for review afterwards.

[aminvakil]

It's working correctly as expected , but what about having flush-dns-on-fail instead of kill-on-fail?

This is better I think, it seems relay handles SIGTERM perfectly well.

[aminvakil]

cc @aldy505

[aldy505]

Makes sense to me

[jjbayer]

I believe terminating the process on bad health checks should be the responsibility of the orchestrator, not the process itself (see https://docs.docker.com/reference/compose-file/services/#healthcheck). This allows you to define a number of retries, etc.

[aminvakil]

I believe terminating the process on bad health checks should be the responsibility of the orchestrator, not the process itself (see https://docs.docker.com/reference/compose-file/services/#healthcheck). This allows you to define a number of retries, etc.

Correct, there are already healthcheck directives in self-hosted for relay, but

docker-compose does not restart services if they become unhealthy for any reason whether it's web container not accessible and returning NXDOMAIN or any another reason

Unfortunately it cannot be done on docker-compose.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}