feat: named child runs that survive migration and resurrection by barjin · Pull Request #759 · apify/apify-sdk-js

barjin · 2026-09-30T09:44:41Z

Adds a runName option to Actor.start, call and callTask that reattaches to the recorded child run after a restart, and Actor.childRuns() to read them.

Closes #738, closes #740

…process

…Actor.childRuns()

…med out ones

…es restarts

github-actions · 2026-09-30T09:46:56Z

⚠️ There are broken links in the documentation.

See more at https://github.com/apify/apify-sdk-js/actions/runs/37911125694#summary-113756252850

barjin

A few ideas / comments where this implementation differs from the Orchestrator ⬇️

barjin · 2026-09-30T09:52:09Z

+        const tracked = await this.#childRunTracker.get(runName);
+        const trackedRun = tracked ? await client.run(tracked.runId).get() : undefined;
+
+        if (trackedRun && ['SUCCEEDED', 'READY', 'RUNNING'].includes(trackedRun.status)) {


Every other run status (aborting / timed-out) is currently replaced by a completely new run instance.

Should we attempt to resurrect these?

Bonus question for resurrected runs - shall we try implementing the 'inherit' timeout mode (pass the remaining timeout to the resurrected child run)?

There's no Actor.resurrect() method in SDK now (is this intentional?)

Every other run status (aborting / timed-out) is currently replaced by a completely new run instance. Should we attempt to resurrect these?

Yes, I'd rather resurrect them.

Bonus question for resurrected runs - shall we try implementing the 'inherit' timeout mode (pass the remaining timeout to the resurrected child run)?

It makes sense. Let's try to implement it.

There's no Actor.resurrect() method in SDK now (is this intentional?)

I'm not aware of this being intentionally missing, so I'd add it and use it (Python SDK is missing it as well).

Co-authored-by: Martin Adámek <banan23@gmail.com>

`Date` and `URL` inputs were hashed as `{}` and function inputs were dropped, so inputs differing only in those values got the same checksum and the earlier run was resumed. Hash the JSON form `apify-client` sends instead, and sort keys by code units, as `localeCompare` depends on the locale.

B4nan · 2026-10-06T13:28:12Z

Lets compare this with apify/apify-sdk-python#1149

B4nan · 2026-10-06T15:11:01Z

Compared with the Python counterpart, which is split into apify/apify-sdk-python#1149 (named runs) and apify/apify-sdk-python#1151 (Actor.child_runs()):

	JS (this PR)	Python
Option	`runName`	`name`
Methods	`start`, `call`, `callTask`	`start`, `call` (no `call_task`)
KVS key	`CHILD_RUNS`	`APIFY_CHILD_RUNS`
Stored record	`runId, status, startedAt, checksum, history[{runId, status, startedAt}]`	`actorId, runId, previousRunIds`
Name reused for a different request	throws if the Actor/task or input differs	`ValueError` only if the actor ID differs, a new input is accepted
`ABORTED` / `TIMED-OUT`	replaced by a new run	resurrected (with build, memory, timeout and max charge passed on)
`ABORTING` / `TIMING-OUT`	a new run starts right away, while the old one is still finishing	waits for it to finish, then resurrects it
`FAILED` / run gone	new run, old one kept in history	same
Concurrent calls with the same name	per-name promise chain	per-name `asyncio.Lock`
`call` log on a resumed run	`fromStart: !resumed`	same, plus a status message watcher; a `SUCCEEDED` run returns without streaming
`childRuns()`	stored snapshot with the last observed status, no API calls	fetches every run from the API; `run: None` if it's gone
Malformed stored data	not checked	raises `ValueError`
Tests	unit	unit + e2e

Things we should settle across both SDKs:

KVS key and record format. These should match, so the stored data looks the same whichever SDK wrote it.
Resurrect or replace aborted/timed-out runs (your question above). Either way, we should wait out ABORTING / TIMING-OUT before starting a replacement, otherwise the old and new runs overlap.
Whether the name check compares input (JS) or just the Actor (Python).
Whether childRuns() fetches fresh status from the API.
Python has no named call_task yet.

vdusek · 2026-10-07T07:09:48Z

Python PRs (apify/apify-sdk-python#1149, apify/apify-sdk-python#1151) have been changed to match JS behavior in two places:

the option is now named run_name,
and call_task supports it as well.

I'm still thinking about the rest.

janbuchar

I have a bunch of code cleanup suggestions, but nothing critical. The PR looks fine to me.

…plementation

…tter

vdusek

See this separate comment #759 (comment).

vdusek · 2026-10-09T13:17:27Z

+
+        return this.locks.get(runName)!.runExclusive(async () => {
+            const tracked = await this.get(runName);
+            const trackedRun = tracked ? await client.run(tracked.runId).get() : undefined;


A 404 here right after a start (e.g. a concurrent call, or a migration moments after start()) can just be a lagging API replica, and we would start a duplicate run. Python retries get() for 3 s before treating the run as LOST.

See apify/apify-sdk-python#1149 (comment).

✍️ Drafted by Claude Code

vdusek · 2026-10-09T13:17:27Z

+            const tracked = await this.get(runName);
+            const trackedRun = tracked ? await client.run(tracked.runId).get() : undefined;
+
+            if (trackedRun && ['SUCCEEDED', 'READY', 'RUNNING'].includes(trackedRun.status)) {


An ABORTING / TIMING-OUT run falls through to start(), so the old and new runs overlap for a while. Python waits for such a run to finish (waitForFinish()) and then handles it as ABORTED / TIMED-OUT.

✍️ Drafted by Claude Code

vdusek · 2026-10-09T13:17:27Z

+            const trackedRun = tracked ? await client.run(tracked.runId).get() : undefined;
+
+            if (trackedRun && ['SUCCEEDED', 'READY', 'RUNNING'].includes(trackedRun.status)) {
+                await this.verifyRequest(runName, request);


The checksum is only checked on this reattach path, so a name whose recorded run failed or got lost accepts a different input, and the new checksum overwrites the old one. Python raises in that case too, so a name stays bound to one Actor/task + input. Which way we want to go?

✍️ Drafted by Claude Code

vdusek · 2026-10-09T13:17:27Z

+                return { run: trackedRun, resumed: true };
+            }
+
+            const run = await start();


ABORTED / TIMED-OUT runs end up here and get replaced by a new run, but we agreed to resurrect them. Python resurrects them with the call's build, memory, timeout and max charge. JS needs an Actor.resurrect() for that first.

This can be addressed once this #769 is merged.

✍️ Drafted by Claude Code

vdusek · 2026-10-09T13:50:10Z

Compared with the Python counterpart (apify/apify-sdk-python#1149 for named runs, apify/apify-sdk-python#1151 for child_runs). Python now matches this PR on the option name, supported methods, KVS key, record format, checksum, per-name locking, and the childRuns shape (sync, run clients keyed by name, custom-token client kept in-process).

Full comparison table:

Area	JS (this PR)	Python	Aligned	Notes
Option name	`runName`	`run_name`	✅	Python renamed `name` → `run_name`
Supported methods	`start`, `call`, `callTask`	`start`, `call`, `start_task`, `call_task`	❌	JS has no `Actor.startTask()` yet, see #764
Record format	`runId, status, startedAt, checksum, history[]`	same	✅
Checksum	sha256 of sorted-key JSON of `{type, id, input}`	same, byte-equal	✅
Write timing	right after start	right after start	✅
`READY` / `RUNNING`	reattach	reattach	✅
`SUCCEEDED`	return as is	return as is	✅
`FAILED` / run gone	new run, old one moved to `history`	same	✅
`ABORTED` / `TIMED-OUT`	new run	resurrected with the call's build, memory, timeout and max charge	❌	Resurrecting is cheaper and was agreed, see r4230501550
`ABORTING` / `TIMING-OUT`	new run right away, overlapping the old one	waits for it to finish, then resurrects	❌	Avoids two children running under one name, see r4230501533
`Actor.resurrect()`	missing	added in apify/apify-sdk-python#1165	❌	Needed for the `ABORTED` / `TIMED-OUT` row
404 on the recorded run	`LOST` immediately	retried for 3 s first	❌	A lagging API replica can otherwise start a duplicate, see r4230501519
Different input, run reattached	throws	raises	✅
Different input, run replaced	allowed, checksum overwritten	raises	❌	Open question whether a name stays bound to one input, see r4230501542
Concurrent calls under one name	per-name lock	per-name lock	✅
`childRuns` API	sync getter, `Record<string, RunClient>`	sync property, `dict[str, RunClientAsync]`	✅	Python aligned in apify/apify-sdk-python#1151
Registry load	in `init()`	in `__aenter__`	✅
Custom `token` in `childRuns`	kept in-process, default client after migration	same	✅	ported from Python
History in `childRuns`	KVS record only	KVS record only	✅
Access before init	warns, returns `{}`	raises	✅	each SDK's existing convention, fine as is
Malformed record	not validated	`ValueError` at init	❌	minor
Max charge / `max_items`	forwarded to the start	forwarded to the start and to the resurrection	✅
Tests	unit	unit + e2e (reboot → reattach)	❌

✍️ Drafted by Claude Code

`Actor.start`, `Actor.call`, `Actor.start_task` and `Actor.call_task` accept an optional `run_name`. Right after the child starts, the SDK records it in the default key-value store under `__ACTOR_CHILD_RUNS`. After a migration or resurrection of the parent, the same call finds the recorded run and reuses it: - `READY` / `RUNNING`: reattaches to it - `SUCCEEDED`: returns it as is - `ABORTED` / `TIMED-OUT`: resurrects it with the call's build, memory, timeout, `max_items` and max charge (an `ABORTING` / `TIMING-OUT` run is waited for first) - `FAILED`, or the run no longer exists: starts a new run and moves the old one to `history` The record has the same shape as in the JS SDK (apify/apify-sdk-js#759): `runId`, `status`, `startedAt`, `checksum` and `history`. The checksum covers the Actor or task ID and the input, computed the same way as in JS. Reusing a name for a different Actor, task or input raises `ValueError`. Concurrent calls under one name in the same process share a single run. A named `call` on a reattached or resurrected run streams only new log lines. If the API reports a recorded run as missing, the SDK retries for 3 seconds before replacing it, because a run started moments ago may not be on every API replica yet. A hard kill between the platform starting the child and the registry write can still orphan the child. Closing that gap needs an idempotency key on the run-start endpoint. Closes: #1127 *✍️ Drafted by Claude Code*

`Actor.child_runs` is a sync property that maps each run name to a `RunClientAsync` for the current run under that name. It has the same shape as `Actor.childRuns` in the JS SDK (apify/apify-sdk-js#759). The client covers the run and its storages: ```python run = await Actor.child_runs['scrape-eu'].wait_for_finish() ``` The SDK reads the registry from the default key-value store when the Actor initializes, so runs recorded before a migration or resurrection are included and reading the property makes no API calls. That adds one record read to every init. A malformed record fails the init and leaves the Actor uninitialized. Runs started without a `run_name` aren't tracked. Runs that were replaced under a name stay in the `history` of the `__ACTOR_CHILD_RUNS` record and aren't exposed. A run started or reattached with a custom `token` gets a client with that token. A run recorded before a migration and not reattached since gets the default client, because the token isn't stored. It also fixes a duplicate `max_items` deprecation warning from #1149: a named start that resurrects its recorded run no longer warns a second time from inside the SDK. Docs are in #1160. Closes: #1129 *✍️ Drafted by Claude Code*

B4nan · 2026-10-09T15:42:50Z

+        const { waitSecs, log, ...startOptions } = rest;
+        const { run, resumed } = await this.#childRunTracker.startOrResumeChildRun(
+            client,
+            runName,
+            { type: 'actor', id: actorId, input },
+            async () => client.actor(actorId).start(input, { ...startOptions, runTimeoutSecs }),
+        );
+
+        // The earlier part of a resumed run's log was already redirected before the migration.
+        const streamedLog = await client.run(run.id).getStreamedLog({ toLog: log, fromStart: !resumed });
+        streamedLog?.start();
+        try {
+            const finishedRun = await client.run(run.id).waitForFinish({ waitSecs });


[medium] With runName, signal and timeoutSecs only reach start(). The unnamed path (client call()) also passes them to waitForFinish / getStreamedLog and defaults timeoutSecs: 'noTimeout'. Aborting the signal doesn't stop Actor.call(..., { runName, signal }); it hangs until the child finishes. And start() falls back to the 'medium' request timeout, which can cut off a start with waitForResources. Same in callTask() below. Fix: forward { waitSecs, timeoutSecs, signal } to waitForFinish, signal to getStreamedLog, and default timeoutSecs like the client does.

Repro (with runName times out, without passes)

import { createServer } from 'node:http'; import type { AddressInfo } from 'node:net'; import { MemoryStorageBackend } from '@crawlee/core'; import { Configuration } from 'apify'; import type { Run as ActorRun } from 'apify-client'; import { ActorClient } from 'apify-client'; import { createIsolatedActor } from '../createIsolatedActor.js'; const server = createServer(() => {}); let apiBaseUrl: string; beforeAll(async () => { await new Promise<void>((resolve) => server.listen(0, '127.0.0.1', resolve)); apiBaseUrl = `http://127.0.0.1:${(server.address() as AddressInfo).port}/`; }); afterAll(() => { server.closeAllConnections(); server.close(); }); const run = { id: 'child-run', status: 'RUNNING', startedAt: new Date() } as ActorRun; test.each([['without runName', {}], ['with runName', { runName: 'child' }]])( 'call() %s rejects once its abort signal fires', async (_label, extra) => { vitest.spyOn(ActorClient.prototype, 'start').mockResolvedValue(run); const { actor } = createIsolatedActor({ config: new Configuration({ apiBaseUrl, token: 'token' }), storageClient: new MemoryStorageBackend(), }); const signal = AbortSignal.timeout(200); await expect(actor.call('some-actor', {}, { signal, log: null, ...extra })).rejects.toThrow(); }, 3000, );

B4nan · 2026-10-09T15:42:50Z

+        const { waitSecs, ...startOptions } = rest;
+        const { run } = await this.#childRunTracker.startOrResumeChildRun(
+            client,
+            runName,
+            { type: 'task', id: taskId, input },
+            async () => client.task(taskId).start(input, { ...startOptions, runTimeoutSecs }),
+        );
+        const finishedRun = await client.run(run.id).waitForFinish({ waitSecs });


[medium] Same as in call(): signal / timeoutSecs never reach waitForFinish.

B4nan · 2026-10-09T15:42:50Z

        });
        log.debug(`Default storages purged`);

+        await this.#childRunTracker.load();


[low] This adds a getRecord('__ACTOR_CHILD_RUNS') (plus a store get) to every Actor.init() on the platform, also for Actors that never use runName, and init fails if the read fails. Is that cost intended for the sync childRuns getter? Alternative: load lazily on the first named start/call/callTask.

B4nan · 2026-10-09T15:42:50Z

+     *
+     * If a `READY`, `RUNNING` or `SUCCEEDED` run is already tracked under this name, it is returned instead of
+     * starting a new one, including after a migration or resurrection of this run. A run that failed, was aborted
+     * or timed out is replaced by a new one. Reusing the name for a different Actor, task or input throws.


[low] It only throws while the tracked run is returned (READY/RUNNING/SUCCEEDED). Otherwise it's silently replaced, which the "allowed when the tracked run is replaced" test pins down. Suggest "...throws while the tracked run is returned."

B4nan · 2026-10-09T15:42:50Z

    static #instance?: Actor;

+    /**
+     * Tracks child runs of this Actor instance.


[nit] Restates the field name, can be dropped.

B4nan · 2026-10-09T15:42:50Z

 }
+
+/**
+ * A FIFO mutex that a critical section may re-enter from a nested call — the charge lock is taken by


[nit] Doc still describes the charge lock / pushData(), but it's a shared util now (also per-name child-run locks), so make it general. Also the node:async_hooks import on L18 should go with the other builtins.

B4nan · 2026-10-09T15:42:50Z

+        );
+    }
+
+    private async persist(trackedRuns: Record<string, TrackedChildRun>) {


[nit] persist() always gets this.loadedRuns, so the param is redundant. trackedRuns (promise) and loadedRuns also hold the same state, could be one field.

B4nan · 2026-10-09T15:42:50Z

+import log from '@apify/log';
+
+import type { TrackedChildRun } from '../../src/child_run_tracker.js';
+import { createIsolatedActor } from '../createIsolatedActor';


[nit] Missing .js extension.

Holy guacamole, can we set up a lint rule for this?

I've set up a hook for myself months ago, so it didn't bother me, but sure, we can: #770

barjin added 10 commits September 30, 2026 10:54

feat: resumable start and call implementions

187354e

fix: do not resume child runs that are aborting or timing out

30114ed

refactor: move ChildRunTracker to its own file

075b9f7

feat: reject reusing a child run name for a different request in one …

c3c0718

…process

feat: resurrect aborted child runs and expose tracked child runs via …

3162daa

…Actor.childRuns()

refactor: start a new child run instead of resurrecting aborted or ti…

81815f2

…med out ones

docs: describe the public ChildRunTracker methods

a337bb0

style: use multiline jsdoc for ChildRunTracker methods

c9922a1

refactor: simplify child run tracking

1940b86

feat: persist the child run request checksum so the name check surviv…

7751f4c

…es restarts

barjin commented Sep 30, 2026

View reviewed changes

barjin self-assigned this Sep 30, 2026

barjin marked this pull request as ready for review September 30, 2026 09:55

barjin requested review from B4nan and janbuchar September 30, 2026 09:55

This was linked to issues Sep 30, 2026

Named child runs that survive migration and resurrection #738

Open

Expose tracked child runs as a readable API #740

Open

apify-service-account added tested Temporary label used only programatically for some analytics. t-tooling Issues with this label are in the ownership of the tooling team. labels Sep 30, 2026

B4nan reviewed Oct 5, 2026

View reviewed changes

Comment thread src/actor.ts Outdated

Comment thread src/child_run_tracker.ts

Comment thread src/child_run_tracker.ts

Comment thread src/child_run_tracker.ts Outdated

Comment thread src/actor.ts

barjin and others added 4 commits October 6, 2026 13:35

docs: extend the JSDoc for the runName option

f4cfdc4

Co-authored-by: Martin Adámek <banan23@gmail.com>

fix: clear trackedRuns cache on a KVS load error

29f7d13

Co-authored-by: Martin Adámek <banan23@gmail.com>

fix: add a per-name mutex for named run start and tracking

ca71dd2

Co-authored-by: Martin Adámek <banan23@gmail.com>

barjin requested a review from B4nan October 6, 2026 12:56

vdusek mentioned this pull request Oct 7, 2026

Add Actor.resurrect() apify/apify-sdk-python#1162

Closed

vdusek reviewed Oct 7, 2026

View reviewed changes

Comment thread src/child_run_tracker.ts Outdated

vdusek reviewed Oct 7, 2026

View reviewed changes

Comment thread src/actor.ts Outdated

This was referenced Oct 7, 2026

feat: reattach named child runs after migration or resurrection apify/apify-sdk-python#1149

Merged

Add Actor.resurrect() #765

Open

refactor: store tracked child runs under the __ACTOR_CHILD_RUNS key

aebd5a4

vdusek reviewed Oct 7, 2026

View reviewed changes

Comment thread src/child_run_tracker.ts

vdusek reviewed Oct 7, 2026

View reviewed changes

Comment thread src/child_run_tracker.ts

barjin mentioned this pull request Oct 7, 2026

Expose tracked child runs as a readable API #740

Open

janbuchar reviewed Oct 8, 2026

View reviewed changes

barjin added 8 commits October 8, 2026 13:25

refactor: split off the child run specific tests into a separate file

37193a0

chore: improve test naming

1a9dbe7

chore: use the ReentrantAsyncLock class instead of a custom lock im…

bb4d73f

…plementation

chore: fix lint error

2e974c6

docs: document the LOST runs better

452d7f6

chore: lock apify-client version

a76459d

Merge branch 'master' into feat/track-child-runs

bd8ec9a

chore: fix failing build

c83bcc6

barjin force-pushed the feat/track-child-runs branch from bc1cc0b to c83bcc6 Compare October 9, 2026 09:01

feat: expose tracked child runs as a synchronous Actor.childRuns ge…

8143fd7

…tter

vdusek mentioned this pull request Oct 9, 2026

feat: add Actor.child_runs with clients for named child runs apify/apify-sdk-python#1151

Merged

vdusek reviewed Oct 9, 2026

View reviewed changes

B4nan requested changes Oct 9, 2026

View reviewed changes

B4nan mentioned this pull request Oct 9, 2026

chore: enforce file extensions in relative imports #770

Open

Conversation

barjin commented Sep 30, 2026

Uh oh!

github-actions Bot commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

barjin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

B4nan commented Oct 6, 2026

Uh oh!

B4nan commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vdusek commented Oct 7, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

janbuchar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vdusek left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vdusek commented Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Sep 30, 2026 •

edited

Loading

B4nan commented Oct 6, 2026 •

edited

Loading

vdusek left a comment •

edited

Loading

vdusek commented Oct 9, 2026 •

edited

Loading