Notifications module owns the dispatch pipeline#92
Draft
machielvdw wants to merge 9 commits into
Draft
Conversation
…page. Request: The notification landing page is module territory — the Lowdefy framework's email-notifications feature now composes landing links and this module's link page should accept them. Motivation: Lowdefy's notifications: config section composes email links as ?_id=<record>&option=<dataPath>; the landing flow (auth check, mark-as-read, redirect) already lives here, so the page just needs to resolve targets from the framework record shape. Decisions: - Coalesce both record shapes: target = get(record.data, option ?? 'links.button') with fallback to the legacy top-level links.button — Lambda-produced records keep working unchanged. - Absolute URL targets navigate via Link url; page targets via pageId/urlQuery/input. - Inbox list falls back to the framework record's preview when description is absent. Full record-shape migration (event_type, send_routine) is a separate effort. Changes: - modules/notifications/pages/link.yaml: set_target resolution step + url/pageId Link branches - modules/notifications/components/list-notifications.yaml: description ?? preview - docs/notifications/index.md: framework link contract + app.notificationLandingPage wiring - .changeset/framework-landing-links.md: minor bump Tags: notifications, landing-page, deep-link, lowdefy-framework, mark-as-read $30m
Request: Implement Option B of the framework-boundary decision — the Lowdefy
framework renders notification emails; this module owns the pipeline around
the render (insert with dedup, send, retry bookkeeping) and the record
convention.
Motivation: The framework's SendNotification step and notificationAdapter were
torn down to a RenderNotification primitive (lowdefy#2234). Every orchestration
op is an ordinary request, and the modules layer is where opinionated
composition belongs — the adapter existed only because the pipeline was
framework JS.
Decisions:
- Per-item pipeline is its own InternalApi called via CallApi: step results
are index-keyed inside :for and :catch cannot see the thrown error, so each
item needs fresh steps/state and dedup detection re-queries by key.
- Insert before send claims the dedup key; requires the app-managed unique
partial index notification_key_unique (module docs) — modules have no index
tooling, so the index is documented, not created.
- Record data stores the ORIGINAL item — the link page resolves targets from
data at the ?option dot-path; a resolved copy would redirect to itself.
- A send failure never fails the dispatch: $inc send_attempts + last_attempt,
record stays sent: false for a drain retry.
- public_link_types var replaces the hardcoded invite-user checks; read paths
coalesce legacy fields (description ?? preview, event_type ?? type).
- send_routine keeps its { event_ids } contract as the event-shaping hook;
full record-shape migration folded in (was deferred in the previous commit).
Changes:
- modules/notifications/api/dispatch-notifications.yaml: batch entry point
- modules/notifications/api/dispatch-notification-item.yaml: render → insert
with dedup → send gate → send + bookkeeping
- modules/notifications/connections/notifications-email.yaml: SMTP fed by
email.* vars, remappable
- modules/notifications/module.lowdefy.yaml: server_url, public_link_types,
email namespace vars; exports; NOTIFICATIONS_SMTP_PASS secret
- modules/notifications/pages/link.yaml: pre-auth guards via public_link_types,
invite target falls back through data.links.button
- components + requests: event_type ?? type coalescing, body ?? text detail
- apps/demo: notifications: template configs, SMTP vars (SendGrid relay),
send-routine shapes items and dispatches, enum entries, guide rewrite
- apps/workflows-test/modules/notifications/send-routine.yaml: canonical field
names on the store-only mock (type: workflow-error e2e contract unchanged)
- docs/notifications/index.md: pipeline, record convention, required indexes
- .changeset/framework-landing-links.md: rewritten for the pipeline
Tags: notifications, dispatch-pipeline, render-notification, dedup, smtp,
record-convention, framework-boundary, modules
$30m
machielvdw
changed the title
Request: Bump the module apps onto the newly published experimental release (0.0.0-experimental-20260706093008) that carries the RenderNotification teardown, and validate the demo builds against it. Motivation: The dispatch pipeline depends on the framework's RenderNotification step, which only exists from this release; the demo could not build until the pin moved. Decisions: - Demo + workflows-test pinned to the new experimental (package.json + lowdefy.yaml), lockfile updated. - Email vars host/user/pass/from are untyped: apps commonly source them from a _secret operator, which stays an unevaluated object at build time and fails a `type: string` check. Only port/secure (never secrets) keep types. Surfaced by the real build failing on email.from. - A module can only reference secrets it declares — the build enforces this, so an app cannot point a var at an undeclared secret name. Corrected the pass var description and secrets.md: set NOTIFICATIONS_SMTP_PASS, or remap the notifications-email connection to an app connection with its own secrets. - Demo now follows that contract: from is a plain string, pass uses the module's NOTIFICATIONS_SMTP_PASS default (set in the deploy env). Changes: - apps/demo, apps/workflows-test: lowdefy pin + lockfile - modules/notifications/module.lowdefy.yaml: untype email string vars; fix pass description - apps/demo/modules/notifications/vars.yaml: use the module secret contract - docs/shared/secrets.md: correct the reuse-a-secret guidance Validated: both apps build against the published release; the demo's dispatch-notification-item artifact contains the full pipeline (RenderNotification, MongoDBInsertOne, MongoDBFindOne dedup, SMTPMailSend, MongoDBUpdateOne) and both notification configs resolve. Tags: notifications, experimental-release, module-vars, secrets, smtp, demo $30m
Request: Two runtime bugs surfaced by an integration spike on a real app (encircle-mvp) against framework 0.0.0-experimental-20260706135208. Motivation: The record-shape migration introduced coalescing chains and null cc/bcc defaults that the framework operators/schema reject at runtime. Decisions: - _if_none is a strict 2-arg coalesce (throws on any other length), so the new-shape → legacy → default chains must nest two 2-item _if_none, not pass a 3-item array. The 3-item form threw at runtime and broke the inbox list (/notifications/all) and the deep-link landing page (/notifications/link). - SMTPMailSend's schema accepts an address list but not null for cc/bcc, so the dispatch send step must default absent cc/bcc to [] (not null), or every item without cc/bcc fails send-step validation and the record persists sent:false with send_attempts bumped. Changes: - components/list-notifications.yaml: nest description ?? preview ?? '' as two 2-item _if_none. - pages/link.yaml: nest the pageId/urlQuery/input data → legacy → default chains (link_invite step) as two 2-item _if_none each. - api/dispatch-notification-item.yaml: send step cc/bcc default to [] instead of null (the stored record fields keep null — readers don't use them). Tags: notifications, if-none, coalesce, smtp, cc-bcc, dispatch, encircle-spike $30m
Adds docs/notifications/email-transport.md: why the dispatch pipeline sends over SMTP (requests are bound to a connection type; the routine hardcodes SMTPMailSend, so notifications-email must be an SMTP connection), why remapping to a SendGrid connection can't work (remap swaps the connection instance, not the request type), how to use SendGrid anyway via its SMTP relay, and how a future module send-step slot could support provider HTTP APIs. Linked from the notifications index; llms.txt regenerated. Tags: notifications, docs, smtp, sendgrid, email-transport, connections $15m
Request: "What would it take to also allow the same behaviour on the sendgrid connection instead of just on the smtp connection" Motivation: The dispatch pipeline hardcoded SMTPMailSend, so email could only go out over an SMTP-type connection — SendGrid worked only via its SMTP relay, and a connection remap alone can never switch the request type. Decisions: - transport switch var over an injected send-step slot: a typed `transport: smtp | sendgrid` var with two baked-in send steps is more discoverable and keeps the module in control of the send contract; chosen explicitly over the free-form slot pattern. - Build-time folding: _module.var resolves at build, so the transport :if condition and the misconfiguration guard fold to literals — only one branch is live at runtime, and a typo'd transport throws on dispatch (outside the send :try so it is not swallowed into retry bookkeeping). - Always-registered inert SendGrid connection: connection schemas validate only when a request targets the connection, so notifications-email-sendgrid with null apiKey is safe under transport: smtp (same pattern as the unset email vars in workflows-test). - replyTo folds to sendgrid.from via _if_none because the SendGridMail schema rejects null replyTo; cc/bcc keep the [] defaults since SendGrid's schema also rejects null there. - Distinct step ids (send_mail_smtp / send_mail_sendgrid): duplicate-stepId validation is disabled in the framework today but intended; _step on a never-run step returns null so the shared mark_sent coalesces messageId (SMTP-only) and records email_result.transport. Changes: - modules/notifications/module.lowdefy.yaml: new transport + sendgrid.* vars (api_key/from/reply_to/filter/sandbox), notifications-email-sendgrid connection registered and exported, NOTIFICATIONS_SENDGRID_KEY secret - modules/notifications/connections/notifications-email-sendgrid.yaml: new SendGridMail connection wired from the sendgrid vars, remappable like notifications-email - modules/notifications/api/dispatch-notification-item.yaml: transport guard + :if branch between SendGridMailSend and SMTPMailSend inside the existing :try; mark_sent records email_result.transport - apps/demo/modules/notifications/vars.yaml: demo switches from the SendGrid SMTP relay to transport: sendgrid (in-repo consumer of the new path) - docs/notifications/email-transport.md: rewritten as "SMTP or SendGrid" — the transport var, sendgrid vars, remap semantics, and the relay as the alternative - docs/notifications/index.md, docs/shared/secrets.md, docs/notifications/reference/vars.md (generated), docs/llms.txt (generated): quickstart, remap, and secret coverage - .changeset/framework-landing-links.md: transport section added to the pending minor changeset - pnpm-lock.yaml: app server builds now install @lowdefy/connection-sendgrid Tags: notifications, email, sendgrid, smtp, transport, dispatch-pipeline, module-vars, connection-remap Refs: #92 $30m
…#92) Request: "in our apps we call it SENDGRID_API_KEY" Motivation: Apps already hold the SendGrid key under SENDGRID_API_KEY (auth email flows use the same secret); a namespaced NOTIFICATIONS_SENDGRID_KEY default would force either a duplicate env var or a connection remap in every consumer. Decisions: - Break the NOTIFICATIONS_* secret naming convention for this one secret: a SendGrid API key is an account-level credential shared across features, unlike the transport-specific SMTP password. Changes: - modules/notifications/module.lowdefy.yaml: sendgrid.api_key defaults to the SENDGRID_API_KEY secret; secrets list renamed - docs/notifications/email-transport.md, docs/notifications/index.md, docs/shared/secrets.md: secret name updated - .changeset/framework-landing-links.md, apps/demo/modules/notifications/vars.yaml: secret name updated Tags: notifications, sendgrid, secrets, sendgrid-api-key, module-vars Refs: #92
Request: Adopt lowdefy's new module-level notifications (PR lowdefy#2248, release 0.0.0-experimental-20260707143804) — the user-admin module ships its own invite templates so apps stop hand-writing them.
Motivation: user-admin dispatched invite events but could not ship the templates; every consuming app duplicated invite-user/resend-user-invite templates and a send_routine shape branch.
Decisions:
- Direct dispatch over the event/send_routine path: the recipient is known at invite time, so the endpoints shape the item inline and CallApi dispatch-notifications with _module.notificationId — removing the app's entire invite obligation. Invite events are still logged; they just no longer flow through send-notification.
- Dedup key {eventId}:{contactId}: each resend logs a new event, so deliberate resends are never suppressed.
- New login_page_id var (default user-account/login): the sign-in button target is app territory; the default matches the ecosystem's conventional user-account entry.
- public_link_types default gains the conventional scoped types (user-admin/invite-user, user-admin/resend-user-invite) so pre-auth landing links work out of the box; custom entry ids override the var.
Changes:
- modules/user-admin/notifications/{invite-user,resend-user-invite}.yaml: module-shipped NotificationEmail templates with {{ inviter_name }} copy and testData
- modules/user-admin/module.lowdefy.yaml: notifications: section, login_page_id var
- modules/user-admin/api/{invite-user,resend-invite}.yaml: send-notification CallApi replaced with direct dispatch-notifications dispatch (scoped notification_id, inline item, login link with email hint)
- modules/notifications/module.lowdefy.yaml: public_link_types default includes the scoped user-admin types
- docs/user-admin/index.md (new Invite emails section), docs/notifications/index.md, generated vars.md/llms.txt
- .changeset/user-admin-module-notifications.md (new), .changeset/framework-landing-links.md (module-templates section)
Tags: user-admin, notifications, module-notifications, invites, module-notificationid, public-link-types, login-page-id
Refs: #92
$30m
Request: Bump to the release containing module-level notifications so the user-admin invite templates build.
Motivation: The previous pin (20260706093008) predates the notifications: module manifest section and _module.notificationId.
Decisions:
- layout menu var untyped: the new release's typed-var validation rejects runtime operators in typed vars, and the menu default is a runtime _menu operator that must pass through to the page block and resolve on the client (same reasoning as the notification module's untyped email vars).
- workflows-test gains slug: the new release requires a top-level slug on apps (demo already had one).
- Demo drops its hand-written user-invite template and shape_invite/dispatch_invite send_routine branch — invites are now module-shipped and dispatched by user-admin directly; event_types enum gains the scoped user-admin/* badge keys.
Changes:
- apps/demo/{lowdefy.yaml,package.json}, apps/workflows-test/{lowdefy.yaml,package.json}, pnpm-lock.yaml: pins → 0.0.0-experimental-20260707143804
- modules/layout/module.lowdefy.yaml: menu var untyped with rationale comment
- apps/workflows-test/lowdefy.yaml: slug: workflows-test
- apps/demo/lowdefy.yaml, apps/demo/modules/notifications/send-routine.yaml: invite template + shape branch removed with pointer comments
- apps/demo/modules/events/event_types.yaml: scoped user-admin/invite-user + user-admin/resend-user-invite badge entries
- apps/demo/.claude/guides/notifications.md: reference files section reflects module-shipped invites
Tags: lowdefy-version-bump, module-notifications, layout, menu-var, slug, demo, invites
Refs: #92
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The notifications module now owns the full notification dispatch pipeline. This is the modules-mongodb half of the framework-boundary decision implemented in lowdefy/lowdefy#2234: the Lowdefy framework (≥ 5.4) only renders notification emails (the
notifications:config section and theRenderNotificationroutine step); this module composes everything around the render — inserting the record, deduplication, sending, retry bookkeeping — and owns the notification record convention.The dispatch pipeline
Two new
InternalApiendpoints:dispatch-notifications(exported) — payload{ notification_id, items }; validates and loops items viaCallApito the per-item endpoint. Per-item calls are structurally required: step results are index-keyed inside:for, so each item needs fresh steps/state.dispatch-notification-item(internal) — the pipeline: mint the record id (_uuid, before render — landing links embed it) →RenderNotification(renders the app's template config;{ pageId, urlQuery }links resolve to?_id=<record>&option=<dataPath>landing URLs through this module'slinkpage) → insert before send in:try/:catch(claims the dedupkey, so concurrent dispatches cannot double-send; duplicate key → skip — detection re-queries bykeysince:catchcannot inspect the error) → send gate (send_email, address validity) →SMTPMailSendover the newnotifications-emailconnection → marksent+email_result. A send failure never fails the dispatch:$inc send_attempts+last_attempt, record stayssent: falsefor a drain retry.send_routinekeeps its{ event_ids }contract as the app's event-shaping hook — existing callers (user-admin invites, the WorkflowAPI plugin) are untouched; routines now typically end in aCallApitodispatch-notifications.New module surface
server_url(origin for email link URLs),emailnamespace (SMTP transport fornotifications-email;passdefaults to the newNOTIFICATIONS_SMTP_PASSsecret),public_link_types(pre-auth link types, replacing hardcoded invite checks)notifications-email(SMTP) — apps can remap it to their own email connection instead of setting the vars{ key: 1 }, { unique: true, partialFilterExpression: { key: { $type: 'string' } }, name: 'notification_key_unique' }— the no-double-send guarantee does not exist without itRecord convention + legacy coalescing
Pipeline records write
type(the notification config id),preview, renderedsubject/body/text, anddataholding the original item (link targets stay{ pageId, urlQuery }objects — the link page reads them back at the?optiondot-path; a resolved copy would redirect the landing page to itself). Read paths coalesce legacy Lambda-era records: inboxdescription ?? preview, badges/filtersevent_type ?? type, link page falls back to top-levellinks.button, detail viewbody ?? text.Demo app = reference implementation
notifications:template configs (quote-approved,user-invite), SMTP vars reusing the SendGrid relay credentials, a rewritten send-routine (aggregations shape items with embedded recipient contacts, then dispatch), and enum entries for the new type badges. The Lambda-era guide (apps/demo/.claude/guides/notifications.md) is rewritten for the new architecture. workflows-test keeps its store-only$mergemock (deliberately exercising engine dispatch without email) with field names aligned to the convention — its e2e contract (type: workflow-error) is unchanged.Verification
dispatchNotifications.test.js— thesend-notification{ event_ids }contract is untouched)pnpm docs:gen+pnpm docs:checkpass (vars reference + llms.txt regenerated)apps/demopins a publishedlowdefyversion that predatesRenderNotification. Once the release is published fromfeat/email-notifications, bump the pin andpnpm --dir apps/demo ldf:b+ a runtime smoke (approve quote → record → inbox → landing link → duplicate skip) validate end-to-end. The framework-side composed pipeline (render → insert → send → update) is already e2e-verified against a live server in lowdefy#2234.Companion PR: lowdefy/lowdefy#2234.
🤖 Generated with Claude Code