Sidebar ads: target by page type alongside keywords

[relyks]

Description

Sidebar ads are the small promotional boxes that appear in the sidebar of many pages. Until now, they could only be targeted with keywords. Keywords describe what the reader is looking at — a book name, a category, a topic. They cannot describe what kind of page the reader is on. For example, a Talmud book's table of contents and the Talmud category browse page both produce the keyword talmud. An editor had no way to say "show this ad only on tables of contents" or "only on the calendars page."

This branch adds that second way to target. Each sidebar ad gets a new Page Type dropdown in Strapi. The ad only shows when the current page's kind matches the chosen value. This new check is added on top of all the existing checks (keywords, language, dates, audience). Include-keyword behavior is unchanged; the empty-field and exclusion rules were deliberately revised during review (details below).

The work is split into two commits:

1. First commit: the test suite stops replaying recorded network traffic. The old tests replayed .har files (recordings of real Strapi responses). The replay tool matches on the exact request body — so adding even one field to our GraphQL query would break all fourteen recordings at once. Now, every recorded scenario is rebuilt in code with the existing payload factory. A new test proves that each rebuilt payload is byte-for-byte equal to its recording. The .har files stay in the repo forever as reference material, but nothing replays them anymore.

2. Second commit: the feature. Seventeen values, defined in one new file (static/js/sefaria/pageTypes.js):

   all_pages homepage category_toc book_toc topic_page author_page portal_page topic_category_toc topics_landing all_topics calendars translations collection_page public_collections user_library notifications voices_home

   The current page's kind is computed from the reader's panel state (which menu is open, which categories are set). There is one special case: for a topic page, we cannot know if it is a plain topic, an author, or a sponsor portal until the topic's data has been fetched from the server. So TopicPage — the component that fetches that data — passes the answer to the ad component through a new pageTypeOverride prop, and only after the data has loaded.

   Six pages never had an ad slot at all. They get one now, so that every dropdown value actually works somewhere: the topics landing page, the Voices home page, notifications, translations, collection pages, and the public collections directory. Portal pages (sponsor-branded topic pages) also get a slot, placed below the sponsor's own content.

   There is deliberately NO fallback for old Strapi servers. If a Strapi environment does not have the new field yet, GraphQL rejects the entire query, no promotions render at all, and the browser console names the problem (LIKELY SCHEMA MISMATCH inside the fetch error). The goal is always a compatible frontend and Strapi; a brief dark window during a deploy is safer than a fallback whose responses could be cached and served to healthy visitors.

   The keyword rules were also tightened alongside the new field (see item 2 below): an empty keywords field now means "no keyword restriction," and exclusion-only keywords no longer double as a way to say "everywhere."

   One more addition (fourth commit): the ad body now renders markdown, the same way banner and modal body text already does. An editor can write bold text, links, and line breaks in a sidebar ad's body and they format instead of appearing as literal ** and []() characters. The title stays plain, like a banner or modal header.

What could warrant closer review

1. The Strapi field must be added before this frontend deploys — and the failure mode is now a hard stop.

   The pageType dropdown (17 values, default all_pages) must exist on the Side Bar Ad content type in every Strapi environment first. If it doesn't, GraphQL rejects the whole query and NO promotions render — banners, modals, and ads all stay dark until the environments match. The console error contains LIKELY SCHEMA MISMATCH so the cause is findable.

   An earlier draft had a softer fallback: on error, retry once with the old query. It was removed because it could poison the shared cache: the server cache key did not include the query body, so one browser's transient error could cache a pageType-less response that every visitor then received for up to 7 days — turning every restricted campaign into a site-wide one (the exact fail-open outcome the design forbids). Instead, the cache key now also hashes the query body (so even a stale browser bundle during a deploy cannot poison the slot new clients read), a pytest pins that separation, and the browser test asserts exactly ONE query is sent (the observable that distinguishes "no fallback" from "fallback that also failed").

2. Keyword semantics changed, and live exclusion-only campaigns need one edit.

   The pages this feature makes targetable (homepage, calendars, notifications, the six new slots…) produce NO keywords — they are not "about" anything. Under the old rule, an ad whose keywords were exclusion-only (like !social-issues) matched any page without that keyword, including keyword-less pages; that was also the editor hack for "show everywhere" (!nowhere). The revised rule (user-ratified): an EMPTY keywords field means "no keyword restriction" (the honest way to say any page), exclusion keywords only subtract from keyword-BEARING pages, and an ad with both includes and exclusions must satisfy both (the old rule needed only one).

   Consequence for live content: an existing exclusion-only ad keeps running on reader/category/topic pages but stops appearing on the homepage and other keyword-less pages until an editor clears its keywords field (and picks a Page Type if it shouldn't run absolutely everywhere). Ads with include keywords, and ads with empty keywords, are unaffected.

3. Collection pages and category pages share one value on purpose.

   A "collection" browse page (like /texts/.../Covenant and Conversation) and a normal category page (like /texts/Tanakh/Torah) look different to a person, but in the database they are the same thing: plain Category records, rendered by the same component. The client cannot tell them apart, so both are category_toc. To target one specific collection, combine the page type with a keyword: category_toc + keyword covenant and conversation. Multi-word keywords work — only commas split the keyword list — and tests pin this at both the unit and browser level.

4. Portal pages match only portal_page, even when the topic is also an author.

   Both current portals (Rabbi Sacks, Rabbi Steinsaltz) happen to be authors. But the portal_slug field lives on the base Topic model, so a future portal might not be an author at all (it could be an organization). Classification therefore checks portal first and returns only portal_page. The practical effect: an author_page campaign will never appear on a sponsor-branded page. Editors must choose portals on purpose.

5. The "frozen recordings" model changes what the contract test means.

   The contract test still compares the payload factory's fields against the recordings. But fields added after the recordings were made (like pageType) are declared in a list called FIELDS_ADDED_SINCE_RECORDING, and the comparison skips them. A second test fails if a declared field ever shows up inside a recording — so the list cannot quietly become an excuse for mismatches. The trade-off: as the query grows over time, the recordings verify a smaller share of it. If the list gets long, the plan is to capture one fresh reference recording as documentation.

6. Page types from all open panels are merged together.

   If a reader has a book's table of contents open in any panel, a book_toc ad can show. This mirrors how keywords already merge across panels, so editors only need one mental model.

7. The server cache key now includes a hash of the query text.

   The cache version was bumped (v5 → v6) AND the key gained a 12-character hash of the query body. Different query shapes get different cache slots, so a browser running an older bundle during a deploy window can never cache a smaller response under the slot up-to-date clients read. Error responses were already never cached. A new pytest proves two different query bodies with the same dates use separate slots.

Key Design Decisions

• Additive for page types; deliberate for keywords. An ad with no pageType (any pre-feature document) is treated as all_pages, so the new dropdown breaks nothing. The keyword gate was revised on purpose (empty = unrestricted, strict exclusions, include AND exclude) and lives in pure helpers with a Jest truth table — review item 2 explains the one edit live exclusion-only ads need.

• Unknown values fail closed. If Strapi ever sends a value the client does not recognize (say, a typo), the ad shows nowhere — not everywhere. A campaign that mysteriously never appears gets noticed and fixed. A typo that broadcast a campaign site-wide would not.

• Never guess before the data arrives. For topic pages, the classifier deliberately answers "unknown" (null), because the real answer needs fetched data. TopicPage already waits for that data before mounting the ad component, so an ad can never appear under a provisional guess and then vanish when the real answer arrives. Worst case, the ad appears a moment late.

• The portal slot hardcodes its own page type. The portal sidebar component only ever renders on portal pages, so it passes portal_page directly. No data threading needed — the component's very existence proves the classification.

• The test migration is proven, not trusted. The rebuilt payloads were generated by comparing each recording against the factory's defaults, and a fidelity test re-checks byte equality on every run. A wrong date, a missing document, or a reordered row fails the suite.

Code Changes

File	Purpose
static/js/sefaria/pageTypes.js (new)	The 17-value list plus the pure functions that classify pages and match ads: classifyPanel, classifyPanels, topicPageTypeOf (portal first, then author), normalizePageType, adMatchesPageTypes.
static/js/ReaderApp.jsx	Adds the current page kinds (pageTypes) to the ad-targeting context.
static/js/Promotions.jsx	Accepts the pageTypeOverride prop, adds the page-type and revised keyword checks to the ad-matching filter, and renders the ad BODY as markdown through the same InterfaceText path banners and modals use (titles stay plain).
static/js/sefaria/sidebarAds.js	Copies pageType from the Strapi document onto each ad, with the all_pages default.
static/js/context.js	Adds pageType to the GraphQL query (now built from one list so the legacy version cannot drift) and implements the one-shot legacy retry with its clear log prefix.
static/js/TopicPage.jsx	Passes the resolved page type once topic data is loaded; the portal sidebar appends an ad slot after the sponsor's modules.
static/js/NavSidebar.jsx	The Promo sidebar module now forwards pageTypeOverride using the existing module-props mechanism.
TopicsLandingPage.jsx, SheetsHomePage.jsx, NotificationsPanel.jsx, TranslationsPage.jsx, CollectionPage.jsx, PublicCollectionsPage.jsx	One-line additions of the previously missing ad slot.
sefaria/views.py	Cache version v5 → v6, with a comment on why the shared cache slot is safe.
e2e-tests/tests/strapi.scenario-payloads.js (new)	Code-built copies of all fourteen recordings, attached to each test scenario.
e2e-tests/tests/strapi-scenario-payload-fidelity.spec.js (new)	Proves each copy equals its recording, byte for byte. Runs without a server.
e2e-tests/tests/strapi-payload-contract.spec.js	Learns about declared post-recording fields, and fails if a declared field appears inside a recording.
e2e-tests/support/strapi-payload-factory.js	Adds the pageType default and the FIELDS_ADDED_SINCE_RECORDING list.
e2e-tests/support/strapi-har-fixture.js	Retired in place — no test imports it; kept as documentation of how the recordings were made.
14 × strapi-{banner,modal,sidebar-ad}*.spec.js	Routing swapped from HAR replay to the code-built payloads. Assertions unchanged. Comments that described the old recording workflow were rewritten.
e2e-tests/tests/strapi-sidebar-ad-page-type.spec.js (new)	The feature's browser test suite — described below.
e2e-tests/tests/strapi-sidebar-ad-markdown.spec.js (new)	Markdown in the ad body renders as elements (bold, links) with no raw syntax left behind; markdown in the title stays literal — pinning both sides of the plain-title boundary.
e2e-tests/support/seed_topic_pools.py (new)	A small script that fills the local topic-pool table (Postgres). Without it, every /topics/<slug> page returns 404 on a fresh local setup. The script explains why the Django server must be restarted after running it.
static/js/sefaria/tests/pageTypes.test.js (new), sidebarAds.test.js	Unit tests, described below.
e2e-tests/tests/README.md, e2e-tests/CLAUDE.md, strapi.fixtures.js	Documentation updated for the new synthetic-only testing model.

Tests

All green: 384 Jest unit tests, 177 chrome-strapi browser tests, and 16 strapi-cache pytest tests (including the new cache-slot separation test). The browser tests use the suite's existing deterministic setup: Strapi responses are served in the browser from generated payloads, the clock is pinned, and a guard fails any test whose page never actually requested the Strapi data.

Unit: pageTypes.test.js (37 tests) + sidebarAds.test.js additions

• classifyPanel — one test per branch: homepage vs. category on the navigation menu, book TOC (both menu spellings), the three topic-menu cases (category list / specific topic / landing page), every simple menu mapping, and null for reader panels, search, profile, admin, and missing input. The "specific topic returns null" case has a comment explaining that TopicPage supplies the real answer.
• topicPageTypeOf — portal beats author on a topic that is both; author detected via subclass; plain topics; missing data does not throw. The tests document why indexes.length is not used: an author with no cataloged works is still an author (true of the local database's own data).
• normalizePageType — null/undefined/empty become all_pages; unknown strings pass through so they can never match; known values are unchanged.
• adMatchesPageTypes — all_pages passes everywhere, including on pages that classify to nothing; specific values pass only when the page kind is active; unknown values never pass.
• sidebarAds.test.js — the trigger's pageType defaults correctly for old documents and carries the Strapi value when present; multi-word keywords keep their inner spaces; every pre-existing keyword parsing test is untouched and still green (the proof that keyword behavior did not change).

Browser: strapi-sidebar-ad-page-type.spec.js (26 tests)

• One "shows up" test per page type, each on its real page — homepage, a real category page, a real book's table of contents, seeded real topic and author pages, the topic category page, topics landing, the A–Z list, calendars, translations, both collection surfaces and the Voices home (visited on the Voices module host), both portals, and notifications (logged in via the suite's stored session). Each test fails only if that page type's chain — classification, ad slot, matching — breaks.
• Both portals on purpose — the Sacks portal publishes only one sponsor block while Steinsaltz publishes all four, so together they prove the ad slot works regardless of how the sponsor sidebar is configured.
• Exclusivity — with portal, author, and topic ads all in one payload: only the portal ad shows on a portal page; only the author ad on an author page; only the topic ad on a plain topic page.
• Boundaries — an ad with a null pageType (an old document) still shows; a book_toc ad does not show on a category page; a homepage ad does not show on a topic category page; an unknown value shows nowhere. Every "does not show" assertion first proves the payload arrived and the page rendered, so it cannot pass by accident.
• The combined targeting case — a category_toc ad with the keyword covenant and conversation shows on that collection's real page and does not show on /texts/Tanakh/Torah — proving the keyword narrows within the page type.
• No flash — the topic API response is held back while the page loads. While it is held, zero ads render (the ad component is not mounted yet). When released, only the correctly typed ad appears.
• Schema mismatch — the test plays an old Strapi: it rejects the query with GraphQL errors inside an HTTP 200. The test asserts the page rendered, the console names the mismatch (LIKELY SCHEMA MISMATCH), NOTHING promotional rendered — no ad, no banner, no modal — and exactly ONE query was sent (the count is what proves no fallback retry exists).
• The real query is checked — the synthetic route matches the URL alone and serves every field regardless of what the client asked for, so one test asserts on the intercepted POST body: the production GraphQL query must actually name pageType (with keywords and startTime as canaries). Without this, dropping the field from the query would leave every other test green while real ads quietly lost their targeting.
• Reactivity — a homepage-targeted ad is visible on /texts, disappears when the reader clicks into a category (a client-side navigation — the Strapi request counter proves no reload happened), and returns on browser back. Every other test is a cold page load; this is the one that proves matching reacts to in-app navigation.
• Authenticated pages — notifications on /notifications and user_library on /saved (the real saved-content route; the similar-looking /texts/saved is just a category-path navigation page), using the suite's stored login session.
• Tests with local prerequisites skip with a message naming the fix (the seed script, or the /etc/hosts entry for the Voices host) — never silently.

Migration guards: fidelity (15 tests) + contract additions

• Every rebuilt payload equals its recording exactly — same documents, languages, dates, and row order, including the extra co-published content some recordings carry.
• A completeness test fails if a recorded scenario has no rebuilt copy, or a copy has no recording.
• The new contract test fails if a declared post-recording field ever appears inside a recording, which keeps the exemption list honest.
• All fourteen migrated test files run unchanged — the behavioral proof that the rebuilt payloads are true drop-in replacements.

Testing Patterns

"Shows up" tests visit real pages with real data. The tests use actual database content — a real multi-word category, seeded topic pages, both real portal documents, the real Voices host — instead of mocked page shells. A bug that only appears against real page state fails here.

Missing prerequisites skip loudly. If the topic pools are not seeded or the Voices host is unreachable, the affected tests skip with a message that names the fix. A skipped test that looks green but explains itself is safer than a passing test that proved nothing.

Fallback behavior is proven by its own log line. The old-Strapi test checks the console message in addition to the rendered ad, because a rendered ad alone could also mean the fallback was never needed.

The migration is checked on every run. Byte equality between every rebuilt payload and its recording runs as part of the suite, so the fixtures can never drift from reality without a test failing.

[gitvelocity-reviewer]

📊 Code Quality Score: 70/100

Base Score 70 × ESF 1.0 (Extra Large: 2082 effective lines, 39 files) = 70

Category	Score	Factors
🔭 Scope	16/20	39 files across frontend JS (pageTypes.js, Promotions.jsx, NavSidebar.jsx, TopicPage.jsx, context.js, sidebarAds.js, ReaderApp.jsx, six page components), E2E test infrastructure (strapi.scenario-payloads.js, strapi-scenario-payload-fidelity.spec.js, strapi-sidebar-ad-page-type.spec.js, strapi-payload-contract.spec.js, fourteen existing spec files migrated), Python seeding (seed_topic_pools.py), Django views (views.py), and the CLAUDE.md memory file. New public exports: PAGE_TYPE, classifyPanel, classifyPanels, topicPageTypeOf, adMatchesPageTypes, normalizePageType, FIELDS_ADDED_SINCE_RECORDING.
🏗️ Architecture	14/20	pageTypes.js is a new module that sits between panel state (in ReaderApp.getUserContext) and ad matching (in Promotions.getCurrentMatchingAds), introducing a classifyPanels → AdContext.pageTypes → Promotions → adMatchesPageTypes dependency chain. SIDEBAR_AD_FIELD_LIST replaces the sidebarAdFields template string in context.js, enabling the legacy-Strapi retry to derive a filtered selection from the same list. TopicPage.jsx now passes pageTypeOverride down to Promotions, creating a new prop-based override path alongside the context-based one. No existing module boundary is removed or merged.
⚙️ Implementation	15/20	classifyPanel handles eight distinct menuOpen values plus the navigation and topics menus with sub-state discrimination, returning null for unclassifiable panels (text reader, search, specific topic) rather than guessing. topicPageTypeOf applies portal exclusivity before author detection, with portal_slug winning over subclass. The legacy-Strapi retry in context.js refactors the fetch into fetchStrapiQuery, calls it with the full query, detects errors in the 200 response, and retries once with legacySidebarAdFields derived by filtering SIDEBAR_AD_FIELD_LIST. FIELDS_ADDED_SINCE_RECORDING in the factory drives both the scenario replica stripping in scenarioPayload() and the contract spec's anti-gaming companion test that fails if a declared field appears in a recording.
⚠️ Risk	10/20	STRAPI_SCHEMA_VERSION bumped from v5 to v6, invalidating all cached Strapi responses on deploy — every user's first page load after deploy fetches fresh. The cache key is shared between full-query and legacy-retry responses (comment in views.py acknowledges this): a cached legacy payload (without pageType) served to a full-featured client silently loses page-type targeting until the cache expires. The legacy retry adds one extra fetch on any GraphQL error, not only unknown-field errors, adding latency on transient failures. The change is additive — all_pages default means existing ads are unaffected — and the pageType gate ANDs with existing gates, so a misconfigured ad shows nowhere rather than everywhere.
✅ Quality	12/15	pageTypes.test.js (159 lines) drives all six exported functions: classifyPanel covers eight menu values plus navigation sub-states, null inputs, and unclassifiable menus; topicPageTypeOf covers portal exclusivity over author, author with no indexes, plain topic, and undefined input; normalizePageType covers null/undefined/empty and the pass-through of unknown values; adMatchesPageTypes covers all_pages, specific match, specific non-match, and unknown values. sidebarAds.test.js adds three tests: multi-word keyword preservation, pageType defaulting to all_pages for pre-field documents, and pageType carry-through. strapi-scenario-payload-fidelity.spec.js proves byte-equality between each SCENARIO_PAYLOADS entry and its .har recording. strapi-sidebar-ad-page-type.spec.js covers 16 page types in the shows-up matrix, four boundary tests (null, negative, unknown, conjunction), author/topic/portal exclusivity, the no-flash guarantee via held topic response, legacy-Strapi degradation with console assertion, and the authenticated notifications path. No unit test drives Promotions.jsx directly.
🔒 Perf / Security	3/5	The legacy-Strapi retry adds at most one extra fetch per page load on error paths; normal paths are unaffected. The fail-closed unknown-value behavior in normalizePageType (pass through unchanged, never match) prevents a CMS typo from becoming a site-wide campaign. STRAPI_SCHEMA_VERSION v6 prevents cross-version cache collisions between the old and new GraphQL shapes.

Was this score accurate? 👍 Yes · 👎 No

How this was scored →

_{Scored by GitVelocity · How are scores calculated?}