Skip to content

Commit 79aa2f4

Browse files
fix(consent): load Ketch first and hold Heap and GA4 until consent (#143)
## What cli.deepgram.com and cli.staging.deepgram.com load Heap and GA4 (G-TYPC1TBCKT) before any consent, including for visitors in GDPR regions and when Ketch is blocked, and have no Ketch and no privacy control. This brings the page in line with developers.deepgram.com: - `web/src/consent/ketch.js`: docs' `fern/assets/ketch.js`, copied verbatim (the Ketch loader for `deepgram_marketing_site` plus the `dgOnConsent` / `dgOnRevoke` helpers), inlined as the first script in `<head>`. - `web/public/consent/dg-consent.js`: docs' built consent banner, unchanged. It mounts the footer "Your Privacy Choices" control into `<div id="fern-footer">` (the id the bundle looks for), and shows a Privacy Policy link instead when Ketch is unavailable. Source commit and hashes are in `web/src/consent/SOURCE.json`. - `web/src/consent/trackers.js`: Heap (docs' snippet, which also fixes the old `heap[o]=` stub bug) and GA4 start only after Ketch grants `targeted_advertising`, and stop if it's withdrawn (Heap `stopTracking`, GA `ga-disable-<id>`, then the banner's reload on withdrawal). Neither starts if Ketch never loads. - `<html class="dark">` makes the banner use its dark theme to match the site. The page has no `dark:` utilities, so nothing else changes. **GA4 purpose:** `targeted_advertising`. The live Ketch config (`production/gdpr/en/config.json`) lists `_ga` and `_ga_*`, along with Heap's `_hp2_*`, under `targeted_advertising`. Nothing related is listed under `analytics`. ## Checked Built with `npm ci && npm run build` (node:22, Docker), served `dist/`, then ran Playwright with a fresh profile each time and Ketch's location stubbed: | | Before (live) | After (local build) | |---|---|---| | Ketch first script in `<head>` | no Ketch | yes | | GB, no choice | Heap, GA4, doubleclick, `_ga` cookies | nothing loads, no cookies, banner auto-opens: Reject All · Customize · Accept All | | GB Reject All / Accept All | n/a | nothing loads / Heap and GA4 load | | US-CA (opt-out state) | everything | no auto-open; Heap and GA4 load (granted by default, as on docs) | | Ketch blocked | everything | nothing loads, footer shows Privacy Policy | ## Notes - Production deploys only from a GitHub release (`web-production.yml`), so merging this updates staging only. cli.deepgram.com keeps the ungated trackers until the next release. - deepgram/deepgram-docs#1302 plans to retire cli.deepgram.com. If the redirect ships first, this goes away with it. - To update these files, copy them again from the same docs commit. Don't edit them here. ## Release plan Merging to main only deploys staging. Production (`web-production`) runs on a root `v*` release, and prod still loads Heap and gtag without Ketch today. Plan: request a patch release right after merge, unless docs#1302 retires cli.deepgram.com first. ## Tests `web/tests/consent-gate.test.cjs` (run by the new `web-ci.yml` on PRs touching `web/**`) covers: nothing loads before consent, after rejection, or with Ketch unavailable; Heap and GA4 load only on `targeted_advertising`; withdrawal disables GA4 and calls Heap `stopTracking`. ## Known gap `_ga` / `_ga_*` cookies persist after withdrawal until they expire. GA is disabled but the cookies are not deleted (same as developers.deepgram.com). ## Maintenance `web/src/consent/SOURCE.json` pins the vendored Ketch files by sha256, and the test checks it. If cli rebrands, re-copy both files from the same docs commit and update `SOURCE.json`, or the hash test will fail.
2 parents b5beda0 + 5cd0725 commit 79aa2f4

11 files changed

Lines changed: 489 additions & 22 deletions

File tree

‎.github/workflows/web-ci.yml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
name: Web CI
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- "web/**"
7+
- "pyproject.toml"
8+
- ".github/workflows/web-ci.yml"
9+
10+
permissions:
11+
contents: read
12+
13+
concurrency:
14+
group: web-ci-${{ github.ref }}
15+
cancel-in-progress: true
16+
17+
jobs:
18+
test:
19+
name: Test & build web
20+
runs-on: ubuntu-latest
21+
steps:
22+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
23+
24+
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
25+
with:
26+
node-version: 22
27+
cache: npm
28+
cache-dependency-path: web/package-lock.json
29+
30+
- name: Install dependencies
31+
run: npm ci
32+
working-directory: web
33+
34+
- name: Test
35+
run: npm test
36+
working-directory: web
37+
38+
- name: Build
39+
run: npm run build
40+
working-directory: web

‎web/package-lock.json‎

Lines changed: 20 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎web/package.json‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,13 @@
99
"dev": "astro dev",
1010
"build": "astro build",
1111
"preview": "astro preview",
12-
"astro": "astro"
12+
"astro": "astro",
13+
"test": "node --test \"tests/*.test.cjs\""
1314
},
1415
"dependencies": {
1516
"@astrojs/sitemap": "^3.7.3",
17+
"@fontsource-variable/inter": "5.3.0",
18+
"@fontsource/fira-code": "5.3.0",
1619
"@resvg/resvg-js": "^2.6.2",
1720
"@tailwindcss/vite": "^4.3.3",
1821
"astro": "^7.3.3",

‎web/public/consent/dg-consent.js‎

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎web/src/consent/SOURCE.json‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"note": "Vendored verbatim from deepgram/deepgram-docs so cli.deepgram.com matches developers.deepgram.com. Do not edit by hand; re-copy both files from the same docs commit.",
3+
"repo": "https://github.com/deepgram/deepgram-docs",
4+
"commit": "b807ab4613f9b138c761a7f21339975108c2e00b",
5+
"files": {
6+
"web/src/consent/ketch.js": "fern/assets/ketch.js",
7+
"web/public/consent/dg-consent.js": "fern/assets/dg-consent.js (built from tools/consent-banner)"
8+
},
9+
"sha256": {
10+
"ketch.js": "00348a71e084e22ca2096d50f5d9179f572b7a753220436203c2c2590224f2b0",
11+
"dg-consent.js": "a9ba240426d3299fde1bc365897c677dfde1cf7403ac11e5ae5a56fb6d30eee0"
12+
}
13+
}

‎web/src/consent/ketch.js‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
// Ketch consent manager loader. Must run before Google Tag Manager and every
2+
// other tracking script, so it is the first `beforeInteractive` entry in
3+
// docs.yml and GTM is loaded from ./gtm.js right after it (not from Fern's
4+
// built-in `analytics.gtm`, which Fern always renders ahead of custom JS).
5+
// The line below is Ketch's snippet verbatim, minus the <script> wrapper.
6+
!function(){window.semaphore=window.semaphore||[],window.ketch=function(){window.semaphore.push(arguments)};var e=document.createElement("script");e.type="text/javascript",e.src="https://global.ketchcdn.com/web/v3/config/deepgram/deepgram_marketing_site/boot.js",e.defer=e.async=!0,document.getElementsByTagName("head")[0].appendChild(e)}();
7+
8+
// Consent helpers shared by gtm.js, component.js (Heap) and sentry.js (Sentry).
9+
// One `consent` subscription keeps Ketch's latest answer. Ketch fires
10+
// `consent` when it resolves the visitor's choice and again whenever the
11+
// visitor changes it, including a withdrawal, and replays the current value to
12+
// late subscribers. Purpose codes come from the deepgram_marketing_site
13+
// property: `analytics` and `targeted_advertising` (opt-in under gdpr, where
14+
// unconsented purposes are left out of the payload; granted by default
15+
// elsewhere). If Ketch never loads (blocked, offline), nothing counts as
16+
// granted: trackers fail closed.
17+
(function (w) {
18+
var granted = {};
19+
var waiting = [];
20+
var revokers = [];
21+
22+
function hasAll(purposes) {
23+
for (var i = 0; i < purposes.length; i++) {
24+
if (granted[purposes[i]] !== true) return false;
25+
}
26+
return true;
27+
}
28+
29+
function run(cb) {
30+
// A throwing callback must not stop the others; rethrow it on its own so
31+
// Sentry still sees it. It is not retried: a callback that failed partway
32+
// could load GTM or add Replay twice if run again, which is worse than
33+
// leaving that one tracker off until the next page load.
34+
try {
35+
cb();
36+
} catch (err) {
37+
setTimeout(function () {
38+
throw err;
39+
});
40+
}
41+
}
42+
43+
w.ketch("on", "consent", function (consent) {
44+
granted = (consent && consent.purposes) || {};
45+
var ready = [];
46+
var still = [];
47+
for (var i = 0; i < waiting.length; i++) {
48+
(hasAll(waiting[i].purposes) ? ready : still).push(waiting[i]);
49+
}
50+
waiting = still;
51+
for (var j = 0; j < ready.length; j++) run(ready[j].cb);
52+
for (var k = 0; k < revokers.length; k++) {
53+
var now = hasAll(revokers[k].purposes);
54+
if (revokers[k].had && !now) run(revokers[k].cb);
55+
revokers[k].had = now;
56+
}
57+
});
58+
59+
// True only while Ketch's latest answer grants every purpose in `purposes`.
60+
// Check it at the moment data would leave the page, so a withdrawal stops
61+
// anything that forwards data per event.
62+
w.dgHasConsent = function (purposes) {
63+
return hasAll(purposes);
64+
};
65+
66+
// Run `cb` once, the first time every purpose in `purposes` is granted, so a
67+
// later opt-in still runs it. To stop a tracker on withdrawal, pair it with
68+
// dgOnRevoke; anything that can't pause stops on the next full page load.
69+
w.dgOnConsent = function (purposes, cb) {
70+
if (hasAll(purposes)) run(cb);
71+
else waiting.push({ purposes: purposes, cb: cb });
72+
};
73+
74+
// Run `cb` each time a visitor withdraws any purpose in `purposes` after
75+
// granting all of them, so a tracker that can pause (Heap, Sentry Replay)
76+
// stops collecting on the same page. A tracker stopped this way stays off
77+
// if the visitor grants consent again on the same page, until the next
78+
// page load. GTM tags rely on the container's own Ketch consent settings
79+
// to stop on withdrawal.
80+
w.dgOnRevoke = function (purposes, cb) {
81+
revokers.push({ purposes: purposes, cb: cb, had: hasAll(purposes) });
82+
};
83+
84+
// Ketch's `needsConsent` verdict for this page load: true while consent is
85+
// missing or incomplete, false once the visitor has chosen, null until Ketch
86+
// says. Ketch emits it once, before `consent`, and does not replay it to late
87+
// subscribers. dg-consent.js subscribes after hydration, after Ketch has
88+
// emitted it, so it reads the verdict recorded here to decide auto-open.
89+
var needsConsent = null;
90+
w.ketch("on", "needsConsent", function (value) {
91+
if (typeof value === "boolean") needsConsent = value;
92+
});
93+
w.dgKetchNeedsConsent = function () {
94+
return needsConsent;
95+
};
96+
})(window);

‎web/src/consent/trackers.js‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
// Heap and Google Analytics 4 for cli.deepgram.com, held until Ketch consent
2+
// through the dgOnConsent / dgOnRevoke helpers in ./ketch.js, as on
3+
// developers.deepgram.com (fern/assets/component.js). Both wait for
4+
// `targeted_advertising`: the Ketch config for deepgram_marketing_site lists
5+
// Heap's _hp2_* and GA's _ga / _ga_* cookies under that purpose. If Ketch
6+
// never loads (blocked, offline), neither starts.
7+
(function (w, d) {
8+
if (typeof w.dgOnConsent !== "function") return;
9+
var PURPOSES = ["targeted_advertising"];
10+
var GA_ID = "G-TYPC1TBCKT";
11+
12+
function insertHeap() {
13+
if (d.getElementById("heap-script")) return;
14+
var script = d.createElement("script");
15+
script.id = "heap-script";
16+
script.text =
17+
'window.heapReadyCb=window.heapReadyCb||[],window.heap=window.heap||[],heap.load=function(e,t){window.heap.envId=e,window.heap.clientConfig=t=t||{},window.heap.clientConfig.shouldFetchServerConfig=!1;var a=document.createElement("script");a.type="text/javascript",a.async=!0,a.src="https://cdn.us.heap-api.com/config/"+e+"/heap_config.js";var r=document.getElementsByTagName("script")[0];r.parentNode.insertBefore(a,r);var n=["init","startTracking","stopTracking","track","resetIdentity","identify","getSessionId","getUserId","getIdentity","addUserProperties","addEventProperties","removeEventProperty","clearEventProperties","addAccountProperties","addAdapter","addTransformer","addTransformerFn","onReady","addPageviewProperties","removePageviewProperty","clearPageviewProperties","trackPageview"],i=function(e){return function(){var t=Array.prototype.slice.call(arguments,0);window.heapReadyCb.push({name:e,fn:function(){heap[e]&&heap[e].apply(heap,t)}})}};for(var p=0;p<n.length;p++)heap[n[p]]=i(n[p])};' +
18+
'heap.load("765739241");';
19+
d.head.appendChild(script);
20+
}
21+
22+
function stopHeap() {
23+
if (w.heap && typeof w.heap.stopTracking === "function") w.heap.stopTracking();
24+
}
25+
26+
function consentState(value) {
27+
return {
28+
ad_storage: value,
29+
ad_user_data: value,
30+
ad_personalization: value,
31+
analytics_storage: value,
32+
};
33+
}
34+
35+
function insertGa() {
36+
if (d.getElementById("ga4-script")) return;
37+
w.dataLayer = w.dataLayer || [];
38+
w.gtag = function () {
39+
w.dataLayer.push(arguments);
40+
};
41+
// Google Consent Mode v2: queue the denied default ahead of everything else
42+
// so no hit can go out before the state is declared, then record the grant
43+
// that got us here. gtag.js reads the dataLayer in order.
44+
w.gtag("consent", "default", consentState("denied"));
45+
w.gtag("consent", "update", consentState("granted"));
46+
w.gtag("js", new Date());
47+
w.gtag("config", GA_ID);
48+
var script = d.createElement("script");
49+
script.id = "ga4-script";
50+
script.async = true;
51+
script.src = "https://www.googletagmanager.com/gtag/js?id=" + GA_ID;
52+
d.head.appendChild(script);
53+
}
54+
55+
// Google's documented per-property opt-out; the page also reloads on a
56+
// withdrawal (dg-consent.js), so nothing restarts until consent is granted.
57+
function stopGa() {
58+
w["ga-disable-" + GA_ID] = true;
59+
if (typeof w.gtag === "function") w.gtag("consent", "update", consentState("denied"));
60+
}
61+
62+
w.dgOnConsent(PURPOSES, insertHeap);
63+
w.dgOnConsent(PURPOSES, insertGa);
64+
w.dgOnRevoke(PURPOSES, stopHeap);
65+
w.dgOnRevoke(PURPOSES, stopGa);
66+
})(window, document);

‎web/src/pages/index.astro‎

Lines changed: 19 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
---
22
import '../styles/global.css';
3+
// Self-hosted fonts: no request to Google before (or without) consent.
4+
import '@fontsource-variable/inter/opsz.css';
5+
import '@fontsource/fira-code/400.css';
6+
import '@fontsource/fira-code/500.css';
37
import { readFileSync } from 'node:fs';
48
import { join } from 'node:path';
59
@@ -208,12 +212,20 @@ const commandCards: CommandCard[] = [
208212
desc: 'Sits in front of a stream so you can see exactly what reaches Deepgram.',
209213
lines: ['dg debug probe'] },
210214
];
215+
import ketchJs from '../consent/ketch.js?raw';
216+
import consentTrackersJs from '../consent/trackers.js?raw';
211217
---
212218

213-
<html lang="en" prefix="og: https://ogp.me/ns#">
219+
<html lang="en" class="dark" prefix="og: https://ogp.me/ns#">
214220
<head>
215221
<!-- ── Essentials ──────────────────────────────────────── -->
216222
<meta charset="utf-8" />
223+
<!--
224+
Ketch consent manager plus the dgOnConsent / dgOnRevoke helpers, vendored
225+
from developers.deepgram.com (src/consent/SOURCE.json). Must stay the
226+
FIRST script in <head>: Heap and GA4 below wait for its consent.
227+
-->
228+
<script is:inline set:html={ketchJs}></script>
217229
<meta name="viewport" content="width=device-width, initial-scale=1" />
218230
<meta name="generator" content={Astro.generator} />
219231
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
@@ -254,11 +266,6 @@ const commandCards: CommandCard[] = [
254266
<meta name="theme-color" content="#13ef95" />
255267
<meta name="msapplication-TileColor" content="#050506" />
256268

257-
<!-- ── Fonts ──────────────────────────────────────────── -->
258-
<link rel="preconnect" href="https://fonts.googleapis.com" />
259-
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
260-
<link href="https://fonts.googleapis.com/css2?family=Inter:ital,opsz,wght@0,14..32,300;0,14..32,400;0,14..32,500;0,14..32,600;0,14..32,700;0,14..32,800;0,14..32,900&family=Fira+Code:wght@400;500&display=swap" rel="stylesheet" />
261-
262269
<!-- ── DNS prefetch ───────────────────────────────────── -->
263270
<link rel="dns-prefetch" href="https://deepgram.com" />
264271
<link rel="dns-prefetch" href="https://developers.deepgram.com" />
@@ -438,20 +445,8 @@ const commandCards: CommandCard[] = [
438445
]
439446
})} />
440447

441-
<!-- Heap Analytics — product analytics for cli.deepgram.com -->
442-
<script is:inline>
443-
window.heap=window.heap||[],heap.load=function(e,t){window.heap.appid=e,window.heap.config=t=t||{};var r=document.createElement("script");r.type="text/javascript",r.async=!0,r.src="https://cdn.us.heap-api.com/js/heap-"+e+".js";var a=document.getElementsByTagName("script")[0];a.parentNode.insertBefore(r,a);for(var n=function(e){return function(){heap.push([e].concat(Array.prototype.slice.call(arguments,0)))}},p=["addEventProperties","addUserProperties","clearEventProperties","identify","resetIdentity","removeEventProperty","setEventProperties","track","unsetEventProperty"],o=0;o<p.length;o++)heap[o]=n(p[o])};
444-
heap.load("765739241");
445-
</script>
446-
447-
<!-- Google Analytics 4 — *.deepgram.com property (266000242 / 372400484) -->
448-
<script is:inline async src="https://www.googletagmanager.com/gtag/js?id=G-TYPC1TBCKT"></script>
449-
<script is:inline>
450-
window.dataLayer = window.dataLayer || [];
451-
function gtag(){dataLayer.push(arguments);}
452-
gtag('js', new Date());
453-
gtag('config', 'G-TYPC1TBCKT');
454-
</script>
448+
<!-- Heap and Google Analytics 4 (G-TYPC1TBCKT), started only after Ketch grants targeted_advertising -->
449+
<script is:inline set:html={consentTrackersJs}></script>
455450
</head>
456451
<body class="bg-dg-black text-dg-text font-sans overflow-x-hidden" style="font-feature-settings: 'cv02','cv03','cv04','cv11';">
457452

@@ -913,10 +908,14 @@ const commandCards: CommandCard[] = [
913908
</div>
914909
<div class="pt-6 text-[11px] text-dg-pebble flex flex-col sm:flex-row justify-between gap-2" style="border-top: 1px solid #2c2c3330;">
915910
<span>© {new Date().getFullYear()} Deepgram, Inc. All rights reserved.</span>
911+
<!-- dg-consent.js mounts the "Your Privacy Choices" control here (Privacy Policy if Ketch is unavailable). -->
912+
<div id="fern-footer"></div>
916913
</div>
917914
</div>
918915
</footer>
919916

917+
<!-- Cookie consent banner and footer control, vendored from developers.deepgram.com. Loads no tracker. -->
918+
<script is:inline src="/consent/dg-consent.js" defer></script>
920919
</body>
921920
</html>
922921

‎web/src/styles/global.css‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
--color-dg-fog: #edede2;
1313
--color-dg-muted: #949498;
1414

15-
--font-sans: "Inter", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
15+
--font-sans: "Inter Variable", "Inter", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
1616
--font-mono: "Fira Code", Monaco, Consolas, "Courier New", monospace;
1717
}
1818

‎web/tests/build-output.test.cjs‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
/**
2+
* Build-output check: the landing page must not reference Google font hosts,
3+
* because a font request before consent leaks the visitor's IP. Fonts are
4+
* self-hosted via @fontsource. Builds the site into a temp dir (needs
5+
* node_modules, i.e. npm ci first) and scans the output.
6+
*/
7+
const test = require('node:test');
8+
const assert = require('node:assert');
9+
const { execFileSync } = require('node:child_process');
10+
const fs = require('node:fs');
11+
const path = require('node:path');
12+
13+
const web = path.join(__dirname, '..');
14+
const walk = (d) => fs.readdirSync(d, { withFileTypes: true }).flatMap((e) =>
15+
e.isDirectory() ? walk(path.join(d, e.name)) : [path.join(d, e.name)]);
16+
17+
test('built site has no Google font references and ships Inter locally', { timeout: 120000 }, () => {
18+
// Inside web/ (not os.tmpdir) so Astro's rename stays on one filesystem.
19+
const cache = path.join(web, 'node_modules', '.cache');
20+
fs.mkdirSync(cache, { recursive: true });
21+
const out = fs.mkdtempSync(path.join(cache, 'build-output-'));
22+
try {
23+
execFileSync('npx', ['astro', 'build', '--outDir', out], { cwd: web, stdio: 'pipe' });
24+
const all = walk(out);
25+
const text = all.filter((f) => /\.(html|css|js|xml|json|webmanifest)$/.test(f));
26+
assert.ok(text.some((f) => f.endsWith('index.html')));
27+
for (const f of text) {
28+
const body = fs.readFileSync(f, 'utf8');
29+
assert.ok(!/fonts\.googleapis\.com|fonts\.gstatic\.com/.test(body), `Google font host in ${path.relative(out, f)}`);
30+
}
31+
assert.ok(all.some((f) => f.endsWith('.woff2') && /inter/i.test(f)), 'Inter woff2 is bundled');
32+
assert.ok(all.some((f) => f.endsWith('.woff2') && /fira-code/i.test(f)), 'Fira Code woff2 is bundled');
33+
} finally {
34+
fs.rmSync(out, { recursive: true, force: true });
35+
}
36+
});

0 commit comments

Comments
 (0)