Privacy-First Web Analytics Platform (Plausible Alternative)
A cookie-free, open-source web analytics tool matching Plausible's features, sold as a hosted lifetime deal and self-hosted.
13,122 lines · 203,302 words · 21 sections · Nov 3, 2026
Quietstat — Privacy-First Web Analytics Platform: Product Specification #
Quietstat is a cookie-free, open-source web analytics product that matches Plausible Analytics feature for feature. It counts visitors with a daily rotating salted hash instead of cookies, never stores raw IP addresses or User-Agent strings, and shows traffic, conversions, funnels, user journeys and revenue on a single-page dashboard. One codebase ships two editions: a multi-tenant hosted service sold as a one-time lifetime deal in three tiers with fixed site and monthly event caps, and a free self-hosted edition deployed with Docker Compose.
This document is the complete build specification. It is written so an AI coding agent or an engineering team can build the product from it alone, without clarifying questions: every default is decided, each concern has one owning section, and other sections refer to that owner by number. "Quietstat" is a working name controlled by a single setting (Section 16.4). Start with Section 1, then follow the reading order in Section 20.
Table of Contents #
- Before You Start
- Product Overview & Vision
- Technology Stack & Architecture
- Conventions & Cross-Cutting Standards
- Postgres Data Model
- ClickHouse Event Store
- Tracking Script & Integrations
- Ingestion Pipeline & Privacy Engine
- Stats Dashboard & Reports
- Goals, Custom Properties, Funnels, User Journeys & Revenue
- Accounts, Authentication, Teams, Roles & Permissions
- Sharing, Embeds, Email Reports & Notifications
- Public APIs: Stats API, Events API, Sites API, API Keys & Rate Limits
- Data Import & Export
- Lifetime Deal Licensing, Payments, Usage Caps & Operator Console
- Editions, Deployment, Configuration & Operations
- Security, Privacy & Compliance
- Testing & Quality Strategy
- Milestones & Execution Plan
- Executor Instructions
1. Before You Start #
This document is the complete specification for building Quietstat, a privacy-first, cookie-free web analytics product. You are expected to build it from this document alone. Every default, limit, validation rule and error is decided here; there is nothing to ask the owner before you begin.
1.1 How to read this document #
- Section numbers are the addresses. Each concern has exactly one owning section (table below). When two sections appear to disagree, the owning section wins. Cross-references such as "Section 8.3" are binding.
- Normative words. MUST / MUST NOT mean the build fails acceptance if violated. SHOULD means follow it unless a concrete technical blocker exists, and record the deviation in the pull request. MAY means optional.
- Never block on a missing value. Section 1.2 lists every owner-supplied value with a default. If a value is unset at runtime, the product uses the default and behaves as described in the "Behavior when unset" column. You never stop to ask.
- Names are literal. Table names, column names, route paths, environment variable names, error codes, enum values and job names written in
codefont are exact. Do not rename them. - Out-of-scope means do not build. Section 2.5 lists exclusions. Do not add stubs, flags or UI for them.
- Versions are stated only in Section 3.1. Every other section says "per Section 3".
- Order of work. Section 19 gives the milestone order and Section 20 gives the rules for executing. Read Sections 1 to 4 fully before writing code, because they define the vocabulary, the repository layout, the response envelopes and the design system that every other section assumes.
- Examples are normative where they show shapes. A JSON example that appears in a section is the exact wire shape; sample values inside it are illustrative only.
1.1.1 Canonical ownership map #
| Concern | Owning section |
|---|---|
| Product constants, glossary, scope | 2 |
| Dependency versions, repository layout, flows, caching, performance budgets | 3 |
| Naming, IDs, envelopes, error codes, pagination, time, money, number formats, UI design system, SPA route map, accessibility | 4 |
| Postgres tables, columns, indexes, seeds, row-count service limits (5.16) | 5 |
| ClickHouse tables, engines, TTL, migrations | 6 |
| Tracker script, npm package, WordPress plugin, GTM template, setup guides | 7 |
POST /api/event processing, visitor hash, sessions, geo, UA, channels, bot filter, exclusions |
8 |
| Metric definitions, reports, time ranges, comparison, realtime, annotations, filters, segments UI | 9 |
| Goals, custom properties, funnels, journeys, revenue | 10 |
| Users, authentication, 2FA, teams, roles, permission matrix, audit event catalogue (11.20) | 11 |
| Shared links, embeds, email reports, spike notifications, email template inventory (12.8) | 12 |
| Stats API, Events API documentation, Sites API, API keys, query JSON and query engine | 13 |
| CSV export, CSV import, GA4 import | 14 |
| Lifetime-deal tiers, checkout, licenses, codes, caps, invoices, operator console | 15 |
| Environment variables (16.4), Docker Compose, hosted topology, backups, observability, upgrades, background jobs: names, schedules and retries (16.6) | 16 |
| Security controls, privacy compliance, all rate limits (17.6), request body limits and CORS/CSP rules, audit log storage | 17 |
| Test strategy | 18 |
| Milestones | 19 |
| Rules for the executing agent | 20 |
1.2 Owner customization questions and defaults #
Each row is a decision the product owner may want to change. The Default is used when the owner supplies nothing. Environment variable names are canonical in Section 16.4, which lists types, validation and .env.example values; this table lists names only. "Seed" means the value is inserted by a database migration or seed script and can be changed afterwards in the UI named in the row.
| # | Question | Default | Controlled by | Behavior when unset or left at default |
|---|---|---|---|---|
| 1 | What is the product called? | Quietstat |
PRODUCT_NAME |
Used in the UI, emails, page titles, invoices and the From display name. The package scope @quietstat/* and the Docker image name do not change. |
| 2 | Which edition does this deployment run? | selfhosted |
EDITION (hosted or selfhosted) |
selfhosted: no licensing, caps, billing UI or operator console. hosted enables them (Section 15). |
| 3 | What is the public URL of the app (scheme, host, port)? | None: required. The shipped .env.example sets http://localhost:8000 |
BASE_URL |
Used to build links in emails, shared links, the tracker snippet, OAuth redirect URIs and Stripe return URLs. Startup fails if it is unset or not a valid absolute http(s) URL without a path. |
| 4 | Which secret signs sessions and cursors and encrypts stored secrets? | None: required in both editions, at least 64 characters. The shipped .env.example explains how to generate one |
SECRET_KEY_BASE (rotation: SECRET_KEY_BASE_PREVIOUS, Section 17.7.4) |
The process refuses to start without it and names the missing variable in the startup log. All signing, HMAC and encryption keys are derived from it (Section 17.7). |
| 5 | Where are the databases? | Docker Compose service hostnames in the shipped compose file | DATABASE_URL, CLICKHOUSE_URL, VALKEY_URL |
Compose file wires these automatically. Outside Compose they are required; the process exits with a clear message naming the missing variable. |
| 6 | Which SMTP server sends email? | None in self-hosted; required in hosted | SMTP_HOST, SMTP_PORT (default 587), SMTP_USER, SMTP_PASSWORD, SMTP_SECURE (starttls default, tls or none), REQUIRE_EMAIL_VERIFICATION |
selfhosted with no SMTP_HOST: the mailer does not send; it logs one email_not_sent line (template name and recipient domain only) and, when NODE_ENV=development, prints the plain-text part to the console so links can be copied. Email verification is then treated as satisfied at signup (Section 11) in selfhosted only, and email-based features show a "Email is not configured" notice (email reports, invitations by email, spike alerts). Invitation links can still be copied manually. hosted refuses to start without SMTP_HOST (Section 16.4). |
| 7 | What is the sender address? | None; required when SMTP_HOST is set |
MAILER_FROM, MAILER_REPLY_TO (optional), MAILER_FOOTER_ADDRESS (optional postal address line for email footers) |
MAILER_FROM is used for every outgoing email as From, with PRODUCT_NAME as the display name when the value is a bare address. |
| 8 | Which address do users contact for support? | The address part of MAILER_FROM in self-hosted; required in hosted |
SUPPORT_EMAIL |
Shown in error pages, invoices and email footers. |
| 9 | Stripe secret API key (hosted only) | None | STRIPE_SECRET_KEY |
Checkout endpoints return 503 integration_not_configured with details.integration = "stripe"; the activate page shows "Payments are not configured" and still allows code redemption. Prices are created inline with price_data, so no Stripe dashboard products or price IDs are needed. |
| 10 | Stripe webhook signing secret (hosted only) | None | STRIPE_WEBHOOK_SECRET |
The webhook route rejects every request with 400 when unset and the operator console shows a red "Webhook not configured" status. |
| 11 | Collect tax through Stripe Tax? | false |
STRIPE_TAX_ENABLED |
When true, Checkout Sessions enable automatic tax and collect billing address. |
| 12 | Seller details printed on invoices | None in self-hosted (no invoices exist). In hosted: required when Stripe is configured | SELLER_NAME, SELLER_ADDRESS_LINE1, SELLER_ADDRESS_LINE2, SELLER_CITY, SELLER_REGION, SELLER_POSTAL_CODE, SELLER_COUNTRY, SELLER_TAX_ID, SELLER_EMAIL |
Optional fields (SELLER_ADDRESS_LINE2, SELLER_REGION, SELLER_TAX_ID) are omitted from the invoice PDF when blank; SELLER_EMAIL falls back to SUPPORT_EMAIL. Hosted startup with Stripe configured fails if a required seller variable (Section 16.4.8) is missing. |
| 13 | Invoice number prefix | QS |
INVOICE_PREFIX |
Numbers are {prefix}-{YYYY}-{6-digit sequence}, sequence continuous per year (Section 15). |
| 14 | Tier names, prices, site caps and monthly billable-event caps | Solo $59 / 5 sites / 100,000; Pro $149 / 25 / 500,000; Agency $349 / 100 / 2,000,000 | Seed rows in license_tiers (Section 5.12): machine codes solo, pro, agency, prices stored in minor units 5900, 14900, 34900 USD; editable in the operator console (Section 15) |
Changes apply to new purchases and to cap checks immediately; existing licenses reference the tier row, so a cap change affects existing holders of that tier. Prices are USD only. |
| 15 | Refund window | 60 days | REFUND_WINDOW_DAYS |
Refunds after the window are refused in the operator console with 422 refund_window_closed. |
| 16 | Marketplace codes | Format QS-XXXX-XXXX-XXXX (Crockford base32); max 3 stacked per team |
Operator console batch generator (Section 15) | Batch label is free text chosen by the operator at generation time (typically the marketplace name). The format and stacking maximum are constants, not settings. |
| 17 | Cap thresholds | 80% warning, 100% reached, 110% hard stop | Constants in packages/shared |
Not owner-configurable. |
| 18 | GeoIP database | Bundled DB-IP "IP to City Lite" MMDB, refreshed monthly by the geoip-update job |
GEOIP_DB_PATH, GEOIP_AUTO_UPDATE |
GEOIP_DB_PATH unset: the bundled file inside the image is used and GEOIP_AUTO_UPDATE defaults to true (newer monthly files are downloaded to the data directory, Section 16.2.2). Set: any MaxMind-compatible City MMDB (GeoLite2-City or GeoIP2-City) is used instead and GEOIP_AUTO_UPDATE defaults to false. |
| 19 | GeoIP attribution text | IP geolocation by DB-IP when the loaded MMDB is a DB-IP file |
GEOIP_ATTRIBUTION |
Rendered in the dashboard footer. For any other vendor's file the default is empty and the owner SHOULD set the text that vendor's license requires; the app does not otherwise detect the database vendor. |
| 20 | Hosting region label (hosted) | EU (Germany) |
HOSTING_REGION_LABEL |
Display text only (privacy notices in the app, invoices). The actual region is an infrastructure choice described in Section 16.2; the hosted reference topology targets a single EU region. |
| 21 | Who are the super admins? | None in production; admin@example.com in the shipped .env.example for development |
SUPER_ADMIN_EMAILS (comma-separated; hosted edition only, ignored in selfhosted) |
Matching is case-insensitive on the account email. Super-admin status requires a verified email (verified by consuming an email token, never by the no-SMTP shortcut of row 6) and an enrolled two-factor method; without it the console answers 403 admin_2fa_required (Section 15). Empty list: no operator console access; the process logs a warning at start in hosted edition. Removing an address takes effect after the web processes restart, because the list is read at start. |
| 22 | Who may register? | open (hosted default); invite_only (self-hosted default) |
REGISTRATION_MODE (open, invite_only, disabled), SETUP_TOKEN |
There is no first-signup bypass. While no user exists, POST /api/app/v1/auth/signup is refused with 403 setup_required in every mode, and the first account is created only through POST /api/app/v1/setup with the one-time setup token. The token is SETUP_TOKEN when set; otherwise it is generated at first start and printed once to the worker log (Section 16.3.5). The setup insert takes a Postgres advisory lock so two concurrent first requests cannot both succeed. After setup, open allows anyone to sign up, invite_only allows signup only through a valid invitation, and disabled blocks all signup (existing users keep working). Rules: Section 11.2. |
| 23 | How long is event data kept? | Forever | DATA_RETENTION_MONTHS (0 means forever; otherwise 1 to 240) |
When greater than 0, the migrate command applies a ClickHouse TTL in months to the events and sessions tables; the imported_* tables are exempt (Section 6.6). Unlicensed teams in the hosted edition are purged after 30 days regardless (Section 15). Audit-log rows are governed by AUDIT_RETENTION_MONTHS (row 35). |
| 24 | Default site time zone | UTC |
Constant; the add-site form pre-selects the browser's zone | Sections 4.6 and 5. |
| 25 | Which reverse proxies are trusted for client IP? | None | TRUSTED_PROXIES (comma-separated CIDRs) |
Unset: the socket peer address is the client IP and X-Forwarded-For is ignored (Section 8). The shipped compose file sets it to the private ranges used by its bundled proxy. |
| 26 | Google OAuth client for GA4 import | None | GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET |
The GA4 import option is shown disabled with the message "Google import is not configured" and CSV import still works (Section 14). |
| 27 | Legal page links | None; all three are required in hosted | TERMS_URL, PRIVACY_URL, DPA_URL |
Unset (self-hosted): the footer, signup form and invoices omit the links. Writing legal copy is out of scope. |
| 28 | Where are uploads, generated exports and invoice PDFs stored? | /data inside the container (a named volume in compose) |
DATA_DIR |
Created at start if missing. |
| 29 | Log verbosity | info |
LOG_LEVEL |
Allowed: fatal, error, warn, info, debug, trace. |
| 30 | Datacenter-IP filtering for new sites | On | site_settings.datacenter_filter_enabled, default true (Section 5.5.2); per-site toggle (Section 8) |
The bot user-agent filter cannot be disabled. A per-site hostname allowlist (site_settings.allowed_hostnames, default empty = the site domain and all its subdomains) drops events sent for other hostnames (Section 8). |
| 31 | Logo and brand color | Text wordmark PRODUCT_NAME; brand color #4338CA |
Replace apps/web/public/brand/logo.svg and edit the --color-brand token (Section 4.14) |
Absent logo file falls back to the text wordmark. |
| 32 | Telemetry | None, in both editions | Not configurable | The product never phones home. |
| 33 | Interface language | English (en-US) |
Not configurable in v1 | Strings are externalized (Section 4.11) so a later release can add locales. |
| 34 | Security contact published at /.well-known/security.txt |
None; required in hosted | SECURITY_CONTACT (a mailto: or https: URI) |
Unset: /.well-known/security.txt returns 404 (Section 16.4). |
| 35 | How long are audit-log rows kept? | 24 months (hosted); forever (self-hosted) | AUDIT_RETENTION_MONTHS (6 to 120; hosted only) |
The audit-prune job deletes older rows in the hosted edition; the self-hosted edition keeps audit rows indefinitely. |
| 36 | Must email addresses be verified before sites can be created? | true in hosted; in self-hosted true when SMTP_HOST is set, else false |
REQUIRE_EMAIL_VERIFICATION |
When false, accounts are marked verified at signup (row 6). Super-admin accounts always require genuine verification (row 21). |
| 37 | Are new passwords checked against known breaches? | local: a bundled list of the most common breached passwords |
PASSWORD_BREACH_CHECK (local or hibp) |
hibp additionally queries the k-anonymity range API of Have I Been Pwned (only the first five characters of the SHA-1 hash leave the server); it is opt-in because it is an outbound call (Section 11). |
1.3 Decisions log #
These are the major decisions already made. Do not re-open them while building. "Where" names the section that specifies the detail.
| ID | Decision | Rationale | Where |
|---|---|---|---|
| D-01 | One TypeScript monorepo (pnpm workspaces) containing server, worker, SPA, shared, tracker, query engine and integrations | Single toolchain, shared Zod schemas, atomic changes across API and UI | 3.2 |
| D-02 | Fastify server serves the dashboard API, public APIs, ingestion and the SPA build | One deployable web process; Fastify has the best throughput for the ingestion hot path | 3.1, 3.4 |
| D-03 | PostgreSQL for accounts and settings; ClickHouse for events | Relational integrity for settings; columnar scans for analytics at low cost | 5, 6 |
| D-04 | Valkey (Redis protocol, BSD license) for sessions cache, rate limits, realtime counters, ingestion locks | Fully open license, drop-in protocol, no new paradigm | 3.6 |
| D-05 | pg-boss for background jobs (Postgres-backed) | No extra queue service to run or secure | 3.4, 16 |
| D-06 | One Docker image, three commands: web, worker, migrate |
Identical artifact for both editions and all environments | 3.4, 16 |
| D-07 | Same codebase for both editions, switched by EDITION |
One test matrix; hosted features are additive modules | 4.10, 16 |
| D-08 | Visitor identity is a daily rotating salted SipHash of site, IP and User-Agent; no cookies, no local storage, no fingerprints beyond that hash | Cookie-free, no consent banner, no cross-day or cross-site tracking | 8 |
| D-09 | Two salts overlap for 30 minutes after midnight UTC so live sessions are not split | Avoids a visitor spike at midnight | 8 |
| D-10 | Session ends after 30 minutes of inactivity | Industry-standard definition | 8, 9 |
| D-11 | Raw IP and User-Agent are never stored or logged | Core privacy promise | 8, 17 |
| D-12 | Location is stored only as country, region and city identifier | Matches the privacy promise; no coordinates | 8 |
| D-13 | Known-bot, headless, spam-referrer and datacenter-IP filtering is on by default; only the datacenter filter can be turned off per site | Keeps numbers clean without silent over-filtering of real users | 8 |
| D-14 | Bundled GeoIP is DB-IP Lite (CC BY 4.0); operators may substitute a MaxMind MMDB | Freely redistributable in the Docker image | 8, 17 |
| D-15 | Three lifetime-deal tiers (Solo, Pro, Agency) that differ only in site count and monthly billable-event cap | Simple to explain; no feature gating | 15 |
| D-16 | "Lifetime" means the life of the hosted product, with no recurring fees | Honest definition stated in the UI and terms-facing copy | 15 |
| D-17 | Billable events are accepted pageviews and custom events; engagement events are not billable |
Engagement pings would inflate usage unfairly | 8, 15 |
| D-18 | Warn at 80% and 100% of the monthly cap; drop events beyond 110% until the next UTC month or an upgrade | A 10% grace avoids losing data at the boundary | 15 |
| D-19 | Stripe Checkout in one-time payment mode; prices built inline; marketplace codes redeemable and stackable up to 3 |
No subscription objects to manage; supports both direct and marketplace sales | 15 |
| D-20 | Upgrades pay the price difference; downgrades are not possible | Matches lifetime semantics | 15 |
| D-21 | Hosted-only: licensing, Stripe, caps and operator console; self-hosted has none of them and no limits | Self-hosting is free and unrestricted | 15, 16 |
| D-22 | JSON field names are snake_case; TypeScript code is camelCase | Matches Plausible-style APIs and ClickHouse columns | 4.1 |
| D-23 | Postgres primary keys are uuid defaulting to uuidv7(); public site identifier in APIs is the site domain |
Time-ordered keys; ergonomic API addressing | 4.2 |
| D-24 | All timestamps stored in UTC; all date bucketing uses the site's IANA time zone; usage months use UTC | Predictable storage, user-correct reporting, unambiguous billing | 4.6 |
| D-25 | Money for billing is integer minor units plus an ISO 4217 code; event revenue in ClickHouse is Decimal(18,3) in major units plus a currency code (Section 6.3) |
No floating-point error; tracked revenue keeps sub-cent precision | 4.7, 6.3 |
| D-26 | Zod schemas in @quietstat/shared validate on both client and server |
One source of truth for shapes | 4.8 |
| D-27 | Cursor pagination for management lists; limit/offset in the Stats API body | Stable lists; analytic queries need offsets | 4.5, 13 |
| D-28 | Hard delete for sites, goals, funnels, segments, annotations, shared links, API keys, invitations; soft delete for users and teams; licenses are never deleted | Minimizes retained data while protecting accounts and billing records | 4.4, 11, 15 |
| D-29 | argon2id for passwords; optional TOTP two-factor with recovery codes | Current best practice without SSO | 11 |
| D-30 | English-only UI with externalized strings | Smallest v1 surface that does not block later locales | 4.11 |
| D-31 | Licensing of components: core application AGPL-3.0-only; tracker script and @quietstat/tracker npm package MIT; WordPress plugin GPL-2.0-or-later; Google Tag Manager template Apache-2.0 |
Keeps the service open while letting the tracker be embedded anywhere; the plugin and GTM licenses follow their directories' requirements | 3.1 |
| D-32 | No SSO, no recurring billing, no Search Console, Looker Studio or Slack integrations, no mobile SDKs | Explicitly out of scope | 2.5 |
| D-33 | The hosted edition is deployed in the EU by default | Matches the privacy positioning; the label is configurable | 16 |
| D-34 | The authenticated dashboard stores UI preferences only in the browser (localStorage or sessionStorage under qs-prefixed keys: theme, panel tabs, remembered period and comparison, dismissed banners, sort choices, last-used setup guide). Shared and embedded dashboards read and write no browser storage and set no cookies (theme through the theme query parameter). The tracker writes nothing; it only reads a site-owner opt-out flag |
The cookie-free promise applies to tracked visitors; third-party pages that embed a dashboard must not receive storage writes | 4.14, 7, 12 |
| D-35 | The salts table is a Postgres UNLOGGED table (excluded from WAL, point-in-time recovery and replicas) and is excluded from logical dumps (pg_dump --exclude-table-data=salts). After a crash or restore it is empty and a fresh salt is generated, so visitors may be double-counted for that day |
A backup must never contain a salt, which would let a day's visitor hashes be confirmed by brute force | 5, 8, 16, 17 |
| D-36 | A fresh self-hosted instance has no first-signup bypass: the first account is created only through POST /api/app/v1/setup with a one-time setup token |
Prevents whoever reaches a new public instance first from owning it | 11, 16 |
| D-37 | A site's domain is stored lowercase, without scheme, port, path or a leading www.; ingestion strips a leading www. and any port from d before lookup; uniqueness is installation-wide on the normalized domain |
example.com and www.example.com are one site and local development uses localhost |
5, 8 |
2. Product Overview & Vision #
2.1 Problem #
Mainstream web analytics tools set cookies, collect personal data, require consent banners, ship heavy scripts and present hundreds of reports that most site owners never read. Cookie-free alternatives exist, but they are subscription products: owners pay every month for as long as they run a site, and agencies multiply that cost across clients.
Quietstat is a privacy-first web analytics product that:
- measures traffic without cookies, without writing to browser storage (the tracker only reads a site-owner opt-out flag), without fingerprinting and without storing personal data by design, so no consent banner is needed for it. Page paths and referrer paths are stored as sent (query strings removed), so integrators must keep personal data out of URL paths or scrub it with the tracker's
transformRequesthook (Section 17.9.2); - shows everything on one simple dashboard that loads in under a second;
- matches the feature set of the leading cookie-free analytics product (parity table in 2.6);
- is sold on the hosted edition as a one-time lifetime deal in three tiers, with no recurring fees, and is also available as a free, unrestricted, self-hosted edition from the same codebase.
2.2 Target users #
| Persona | Who they are | Typical scale | Main needs |
|---|---|---|---|
| Maya, the site owner | Blogger, small-business owner or publisher | 1 to 5 sites, 1,000 to 100,000 pageviews per month | A glance-able dashboard, no cookie banner, weekly email report, simple goals |
| Diego, the agency owner | Marketer or agency running analytics for clients | 10 to 100 sites, several team members, clients who want read access | Team roles, per-site guest access, shared links, scheduled email reports, an all-sites overview |
| Priya, the developer | Developer or small team that self-hosts | Own infrastructure, 1 to 50 sites | Docker Compose install, Stats API and Events API, imports, no limits, easy upgrades |
| Sam, the store marketer | Ecommerce marketer | 1 to 3 shops | Revenue goals, funnels, revenue by source, campaign (UTM) reports |
| Olivia, the operator | Person who runs the hosted edition and sells the lifetime deal | Hundreds to thousands of accounts | Sell tiers, generate marketplace codes, see usage, refund purchases, impersonate read-only for support |
2.2.1 Jobs to be done #
| Persona | When | I want to | So that | Primary sections |
|---|---|---|---|---|
| Maya | I publish a post | see how many people read it and where they came from | I know what to write next | 9 |
| Maya | I install analytics | add one snippet (or a WordPress plugin) and see data within minutes | I do not need a developer | 7 |
| Maya | I get a weekly email | read the headline numbers without logging in | I stay informed with no effort | 12 |
| Diego | I onboard a new client | create the site, install the snippet and invite the client as a read-only guest | the client sees their own data only | 7, 11 |
| Diego | A client asks for a report | send a password-protected link limited to one segment | I share exactly what I intend | 12 |
| Diego | Monday morning | scan every site on one screen | I spot sites with traffic drops | 9 |
| Priya | I deploy | run docker compose up and have a working instance |
I own my data | 16 |
| Priya | I integrate | query stats programmatically and send server-side events | dashboards and apps show analytics | 13 |
| Priya | I migrate | import Google Analytics 4 history or a CSV | I do not lose history | 14 |
| Sam | I run a campaign | see conversions and revenue by UTM campaign | I put budget where it pays | 9, 10 |
| Sam | The checkout loses buyers | build a funnel and compare periods | I find the weak step | 10 |
| Olivia | A customer buys | have Stripe create the license automatically and email an invoice | no manual work | 15 |
| Olivia | A marketplace sells codes | generate a batch and export a CSV | the marketplace can distribute it | 15 |
| Olivia | A customer exceeds their cap | see usage and have the product warn and then stop ingestion safely | costs and fairness are controlled | 15 |
2.3 Editions #
One codebase builds both editions. The edition is chosen at runtime by EDITION (Section 1.2, row 2).
| Aspect | Hosted edition (EDITION=hosted) |
Self-hosted edition (EDITION=selfhosted) |
|---|---|---|
| Operator | The product owner, multi-tenant | The customer, usually single tenant |
| Price to the user | One-time lifetime deal (Solo, Pro, Agency) | Free |
| License of the software | AGPL-3.0-only (the code is the same) | AGPL-3.0-only |
| Sites | Capped by tier (5, 25, 100); unlicensed accounts: 1 site | Unlimited |
| Events | Capped by tier per UTC month (100,000, 500,000, 2,000,000 billable); hard stop at 110% | Unlimited |
| Licensing, checkout, codes, invoices, usage meter | Yes (Section 15) | None; the UI and routes do not exist |
Operator console /admin |
Yes, for SUPER_ADMIN_EMAILS |
No |
| Feature set | All features in every tier | All features |
| Registration | Open | invite_only by default; configurable |
| Operator-provided SMTP | Optional SMTP; degraded behavior described in Section 1.2 row 6 | |
| Stripe | Required for direct sales | Not used |
| Hosting location | EU by default | Wherever the owner runs it |
| Data retention | Forever; unlicensed teams purged after 30 days | Owner-configurable (DATA_RETENTION_MONTHS) |
| Updates | Operator deploys | Owner pulls the new image and runs migrate (Section 16) |
| Telemetry | None | None |
| Support | Operator's support address | Community issue tracker |
There is no feature gating: every feature is available in every tier and in both editions. Tiers differ only in site count and monthly billable-event cap.
2.4 Core features #
| # | Feature | Summary | Owning sections |
|---|---|---|---|
| 1 | Cookie-free tracking script | One small script records pageviews, SPA and hash-route navigations, outbound link clicks, file downloads, form submissions, 404 pages, scroll depth and engagement time | 7, 8 |
| 2 | Privacy-preserving visitor counting | Daily rotating salted hash of site, IP and User-Agent; raw IP and UA never stored; location limited to country, region, city | 8, 17 |
| 3 | Single-page stats dashboard | Visitors, pageviews, visits, bounce rate and visit duration; time-series chart; period comparison; real-time view; annotations; reports for sources, channels, UTM campaigns, pages, locations, devices and AI assistant traffic | 9 |
| 4 | Filters and saved segments | Combinable filters on source, page, location, device, goal and custom property; savable as segments | 9 |
| 5 | Goals, custom events and properties | Custom event, pageview and scroll-depth goals; custom properties on events and pageviews; goals narrowed by up to 3 properties; revenue values with currency | 10, 7 |
| 6 | Funnels and user journeys | Ordered, optional-step and strict-order funnels with period comparison and revenue per step; visual map of visitor paths | 10 |
| 7 | Ecommerce revenue reporting | Multi-currency revenue goals; total revenue, average order revenue and revenue by source | 10, 9 |
| 8 | Accounts, teams, roles | Email and password login, optional TOTP two-factor, multiple sites per account, teams with owner, admin, editor and viewer roles, invitations, per-site guests | 11 |
| 9 | Sharing, embeds, email reports | Shared dashboard links with optional password or fixed-segment limit; embeddable dashboards; scheduled email reports; traffic spike notifications | 12 |
| 10 | Stats API and Events API | Bearer-key query API with aggregate, breakdown and time-series results; unauthenticated event ingestion for server-side and app events; Sites API | 13 |
| 11 | Data import and export | CSV export of any report, CSV import, Google Analytics (GA4) import | 14 |
| 12 | Platform integrations | WordPress plugin, Google Tag Manager template, npm package, plain script tag, proxy endpoint option, setup guides for common frameworks | 7 |
| 13 | Traffic quality | Bundled MaxMind-compatible GeoIP database; default-on known-bot and datacenter-IP filter; page and IP exclusions | 8 |
| 14 | Lifetime-deal purchase and licensing | One-time payment tiers with fixed site and monthly billable-event caps, checkout, license code redemption, usage meter, cap warnings, invoices and tier upgrades | 15 |
| 15 | Hosted and self-hosted editions | One codebase; multi-tenant hosted with lifetime-deal accounts; free Docker Compose self-hosted edition | 16, 3 |
2.5 Out of scope #
The following MUST NOT be built, stubbed or hinted at in the UI:
- Recurring subscriptions, monthly or annual billing, and free trials.
- Google Search Console integration.
- Looker Studio connector.
- Slack reports (email is the only report channel).
- Single sign-on of any kind (Google Workspace, Microsoft Entra, Okta, SAML, OIDC).
- Native mobile SDKs for iOS and Android. Apps send events through the Events API (Section 13).
- Feature gating between tiers. No code path may branch on the tier except for the site and event limits.
- Marketing website, blog and documentation-site copy. In-app UI text and the framework setup guides inside the app are in scope.
- Nonprofit and education discounts.
- Session replay, heatmaps and A/B testing.
- Data-residency selection settings. The hosted edition runs in one region (Section 1.2, row 20).
- Cross-device, cross-day or cross-site visitor identification of any kind.
- An interactive in-app API playground. Section 13 provides copyable request examples instead.
- Legal copy: terms of service, privacy policy and data-processing-agreement text. The product only links to URLs supplied by the owner.
2.6 Plausible parity table #
Each capability of the reference product is mapped to the section that specifies it, or marked as excluded.
| Reference capability | Quietstat | Section |
|---|---|---|
| Lightweight init-based snippet; no cookies, no browser-storage writes (opt-out flag read only) or persistent identifiers | Included | 7, 8 |
| Automatic pageviews and manual pageviews with optional custom URL | Included | 7 |
| Single-page-app support and hash-based routing | Included | 7 |
| Outbound link, file download, form submission and 404 tracking | Included | 7 |
| Custom events from code, link clicks and element clicks | Included | 7, 10 |
| Custom properties on events and pageviews (global or per event) | Included | 7, 10 |
| Revenue values with currency on events | Included | 7, 10 |
| Scroll depth and engagement time | Included | 7, 8, 9 |
| UTM and referral parameter capture; other query parameters discarded | Included | 8 |
| Localhost capture toggle | Included | 7 |
transformRequest hook |
Included | 7 |
Proxy endpoint option for ad-blocker resilience |
Included | 7, 16 |
| Script logging toggle | Included | 7 |
| Script tag, npm package, WordPress plugin, Google Tag Manager template, framework guides | Included | 7 |
| Events API (server-side events) | Included | 13 |
| Daily salted visitor hash, no cross-day tracking, raw IP and UA never stored | Included | 8, 17 |
| Location only to country, region, city; versions to major.minor | Included | 8 |
| Permanent deletion of account and site data | Included | 5, 11, 17 |
| EU hosting, open source, self-hostable | Included | 16 |
| Data processing agreement and compliance pages | Excluded by scope (legal and marketing copy) | 2.5 |
| Single-page dashboard with time series and five headline metrics | Included | 9 |
| Real-time dashboard | Included | 9 |
| Time ranges including Last 24 Hours, keyboard shortcuts, period comparison | Included | 9 |
| Reports: sources, channels, campaigns, pages, entry pages, exit pages, countries, regions, cities, browsers, operating systems, devices, screen sizes | Included | 9 |
| AI Assistants channel | Included | 8, 9 |
| Combinable filters | Included | 9 |
| Saved segments | Included | 9 |
| Annotations (personal or shared) | Included | 9 |
| Goal and conversion columns; revenue columns | Included | 9, 10 |
| Consolidated all-sites view; sort sites by traffic or domain | Included | 9 |
| Traffic spike notifications | Included | 12 |
| Page and IP exclusions | Included | 8 |
| Custom event, pageview and scroll-depth goals; goals narrowed by up to 3 properties | Included | 10 |
| Funnels: ordered, optional steps, strict order, period comparison, revenue per step | Included | 10 |
| User journeys | Included | 10 |
| Ecommerce revenue: multi-currency, total and average, revenue by source | Included | 10 |
| Shared links, password protection, segment-limited sharing | Included | 12 |
| Embeddable dashboards | Included | 12 |
| Scheduled email reports | Included | 12 |
| Slack reports | Excluded by scope | 2.5 |
| Team roles, ownership transfer, agency and freelancer access | Included (team roles, transfer, per-site guests) | 11 |
| Team API keys | Included | 13 |
| SSO (Google Workspace, Entra, Okta) | Excluded by scope | 2.5 |
| Two-factor authentication; team-level 2FA enforcement | Included | 11 |
| Account settings: email change, login management, appearance, password reset, account deletion | Included | 11, 4.14 |
| Google Search Console integration | Excluded by scope | 2.5 |
| Looker Studio connector | Excluded by scope | 2.5 |
| Stats API (aggregate, breakdown, time series) | Included | 13 |
| API playground | Excluded by scope (copyable examples instead) | 2.5, 13 |
| Sites API | Included | 13 |
| Import from Google Analytics and CSV; CSV export | Included | 14 |
| Free trial | Excluded by scope | 2.5 |
| Plans by monthly pageview volume and plan changes | Replaced by lifetime tiers with caps and price-difference upgrades | 15 |
| Invoices | Included | 15 |
| Nonprofit and education pricing | Excluded by scope | 2.5 |
| Security, procurement, status and changelog pages | Excluded by scope (marketing copy); security controls are specified in 17 | 2.5, 17 |
2.7 Success criteria #
The product is accepted when all of the following measurements are met on the reference environment defined in Section 3.8 (4 vCPU, 16 GB RAM, NVMe storage for the single node running all services).
| Area | Metric | Target |
|---|---|---|
| Tracker size | Core tracker (core.js), gzip level 9 |
At most 1,500 bytes, with the per-module budgets and the generic script.js budget of 4,800 bytes defined in Section 7.8.1. Section 7 enforces them in CI. |
| Tracker cost | Main-thread work on page load | Under 5 ms on a mid-range mobile profile; no network request except the event beacon |
| Ingestion throughput | Sustained POST /api/event rate on one web process |
At least 2,000 events per second with p99 response time under 25 ms |
| Ingestion burst | Burst absorbed without data loss while ClickHouse is healthy | 10,000 events per second for 30 seconds; buffer flush within 5 seconds after burst |
| Ingestion durability | Events lost on graceful shutdown | Zero (the buffer is flushed on SIGTERM). On abrupt crash every row still in the in-process buffer is lost: normally at most one flush interval (2 seconds), and up to the 100,000-row buffer bound during a ClickHouse outage (Section 8.15) |
| ClickHouse outage tolerance | Events lost while ClickHouse is unreachable | None for an outage of up to 45 seconds at 2,000 events per second per web process; beyond the 100,000-row buffer bound new events are dropped with a 202 response and reason buffer_full, and a warning is logged (Section 8.15.3) |
| Dashboard latency | Time to render all top-level dashboard widgets for a 30-day range, server-side, on a site with 50 million events in the period | p95 under 1.2 s |
| Single report latency | One breakdown query (any report) over a 12-month range, 100 million events in the table | p95 under 500 ms |
| Real-time view | Time to refresh | p95 under 300 ms |
| Management API latency | Any /api/app/v1 call that does not query ClickHouse |
p95 under 150 ms |
| SPA weight | Initial JavaScript for the dashboard route, gzip | At most 250 KB; each additional route chunk at most 120 KB |
| SPA speed | Largest Contentful Paint of the dashboard on a cold cache over a throttled fast 4G profile | Under 2.0 s |
| Accuracy | Visitor and pageview counts compared with the same synthetic traffic fed through a reference script | Exactly equal for deterministic test traffic |
| Privacy | Raw IP or User-Agent present in any database row, log line or file | Zero occurrences (verified by Section 18 tests) |
| Parity | Rows marked "Included" in 2.6 | 100% implemented and covered by at least one automated test |
| Accessibility | Automated axe checks and manual keyboard pass on every route in 4.13 | Zero serious or critical violations; WCAG 2.2 AA |
| Install time (self-hosted) | From git clone to a working dashboard with a first pageview |
Under 10 minutes following the README |
| Recovery | Time to restore from the nightly backup on the reference environment | Under 30 minutes for a 50 GB dataset |
2.8 Glossary #
| Term | Definition |
|---|---|
| Site | One tracked website, identified publicly by its domain and internally by a UUID. Belongs to exactly one team. |
| Team | The owning unit of sites, members, API keys and (hosted) a license. A user may belong to many teams. |
| Visitor | A unique user_id hash within the reporting period for one site. The same person on two days counts as two visitors; the same person on two devices counts as two visitors. |
user_id |
The 64-bit daily hash described in Section 8. It is not a persistent identifier and never leaves the server. |
| Visit / session | A series of events from one user_id on one site with no gap of 30 minutes or more. "Visit" is the dashboard term; "session" is the storage term. They mean the same thing. |
| Pageview | An event named pageview: one load or one SPA/hash navigation to a URL. |
| Event | Any record sent to the ingestion endpoint: a pageview, an engagement event or a custom event. |
| Custom event | An event whose name is not pageview or engagement, such as signup or Outbound Link: Click. |
| Engagement event | A non-interactive event named engagement carrying scroll depth and engagement time. It is not billable and does not count as an interaction. |
| Interactive event | An event that counts as user interaction for bounce purposes. All custom events are interactive by default (i defaults to true); engagement is not. |
| Property (custom property) | A key and string value attached to an event or pageview. Up to 30 per event. Only keys on a site's allowlist are reported (Section 10). |
| Goal | A saved definition of a conversion: a custom event name, a pageview path pattern or a scroll depth threshold on a path, optionally narrowed by up to 3 property conditions. |
| Conversion | A visitor who completed a goal at least once in the selected period and filters. |
| Conversion rate | Converting visitors divided by total visitors in the same scope. |
| Group conversion rate | In a breakdown, the share of visitors in the group who converted. |
| Revenue | Money attached to an event in the $ field, stored as an exact decimal (Decimal(18,3), major units) with a currency code (Section 6.3). |
| Funnel | An ordered list of 2 to 8 steps (pages or events) and the share of visitors who reach each step. |
| User journey | A diagram of the paths visitors take through pages and events, drawn as a Sankey. |
| Bounce | A visit with exactly one pageview and no interactive custom event. |
| Bounce rate | Bounced visits divided by visits. |
| Visit duration | Time from the first to the last event of a visit, in seconds. |
| Time on page | Engagement time recorded for a page (Section 9). |
| Scroll depth | Maximum percent of the page scrolled, 0 to 100. |
| Source | The referring site or tagged origin of a visit, such as google.com or a utm_source value. |
| Channel | The classification of a visit's source into one of the 18 channels in Section 8, including AI Assistants. |
| UTM parameters | utm_source, utm_medium, utm_campaign, utm_term, utm_content; stored in dedicated columns. |
| Entry page / exit page | The first or last pageview of a visit. |
| Filter | A condition on a dimension that narrows every report on the dashboard. |
| Segment | A saved, named set of filters, personal or site-wide. |
| Annotation | A dated note shown on the time-series chart. |
| Period comparison | Showing the selected period beside the previous period or the same period last year. |
| Realtime | The live view covering the last 30 minutes of activity in minute buckets (Section 9.12). |
| Current visitors | Distinct visitors with activity in the last 5 minutes (Section 9.14). |
| Shared link | A tokenized URL giving read-only dashboard access, optionally password-protected or restricted to a segment. |
| Embed | A shared link rendered inside an iframe on another site. |
| Spike notification | An email sent when real-time visitors cross an owner-set threshold. |
| Billable event | An accepted pageview or custom event counted against the monthly cap. Engagement events are excluded. |
| Cap | The monthly billable-event allowance of a license tier. |
| Grace | The 10% above the cap (cap to 110%) during which events are still recorded. |
| License | The team-level record that grants a tier. One active license per team. |
| Tier | One of Solo, Pro, Agency. |
| Code | A marketplace-redeemable string QS-XXXX-XXXX-XXXX. Redeemed codes stack: 1 = Solo, 2 = Pro, 3 = Agency. |
| Unlicensed | A hosted team with no active license (never purchased, or revoked). Events are dropped. |
| Daily salt | A random 16-byte value used for one UTC day to hash visitors; deleted after use. It lives only in an UNLOGGED Postgres table that is excluded from backups (Section 5.9.3). |
| Tracker | The browser script and its npm package. |
| Ingestion | Receiving, validating, enriching and storing an event (Section 8). |
| Proxy endpoint | A route on the customer's own domain that forwards tracker traffic to the app, reducing ad-blocker interference. |
| Imported data | Aggregated historical data from CSV or GA4 stored in the imported_* tables and merged into reports only for dates strictly earlier than the site's first native day (Section 13.4.10). |
| Stats API | The authenticated query API at POST /api/v2/query. |
| Events API | The unauthenticated ingestion endpoint POST /api/event, usable server-to-server. |
| API key | A team-scoped Bearer token for the Stats API and Sites API. |
| Guest | A user granted guest_viewer or guest_editor access to a single site without team membership. |
| Super admin | A hosted-edition operator whose email is in SUPER_ADMIN_EMAILS. |
| Operator | The person or company that runs a deployment. |
| Edition | hosted or selfhosted, selected by EDITION. |
| Request ID | The req_... identifier attached to every API response and log line. |
3. Technology Stack & Architecture #
3.1 Version table #
This table is the only place in the document that states dependency versions. Every other section says "per Section 3".
These version lines are a known-good floor, not a lockfile. At build time, install the current stable release of each dependency (npm install <pkg>@latest, or your ecosystem's equivalent), confirm the major line still matches, and let the lockfile record the exact resolved versions.
| Layer | Component | Version line | Purpose |
|---|---|---|---|
| Runtime | Node.js | 26.x LTS | Server, worker and tooling runtime |
| Language | TypeScript | 7.x (strict) |
All first-party code except the WordPress plugin |
| Package manager | pnpm | 12.x (workspaces) | Monorepo installs and scripts |
| HTTP server | Fastify | 5.x | API, ingestion, static serving |
| @fastify/cookie | 11.x | Session cookie parsing and signing | |
| @fastify/rate-limit | 11.x | Rate limiting backed by Valkey | |
| SPA | React | 19.x | Dashboard UI |
| Vite | 8.x | SPA build and dev server | |
| React Router | 8.x | Client routing | |
| TanStack Query | 5.x | Server-state caching | |
| Tailwind CSS | 4.x | Styling via design tokens | |
| Recharts | 3.x | Time-series, bar and funnel charts | |
| d3-sankey | 0.x | User journeys diagram | |
| Relational DB | PostgreSQL | 18.x | Accounts, settings, licenses, jobs |
| Drizzle ORM | 0.x (pre-1.0 line) | Typed queries; schema definitions live in @quietstat/shared |
|
| drizzle-kit | 0.x | Postgres migration generation | |
| Analytics DB | ClickHouse | 26.8 LTS line | Events and sessions |
| @clickhouse/client | 1.x | ClickHouse access | |
| Cache and counters | Valkey | 9.x | Session cache, rate limits, realtime counters, locks |
| iovalkey | 0.x | Valkey client | |
| Jobs | pg-boss | 12.x | Background jobs and cron |
| Validation | Zod | 4.x | Shared schemas in @quietstat/shared |
| Auth | argon2 (npm) | 0.x (argon2id) | Password hashing |
| otplib | 13.x | TOTP two-factor | |
| Nodemailer | 10.x | SMTP delivery | |
| Logging | pino | 10.x | JSON logs |
| Traffic parsing | ua-parser-js | 2.x | User-Agent parsing (AGPL-3.0, compatible with the core license) |
| isbot | 5.x | Bot user-agent detection | |
| GeoIP | maxmind (npm) | 5.x | MMDB reader |
| DB-IP "IP to City Lite" | monthly release | Bundled MMDB (CC BY 4.0); refreshed at runtime by geoip-update (Section 16.2.2) |
|
| Payments (hosted) | stripe (npm) | 23.x | Checkout, webhooks, refunds |
| Invoices (hosted) | pdf-lib | 1.x | Pure-JavaScript invoice and credit-note PDF generation |
| Tests | Vitest | 5.x | Unit and integration tests |
| Playwright | 1.x | End-to-end and accessibility tests | |
| Tracker build | esbuild | 0.x | Bundles and minifies the tracker |
| Lint and format | ESLint (flat config), typescript-eslint, Prettier | Current stable major at build time; record in the lockfile | Section 4.12 |
| WordPress plugin | PHP | 8.x (8.1 is the minimum supported); tested up to the current WordPress 7.x | integrations/wordpress |
Component licenses:
| Component | License |
|---|---|
| Core application (everything outside the rows below) | AGPL-3.0-only |
Tracker script and the @quietstat/tracker npm package |
MIT |
| WordPress plugin | GPL-2.0-or-later |
| Google Tag Manager template | Apache-2.0 (a Tag Manager gallery requirement) |
Each package directory contains its own LICENSE file with the matching text; the repository root contains the AGPL text and a LICENSING.md stating this table.
3.1.1 Why each choice #
| Choice | Reason |
|---|---|
| Node.js + TypeScript everywhere | One language across server, worker, SPA, tracker and query engine lets Zod schemas and types be shared with no code generation |
| pnpm workspaces | Strict dependency isolation, fast installs, workspace protocol for internal packages |
| Fastify | Highest throughput of the mainstream Node frameworks; schema-first; first-class logging with pino; encapsulated plugins keep route families separate |
| React + Vite + React Router + TanStack Query | Mainstream, well understood by AI and human developers; TanStack Query gives caching, retries and refetch intervals needed by dashboards and the realtime view |
| Tailwind CSS 4 | Design tokens map to CSS variables, so light and dark themes switch without rebuilding styles |
| Recharts + d3-sankey | Recharts covers line, area and bar charts with accessible markup; d3-sankey is the standard layout engine for journeys |
| PostgreSQL | Transactional integrity for accounts, memberships, licenses and purchases; native uuidv7() gives time-ordered primary keys without extensions |
| Drizzle ORM | Typed SQL close to the metal; migrations are plain SQL files that can be reviewed |
| ClickHouse | Column store built for event analytics; compression makes retention cheap; VersionedCollapsingMergeTree supports updating session rows without mutations |
| In-repo ClickHouse migration runner | No mature ORM-level migrator is needed; plain .sql files plus a schema_migrations table are auditable and deterministic |
| Valkey | Redis-protocol compatibility with a permissive license; used for volatile state only, so a flush never loses durable data |
| pg-boss | Jobs, retries, cron and singleton keys on the database already required; one less service to operate |
| Zod | Runtime validation with inferred static types; the same schema validates forms and requests |
| argon2id | Memory-hard password hashing recommended by current guidance |
| otplib | Maintained TOTP implementation; no vendor lock-in |
| Nodemailer | De facto SMTP library; works with any provider; templates remain plain TypeScript functions so there is no template-engine dependency |
| pino | Fast structured JSON logging with built-in redaction, required to guarantee that IP and User-Agent never reach logs |
| ua-parser-js + isbot + datacenter CIDR list | Pure in-process classification; no external lookup service, so no personal data leaves the server |
DB-IP Lite via maxmind |
Redistributable inside the Docker image; the MMDB format lets operators swap in a MaxMind database with one variable |
| Stripe Checkout, one-time mode | Hosted payment page, tax and receipts handled by Stripe; no PCI scope; no subscription lifecycle to model |
| Vitest + Playwright | Fast unit and integration tests plus real-browser end-to-end and accessibility checks |
| esbuild for the tracker | Deterministic minification with precise size control for the byte budget |
3.2 Monorepo layout #
quietstat/
├── package.json # root scripts: dev, build, lint, typecheck, test, format
├── pnpm-workspace.yaml # apps/*, packages/*, tools/*, integrations/gtm (JS tooling only)
├── tsconfig.base.json # strict base config extended by every package
├── eslint.config.js # flat config, one for the whole repository
├── .prettierrc.json
├── .editorconfig
├── .env.example # every variable from Section 16.4 with defaults
├── Dockerfile # multi-stage; produces quietstat/quietstat
├── LICENSE # AGPL-3.0-only
├── LICENSING.md # component license table
├── CHANGELOG.md
├── README.md # install, develop, upgrade
├── apps/
│ ├── server/ # @quietstat/server (Fastify)
│ │ ├── package.json # "exports" map exposes modules to the worker
│ │ ├── src/
│ │ │ ├── main.ts # command dispatcher: web | worker | migrate
│ │ │ ├── app.ts # builds the Fastify instance
│ │ │ ├── config/ # the only reader of process.env; parses it with the schema in packages/shared/src/config.ts (Section 16.4.1)
│ │ │ ├── plugins/ # request id, error handler, auth, csrf, rate limit, static SPA
│ │ │ ├── routes/
│ │ │ │ ├── app/ # /api/app/v1/* (dashboard API, one folder per domain)
│ │ │ │ ├── admin/ # /api/app/v1/admin/* (operator console API)
│ │ │ │ ├── public-v1/ # /api/v1/* (Sites API)
│ │ │ │ ├── public-v2/ # /api/v2/* (Stats API)
│ │ │ │ ├── share/ # /api/share/v1/*
│ │ │ │ ├── ingest/ # POST /api/event
│ │ │ │ ├── webhooks/ # /api/webhooks/stripe
│ │ │ │ └── health.ts # /healthz, /readyz
│ │ │ ├── modules/ # business logic, one folder per domain:
│ │ │ │ │ # auth, users, teams, sites, goals, funnels, segments,
│ │ │ │ │ # annotations, sharing, reports, api-keys, imports, exports,
│ │ │ │ │ # licensing, billing, usage, audit
│ │ │ │ │ # (licensing/entitlement.ts is the ingest entitlement decision used by Section 8.9)
│ │ │ ├── ingestion/ # pipeline, visitor hash, sessions, enrichment, filters, buffer
│ │ │ ├── db/
│ │ │ │ ├── postgres/ # Drizzle client and repositories (the schema lives in @quietstat/shared/db)
│ │ │ │ ├── clickhouse/ # client, inserts, migration runner
│ │ │ │ └── valkey/ # client and key helpers
│ │ │ ├── security/ # guardedFetch outbound HTTP helper (Section 17.4) and CSRF middleware (Section 17)
│ │ │ ├── mail/ # transport, layout, template functions, plain-text builders
│ │ │ ├── jobs/ # job names, payload schemas, handlers registered by the worker
│ │ │ └── lib/ # errors, ids, cursor, hmac, time helpers used only server-side
│ │ ├── data/ # bundled DB-IP MMDB only (other reference data lives in packages/shared/data)
│ │ └── test/
│ ├── worker/ # @quietstat/worker (pg-boss consumers and cron)
│ │ └── src/main.ts # imports handlers from @quietstat/server/jobs
│ └── web/ # @quietstat/web (React SPA)
│ ├── index.html
│ ├── vite.config.ts
│ ├── public/ # brand/, fonts/, favicon, robots.txt, assets/theme-init.js (served at /assets/theme-init.js)
│ ├── e2e/ # Playwright specs for UI flows
│ └── src/
│ ├── main.tsx
│ ├── routes/ # route modules, one per entry in Section 4.13
│ ├── features/ # feature folders: dashboard, goals, funnels, teams, licensing...
│ ├── components/ # design-system components from Section 4.14
│ ├── api/ # typed fetch client, query keys, TanStack Query hooks
│ ├── i18n/ # t() helper and locales/en.json
│ ├── styles/ # tokens.css, base.css
│ └── lib/ # formatting, theme, keyboard shortcuts
├── packages/
│ ├── shared/ # @quietstat/shared: Zod schemas, types, constants, error catalogue,
│ │ │ # formatting helpers, time helpers, currency table, config schema
│ │ ├── src/
│ │ │ ├── config.ts # env schema (Section 16.4.1)
│ │ │ └── db/schema/ # Drizzle table definitions (Section 5), exported as @quietstat/shared/db
│ │ ├── migrations/
│ │ │ ├── postgres/ # drizzle-kit generated SQL, immutable once merged
│ │ │ └── clickhouse/ # NNNN_name.sql files applied in order (Section 6)
│ │ └── data/ # spam-referrers.txt, datacenter-ranges.txt, subdivisions.json,
│ │ # cities.json, currencies.json (reference data bundled in the image)
│ ├── query/ # @quietstat/query: query JSON -> ClickHouse SQL (Section 13)
│ │ └── src/
│ └── tracker/ # @quietstat/tracker: tracker source, build script, npm package
│ ├── src/
│ ├── build/ # esbuild script, check-size.ts and size-budget.json (Section 7.8)
│ └── dist/ # generated: script bundles served by the server
├── integrations/
│ ├── wordpress/ # PHP plugin (own readme.txt, GPL-2.0-or-later)
│ └── gtm/ # Tag Manager template (template.tpl, metadata.yaml, Apache-2.0)
├── deploy/
│ ├── docker-compose.yml # self-hosted stack: web, worker, postgres, clickhouse, valkey, proxy
│ ├── docker-compose.hosted.yml # overrides for the hosted topology
│ ├── caddy/ # reverse-proxy sample config with automatic TLS
│ └── nginx/ # alternative proxy sample config
├── tests/
│ ├── e2e/ # cross-app Playwright scenarios (ingestion to dashboard)
│ └── load/ # throughput and latency scripts for Section 2.7
├── tools/
│ └── eslint-rules/ # custom lint rules: package boundaries (3.2 rule 2), no process.env outside config,
│ # no literal JSX text (4.11)
└── scripts/ # dev helpers: seed demo data, generate synthetic trafficRules:
- Package names:
@quietstat/server,@quietstat/worker,@quietstat/web,@quietstat/shared,@quietstat/query,@quietstat/tracker. All first-party imports between packages use these names with the workspace protocol; relative imports that cross package boundaries are forbidden. - Allowed dependency directions:
shareddepends on no first-party package (it may depend ondrizzle-ormandzod);querydepends onshared;trackerdepends on nothing first-party (it must stay dependency-free to meet its byte budget; shared constants are copied at build time by a script, not imported at runtime);serverdepends onsharedandquery;workerdepends onserver;webdepends onshared.webMUST NOT import fromserver,workerorquery. A custom lint rule intools/eslint-rules/enforces this. - The worker reuses server modules through the
exportsmap of@quietstat/server(./modules/*,./jobs,./db/*,./mail,./config,./lib/*). It never imports route files. - Migrations are append-only. Editing a merged migration file is forbidden; fix forward with a new file. Postgres migrations and ClickHouse migrations live in
packages/shared/migrations/{postgres,clickhouse}, and the Drizzle table definitions inpackages/shared/src/db/schema. apps/server/data/ships inside the image with the bundled GeoIP file. Newer GeoIP files are downloaded toDATA_DIR/geoipby thegeoip-updatejob, and refreshed datacenter and spam-referrer lists are written toDATA_DIR/listsbylists-refresh(Section 16.2.2). The image copy is the fallback when no downloaded file exists.
3.3 Component diagram #
┌───────────────────────────────────────────────┐
Visitor browser │ Reverse proxy (TLS) │
┌──────────────┐ beacon │ Caddy or nginx; forwards X-Forwarded │
│ q-{id}.js ├───────────►│ │
└──────────────┘ └───────┬─────────────────────────┬─────────────┘
Customer server │ │
┌──────────────┐ Events API ▼ ▼
│ HTTP client ├───────────► ┌──────────────────────────────────────────┐
└──────────────┘ │ web process (stateless, N copies) │
Dashboard user │ ┌────────────┐ ┌────────────────────┐ │
┌──────────────┐ SPA + API │ │ Routes: │ │ Ingestion pipeline │ │
│ React SPA ├────────────►│ │ app, admin │ │ validate → filter │ │
└──────────────┘ │ │ v1, v2, │ │ → enrich → hash → │ │
API client (Bearer key) │ │ share, hook│ │ session → buffer │ │
┌──────────────┐ /api/v2 │ └─────┬──────┘ └─────────┬──────────┘ │
│ Script / BI ├────────────►│ │ query engine │ batch insert│
└──────────────┘ └────────┼───────────────────┼─────────────┘
│ │
┌─────────────────────────────┼───────────────────┼──────────────┐
│ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ PostgreSQL │ │ Valkey │ │ ClickHouse │ │
│ │ accounts, │ │ sessions, │ │ events, │ │
│ │ settings, │ │ rate limits, │ │ sessions, │ │
│ │ licenses, │ │ realtime, │ │ imported_* │ │
│ │ pg-boss jobs │ │ locks, cache │ └──────▲───────┘ │
│ └──────▲───────┘ └──────▲───────┘ │ │
│ │ │ │ │
│ ┌──────┴──────────────────┴───────────────────┴──────────┐ │
│ │ worker process (1..N, pg-boss) │ │
│ │ email reports, spike checks, salt rotation, usage │ │
│ │ rollups, imports, exports, purges, GeoIP refresh │ │
│ └───────┬─────────────────────────────┬──────────────────┘ │
└───────────┼─────────────────────────────┼──────────────────────┘
▼ ▼
SMTP server Google Data API (GA4 import)
web also talks to: Stripe API + Stripe webhooks (hosted edition only)3.4 Process types #
One Docker image, quietstat/quietstat, selects its role from its command. The same image is used by both editions.
| Command | Role | Scaling | Listens | Notes |
|---|---|---|---|---|
web |
Fastify server: dashboard API, public APIs, ingestion, shared-link API, Stripe webhook, health endpoints, static SPA and tracker files | Stateless; run 1 to N replicas behind a load balancer | PORT (default 8000) |
Holds an in-process ingestion buffer; drains it on SIGTERM within 10 s |
worker |
pg-boss consumers and cron schedules | 1 to N replicas; pg-boss guarantees each job runs once; cron schedules are registered by an advisory-lock holder | None (no HTTP port) except an optional health file touched every 15 s | Job names and schedules are canonical in Section 16 |
migrate |
Applies Postgres migrations, ClickHouse migrations (tracked in schema_migrations) and seeds; then exits with code 0 or 1 |
One-shot | None | Idempotent. web and worker refuse to start if the schema version is older than the image expects and print the instruction to run migrate |
Startup order in Compose: Postgres, ClickHouse and Valkey healthy; migrate completes; then web and worker. Readiness (/readyz) is false until all three stores answer.
3.5 Request flows #
3.5.1 Ingestion (tracker or Events API to ClickHouse) #
- The browser sends
POST /api/eventwithContent-Type: text/plain(no CORS preflight) or the Events API caller sends JSON. CORS allows any origin. - The
webprocess applies theevent_ingestrate-limit policy of Section 17.6 before reading the body (429 above it), then reads the body with a 32 KB limit (413 above it) and parses JSON; malformed payloads return 400 with a plain-text reason. - Zod validates the payload (Section 8). The domain in
dis normalized (lowercased, any port and one leadingwww.removed) and resolved through the in-process site cache (Section 3.6). Unknown domains, and sites of teams pending deletion, are dropped silently. - For each resolved site, in the order defined in Section 8.1: hostname allowlist (
site_settings.allowed_hostnames, part of site resolution), bot filter, IP exclusions, datacenter filter, page exclusions, spam-referrer filter, then the license and cap check (hosted only). A drop at any step returns 202 and increments the per-site per-day filtered counter in Valkey (Section 8.8.6). - Accepted events are enriched: User-Agent parsing, GeoIP lookup, referrer and channel classification, UTM extraction, URL normalization.
- The visitor
user_idis computed with the daily salt; the session is looked up or created in Valkey (30-minute sliding expiry);session_idis assigned. - The enriched event is appended to the in-process buffer; realtime and billable counters are incremented in Valkey.
- The server answers 202 with an empty body. The response does not wait for ClickHouse.
- The buffer flushes a batch insert to the
eventstable and the corresponding rows tosessionsevery 2 seconds or at 5,000 rows, whichever comes first. Failed inserts retry with exponential backoff from 1 s up to 30 s while the buffer holds up to 100,000 event rows; beyond that bound new events are dropped with a 202 response (reasonbuffer_full) and a warning is logged. The batch size, flush interval and buffer bound are the variablesINGEST_BATCH_ROWS,INGEST_FLUSH_INTERVAL_MSandINGEST_BUFFER_MAX_ROWS(Section 16.4.6). Section 8.15 specifies the failure modes.
3.5.2 Dashboard query #
- The SPA calls a dashboard API route under
/api/app/v1/sites/{domain}/...with the session cookie (andX-CSRF-Tokenfor mutating calls). - The auth plugin resolves the session from Valkey (fallback to Postgres), loads the user and checks the permission for the site (Section 11).
- The route validates the request with Zod, then builds a query object (metrics, dimensions, filters, date range in the site time zone, comparison range).
- The result cache is checked (Section 3.6). On hit, the cached payload is returned.
- On miss,
@quietstat/querygenerates parameterized ClickHouse SQL. The site's UUID predicate is injected by the engine and cannot be omitted. Segments are expanded into filters. Imported data is merged only for dates strictly earlier than the site's first native day (Section 13.4.10). - The query runs under the read-only ClickHouse user (Section 6.1) with a 30-second timeout, a result row cap, a memory cap (
max_memory_usage) and a rows-to-read guard that raises 422query_too_large(the guards are constants inpackages/shared, Section 13.4.11), returns rows, and the route shapes the response in the standard envelope (Section 4.4). - The result is stored in the cache and returned. The SPA keeps it in TanStack Query for 30 seconds.
The Stats API (POST /api/v2/query) uses steps 3 to 7 with an API key instead of a session (Section 13).
3.5.3 Scheduled email report #
- The
email-reports-dispatchcron job in the worker (schedule in Section 16.6) findsemail_report_subscriptionswhose next send time is due and enqueues oneemail-report-sendjob per (site, frequency, period key) with a singleton key of that tuple, so duplicates cannot occur. - The
email-report-sendjob loads the site and recipients, runs the report queries through the shared query engine for the report period in the site time zone, and builds the HTML and plain-text bodies with the template functions (Section 12). - Each message is delivered by an
email-sendjob through Nodemailer and SMTP. Success updates the subscription's last-sent time. Retries and back-off follow theemail-report-sendandemail-sendrows of Section 16.6; after the last failure the job is marked failed and logged. - Without SMTP configured the job completes immediately and logs
email_not_sent(Section 1.2, row 6).
3.5.4 Data import #
- The user uploads a CSV (multipart, size limit in Section 14) or authorizes Google and selects a GA4 property.
- The server stores the upload in
DATA_DIR, inserts animportsrow with its initial status (Section 14.6) and enqueues animport-runjob (CSV) or aga4-import-runjob (GA4); the API returns 202 with the import resource. - The worker streams the file or pages the Google Data API, validates rows with the schemas in Section 14, writes batches into the
imported_*ClickHouse tables, and updates progress on theimportsrow every 5,000 rows. - On completion the import is marked
completed, the site's cache epoch is incremented so reports refresh, and the owner is notified in the UI. On failure the import is markedfailedwith an error code and message; rows already written for it are never visible in reports (Section 14.5) and are removed by theimport-purgejob when the import is deleted or cancelled.
3.5.5 Checkout (hosted edition) #
- In the SPA the user chooses a tier on the activate page. The checkout endpoint (Section 15) validates the team, the role (owner only) and that the tier is a legitimate purchase or upgrade.
- The server creates a Stripe Checkout Session in
paymentmode with inlineprice_data(amount fromlicense_tiers, or the price difference for an upgrade), metadata{team_id, tier, kind}, and the success and cancel URLs underBASE_URL. It returns the Stripe URL; the SPA redirects. - The customer pays on the Stripe-hosted page. Stripe redirects to the success URL (
/activate/return) and sendscheckout.session.completedto/api/webhooks/stripe. - The webhook handler verifies the signature and is idempotent by purchase state transition (Section 5.10; there is no event-ID table): in one Postgres transaction it moves the
purchasesrow frompendingtopaidand creates or upgrades the team'slicensesrow. A replayed event finds the purchase alreadypaidand changes nothing. A paid checkout always ends with either an active license or an automatic refund, never neither (Section 15.6). - The webhook transaction assigns the sequential invoice number (Section 15) and enqueues the confirmation email as an
email-sendjob; the invoice PDF is rendered on demand when downloaded (Section 15). The SPA, pollingGETof the license state on the return page, shows success within seconds. Section 15 specifies failure handling when the webhook is late.
3.6 Caching strategy #
| Cache | Store | Key | TTL | Invalidation |
|---|---|---|---|---|
| Site lookup for ingestion (domain to site record, settings, team license state) | Process memory, LRU of 10,000 entries, with a negative cache for unknown domains | Normalized domain |
30 s (negative entries 30 s); stale entries are served for up to 10 minutes when Postgres is unreachable | Expires; a site or license change publishes a Valkey pub/sub message on channel qs:site-config that every web process uses to evict immediately (Section 8.6) |
| Daily salts | Process memory | Salt id |
Refreshed every 60 s | Per Section 8 |
| GeoIP, user-agent regex tables, bot lists, datacenter CIDRs, spam referrers | Process memory | None | Process lifetime | Reloaded when the file's modification time changes, checked every 60 s (Section 8.10.3) |
| Active session state for visit stitching | Valkey | qs:s:{site_id}:{user_id hex} |
30 min sliding | Natural expiry (Section 8.12.2) |
| Dashboard session records | Valkey, fallback Postgres auth_sessions |
qs:auth:{session_token_hash} |
5 min | Deleted on logout, password change and revoke |
| Query result cache | Valkey | qs:stats:{site_id}:{epoch}:{sha256(query JSON + caller scope)}; the caller scope is the share-link id and enforced segment, so a segment-limited viewer is never served an unrestricted result |
60 s when the range includes today; 10 min when it ends before today (site time zone); 5 s for realtime and current-visitors | The per-site epoch (qs:epoch:{site_id}) is incremented on the changes listed in Section 9.23.2 |
| Realtime visitor counters | Valkey sorted sets | qs:rt:{site_id} (member user_id, score last-seen timestamp) |
Members older than 300 s trimmed on write and read | Natural |
| Rate-limit counters | Valkey | Per limiter key (Section 17.6) | Window length | Natural |
| Usage counters (hosted) | Valkey, reconciled to usage_monthly |
qs:usage:{team_id}:{YYYYMM} |
40 days | Incremented by web at ingestion (Section 8.9) and flushed per Section 15 |
| Filtered-traffic counters | Valkey hash | qs:filt:{site_id}:{yyyymmdd} (one field per drop reason) |
35 days | Natural (Section 8.8.6) |
| Static assets | HTTP caching | Hashed filenames under /assets/ |
Cache-Control: public, max-age=31536000, immutable; index.html and /assets/theme-init.js (not hashed): no-cache; tracker script: public, max-age=600, stale-while-revalidate=86400, stale-if-error=604800 (Section 7.2.2) |
Content-hashed names for SPA assets |
Valkey holds only reproducible or short-lived state. Flushing it MUST NOT lose durable data: sessions fall back to Postgres, counters rebuild from ClickHouse, rate limits restart.
3.7 Performance budgets #
| Budget | Value | Enforced by |
|---|---|---|
POST /api/event handler time (excluding network) |
p50 under 3 ms, p99 under 25 ms | Load test in tests/load |
| Ingestion buffer flush | 2 s or 5,000 rows; max batch 20,000 rows | Unit test |
| ClickHouse insert latency per batch of 5,000 rows | p95 under 200 ms | Monitoring (Section 16) |
| Any dashboard report query | p95 under 500 ms; hard timeout 30 s | Load test and max_execution_time |
| Full dashboard load | p95 under 1.2 s server-side | Load test |
| Management API (Postgres only) | p95 under 150 ms | Integration test thresholds |
| Tracker script | Core (core.js) at most 1,500 bytes gzip -9; per-module budgets and the generic script.js budget of 4,800 bytes per Section 7.8.1 |
CI size check (Section 7.8) |
| SPA initial JS | At most 250 KB gzip | CI bundle check |
| Worker memory | Under 512 MB resident per process | Monitoring |
| Web memory | Under 1 GB resident per process under sustained 2,000 events per second | Load test |
| Ingestion availability | A ClickHouse outage of up to 45 seconds at 2,000 events per second per process loses no events; beyond the 100,000-row buffer bound new events are dropped with buffer_full (Section 8.15.3) |
Failure test in Section 18 |
3.8 Scaling model #
| Dimension | Approach |
|---|---|
| Web throughput | Add web replicas behind a load balancer. Each holds its own buffer; no coordination except Valkey. Session stitching goes through Valkey so any replica can handle any event. |
| Ingestion burst | In-process buffer of up to 100,000 event rows absorbs short ClickHouse slowdowns; above it, new events are dropped (buffer_full) and a warning is logged (Section 8.15.3). |
| Worker | Add replicas; pg-boss distributes jobs. Heavy jobs (imports, exports) run with a concurrency of 1 per worker process and teamConcurrency of 1 per site. |
| ClickHouse | Single node (suits up to roughly 100 million events per month). Replicated clusters are out of scope in v1 (Section 6.1); larger deployments scale the node vertically. |
| PostgreSQL | Single primary; load is small (settings and accounts). Read replicas are not required. |
| Valkey | Single instance; persistence off; loss is tolerated. |
| Storage growth | About 100 bytes per event compressed on disk for planning; the sizing table is in Sections 6.11 and 16.11. |
Reference sizing, used for the acceptance numbers in Section 2.7:
| Profile | Machine | Intended load |
|---|---|---|
| Small self-hosted | 2 vCPU, 4 GB RAM | Up to 10 million events per month |
| Reference (acceptance tests) | 4 vCPU, 16 GB RAM, NVMe, all services on one node | Up to 100 million events per month |
| Hosted production | 2 or more web replicas (2 vCPU, 2 GB each), 1 worker, ClickHouse 8 vCPU / 32 GB, Postgres 2 vCPU / 4 GB, Valkey 1 GB |
Thousands of sites; capacity is bounded by sold caps (Section 15) |
3.9 Third-party services #
| Service | Used for | Edition | Required | Data sent | Failure behavior |
|---|---|---|---|---|---|
| SMTP provider (operator's choice) | All transactional and report email | Both | Optional in self-hosted, required in hosted | Recipient address and message | Jobs retry; user-facing flows that need email show "Email is not configured" or a retry message |
| Stripe | Checkout, receipts, tax, refunds | Hosted | Required for direct sales | Customer email, tier, amount, team ID metadata | Checkout returns 503 integration_not_configured or unavailable; existing licenses unaffected |
| Google (OAuth and Analytics Data API) | GA4 import | Both | Optional | OAuth tokens (stored encrypted), property ID | Import option disabled when GOOGLE_CLIENT_ID is unset; failed imports surface an error message |
| DB-IP | Source of the GeoIP file: bundled at image build; the geoip-update job downloads the monthly file at runtime when GEOIP_AUTO_UPDATE=true (Section 16.2.2) |
Both | Optional at runtime | None (public file download) | The bundled or last downloaded file keeps being used |
| Datacenter-range and spam-referrer feeds | Source of the datacenter CIDR list and spam-referrer list, refreshed weekly by lists-refresh when LISTS_AUTO_UPDATE=true (Section 16.2.2) |
Both | Optional at runtime | None (public file download) | The bundled or last downloaded lists keep being used |
| European Central Bank reference rates | Daily exchange rates for revenue currency conversion (Section 10), fetched by the fx-rates-fetch job |
Both | Optional | None (public file fetch) | Last stored rates are used; if none exist, revenue is reported only in original currencies |
| Have I Been Pwned range API | Breached-password check when PASSWORD_BREACH_CHECK=hibp (Section 11) |
Both | Optional, off by default | First five characters of a SHA-1 password hash only | The check falls back to the bundled local list |
| Container registry | Distribution of quietstat/quietstat |
Both | Build and deploy time | None | Not a runtime dependency |
| npm registry | Distribution of @quietstat/tracker |
Both | Publish time | None | Not a runtime dependency |
| WordPress.org plugin directory and the Tag Manager community template gallery | Distribution of the two plugins | Both | Publish time | None | Not a runtime dependency |
| Marketplace (external sales channel) | Sells codes generated by the operator | Hosted | Offline: the operator exports a CSV | None automated | No runtime integration |
No third-party analytics, fonts, CDN scripts or error-reporting services are loaded by the dashboard. The SPA serves all assets from its own origin.
3.10 Dependency failure behavior #
| Dependency down | Ingestion | Dashboard | Jobs |
|---|---|---|---|
| PostgreSQL | Accepts events using cached site data for up to 10 minutes, then answers 503 (Section 8.6) | Login and settings fail with 503 unavailable; session cache keeps active users reading cached results only |
Paused until restored (pg-boss) |
| ClickHouse | Buffers up to 100,000 event rows and answers 202; beyond that bound new events are dropped with 202 (buffer_full) and a warning is logged (Section 8.15.3) |
Report calls return 503 unavailable; settings pages work |
Report and import jobs retry |
| Valkey | Answers 503 unavailable with Retry-After: 5 and drops the event (Section 8.15.4); rate limits follow Section 17.6 |
Sessions fall back to Postgres; result cache bypassed; bearer-key API routes fail closed with 503 | Unaffected |
| SMTP | Unaffected | Unaffected | Email jobs retry then fail |
| Stripe | Unaffected | Checkout unavailable | Refund and invoice jobs retry |
/readyz returns 503 unless Postgres, ClickHouse and Valkey all respond; /healthz returns 200 as long as the process runs.
4. Conventions & Cross-Cutting Standards #
4.1 Naming #
| Subject | Convention | Example |
|---|---|---|
| HTTP JSON fields (requests, responses, query-string keys) | snake_case |
visit_duration, next_cursor |
| TypeScript variables, functions, properties | camelCase |
getSiteByDomain |
| TypeScript types, interfaces, classes, React components, Zod schema constants | PascalCase; schemas end in Schema |
CreateGoalSchema, SiteSettings |
| TypeScript constants that are true compile-time constants | SCREAMING_SNAKE_CASE |
MAX_FUNNEL_STEPS |
| Files and folders | kebab-case; React component files kebab-case.tsx; tests *.test.ts; Playwright specs *.spec.ts |
create-goal.ts |
| Postgres tables | plural snake_case |
shared_links |
| Postgres columns, ClickHouse tables and columns | snake_case |
created_at, site_id |
| Postgres indexes | {table}_{columns}_idx; unique: {table}_{columns}_key; foreign keys: {table}_{column}_fkey |
goals_site_id_idx |
| Environment variables | SCREAMING_SNAKE_CASE, no prefix except where an owning section defines one |
SMTP_HOST |
| Enum values in JSON and databases | lowercase snake_case strings |
guest_viewer |
| Error codes | lowercase snake_case |
site_limit_reached |
| Job names (pg-boss) | lowercase kebab-case; the single list is the jobs table in Section 16.6 |
email-report-send |
| Valkey keys | colon-separated lowercase segments, versionless; every key starts with qs: |
qs:rt:{site_id} |
| i18n keys | dot-separated area.component.element, camelCase segments |
goals.form.nameLabel |
| Query-string filter and sort parameters | snake_case |
?sort=created_at&order=desc |
| URL paths | lowercase kebab-case, plural nouns for collections |
/api/app/v1/shared-links |
| Git branches | {type}/{scope}-{slug} |
feat/server-goal-limits |
| npm/package names | @quietstat/{dir} |
@quietstat/shared |
Boundary conversion: the Fastify layer receives snake_case JSON validated by Zod schemas that already use snake_case keys; repositories map to camelCase TypeScript objects at the module boundary through a single toCamel / toSnake pair in @quietstat/shared. No handler hand-renames individual fields.
Booleans are named as predicates (is_active, has_password); timestamps end in _at; dates (no time) end in _on in JSON and _date is forbidden; durations end in _seconds or _ms; billing amounts end in _minor (Section 4.7); event revenue columns are the exception: revenue_amount and revenue_reporting_amount hold Decimal(18,3) major units (Section 6.3); counts end in _count; foreign keys end in _id.
4.2 Identifiers #
| Identifier | Format | Where |
|---|---|---|
| Postgres primary key | uuid with default uuidv7() |
Every table in Section 5 |
| Site, public | The site domain string (lowercase, punycode ASCII, no scheme, port, path or leading www.; Section 5.5.1), URL-encoded in route paths |
All route paths (/api/app/v1/sites/{domain}/...) and query payloads (site_id field of the Stats API carries the domain) |
| Site, internal | The site UUID | ClickHouse site_id, foreign keys |
| Team, user, goal, funnel and other resources in the dashboard API | UUID strings in lowercase canonical form | Route parameters and JSON |
| Request ID | req_ followed by 24 lowercase hex characters (12 random bytes) |
X-Request-Id response header, error.request_id, every log line |
ClickHouse user_id |
UInt64 |
Section 8; serialized in JSON as a decimal string; never exposed by public APIs |
ClickHouse session_id |
UInt64 random |
Section 8 |
| Cursor | Opaque string (Section 4.5) | Pagination |
| Tokens (session, API key, share token, email token, invitation token) | Formats and storage owned by Sections 11, 12 and 13. Session, API-key, email and invitation tokens are random, at least 256 bits, and stored only as SHA-256 hashes. Three values are stored in clear because they must be re-displayed or are low-impact: the shared-link slug (192 bits, Section 5.7.1), the report unsubscribe token (192 bits, Section 5.7.2) and the site proxy key (160 bits, Section 7.15.4) | Never logged |
Rules:
- A client may send its own
X-Request-Idrequest header. The server uses it only if it matches^[A-Za-z0-9_-]{8,64}$; otherwise it generates one. The value used is always echoed in the response header. - UUID route parameters are validated with Zod
z.uuid(); a malformed value returns 404not_found(not 400), to avoid distinguishing "bad format" from "absent". - Identifiers in URLs are never reused after deletion because UUIDs are unique; when a site is deleted or its domain is changed, the old domain is reserved for 30 days for the same team (which may add it again); other teams receive 409
domain_takenuntil the reservation expires (Section 5.5.1). - The site
domainis case-insensitive on input and stored lowercase, without scheme, port, path or a leadingwww.; ingestion strips a leadingwww.and any port fromdbefore lookup, sowww.example.comandexample.comare the same site. Its character rules and uniqueness are owned by Section 5.5.1 (unique across the whole installation on the normalized domain); local development useslocalhost.
4.3 Route families and HTTP semantics #
| Family | Prefix | Authentication | CSRF | CORS | Rate-limit class | Owning section |
|---|---|---|---|---|---|---|
| Dashboard API | /api/app/v1/..., including the unauthenticated GET /api/app/v1/bootstrap |
Cookie session | X-CSRF-Token required on POST, PATCH, DELETE (exemptions: Section 11.7) |
Same-origin only (no CORS headers) | app_api_user (per user; Section 17.6) |
5 to 15 (per domain) |
| Operator console API | /api/app/v1/admin/... |
Cookie session of a super admin (hosted only) | Required on mutating calls | Same-origin only | admin_api (Section 17.6) |
15 |
| Stats API | POST /api/v2/query |
Authorization: Bearer <api key> |
Not applicable | None (no CORS headers; server-to-server only, Section 17.5.3) | stats_api_key (Section 17.6) |
13 |
| Sites API | /api/v1/sites... |
Bearer API key | Not applicable | None (Section 17.5.3) | sites_api_key (Section 17.6) |
13 |
| OpenAPI document | GET /api/openapi.json |
None | Not applicable | Access-Control-Allow-Origin: *, no credentials |
openapi (Section 17.6) |
13 |
| Events API and tracker ingestion | POST /api/event |
None | None | Access-Control-Allow-Origin: *; preflight answered |
event_ingest (per IP) and event_ingest_proxy (valid proxy key only) (Section 17.6) |
8, 13 |
| Shared-link and public-dashboard data | /api/share/v1/link/:token/* and /api/share/v1/public/:domain/* (mirrors of the dashboard read endpoints) |
Share token (and password proof where set), or none for a public dashboard | Not applicable (read-only) | None (the embed and share pages are served from the app origin; Section 17.5.3) | share_view, share_password (Section 17.6) |
12 |
| Stripe webhook (hosted) | /api/webhooks/stripe |
Stripe signature | Not applicable | None | None beyond signature verification; idempotent by purchase state transition (Section 15) | 15 |
| Health | GET /healthz, GET /readyz |
None | None | None | None | 16 |
| Static | /, /assets/*, /js/*, /brand/*, SPA fallback for every other GET that does not start with /api/ |
None | None | /js/* allows * |
tracker_files for /js/* (Section 17.6); none for the rest |
3, 7 |
HTTP semantics for the dashboard, Sites and share families:
| Operation | Method and path shape | Success status | Body |
|---|---|---|---|
| List | GET /{collection} |
200 | List envelope |
| Read | GET /{collection}/{id} |
200 | Single envelope |
| Create | POST /{collection} |
201, Location header with the new resource path |
Single envelope |
| Partial update | PATCH /{collection}/{id} |
200 | Single envelope with the updated resource |
| Delete | DELETE /{collection}/{id} |
204 | Empty |
| Action that is not CRUD | POST /{collection}/{id}/{verb} where the verb is a kebab-case imperative |
200 with the resource, 202 when asynchronous, or 204 when there is nothing to return | As stated |
| Asynchronous work (imports, exports, site purge) | POST returns 202 with a resource that has a status field; the client polls GET /{id} |
202 | Single envelope |
Additional rules:
PUTis not used anywhere. Updates arePATCHwith partial bodies; arrays inside aPATCHbody replace the whole array.- Request bodies are
application/json(UTF-8). Any other content type on a JSON route returns 415unsupported_media_type. File uploads usemultipart/form-dataon routes that say so. - Unknown JSON keys in request bodies of the dashboard, Sites and Stats APIs are rejected with 400
validation_failed(strict objects).POST /api/eventignores unknown keys (Section 8). - A
HEADrequest is answered forGETroutes;OPTIONSis answered only where CORS applies. Other method mismatches return 405method_not_allowedwith anAllowheader. - Idempotency keys are not supported in v1. Safe retries are provided by uniqueness constraints and, for Stripe, by idempotency through purchase state transitions (Section 15; there is no event-ID table).
- Every API response carries
X-Request-Id,Cache-Control: no-store(except/healthz, static files and the public tracker script) andX-Content-Type-Options: nosniff. Rate-limited routes also sendX-RateLimit-Limit,X-RateLimit-RemainingandX-RateLimit-Reset(Section 17.6), and a 429 addsRetry-Afterin seconds. - Maximum request body sizes per route family are defined once in the body-limit table of Section 17.4 (including
POST /api/event, the Stats and Sites APIs, the Stripe webhook and the CSV import upload route). A larger body returns 413payload_too_large(plain text forPOST /api/event). - Boolean query parameters use the literals
trueandfalse; arrays use repeated keys (?status=a&status=b); unknown query parameters on the dashboard and Sites APIs are rejected with 400validation_failed. - The SPA never calls the Sites API or the Stats API. The Sites API and Stats API never accept a cookie session.
- Redirect targets supplied by clients (
return_to) are accepted only when they are relative paths starting with a single/and not//,/\or containing a scheme; otherwise they are ignored and the default destination is used.
4.4 Response envelopes and errors #
4.4.1 Success envelopes #
Single resource:
{ "data": { "id": "0194...", "name": "Signup", "created_at": "2026-11-03T14:05:09.123Z" } }List:
{
"data": [ { "id": "0194...", "name": "Signup" } ],
"meta": { "next_cursor": "eyJ2IjoxLC4uLn0", "limit": 50 }
}next_cursor is null on the last page. The meta object of a list MAY contain additional documented keys defined by the owning section; consumers ignore unknown keys.
Exceptions: POST /api/v2/query returns { "results": [...], "meta": {...}, "query": {...} } (Section 13); POST /api/event returns 202 with an empty body; 204 responses have no body.
4.4.2 Error envelope #
Every error from every route family except POST /api/event has this exact shape:
{
"error": {
"code": "validation_failed",
"message": "The request body is invalid.",
"details": { "fields": [ { "path": "steps.1.name", "message": "Required." } ] },
"request_id": "req_9f2c0a41d3b84e57a1c6e2b0"
}
}Rules:
codeis one of the codes in 4.4.3. Codes are stable API contract;messageis human-readable English and MAY change. Clients branch oncodeand HTTP status, never onmessage.messageis a complete sentence ending in a period, at most 200 characters, and never contains secrets, SQL, stack traces, file paths or personal data other than what the caller sent.detailsisnullunless the code defines a shape (listed in 4.4.3). It is never omitted.- Validation failures put every field issue in
details.fields, an array of{ "path", "message" }.pathuses dot notation with array indexes (goals.0.value); a body-level issue haspathequal to an empty string. POST /api/eventreturns, for 400, 405, 408, 413, 429 and 503, a plain-text one-line reason (Content-Type: text/plain; charset=utf-8, at most 200 characters, no JSON). Reason strings are defined in Section 8.3.- 5xx responses never reveal internals:
internal_errorhas the fixed message "Something went wrong on our side." and therequest_idis how an operator finds the cause in logs. - Unhandled exceptions are caught by the global error handler, logged at
errorwith the stack trace, and returned asinternal_error. ThrownAppErrorinstances (class inapps/server/src/lib/errors.ts, constructor(code, details?, messageOverride?)) are mapped using the status in the catalogue. - Authentication failures never reveal whether an account exists: login failure is always
invalid_credentials. - Unknown resources and resources the caller may not know exist are 404
not_found; resources the caller can see but may not act on are 403forbidden. Site-level and team-level resources of other tenants are always 404.
4.4.3 Error code catalogue #
Statuses used: 400, 401, 403, 404, 405, 408 (POST /api/event only, plain text), 409, 413, 415, 422, 429, 500, 503, 507 (CSV import upload only); 402 is never used. This table is the closed list of error codes: domain sections MUST use these codes and MUST NOT invent others. If an owning section needs a condition not listed, it uses the closest generic code and puts the discriminator in details.reason. A build check fails if any AppError code is not a key of ERROR_CATALOG.
Every row-count limit of Section 5.16 other than the site limit (which uses site_limit_reached) returns limit_reached with details.limit_name set to the limit's name in Section 5.16, for example goals, funnels, custom_properties, site_segments, personal_segments, annotations, pinned_sites, shared_links, api_keys, report_subscriptions, spike_recipients, teams_owned, pending_invitations, site_guests. Conditions that name a missing object (goal, funnel, segment, annotation, import, export, member) are always 404 not_found (rule 9 of 4.4.2); a funnel step that points at a goal of another site is 400 validation_failed on steps.N.goal_id.
| Code | Status | Meaning | details shape |
|---|---|---|---|
validation_failed |
400 | Request body, query or path failed schema validation | { "fields": [{ "path", "message" }] } |
malformed_json |
400 | Body is not parseable JSON | null |
invalid_cursor |
400 | Cursor is malformed, tampered or belongs to another query | null |
invalid_query |
400 | Stats API query is structurally valid JSON but semantically unusable (unknown metric, dimension, filter) | { "fields": [{ "path", "message" }] } |
invalid_signature |
400 | Webhook signature verification failed | null |
unauthenticated |
401 | No valid session or API key | null |
session_expired |
401 | Session existed but is expired or revoked | null |
invalid_credentials |
401 | Wrong email or password | null |
two_factor_required |
401 | Password accepted; TOTP or recovery code needed | { "challenge_token": "...", "expires_in": 0 } (expires_in is the remaining lifetime in seconds; the value shown is a placeholder) |
challenge_expired |
401 | The two-factor challenge token expired or was already used; sign in again | null |
invalid_two_factor_code |
401 | TOTP code wrong or reused at login | null |
invalid_recovery_code |
401 | Recovery code wrong or already used | null |
invalid_api_key |
401 | API key missing, malformed, unknown or revoked | null |
share_password_required |
401 | Shared link needs a password proof | null |
share_password_invalid |
401 | Shared-link password wrong | null |
forbidden |
403 | Authenticated but the role does not allow the action | { "required_role": "admin" } where applicable, otherwise null |
csrf_invalid |
403 | Missing or wrong X-CSRF-Token |
null |
email_not_verified |
403 | Action requires a verified email | null |
api_key_scope_insufficient |
403 | API key lacks the needed scope | { "required_scope": "..." } |
impersonation_read_only |
403 | Mutation attempted during a super-admin impersonation session | null |
registration_closed |
403 | Signup not allowed in the current REGISTRATION_MODE |
null |
two_factor_enrollment_required |
403 | The team requires 2FA and the user has not enrolled | { "team_id": "..." } |
two_factor_required_by_team |
403 | The user tries to disable their own 2FA, or belongs to a team that enforces it | { "teams": [{ "id", "name" }] } |
account_suspended |
403 | The operator suspended the account; login is refused after the password check | null |
account_pending_deletion |
403 | The account is in its deletion window; only the restore screen is available | null |
admin_2fa_required |
403 | Operator console access requires an enrolled second factor | null |
reauth_required |
403 | A step-up re-authentication is missing or older than 15 minutes | null |
license_required |
403 | The action needs an active license and the hosted team has none | null |
setup_required |
403 | Signup was attempted while no user exists; the first account is created through first-run setup (Section 16.3.5) | null |
setup_token_invalid |
403 | The first-run setup token is wrong (wrong attempts are rate limited, Section 17.6) | null |
owner_protected |
403 | The action targets the team owner, who changes only through ownership transfer | null |
target_team_forbidden |
403 | The caller lacks owner or admin rights in the destination team of a transfer | null |
invitation_email_mismatch |
403 | The signed-in or signup email differs from the invited address | null |
not_found |
404 | Resource, route or token does not exist (also for hidden resources and hosted-only routes in self-hosted) | null |
method_not_allowed |
405 | HTTP method not supported by the route | null |
request_timeout |
408 | The request body was not received in time; used only by POST /api/event, which answers with a plain-text reason instead of the envelope (4.4.2 rule 5) |
null |
conflict |
409 | State conflict not covered by a specific code | { "reason": "..." } |
email_taken |
409 | Email already registered (changing email or inviting an existing member) | null |
domain_taken |
409 | Site domain already tracked in this installation | null |
site_limit_reached |
409 | Team cannot create more sites under its license (or unlicensed with 1 site) | { "limit": 5, "current": 5 } |
limit_reached |
409 | A numeric limit other than sites was hit; limit_name is the limit's name in Section 5.16 (goals, funnels, custom_properties, personal_segments, site_segments, annotations, pinned_sites, teams_owned, pending_invitations, site_guests, shared_links, report_subscriptions, spike_recipients and every other row-count limit there) |
{ "limit_name": "goals", "limit": 100, "current": 100 } |
member_exists |
409 | User is already a member of the team | null |
invitation_exists |
409 | A pending invitation for that email already exists | null |
owner_required |
409 | The change would leave a team without exactly one owner | null |
goal_exists |
409 | A goal with the same definition already exists on the site | { "goal_id": "..." } |
goal_name_taken |
409 | Goal display name already used on the site (case-insensitive) | null |
goal_in_use |
409 | Goal is referenced by funnel steps | { "funnel_ids": ["..."] } |
funnel_name_taken |
409 | Funnel name already used on the site | null |
segment_name_taken |
409 | Segment name already used in its scope (case-insensitive) | null |
segment_in_use |
409 | Segment is used by shared links or email reports | { "shared_link_ids": ["..."] } |
property_in_use |
409 | Property key is used by goal conditions, funnel step conditions or segments | { "goal_ids": [], "funnel_ids": [], "segment_ids": [] } |
custom_prop_exists |
409 | The property key is already on the site's allowlist | null |
import_in_progress |
409 | Another import for the site is running | null |
import_overlap |
409 | The import's date range intersects an existing import of the site (Section 14.4.1) | { "conflicting_import_id": "..." } |
import_not_cancellable |
409 | The import is not in a status that can be cancelled | { "status": "completed" } |
export_in_progress |
409 | An export is already queued or running for the site | null |
purge_in_progress |
409 | The site's data is being deleted | null |
team_pending_deletion |
409 | The team is scheduled for deletion | { "delete_at": "2026-11-10T00:00:00.000Z" } |
owner_cannot_leave |
409 | The team owner tried to leave; transfer ownership or delete the team | null |
owner_must_transfer |
409 | Account deletion blocked because the user owns teams with other members | { "teams": [{ "id", "name", "member_count" }] } |
transfer_pending |
409 | An ownership or site transfer is already pending | null |
cannot_change_own_role |
409 | A member tried to change their own role | null |
use_leave |
409 | A member tried to remove themselves; use the leave action | null |
already_verified |
409 | The email address is already verified | null |
already_guest |
409 | The user already has guest access to the site | null |
two_factor_already_enabled |
409 | Two-factor is already enabled for the account | null |
two_factor_not_enabled |
409 | Two-factor is not enabled for the account | null |
license_acknowledgement_required |
409 | Deleting a team that holds an active license needs acknowledge_license_loss: true |
null |
api_key_name_taken |
409 | API key name already used in the team (case-insensitive) | null |
recipient_exists |
409 | The address is already a recipient of that report or alert | null |
recipient_unsubscribed |
409 | The recipient unsubscribed and must resubscribe themselves | null |
not_bounced |
409 | A reactivation was requested for a recipient whose status is not bounced |
null |
not_resubscribable |
409 | A resubscribe was requested for a recipient whose status is bounced |
null |
setup_already_completed |
409 | First-run setup was already completed (a user exists) | null |
code_already_redeemed |
409 | License code already used | null |
code_stack_limit_reached |
409 | Team already holds the maximum of 3 stacked codes | null |
license_already_active |
409 | Purchase would duplicate an equal or higher active tier | { "tier": "pro" } |
license_source_conflict |
409 | The purchase or code flow is incompatible with the source of the team's active license | null |
payload_too_large |
413 | Body or upload exceeds the route limit | { "max_bytes": 1048576 } |
unsupported_media_type |
415 | Content type not accepted | null |
unprocessable |
422 | Semantically invalid request not covered by a specific code | { "reason": "..." } |
token_invalid |
422 | Email, reset, invitation or verification token unknown or already used | null |
token_expired |
422 | Token past its expiry | null |
invitation_expired |
422 | The invitation is past its expiry | null |
two_factor_setup_expired |
400 | The pending two-factor enrollment expired; start again | null |
current_password_incorrect |
422 | Confirmation password wrong (step-up, account changes) | null |
two_factor_code_incorrect |
422 | The TOTP code is wrong in a step-up or enable/disable flow | null |
password_too_common |
422 | Password is on the common-password list | null |
password_breached |
422 | Password appears in the breach check | null |
password_unchanged |
422 | The new password equals the current one | null |
email_unchanged |
422 | The new email equals the current one | null |
invalid_role |
422 | Role not allowed for this operation | null |
invalid_target |
422 | The target user is not valid for this operation | null |
use_ownership_transfer |
422 | Changing a role to owner requires an ownership transfer |
null |
confirm_email_mismatch |
422 | The typed confirmation email does not match | null |
confirm_name_mismatch |
422 | The typed confirmation name does not match | null |
same_team |
422 | Source and destination team of a transfer are the same | null |
owner_two_factor_required |
422 | The operation requires the owner to have two-factor enabled | null |
target_two_factor_required |
422 | The target user must enable two-factor first | null |
target_email_unverified |
422 | The target user's email is not verified | null |
cannot_invite_self |
422 | A user cannot invite their own address | null |
code_invalid |
422 | License code does not exist, is malformed or is revoked | null |
downgrade_not_allowed |
422 | Requested tier is lower than, or equal to, the current tier, or the upgrade amount is below the minimum | null |
refund_window_closed |
422 | Purchase older than REFUND_WINDOW_DAYS |
{ "purchased_at": "..." } |
invalid_upgrade_target |
422 | The requested upgrade target is not a higher active tier, or the amount due is below the minimum charge | null |
refund_upgrade_first |
422 | A later upgrade on the same license must be refunded first | null |
license_purged |
422 | The license data was already purged and cannot be reinstated | null |
tier_constraint_violation |
422 | A tier edit breaks the price, site-cap or event-cap ordering across ranks | { "fields": [{ "path", "message" }] } |
import_file_invalid |
422 | Uploaded file fails structural validation | { "reason": "...", "line": 12 } |
import_invalid_file_name |
422 | A CSV file name does not match the required pattern | { "file": "..." } |
import_files_range_mismatch |
422 | CSV files in one upload carry different date ranges | null |
import_visitors_file_missing |
422 | The required visitors file is absent | null |
import_duplicate_table |
422 | The same table appears twice in one upload | null |
import_invalid_header |
422 | A CSV header lacks required columns | { "file": "...", "missing": ["..."] } |
import_too_many_rows |
422 | A CSV file exceeds the row limit | null |
import_archive_rejected |
422 | A ZIP archive fails the safety checks | null |
goal_required |
422 | A conversion metric was requested without a goal in scope | { "metric": "..." } |
unknown_goal |
422 | A Stats API filter names a goal not defined on the site | { "goal": "..." } |
invalid_regex |
422 | A matches pattern does not compile |
{ "pattern": "..." } |
comparison_not_supported |
422 | Comparison was requested with a range that does not support it | { "date_range": "..." } |
too_many_time_buckets |
422 | More than 10,000 time buckets would be returned | { "buckets": 0, "max": 10000 } |
time_dimension_too_fine |
422 | The time dimension is too fine for the range | { "dimension": "...", "max_span": "..." } |
time_labels_requires_time_dimension |
422 | include.time_labels needs exactly one time dimension |
null |
unsupported_combination |
422 | The metric and dimension cannot be combined | { "metric": "...", "dimension": "..." } |
query_too_large |
422 | The query exceeds the scan, memory or row guard | { "limit": "..." } |
invalid_date_range |
422 | Dates in the wrong order, outside the allowed range, unparsable, or span too long | { "reason": "..." } |
invalid_interval |
422 | The interval is not allowed for the range | null |
invalid_comparison |
422 | The comparison mode or its dates are invalid for the request | null |
invalid_filter |
422 | Unknown dimension, empty values, too many filters or values, over-long value, nested segment or missing parent filter | { "fields": [{ "path", "message" }] } |
unsupported_filter_operator |
422 | The operator is not allowed for the dimension | { "dimension": "...", "operator": "..." } |
immutable_field |
422 | Attempt to change a field that is fixed after creation (goal type, revenue_currency) |
{ "field": "..." } |
revenue_not_supported |
422 | revenue_currency on a goal type that cannot carry revenue |
null |
unsupported_currency |
422 | Currency code not in the supported list | null |
conditions_not_supported |
422 | Property conditions on a scroll goal or on a goal-type funnel step | null |
duplicate_property_condition |
422 | Two conditions with the same key | null |
too_many_property_conditions |
422 | More than 3 property conditions | null |
property_not_allowed |
422 | Property key is neither allowlisted nor reserved | { "key": "..." } |
reserved_property_name |
422 | Property key starts with qs_, quietstat_ or $ |
null |
reserved_event_name |
422 | An event goal or funnel event step uses pageview or engagement |
null |
invalid_step_count |
422 | A funnel has fewer than 2 or more than 8 steps | null |
invalid_optional_step |
422 | The first or last funnel step is marked optional | null |
strict_with_optional_steps |
422 | Strict order combined with an optional step | null |
duplicate_adjacent_steps |
422 | Two adjacent funnel steps have an identical definition | null |
segment_immutable |
422 | The segment of an existing shared link cannot be changed (segment_id sent on update) |
null |
segment_not_shared |
422 | A personal segment was given where only a site segment can back a shared link | null |
embed_origins_required |
422 | Embed mode allowlist needs at least one origin |
null |
invalid_embed_origin |
422 | An embed origin is not a valid origin | { "origin": "..." } |
invalid_recipient |
422 | A report or alert recipient is not a valid address or user | null |
rate_limited |
429 | Rate limit exceeded; Retry-After header set |
{ "retry_after_seconds": 12 } |
internal_error |
500 | Unhandled server error | null |
unavailable |
503 | A required dependency (Postgres, ClickHouse, Valkey) is down | { "dependency": "clickhouse" } |
integration_not_configured |
503 | SMTP, Stripe or Google is not configured | { "integration": "stripe" } |
billing_unavailable |
503 | The Stripe API failed or is unreachable | null |
query_timeout |
503 | A ClickHouse query exceeded its time limit; Retry-After set |
null |
insufficient_storage |
507 | A CSV import upload was refused because free space under DATA_DIR would fall below the safety margin |
null |
Values of the error_code field of imports and exports resources (for example export_too_large, ga4_permission_denied, import_validation_failed) are owned by Section 14; they describe a failed asynchronous job, are not API error codes and never appear in the error envelope.
Standard messages (used when no override is given):
| Code | Message |
|---|---|
validation_failed |
"The request is invalid." |
malformed_json |
"The request body is not valid JSON." |
invalid_cursor |
"The pagination cursor is invalid." |
invalid_query |
"The query is invalid." |
unauthenticated |
"Authentication is required." |
session_expired |
"Your session has expired. Sign in again." |
invalid_credentials |
"The email or password is incorrect." |
two_factor_required |
"A two-factor code is required." |
forbidden |
"You do not have permission to do this." |
csrf_invalid |
"The request could not be verified. Reload the page and try again." |
not_found |
"The requested resource was not found." |
rate_limited |
"Too many requests. Try again later." |
internal_error |
"Something went wrong on our side." |
unavailable |
"A required service is temporarily unavailable." |
query_timeout |
"The query took too long. Narrow the date range or filters." |
site_limit_reached |
"Your plan has reached its site limit." |
payload_too_large |
"The request is too large." |
| other codes | One sentence stating the condition, taken from the Meaning column and rewritten to address the user |
The full catalogue, with messages, lives in packages/shared/src/errors.ts as a typed map ERROR_CATALOG: Record<ErrorCode, { status, message }>; the SPA imports the same map to show friendly text.
4.5 Pagination #
Management lists (dashboard API, Sites API, share API):
| Parameter | Type | Default | Rules |
|---|---|---|---|
limit |
integer | 50 | 1 to 200. Values outside the range return 400 validation_failed; there is no silent clamping. |
cursor |
string | none | Value of the previous response's meta.next_cursor. |
sort |
enum | created_at unless the route documents otherwise |
Allowed values per route. |
order |
asc or desc |
desc |
Applied to sort. |
- The response is the list envelope (4.4.1) with
meta.next_cursorandmeta.limit(the limit actually applied). - Keyset pagination: every sort is completed with
idas a tiebreaker so ordering is total. The cursor encodes{ "v": 1, "s": "<sort field>", "o": "<order>", "k": [<last sort value>, "<last id>"] }, JSON-encoded, then base64url, followed by.and the first 16 characters of the base64url HMAC-SHA256 of the encoded part keyed with a key derived fromSECRET_KEY_BASE(Section 17.7). A cursor whose signature fails, whosesorodiffers from the request, or that cannot be decoded returns 400invalid_cursor. - No total count is returned. Routes that need one document a separate count field in
metaunder their own name. - Lists never exceed 200 items per page; endpoints with naturally small collections (members of a team, goals of a site) still paginate using the same envelope.
- Stats queries do not use cursors: the Stats API takes
pagination: { limit, offset }in the body with default and maximum limit 10,000 (Section 13). Report previews on the dashboard request 9 rows; "Details" views load 100 rows per "Load more" click through the samelimit/offsetpair (Section 9.6).
4.6 Time zones and dates #
| Rule | Decision |
|---|---|
| Storage | All instants are UTC: timestamptz in Postgres, DateTime('UTC') in ClickHouse. |
| Event time | The server receive time is the event time. Client clocks are never trusted or sent. |
| API format for instants | ISO 8601 in UTC with millisecond precision and Z: 2026-11-03T14:05:09.123Z. Inputs may use any offset and are normalized. |
| API format for site-local dates | YYYY-MM-DD. JSON keys for dates end in _on; for instants in _at. |
| Site time zone | Every site has an IANA zone in timezone. Default on creation: the browser's zone from Intl.DateTimeFormat().resolvedOptions().timeZone when the add-site form submits, if it is a valid zone as defined next, else UTC. Validation: the value must be UTC or a member of Intl.supportedValuesOf('timeZone') on the server runtime, and is stored in its canonical form (new Intl.DateTimeFormat('en', { timeZone }).resolvedOptions().timeZone). Offset strings such as +05:00 and legacy aliases are rejected because ClickHouse cannot use them; otherwise 400 validation_failed on timezone. A Section 18 test checks that every accepted value is accepted by ClickHouse toTimeZone. |
| Bucketing | Day, week and month buckets and the meaning of "today", "yesterday", "this month" use the site zone. ClickHouse queries use toStartOfDay(timestamp, 'Zone') style functions with the zone as a bound parameter. |
| Weeks | Weeks start on Monday (ISO 8601). Week buckets are labeled by their Monday. |
| Date range semantics | A range is two inclusive local dates [from, to]. It converts to the half-open UTC instant interval [start_of_day(from, zone), start_of_day(to + 1 day, zone)). |
| Daylight saving | A local day may last 23 or 25 hours; day buckets always follow the local calendar. Hour buckets in the shift hour are keyed by the UTC instant, so the repeated hour appears twice with different offsets and the skipped hour does not appear. |
| Rolling ranges | "Last 24 hours" is the rolling interval ending now with hourly buckets; "Realtime" is the last 30 minutes with minute buckets (current visitors are those active in the last 5 minutes, Section 9.14). |
| Changing a site's zone | Allowed at any time. Data is stored in UTC, so history is re-bucketed at query time; the per-site cache epoch is incremented (Section 3.6). |
| Usage caps | Calendar months in UTC regardless of site zone (Section 15). |
| Dashboard display | Site data renders in the site zone. Account-level timestamps (sessions, invoices, audit lists) render in the browser's zone with the UTC value in a tooltip. |
| Emails | Report emails use the site zone and name it (Europe/Berlin); transactional emails show UTC with an explicit "UTC". |
| Server clock | Servers run on UTC; code obtains time only through an injectable clock.now() so tests can freeze it. |
| Implementation | Helpers live in packages/shared/src/time/ built on Intl.DateTimeFormat and Date; no date library is added. Functions required: zonedStartOfDay, addLocalDays, localDateOf, formatInZone, resolveRange. |
Date and time labels in the UI use en-US formats: day buckets Nov 3, month buckets Nov 2026, hour buckets 2 PM, full date-time Nov 3, 2026, 2:05 PM, using a 12-hour clock.
4.7 Money, currency and number formatting #
4.7.1 Money #
| Rule | Decision |
|---|---|
| Storage and internal exchange (billing and dashboard-API money) | Integer minor units plus an ISO 4217 code: amount_minor (bigint) and currency (three uppercase letters). |
| Event revenue (ClickHouse) | The exception to the rule above: revenue_amount and revenue_reporting_amount are Decimal(18,3) in major units with revenue_currency and revenue_reporting_currency (Section 6.3). The value is parsed as decimal text and rounded half away from zero to 3 decimal places. Absent revenue is NULL, never 0. |
| Minor-unit exponent | Taken from a table in packages/shared/src/currencies.ts covering every currency supported by Intl.supportedValuesOf('currency') (0 for JPY, KRW and other zero-decimal currencies; 3 for BHD, KWD and other three-decimal currencies; 2 otherwise). Used for billing amounts, output rounding and formatting. |
| Input | Tracker revenue accepts a decimal string or a JSON number; it is parsed as decimal text (never through binary floating point). The bound is 0 ≤ amount ≤ 999,999,999,999.999 (12 integer digits, at most 3 fractional digits after rounding); negative, non-finite or larger amounts are invalid. The tracker drops invalid revenue client-side before sending (Section 7.6.2); the server rejects an event with invalid revenue with 400 $ is invalid (Section 8.14). Currency codes are uppercased and must be in the table. |
| Public API output | Revenue metrics are objects: { "value": 1234.56, "currency": "USD", "short": "$1.2K", "long": "$1,234.56" }. value is a JSON number in major units, rounded half away from zero to the currency's ISO 4217 exponent at output. |
| Other money in the dashboard API | Objects { "amount_minor": 14900, "currency": "USD" }. |
| Sums | Amounts of different currencies are never added without conversion. Conversion rules (revenue currency per site, exchange rates) are owned by Section 10. |
| Prices | Tier prices are integers in minor units in license_tiers.price_minor with currency USD (seed 5900, 14900 and 34900; Sections 5.10 and 5.12). |
Formatting (formatCurrency in packages/shared/src/format.ts):
| Form | Rule | Example |
|---|---|---|
| Long | Intl.NumberFormat('en-US', { style: 'currency', currency }), exponent per currency |
$1,234.56, ¥1,235, €0.50 |
| Short | Compact when the absolute major value is at least 1,000: one decimal place maximum, trailing .0 removed, suffix K, M, B, preceded by the currency symbol; below 1,000 identical to long without trailing .00 only when the value is whole |
$1.2K, $12M, $999.50, $12 |
4.7.2 Number formatting #
| Kind | Rule | Example |
|---|---|---|
| Counts in tables and tooltips | Full, en-US thousands separators |
12,345 |
| Counts in headline tiles and chart axes | Full below 10,000; compact from 10,000: one decimal maximum, trailing .0 removed, suffix K, M, B |
9,876, 12.3K, 1.2M |
| Percentages | Bounce rate and exit rate: whole percent. Conversion rates and group conversion rates: one decimal. Share-of-total percentages: whole percent. Values that round to zero but are above zero show <1%. |
42%, 2.4%, <1% |
| Ratios | views_per_visit: one decimal |
1.8 |
| Durations | Under 60 s: 42s. Under 1 h: 1m 24s. At least 1 h: 1h 5m. Zero: 0s. |
1m 24s |
| Scroll depth | Whole percent | 75% |
| Change versus comparison period | Whole percent with an arrow icon and a sign in text for assistive tech; previous value 0 and current above 0 shows New; both 0 shows 0%; magnitudes under 1% show <1% |
+12%, -8% |
| Missing value | An em dash — (never 0, never blank) |
— |
| Bytes | Binary units, one decimal: KB, MB, GB |
3.4 MB |
| Ordinals and plurals | Handled by Intl.PluralRules in the i18n helper |
1 site, 2 sites |
Helpers formatNumber, formatCompact, formatPercent, formatDuration, formatCurrency, formatDelta and formatBytes are the only formatting code used by the SPA, the email templates and the CSV export labels (CSV export itself writes raw numbers, Section 14).
4.8 Validation #
- Zod schemas live in
packages/shared/src/schemas/{domain}.ts, named{Thing}Schema, with inferred types exported as{Thing}. A schema used by a route is imported by both the Fastify route (for validation) and the SPA (for forms and typed clients). Fastify routes callschema.parsethrough a sharedvalidate(schema, data)helper that converts aZodErrorintovalidation_failed(4.4.2). - Object schemas are strict (
z.strictObject) except the ingestion payload, which is lenient. - String normalization happens in the schema:
trim(), Unicode NFC normalization, and rejection of control characters (U+0000 to U+001F and U+007F) except line breaks in fields documented as multi-line.NameandShortTextadditionally reject bidirectional control characters (U+202A to U+202E and U+2066 to U+2069) and zero-width characters (U+200B to U+200F). - Standard primitives, defined once in
packages/shared/src/schemas/primitives.tsand reused:
| Primitive | Rule |
|---|---|
Email |
Trimmed, lowercased, 3 to 254 characters, one @, domain part with at least one dot, no spaces, and passes the Zod email check |
Uuid |
Canonical lowercase UUID |
IsoInstant |
ISO 8601 with offset or Z; normalized to UTC |
LocalDate |
YYYY-MM-DD, a real calendar date |
TimeZone |
UTC or a member of Intl.supportedValuesOf('timeZone'), canonicalized through Intl.DateTimeFormat(...).resolvedOptions().timeZone; offset strings and legacy aliases are rejected (Section 4.6) |
CurrencyCode |
Three uppercase letters present in the currency table |
Url |
Absolute http or https URL, at most 2,000 characters |
Name |
1 to 100 characters after trimming |
ShortText |
0 to 500 characters |
LongText |
0 to 5,000 characters, line breaks allowed |
Hostname |
Lowercase, 1 to 253 characters, labels of 1 to 63 characters of letters, digits and hyphens, not starting or ending with a hyphen, at least one dot unless the value is localhost; internationalized names are converted to punycode before validation; a single leading www. is removed, and a port, scheme or path is rejected |
Cidr |
IPv4 or IPv6 address with optional prefix length |
Color |
#RRGGBB hexadecimal |
PageLimit |
Integer 1 to 200 |
- Any field limit that an owning section does not specify defaults to
Namefor names,ShortTextfor short free text andLongTextfor descriptions. - Server-side checks that need the database (uniqueness, existence, limits) throw
AppErrorcodes from 4.4.3; they are not expressed as Zod issues. - The same schema validates a form on blur and on submit in the SPA; the server revalidates every request regardless of client validation.
- Passwords are validated by the rules in Section 11. Passwords are never trimmed, normalized or logged.
4.9 Logging #
- All logging uses pino JSON on stdout, one object per line. The server logger is the Fastify logger; modules use child loggers (
logger.child({ module: 'billing' })). Fastify is created withdisableRequestLogging: true, and thereqserializer is replaced by one that emits onlymethod,route(the pattern) andreq_id. The default request and response log lines, which would carry the raw URL (share and reset tokens travel in paths), the remote address and the host, therefore never exist. One access line per request is written explicitly by the server's own hook with the fields of rule 2. - Standard fields on every line:
level(string label),time(ISO 8601 UTC),msg,module, and when in a request contextreq_id,method,route(the route pattern, not the raw URL),status,duration_ms, plusaccount_user_id,team_idandsite_id(UUIDs) when known. The field is namedaccount_user_idso it can never be confused with the visitor hashuser_id; the visitor hash andsession_idare never logged. - Levels:
fatalthe process is exiting;erroran operation failed unexpectedly (always includeserrwith stack);warndegraded or rejected (rate limits, dependency retries, config fallbacks);infolifecycle events and one access line per dashboard or API request;debugdevelopment detail;tracenever enabled in production. Default level is set byLOG_LEVEL. POST /api/eventis not access-logged atinfo. It logs only errors and, atdebug, a 1-in-1,000 sample of accepted requests with no payload fields.- Redaction (pino
redact, applied to every logger) covers these header paths:req.headers.authorization,req.headers.cookie,req.headers['x-csrf-token'],req.headers['x-forwarded-for'],req.headers['x-real-ip'],req.headers['cf-connecting-ip'],req.headers['true-client-ip'],req.headers.forwarded,req.headers['x-quietstat-client-ip'],req.headers['x-quietstat-proxy-key'],req.headers['user-agent'],req.headers.referer,req.headers.origin,res.headers['set-cookie']. For body and context properties pino does not support "any depth", so the list is explicit paths plus wildcard paths one and two levels deep (*.password,*.*.password, and the same fortoken,secret,api_key,ip,user_agent,email_bodyandauthorization). The redaction list is a constant inapps/server/src/lib/logger.ts. A unit test logs a fixture object containing every forbidden key at the top level, one level and two levels deep and asserts that none of the values survive in the output. Request serializers drop the remote address and the user-agent entirely rather than masking them. - Never logged under any level: raw IP addresses, User-Agent strings, event payloads, URLs from tracked sites, referrers, email bodies, password hashes, TOTP secrets, API keys, share tokens, Stripe secrets and card data. The only deliberate exceptions are the one-time setup token on first start (Section 16.3.5) and, for a self-hosted deployment with no SMTP configured, the
action_urlfield of theemail_not_sentline (Section 12.9.6) so the owner can complete the flow by hand. - Emails are logged as a template name plus the recipient domain only.
- Business audit records (who did what to which resource) go to the
audit_logtable (columns in Section 5, event catalogue in Section 11.20), not to the application log. The application log is operational and may be discarded after 30 days; the audit log is retained perAUDIT_RETENTION_MONTHS(Section 16.4; Section 1.2, row 35). - Each job run logs
job_startedandjob_finishedwithjob,attempt,duration_ms, andjob_failedwith the error. - Client-side errors are never sent to a third party. The SPA logs to the browser console only.
4.10 Feature flags and edition checks #
- There is no feature-flag service and no per-user or per-tier flags. Behavior differences come from exactly two sources: the edition (
EDITION) and the presence of optional configuration (SMTP, Stripe, Google). - The server builds one immutable
RuntimeConfigat start. Its public subset is:
{
"product_name": "Quietstat",
"edition": "hosted",
"base_url": "https://app.example.com",
"registration_mode": "open",
"features": { "licensing": true, "stripe": true, "google_import": false, "email": true },
"geoip_attribution": "IP geolocation by DB-IP",
"terms_url": null,
"privacy_url": null,
"dpa_url": null,
"hosting_region_label": "EU (Germany)"
}- The public subset is the base of the runtime bootstrap, which is delivered two ways: inlined into
index.htmlas a non-executable<script type="application/json" id="qs-bootstrap">element when the server serves the SPA (allowed by the CSP because it is data, not code; the JSON is serialized with<,>,&, U+2028 and U+2029 escaped as\u003c,\u003e,\u0026,\u2028and\u2029), and throughGET /api/app/v1/bootstrap(single envelope) for refresh. Section 11.2 owns the complete response, which MUST contain every key of the subset above unchanged (product_name,edition,base_url,registration_mode,features,geoip_attribution,terms_url,privacy_url,dpa_url,hosting_region_label) plussetup_required,banner,csrf_token,userand the password-policy fields; the response is never cacheable when a session cookie is present. Secrets never appear in it.features.licensingandfeatures.stripeare true only whenEDITION=hosted(and, forstripe, whenSTRIPE_SECRET_KEYis set);features.google_importis true whenGOOGLE_CLIENT_IDis set;features.emailis true whenSMTP_HOSTis set. - Server code checks the edition with
requireEdition('hosted'), a route-level guard. In the self-hosted edition, hosted-only routes (licensing, checkout, codes, usage,/admin, the Stripe webhook) are not registered at all, so they return 404not_found. - The SPA reads the config through a
useRuntimeConfig()hook and renders hosted-only navigation and pages only when the matching feature is true. Hosted-only SPA routes (4.13) redirect to/in self-hosted. - Feature code MUST NOT inspect the license tier except in the limit checks owned by Section 15 (site count, billable events). No UI or API capability may differ between Solo, Pro and Agency.
- The tracker and
POST /api/eventbehave identically in both editions except for the cap and license checks (hosted).
4.11 Internationalization #
- v1 ships English (
en-US) only. There is no language selector and noAccept-Languagenegotiation. The<html lang="en">attribute is fixed. - All user-visible strings in
apps/webare externalized inapps/web/src/i18n/locales/en.json(flat object, keys per 4.1) and read through a typed helpert(key, params?)inapps/web/src/i18n/index.ts. Interpolation uses{name}placeholders; plurals use the keys{key}_oneand{key}_otherchosen byIntl.PluralRules('en-US'); the helper's key type is generated fromen.jsonso a missing key is a compile error. - An ESLint rule (
react/jsx-no-literalswithnoStrings: true, allowing only punctuation and symbol-only strings) forbids literal text in JSX. - Server-generated text: API error messages come from
ERROR_CATALOG(4.4.3); email subject and body strings live inapps/server/src/mail/strings.ts(one exported function per template, Section 12). Neither is inline in handlers. - Dates, numbers, currency and lists are formatted through the helpers in 4.6 and 4.7 using
en-US, never by manual string concatenation. - Technical identifiers shown to users (event names, page paths, property values, goal names, sources) are displayed verbatim and are never translated.
- Text direction is left-to-right. Layout uses logical CSS properties (
margin-inline-start,padding-inline-end) so right-to-left can be added without layout rewrites.
4.12 Code style, linting, formatting and Git #
4.12.1 TypeScript and style #
| Topic | Rule |
|---|---|
| Modules | ESM only ("type": "module"), verbatimModuleSyntax, moduleResolution: "bundler" for web and tracker, "nodenext" for the others |
| Compiler | strict, noUncheckedIndexedAccess, exactOptionalPropertyTypes, noImplicitOverride, noFallthroughCasesInSwitch; no any (lint error @typescript-eslint/no-explicit-any); unknown plus Zod parsing at every boundary |
| Exports | Named exports only; default exports allowed solely for config files that tools require |
| Errors | Throw AppError (4.4.2) or Error subclasses; never throw strings; no empty catch blocks |
| Async | @typescript-eslint/no-floating-promises and no-misused-promises are errors |
| Environment | process.env is read only in apps/server/src/config/; everything else receives a typed config object |
| Time | No Date.now() or new Date() outside the clock helper (4.6) in server code |
| Randomness | Security-sensitive randomness only from node:crypto |
| Size | A file over 400 lines or a function over 80 lines is split unless the owning section's SQL builders are the reason |
| Comments | Explain why, not what; each exported function has a one-line TSDoc comment |
| Imports | Sorted by eslint-plugin-import order rules: node built-ins, external, @quietstat/*, relative; no cycles (import/no-cycle error); package-boundary rule in 3.2 enforced by the custom rule in tools/eslint-rules/ |
| SQL | Parameterized queries only; string-concatenated SQL is forbidden outside @quietstat/query, where identifiers come from allow-lists |
| Formatting | Prettier: 100 columns, 2 spaces, single quotes, semicolons, trailing commas all, LF line endings; .editorconfig mirrors it |
| PHP | WordPress Coding Standards via PHPCS in integrations/wordpress |
Root scripts (names are canonical): pnpm dev (server, worker and web in watch mode), pnpm build, pnpm lint, pnpm typecheck, pnpm test (Vitest), pnpm test:e2e (Playwright), pnpm format, pnpm format:check, pnpm db:generate (drizzle-kit), pnpm db:migrate (runs the migrate command locally). CI runs format:check, lint, typecheck, test, build, in that order; a failure blocks merge.
4.12.2 Git and release conventions #
| Topic | Rule |
|---|---|
| Default branch | main, protected; changes arrive through pull requests with passing CI |
| Branches | {type}/{scope}-{slug} where type is feat, fix, chore, docs, refactor, test or perf |
| Commits | Conventional Commits: type(scope): imperative summary at most 72 characters; body explains why; breaking changes add ! and a BREAKING CHANGE: footer |
| Scopes | server, worker, web, shared, query, tracker, wordpress, gtm, deploy, db, docs, repo |
| Merge style | Squash merge; the squashed message follows the commit rule |
| Migrations | One migration file per change, never edited after merge; the file name carries a sequence number and a short description |
| Versioning | Product releases are tagged vMAJOR.MINOR.PATCH (semantic versioning) and published as the image tags MAJOR.MINOR.PATCH, MAJOR.MINOR and latest. @quietstat/tracker, the WordPress plugin and the GTM template have independent semantic versions recorded in their own manifests |
| Changelog | CHANGELOG.md in Keep a Changelog format, updated in the pull request that changes behavior |
| Secrets | Never committed; .env is git-ignored and only .env.example is tracked |
| Pull request size | Prefer under 600 changed lines excluding generated files and migrations |
| Tracker script release | The tracker script served by the app (/js/*) is built with the image and ships in the vX.Y.Z image release; the same sources are published separately as @quietstat/tracker by a tracker-v* tag. A release fails CI if any tracker byte budget of Section 7.8.1 is exceeded |
| Release automation | Pushing a tag vX.Y.Z builds and publishes the image; tracker-v* publishes @quietstat/tracker to npm with provenance; wordpress-v* deploys the plugin to the WordPress.org SVN repository; gtm-v* attaches the Tag Manager template as a release asset. Each release job requires a manual approval step in CI |
4.13 App shell and navigation map #
4.13.1 Shell #
| Region | Content |
|---|---|
| Skip link | First focusable element: "Skip to main content" |
| Top bar (all authenticated pages) | Logo or wordmark (links to /sites); team switcher (shown when the user belongs to more than one team or is a guest); site switcher (shown inside a site); usage pill (hosted, shown at 80% usage or above); theme toggle; user menu (Profile, Security, Notifications, Teams, Operator console for super admins, Sign out) |
Site sub-navigation (inside /sites/:domain/...) |
Tabs: Dashboard, Realtime, Funnels, Journeys, Settings (Settings only for roles with any settings access) |
| Banners (under the top bar, full width, dismissible only where stated) | Unlicensed (hosted), cap warning at 80%, cap reached at 100%, ingestion paused at 110%, email not configured (admins only), impersonation (not dismissible), maintenance |
| Main | The routed page |
| Footer | Product name and version, geolocation attribution (geoip_attribution from the runtime config, Section 4.10), Terms, Privacy and DPA links when configured |
| Settings layout | Left vertical menu (collapses into a select on narrow screens) plus content column, max width 768 px |
| Public layout | Centered card, no top bar, for auth pages; the shared dashboard layout for /share/:token and /embed/:token |
4.13.2 Guard vocabulary #
| Guard | Meaning (permission matrix in Section 11) |
|---|---|
public |
No session needed |
auth |
Any signed-in user |
site-viewer |
Team owner, admin, editor or viewer of the site's team, or any guest of the site |
site-editor |
Team owner, admin or editor, or a guest_editor of the site |
team-admin |
Team owner or admin |
team-owner |
The team owner |
super-admin |
Email in SUPER_ADMIN_EMAILS (hosted only) |
A route whose guard fails renders the 403 page ("You do not have access to this page") for signed-in users, redirects unauthenticated users to /login?return_to=<path>, and renders the 404 page for resources the user cannot know exist (4.4.2, rule 9).
4.13.3 Routes #
| Path | Page | Purpose | Guard | Edition |
|---|---|---|---|---|
/setup |
First-run setup | Shown while no user exists (the bootstrap response reports setup_required); creates the first account with the one-time setup token; redirects to /login once a user exists |
public |
Both |
/login |
Sign in | Email and password; second step for TOTP or recovery code | public |
Both |
/login/2fa |
Second factor | Six-digit code entry (or recovery code) after the password step | public |
Both |
/register |
Sign up | Create account and personal team; honors REGISTRATION_MODE and is unavailable until setup is complete |
public |
Both |
/forgot-password |
Forgot password | Request a reset email | public |
Both |
/reset-password/:token |
Reset password | Set a new password | public |
Both |
/verify-email/:token |
Verify email | Confirms an email address (signup or change) | public |
Both |
/invitations/:token |
Accept invitation | Shows team, role and inviter; accepts after sign-in or signup | public |
Both |
/share/:token |
Shared dashboard | Read-only dashboard, password prompt when protected | public |
Both |
/embed/:token |
Embedded dashboard | Chrome-less variant for iframes | public |
Both |
/public/:domain |
Public dashboard | Read-only dashboard of a site whose public dashboard is enabled (site_settings.public_dashboard) |
public |
Both |
/unsubscribe/:token |
Unsubscribe | One-click unsubscribe from email reports or spike alerts, with resubscribe | public |
Both |
/ |
Home | Redirects to /sites, or to /login when signed out |
public |
Both |
/sites |
Sites | Sites list with search, sort by visitors or domain, per-site sparkline and visitors | auth |
Both |
/all-sites |
All sites | Consolidated dashboard across every site the user can read | auth |
Both |
/sites/new |
Add site | Domain, time zone, team; then goes to setup | team-admin (or auth when the user has no team yet) |
Both |
/sites/:domain/setup |
Setup | Snippet, framework guide chooser, WordPress and GTM instructions, "waiting for first pageview" detector | site-editor |
Both |
/sites/:domain/setup/:guide |
Setup guide | One framework or platform guide; :guide is a guide slug from Section 7.14 |
site-editor |
Both |
/sites/:domain |
Dashboard | Headline metrics, chart, comparison, reports, filters, segments | site-viewer |
Both |
/sites/:domain/realtime |
Realtime | Live visitors (last 5 minutes) and sources, pages and goals for the last 30 minutes | site-viewer |
Both |
/sites/:domain/:report |
Report details | Expanded report with search, "Load more" and extra columns; :report is one of sources, campaigns, pages, locations, devices, goals, properties (Section 9.6.4). The literal segments realtime, journeys, funnels, settings and setup take precedence over :report |
site-viewer |
Both |
/sites/:domain/funnels/:funnelId |
Funnel | Funnel visualization, comparison, revenue per step | site-viewer |
Both |
/sites/:domain/journeys |
User journeys | Sankey of visitor paths | site-viewer |
Both |
/sites/:domain/settings |
Settings home | Redirects to the first settings page the user may open | site-viewer |
Both |
/sites/:domain/settings/general |
General | Name, time zone, delete or reset data (delete requires team-admin) |
site-editor |
Both |
/sites/:domain/settings/tracking |
Tracking | Snippet, tracker options, proxy endpoint help, installation check | site-editor |
Both |
/sites/:domain/settings/goals |
Goals | Create, edit, delete goals | site-editor |
Both |
/sites/:domain/settings/properties |
Properties | Custom property allowlist | site-editor |
Both |
/sites/:domain/settings/funnels |
Funnels | Create, edit, delete funnels | site-editor |
Both |
/sites/:domain/settings/segments |
Segments | Manage site-wide and personal segments | site-viewer |
Both |
/sites/:domain/settings/annotations |
Annotations | List, edit and delete annotations | site-viewer (own); site-editor (all) |
Both |
/sites/:domain/settings/exclusions |
Exclusions | IP and page exclusions, datacenter filter toggle, filtered-traffic counters | site-editor |
Both |
/sites/:domain/settings/shared-links |
Shared links | Create links, passwords, segment limits, embed code | site-editor |
Both |
/sites/:domain/settings/notifications |
Email reports and alerts | Email report subscriptions and spike notifications | site-viewer (own subscriptions); site-editor (others and spike settings) |
Both |
/sites/:domain/settings/access |
Site access | Guests of this site | team-admin |
Both |
/sites/:domain/settings/data |
Import and export | CSV export panel (every role that can view the site, including guests) and history; the CSV import, GA4 import and import history controls render only for team-admin (Section 14) |
site-viewer |
Both |
/account/profile |
Profile | Name, email change, delete account | auth |
Both |
/account/security |
Security | Password, two-factor, recovery codes, active sessions | auth |
Both |
/account/notifications |
My notifications | All my email report subscriptions across sites | auth |
Both |
/teams |
Teams | Teams the user belongs to; create a team | auth |
Both |
/teams/:teamId/settings |
Team settings | Name, require 2FA, transfer ownership, delete team | team-admin (transfer and delete: team-owner) |
Both |
/teams/:teamId/members |
Members | Members, roles, invitations | team-admin |
Both |
/teams/:teamId/api-keys |
API keys | Create, revoke, copy request examples | team-admin |
Both |
/teams/:teamId/billing |
Billing | Tier, usage meter, purchases, invoices, redeem code, upgrade. View needs billing.view; checkout, upgrade and redeem actions need billing.manage (Section 11 matrix) |
team-admin to view; actions team-owner |
Hosted |
/teams/:teamId/billing/invoices |
Invoices | Invoice and credit-note list with PDF downloads | team-admin (billing.view); PDF downloads team-owner (billing.manage) |
Hosted |
/activate |
Activate | Choose a tier to buy or enter a code | team-owner |
Hosted |
/activate/return |
Payment return | Waits for the webhook, then shows success or failure | team-owner |
Hosted |
/redeem |
Redeem code | Deep link that opens the activate page with a code prefilled from the code query parameter (the SPA moves it into memory and removes it from the address bar at once) |
team-owner |
Hosted |
/admin |
Operator overview | Totals and health | super-admin |
Hosted |
/admin/users |
Users | Search, view, suspend, impersonate (read-only) | super-admin |
Hosted |
/admin/teams |
Teams | Search and view teams, sites and licenses | super-admin |
Hosted |
/admin/licenses |
Licenses | List, grant, revoke | super-admin |
Hosted |
/admin/code-batches |
Code batches | Generate batches, export CSV, revoke codes | super-admin |
Hosted |
/admin/purchases |
Purchases | Purchases, refunds | super-admin |
Hosted |
/admin/usage |
Usage | Billable events per team and month | super-admin |
Hosted |
/admin/audit-log |
Audit log | Search and view audit entries | super-admin |
Hosted |
* |
Not found | 404 page with link home | public |
Both |
Rules:
- Hosted-only routes redirect to
/in the self-hosted edition (4.10). - A site route whose
:domaindoes not exist or is not visible renders the 404 page. After a first login the app opens/sites; when the user has exactly one site it opens that site's dashboard. - Dashboard state lives in the query string (the parameters defined in Section 9.17:
period,comparison,fand the report-specific keys) so any view is linkable and bookmarkable. Changing it replaces history entries within 500 ms of the last change and pushes a new entry on explicit period or filter commits. - Every route module is lazy-loaded; the route table is declared once in
apps/web/src/routes/index.tsxand mirrored by the table above. - Page titles follow
{Page} · {Site or Team} · {PRODUCT_NAME}; a document title is set on every route change and announced by moving focus to the page'sh1. - Dashboard keyboard shortcuts (single keys, ignored while typing in a field and when a modifier key is held) are defined only in Section 9.12.3, together with the period tokens they map to.
?opens the shortcut list (4.14.3KeyboardShortcutsDialog).
4.14 UI design system #
All tokens are CSS custom properties defined in apps/web/src/styles/tokens.css on :root (light) and :root[data-theme="dark"] (dark), surfaced to Tailwind through its theme configuration so utilities such as bg-surface and text-muted exist. Theme choice is system, light or dark, stored in localStorage under qs:theme (default system, which follows prefers-color-scheme) and applied before first paint by /assets/theme-init.js, a small external, non-module, render-blocking <script src> in the <head> of index.html, loaded before the app bundle. There is no inline script anywhere in the app (the CSP is script-src 'self', Section 17.5.2), which is why the theme bootstrap is a static file. Shared, embedded and public dashboards read and write no browser storage: they take the theme from the theme query parameter (light, dark or system) and otherwise follow prefers-color-scheme.
4.14.1 Color tokens #
| Token | Light | Dark | Use |
|---|---|---|---|
--color-bg |
#F8FAFC |
#0B0F1A |
Page background |
--color-surface |
#FFFFFF |
#121826 |
Cards, panels, inputs |
--color-surface-2 |
#F1F5F9 |
#1B2335 |
Table stripes, hovered rows, secondary panels |
--color-border |
#E2E8F0 |
#263046 |
Dividers, card borders |
--color-border-strong |
#94A3B8 |
#5B6A87 |
Input borders (at least 3:1 against surface) |
--color-text |
#0F172A |
#E5E9F2 |
Primary text |
--color-text-muted |
#475569 |
#A7B0C3 |
Secondary text |
--color-text-subtle |
#5B6678 |
#8893A8 |
Hints, placeholders; at least 4.5:1 on surface and surface-2 |
--color-brand |
#4338CA |
#818CF8 |
Primary actions, links, active nav |
--color-brand-hover |
#3730A3 |
#A5B4FC |
Hover state of brand elements |
--color-brand-subtle |
#EEF2FF |
#1E2447 |
Selected rows, active tab background |
--color-on-solid |
#FFFFFF |
#0B0F1A |
Text on solid brand and status backgrounds |
--color-success |
#15803D |
#4ADE80 |
Success text, icons and solid fills |
--color-warning |
#B45309 |
#FBBF24 |
Warning |
--color-danger |
#B91C1C |
#F87171 |
Errors, destructive actions |
--color-info |
#1D4ED8 |
#60A5FA |
Informational |
--color-success-subtle / -warning-subtle / -danger-subtle / -info-subtle |
#DCFCE7 / #FEF3C7 / #FEE2E2 / #DBEAFE |
#0F2E1B / #33260A / #3A1414 / #112849 |
Banner and badge backgrounds |
--color-focus |
#4338CA |
#A5B4FC |
Focus ring |
--color-overlay |
rgb(15 23 42 / 0.5) |
rgb(0 0 0 / 0.6) |
Modal backdrop |
Chart palette (--chart-1 to --chart-6, categorical, distinguishable under common color-vision deficiencies and also differentiated by line style or marker): light #4F46E5, #0E7490, #B45309, #BE185D, #15803D, #6D28D9; dark #818CF8, #22D3EE, #FBBF24, #F472B6, #4ADE80, #A78BFA. The primary series (visitors) uses --chart-1; the comparison series uses the same hue at 45% opacity with a dashed stroke. Gridlines use --color-border. Funnel bars use --chart-1 for reached and --color-surface-2 for dropped.
Contrast requirement: every text and icon color pairing used in the tokens above meets 4.5:1 (3:1 for large text, graphics and input borders). Section 18 includes an automated check that computes these ratios from tokens.css.
4.14.2 Type, spacing, shape and motion #
| Group | Tokens |
|---|---|
| Font families | Sans: Inter Variable self-hosted as a WOFF2 file in apps/web/public/fonts/ (with its license text), falling back to system-ui, -apple-system, "Segoe UI", Roboto, sans-serif. Mono: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace. No external font requests. |
| Type scale (size / line height) | xs 12/16, sm 14/20, base 16/24, lg 18/28, xl 20/28, 2xl 24/32, 3xl 30/36, 4xl 36/40 |
| Weights | 400 body, 500 labels and table headers, 600 headings and stat values, 700 not used |
| Usage | Body text and form controls base; dense tables sm; headline stat values 3xl at weight 600 with font-variant-numeric: tabular-nums; page h1 2xl; card titles lg; captions xs |
| Spacing scale | 4 px base unit: 0, 4, 8, 12, 16, 20, 24, 32, 40, 48, 64 (Tailwind steps 0 to 16) |
| Radii | sm 4 px, md 8 px (default for controls and cards), lg 12 px (modals), full 9999 px (pills, avatars) |
| Elevation | shadow-sm (cards: 0 1px 2px rgb(0 0 0 / 0.06)), shadow-md (popovers, menus: 0 4px 12px rgb(0 0 0 / 0.12)), shadow-lg (modals: 0 12px 32px rgb(0 0 0 / 0.2)); dark theme relies on borders more than shadows |
| Z-index | base 0, sticky 10, dropdown 20, banner 30, modal 40, toast 50 |
| Motion | 120 ms for hover and focus transitions, 200 ms for panels and modals, ease-out; chart draw animation 300 ms. Under prefers-reduced-motion: reduce all transitions are 0 ms and chart animation is disabled. |
| Layout | Page content max width 1280 px, centered; 16 px side padding below 640 px, 24 px from 640 px, 32 px from 1024 px; settings content max width 768 px |
| Focus ring | 2 px solid --color-focus with a 2 px offset, on every focusable element; never removed without replacement |
| Icons | One open-license icon set bundled as tree-shaken SVG components; 16 px in dense UI, 20 px default, 24 px in navigation; decorative icons aria-hidden, meaningful icons have labels |
4.14.3 Component inventory #
Every component below lives in apps/web/src/components/, is keyboard-operable, supports both themes and has all listed states. No page may define a visually equivalent component locally.
| Component | Variants and states | Accessibility notes |
|---|---|---|
Button |
primary, secondary, ghost, danger; sizes sm (32 px), md (40 px), lg (48 px); states default, hover, active, focus, disabled, loading (spinner replaces icon, width kept) | Native button; aria-busy while loading; minimum 24 x 24 px target (md default is 40 px) |
IconButton |
same variants; square | Requires aria-label (compile-time required prop) |
Link |
inline, nav, external (opens in the same tab unless the page says otherwise; external links show an icon and rel="noopener noreferrer") |
Underline on hover and focus |
TextField, Textarea, PasswordField |
default, error, disabled, readonly; with label, hint, error text, prefix, suffix; password has a show/hide toggle | Label always visible; aria-describedby for hint and error; aria-invalid |
NumberField |
integer or decimal, min, max, step | Native input type="text" inputmode="numeric" with parsing |
Select |
native select styled | Native semantics |
Combobox |
single and multi value; async options; creatable | ARIA 1.2 combobox pattern; used for filter values and property keys |
Checkbox, RadioGroup, Switch |
checked, indeterminate, disabled | Switch uses role="switch" |
FormField |
Wraps a control with label, required marker, hint and error | Generates ids and associations |
DateRangePicker |
Presets list plus two-month calendar, comparison selector | Keyboard grid navigation; announces the selected range |
Tabs |
horizontal, with counts | role="tablist" pattern; arrow-key navigation |
DropdownMenu, Popover, Tooltip |
placements auto-flip; tooltip on hover and focus with 400 ms hover delay | Escape closes; focus returns to trigger; tooltips never carry essential information |
Modal, ConfirmDialog, Drawer |
sizes sm, md, lg; destructive confirm requires typing the resource name | Focus trap, aria-modal, Escape closes (except while a mutation is pending), focus returns |
Toast |
success (4 s), info (6 s), warning (8 s), error (stays until dismissed); max 3 visible, stacked bottom-right (bottom-center below 640 px) | aria-live="polite" (error: assertive); action button allowed |
Banner |
info, success, warning, danger; with title, text, action and optional dismiss | role="status" (danger: role="alert") |
Badge |
neutral, brand, success, warning, danger | Text label always present |
Card, PageHeader, SettingsLayout, EmptyState, Skeleton, Spinner |
See 4.17 | |
Table |
sortable columns, sticky header, row actions menu, selectable rows, density compact and normal | Real table markup, aria-sort, caption or aria-label |
ReportCard |
Top-list report: tabs for dimension variants, rows with proportional bar, metric columns, "Details" link | Rows are a labeled list/table; bar is decorative |
StatTile |
label, value, change versus comparison, sparkline; states loading, empty, error | Value and change read as text: "Visitors 12,345, up 12% versus previous period" |
TimeSeriesChart |
Area or line over time; granular tooltip; annotation markers; comparison series; click-and-drag selects a range | Hidden accessible table alternative (4.15); keyboard focus on points with arrow keys |
BarChart, FunnelChart, SankeyDiagram |
Funnel steps with drop-off; journeys with links | Each exposes a data table toggle |
FilterBar, FilterChip |
Chips with dimension, operator and value; add, edit, remove; "Save as segment" | Chips are buttons with full-text labels |
SegmentPicker |
Lists personal and site segments | Combobox pattern |
CodeBlock, CopyField |
Monospace, copy button with confirmation, optional language label | Copy button announces "Copied" |
FileDropzone |
Click or drag; shows file name, size and validation | Keyboard-activatable; drag has a button alternative |
ProgressBar |
Determinate with label; used for usage meter and imports | role="progressbar" with values |
Stepper |
Setup steps | Ordered list with aria-current="step" |
Avatar |
Initials on a deterministic color; image never fetched from third parties | Decorative with adjacent name |
ThemeToggle, TeamSwitcher, SiteSwitcher, UserMenu |
Shell controls | Menu patterns above |
KeyboardShortcutsDialog |
Lists the shortcuts of Section 9.12.3 | Opens with ? |
Pagination |
"Load more" button for cursor lists and for offset report details | Announces new item count |
4.15 Accessibility (WCAG 2.2 AA) #
The dashboard, the shared and embedded dashboards, and every auth page conform to WCAG 2.2 level AA. Requirements:
- Perceivable. Color contrast per 4.14.1 (1.4.3, 1.4.11). Color is never the only means of conveying information: chart series also differ by stroke style and marker, deltas carry arrows and signs, statuses carry text. Text can be resized to 200% and content reflows at 320 CSS px wide without two-dimensional scrolling, except data tables and charts that scroll horizontally inside their container (1.4.4, 1.4.10). Text spacing overrides do not break layout (1.4.12). Images have alternative text; decorative images use empty alt.
- Charts. Every chart has a visible title, a text summary available to screen readers (such as "Visitors per day, 7 days, total 12,345, highest on Nov 3") and a "View as table" toggle that renders the same data in an accessible table. Hover-only information is also available on focus. Tooltips are dismissible with Escape and persistent while hovered (1.4.13).
- Operable. Everything is operable with a keyboard (2.1.1) with no traps (2.1.2); single-character shortcuts can be turned off in the shortcuts dialog (2.1.4); focus order follows reading order; the focus indicator is always visible and never hidden by sticky headers (2.4.7, 2.4.11); a skip link and landmarks (
header,nav,main,footer) are present on every page; page titles and the focusedh1change on each route (2.4.2, 2.4.3). - Pointer targets. Interactive targets are at least 24 x 24 CSS px, with spacing exceptions only for inline text links (2.5.8). Drag interactions (chart range selection, reordering funnel steps) have single-pointer alternatives: date inputs and move-up / move-down buttons (2.5.7).
- Timing. No time limits except session expiry, which warns 2 minutes before and allows extension by any interaction (2.2.1). Auto-refreshing views (realtime) can be paused, and refresh never moves focus.
- Understandable.
lang="en"; consistent navigation and identification (3.2.3, 3.2.4); consistent placement of help and support link in the footer (3.2.6). Form errors are associated with fields, summarized at the top of the form with links to the first invalid field, and describe how to fix the problem (3.3.1, 3.3.3). Previously entered information is not requested again within a flow (3.3.7). Authentication does not rely on cognitive tests: password managers and pasting into password and TOTP fields are allowed (3.3.8). - Robust. Native elements first; ARIA only where needed and per the WAI-ARIA Authoring Practices; status messages use live regions (4.1.3); toasts and banners as in 4.14.3. Valid, unique ids.
- Motion and theming.
prefers-reduced-motionandprefers-color-schemeare honored;forced-colorsmode keeps focus rings and borders visible using system colors. - Verification. Playwright specs run axe-core on every route in 4.13.3 in both themes at 360 px and 1280 px widths; the build fails on any serious or critical violation. A manual checklist (keyboard-only pass and one screen reader pass per release) is part of Section 18.
4.16 Responsive behavior #
| Breakpoint | Range (CSS px) | Layout behavior |
|---|---|---|
base (mobile) |
0 to 639 | Single column; top bar shows logo, site switcher and a menu button that opens a Drawer with the remaining controls; site sub-navigation scrolls horizontally; stat tiles in a 2-column grid; report cards stacked; tables keep their columns but hide secondary metric columns (each report defines its primary column set); filters open in a Drawer; modals become full-screen sheets; period picker presets only (custom range in a sheet) |
sm |
640 to 767 | As mobile with 24 px side padding; toasts bottom-right |
md |
768 to 1023 | Two-column report grid; settings menu becomes a left column; stat tiles in one row of 5 with compact numbers |
lg |
1024 to 1279 | Full top bar; report grid two columns; chart height 320 px |
xl |
1280 and up | Content centered at max 1280 px; report grid two columns; chart height 360 px |
Touch: targets stay at least 40 px high for primary actions at every breakpoint. Hover-only affordances have tap or focus equivalents. No feature is desktop-only; the embed layout is fluid down to 320 px.
4.17 Loading, empty and error state patterns #
4.17.1 Loading #
- Data regions show a
Skeletonplaceholder matching the final layout (tile, chart area, table rows) after a 150 ms delay (nothing is shown before that), and keep it for at least 300 ms to avoid flicker. - When refetching with existing data (period or filter change), the old data stays visible at 60% opacity with
aria-busy="true"on the region until the new data arrives. - Buttons show a
loadingstate during mutations and are disabled; forms cannot be submitted twice. - Routes show a full-page
Spinneronly while the route chunk or the session is loading, and only after 300 ms. - TanStack Query defaults:
staleTime30 s,gcTime5 min,refetchOnWindowFocustrue for dashboard queries and false for settings forms,retry2 times with 1 s then 2 s delays for GET requests on network errors and 5xx responses, no retry on 4xx, no retry for mutations. Realtime queries userefetchInterval10 s and the current-visitors pill 30 s; both pause when the tab is hidden. - Mutations are not optimistic, except toggles (switches) that revert with an error toast on failure.
4.17.2 Empty states #
Every list and report has an EmptyState with an icon, one-sentence explanation and, where it exists, one primary action.
| Situation | Message pattern | Action |
|---|---|---|
| New site with no data yet | "Waiting for your first visit." with the install status | "Open setup instructions" |
| Report empty for the period | "No data for this period." | "Change period" |
| Report empty because of filters | "Nothing matches your filters." | "Clear filters" |
| Zero sites | "Add your first site to start collecting data." | "Add site" |
| Zero goals, funnels, segments, annotations, shared links, API keys, exclusions, reports | One sentence naming the object and why it is useful | "Create {object}" for roles allowed; otherwise no button |
| Not permitted to create | Same message without the action | None |
| Unlicensed (hosted) | "Activate a license to start collecting events." | "Activate" for owners; otherwise "Ask the team owner to activate" |
| Feature needs unconfigured integration | "{Integration} is not configured on this server." | None |
4.17.3 Errors #
| Situation | Presentation |
|---|---|
Field validation (400 validation_failed) |
Mapped by details.fields[].path to the matching form field; unmatched paths appear in a form-level Banner; focus moves to the first invalid field |
| Query failure inside one widget | The widget shows an inline error with the standard message and a "Try again" button; other widgets keep working |
| Page-level load failure | Error page with the message, a "Try again" button and the request_id with a "Copy details" button |
401 unauthenticated or session_expired |
Redirect to /login?return_to=<current path>; unsaved form data is not preserved |
403 forbidden |
The 403 page in the shell |
404 not_found |
The 404 page in the shell |
429 rate_limited |
Error toast "Too many requests. Try again in {n} seconds." and the triggering control stays disabled for retry_after_seconds |
503 unavailable or query_timeout |
Inline error with "Try again"; a persistent offline Banner appears when the browser reports no network (navigator.onLine false) and disappears on recovery |
500 internal_error |
Toast with the message and request_id |
csrf_invalid |
The client refreshes the CSRF token once and retries the request one time; if it fails again it shows the standard message |
| Destructive actions | Always a ConfirmDialog; irreversible actions (delete site, delete team, purge data) require typing the resource name |
| Success feedback | A success toast for actions without a visible result; inline confirmation (changed state) otherwise |
5. Postgres Data Model #
PostgreSQL holds everything that is not an analytics event: accounts, teams, sites and their settings, goals and funnels, sharing, API keys, imports/exports, privacy salts, licensing, usage counters and the audit log. Analytics events live in ClickHouse (Section 6). This section owns every Postgres table, column, constraint, index and retention rule. Other sections reference tables by name and must not redefine them.
5.1 Principles and Conventions #
| Rule | Decision |
|---|---|
| Naming | Tables are plural snake_case; columns snake_case; constraints named <table>_<what>_<kind> (_key unique, _check check, _fkey foreign key, _idx non-unique index). |
| Primary keys | id uuid PRIMARY KEY DEFAULT uuidv7() using the built-in uuidv7() of the PostgreSQL line named in Section 3.1. Exceptions: site_settings (PK is site_id), exchange_rates (composite PK), site_domain_reservations (PK is domain). |
| Timestamps | timestamptz only, stored UTC. Every mutable table has created_at and updated_at (both NOT NULL DEFAULT now()); updated_at is maintained by the trigger in 5.2, never by application code. Append-only tables have created_at only. |
| Enumerations | Text columns with CHECK (col IN (...)), not Postgres ENUM types. Reason: adding a value to a CHECK is a transactional, rewrite-free migration, while ALTER TYPE ... ADD VALUE cannot be used in the same transaction as its first use and removing values is impossible. Every value set is mirrored by a Zod enum in @quietstat/shared; Drizzle columns use text('col', { enum: [...] }). |
| Money | Integer minor units in bigint/integer columns named *_minor, with an ISO 4217 char(3) currency column next to it. No floating point. Exception: analytics event revenue in ClickHouse is Decimal(18,3) in major units (Section 6.3). |
| Hashed secrets | Session cookies and API keys (≥ 256 bits of entropy) are stored as bytea SHA-256 digests (32 bytes); email-verification, password-reset, email-change and invitation tokens (≥ 256 bits) are stored as bytea HMAC-SHA-256 digests (32 bytes) keyed per Section 17.7. Passwords and share-link passwords use argon2id PHC strings (text). TOTP secrets and GA4 OAuth tokens are stored encrypted in the text format of Section 17.7.2 (AES-256-GCM, key derived from SECRET_KEY_BASE). Raw secrets are never stored, with these documented exceptions that must be re-displayed or looked up in clear: the shared-link token (192 bits, 5.7.1), the report and spike unsubscribe_token (256 bits, 5.7.2 and 5.7.3), the site proxy_key (160 bits, 5.5.1) and marketplace license codes (revocable bearer vouchers, Section 15.19). |
| Lowercase invariants | users.email, invitations.email, sites.domain and report recipient emails are stored lowercase and trimmed; a CHECK enforces it so a case-variant duplicate can never exist. |
| Tenant isolation | No Postgres row-level security. Every query goes through repository functions in apps/server that take a verified team_id/site_id scope from the permission layer in Section 11. |
| Deletes | Follow Section 4: hard delete for sites, goals, funnels, segments, annotations, shared links, API keys, invitations; soft delete for users and teams; licenses, purchases and code records are never deleted. |
| FK policy | ON DELETE CASCADE for pure children (owned configuration). ON DELETE SET NULL for "who created this" references. ON DELETE RESTRICT where silent cascade would be dangerous (a site's team, a goal used by a funnel, a segment used by a shared link, tiers). The exact behavior is shown on every FK below. |
| Drizzle mapping | uuid → uuid() with .default(sql\uuidv7()`); timestamptz→timestamp({ withTimezone: true, mode: 'date' }); bytea→ acustomType<{ data: Buffer }>; cidr→cidr(); text[]/uuid[]→.array(); jsonb→jsonb().$type()with the Zod-inferred type from@quietstat/shared; partial/expression indexes and CHECKs use index().on().where(sql`...`)/check()`. Constraint triggers, exclusion constraints and functions are hand-written SQL in custom migrations (5.13). |
| Schema location | Drizzle table definitions live in packages/shared/src/db/schema/*.ts (exported as @quietstat/shared/db, never imported by apps/web), so apps/server, apps/worker and the migrate command share one definition. SQL migrations live in packages/shared/migrations/postgres/. |
| Database roles | One application role owns the schema in self-hosted installs. Hosted installs may run migrations with a separate owner role; the spec requires nothing beyond ordinary table privileges for the runtime role. |
5.2 Shared Database Objects (migration 0000) #
CREATE EXTENSION IF NOT EXISTS btree_gist; -- used by the imports exclusion constraint (5.9)
CREATE FUNCTION app_set_updated_at() RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN NEW.updated_at := now(); RETURN NEW; END $$;
-- Number of keys in a JSON object; -1 when the value is not an object (so a CHECK can reject it).
CREATE FUNCTION app_jsonb_key_count(j jsonb) RETURNS integer LANGUAGE sql IMMUTABLE PARALLEL SAFE AS $$
SELECT CASE WHEN jsonb_typeof(j) = 'object' THEN (SELECT count(*)::integer FROM jsonb_object_keys(j)) ELSE -1 END
$$;
CREATE FUNCTION app_forbid_update() RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN RAISE EXCEPTION '% is append-only', TG_TABLE_NAME USING ERRCODE = '42501'; END $$;The updated_at triggers are attached in migration 0008 to the 20 tables that have an updated_at column: users, teams, team_memberships, sites, site_settings, goals, funnels, segments, annotations, shared_links, email_report_subscriptions, spike_notifications, spike_notification_recipients, imports, license_tiers, licenses, purchases, usage_monthly, usage_site_monthly, exchange_rates.
Migration 0003 additionally creates the function app_sites_forbid_script_id_change and its trigger on sites (5.5.1). Trigger name: trg_<table>_updated_at, BEFORE UPDATE ... FOR EACH ROW EXECUTE FUNCTION app_set_updated_at().
5.3 Identity: users, user_recovery_codes, auth_sessions, email_tokens #
Authentication flows, password policy, lockout thresholds and token lifetimes are owned by Section 11; this section only fixes storage.
5.3.1 users #
Purpose: one row per human account. Soft-deleted via deleted_at, then anonymized in place by user-anonymize 30 days later (the row is never hard-deleted, so financial and audit references stay valid).
CREATE TABLE users (
id uuid PRIMARY KEY DEFAULT uuidv7(),
email text NOT NULL, -- lowercase, trimmed, 3..254 chars
email_verified_at timestamptz, -- NULL until the verify_email token is consumed
password_hash text NOT NULL, -- argon2id PHC string; the literal '!' for anonymized users (matches nothing)
name text NOT NULL, -- 1..100 chars
locale text NOT NULL DEFAULT 'en', -- UI language tag; supported values per Section 4
theme_preference text NOT NULL DEFAULT 'system', -- light | dark | system; source of truth for the UI theme (Section 11)
preferences jsonb NOT NULL DEFAULT '{}'::jsonb, -- per-user UI preferences; key pinned_sites = array of site UUIDs pinned to the top of the site list (max 20, Section 9.19)
last_team_id uuid, -- last selected team (Section 11.14.2); FK added in migration 0002 once teams exists; NULL when the team is deleted
timezone text, -- IANA display preference; NULL = use the browser's
totp_secret_encrypted text, -- AES-256-GCM text format of Section 17.7.2 (v1.<kid>.<base64url(nonce || ciphertext || tag)>, purpose totp-encryption, associated data "users.totp_secret_encrypted:<id>"); a not-yet-confirmed secret is held in Valkey, never here
totp_enabled_at timestamptz,
totp_last_used_step bigint, -- last accepted 30-second TOTP step; rejects replay
last_login_at timestamptz,
suspended_at timestamptz, -- set by the operator console (Section 15.17.3); login is refused with 403 account_suspended
deleted_at timestamptz, -- soft-delete request time
anonymized_at timestamptz, -- set by user-anonymize
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT users_email_check CHECK (email = lower(btrim(email)) AND char_length(email) BETWEEN 3 AND 254 AND email ~ '^[^@[:space:]]+@[^@[:space:]]+$'),
CONSTRAINT users_name_check CHECK (char_length(name) BETWEEN 1 AND 100),
CONSTRAINT users_locale_check CHECK (locale ~ '^[a-z]{2,3}(-[A-Za-z0-9]{2,8})*$'),
CONSTRAINT users_theme_check CHECK (theme_preference IN ('system','light','dark')),
CONSTRAINT users_preferences_check CHECK (jsonb_typeof(preferences) = 'object' AND pg_column_size(preferences) <= 8192
AND (NOT preferences ? 'pinned_sites' OR (jsonb_typeof(preferences->'pinned_sites') = 'array' AND jsonb_array_length(preferences->'pinned_sites') <= 20))),
CONSTRAINT users_totp_check CHECK ((totp_secret_encrypted IS NULL) = (totp_enabled_at IS NULL)),
CONSTRAINT users_anonymized_check CHECK (anonymized_at IS NULL OR deleted_at IS NOT NULL)
);
CREATE UNIQUE INDEX users_email_key ON users (email);
CREATE INDEX users_pending_anonymization_idx ON users (deleted_at) WHERE deleted_at IS NOT NULL AND anonymized_at IS NULL;Notes:
- Email uniqueness covers soft-deleted users until anonymization (the address stays reserved for 30 days, which also blocks a re-signup that would confuse an undo).
- No
is_super_admincolumn. Super-admin status is computed per request and requires all of:email_verified_at IS NOT NULLset by consuming averify_emailtoken (never by the no-SMTP shortcut),totp_enabled_at IS NOT NULL(otherwise operator routes return403 admin_2fa_required), andemailin the case-insensitive set parsed from envSUPER_ADMIN_EMAILS(hosted edition only; ignored whenEDITION=selfhosted). See 5.12. preferences.pinned_sitesholds the user's pinned sites as site UUIDs (at most 20, 5.16). Routes:POST /api/app/v1/me/pinned-siteswith{ "domain" }pins a site (idempotent,200also when already pinned; the user must currently have access to it) andDELETE /api/app/v1/me/pinned-sites/{domain}unpins it (204). There is no foreign key: the service removes an id when the user loses access to the site or the site is deleted, and ignores unknown ids when reading. Exceeding 20 returns409 limit_reachedwithlimit_name: pinned_sites.- Login lockout state is not stored in Postgres: failed-attempt counters and locks live in Valkey with a 15-minute lifetime (Section 11.8).
suspended_atis written only by the operator console; a suspended user cannot log in and has their sessions deleted (Section 15.17.3).- While
deleted_at IS NOT NULL AND anonymized_at IS NULL, a login yields only the restore-only session of Section 11.13.3; existing sessions were deleted at the request. - Anonymization sets
email = 'deleted-' || id || '@deleted.invalid',name = 'Deleted user',password_hash = '!', clears TOTP columns,timezone,preferences,locale→'en', and deletes the user'sauth_sessions,user_recovery_codes,email_tokens,team_memberships,site_guests, personalsegments, personalannotations,email_report_subscriptionsandspike_notification_recipientsrows with thisuser_id(theapi_keysthey created were already deleted when the deletion was requested, Section 11.16.5), and setslicenses.owner_email_snapshotto NULL for licenses of teams they owned. Rows that merely record authorship (created_by_user_id,audit_log.actor_user_id) keep pointing at the anonymized row.
5.3.2 user_recovery_codes #
Purpose: single-use 2FA recovery codes (count, format and regeneration rules per Section 11). Regenerating deletes all existing rows for the user first.
CREATE TABLE user_recovery_codes (
id uuid PRIMARY KEY DEFAULT uuidv7(),
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
code_hash text NOT NULL, -- lowercase hex SHA-256 of the normalized code (uppercase, hyphens removed)
used_at timestamptz,
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT user_recovery_codes_hash_check CHECK (code_hash ~ '^[0-9a-f]{64}$')
);
CREATE UNIQUE INDEX user_recovery_codes_user_hash_key ON user_recovery_codes (user_id, code_hash);5.3.3 auth_sessions #
Purpose: dashboard login sessions. Postgres is the source of truth; Valkey caches resolved sessions (Section 3). Logout and "sign out other devices" delete rows (no revoked_at). The CSRF token is not stored: it is derived as an HMAC of the session id keyed with the csrf key derived from SECRET_KEY_BASE (Section 17.7), so there is no CSRF column. Impersonation sessions (impersonator_user_id set) are hidden from the target user's session list, are excluded from "sign out other devices" and from the per-user session cap, and are re-validated against the operator's current super-admin status on every request (Sections 11.6 and 15.17).
CREATE TABLE auth_sessions (
id uuid PRIMARY KEY DEFAULT uuidv7(),
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
token_hash bytea NOT NULL, -- SHA-256 of the 32-byte random cookie token
previous_token_hash bytea, -- hash of the cookie token before the last rotation; accepted until previous_valid_until so concurrent requests survive a rotation (Section 11.6.3)
previous_valid_until timestamptz,
rotated_at timestamptz, -- time of the last token rotation
device_label text, -- "Chrome 141 on macOS 15", derived at login, <= 200 chars; the raw UA and the IP are not stored
country char(2), -- ISO 3166-1 alpha-2 resolved from the IP at login for the session list; the IP itself is never stored
two_factor_verified_at timestamptz, -- NULL when the user has no 2FA or this session has not passed it yet
reauth_at timestamptz, -- last successful step-up re-authentication (password, plus TOTP code when 2FA is on); sensitive actions require it within the window of Section 11 / 15.17.2
impersonator_user_id uuid REFERENCES users(id) ON DELETE SET NULL, -- operator console read-only impersonation (Section 15)
created_at timestamptz NOT NULL DEFAULT now(),
last_seen_at timestamptz NOT NULL DEFAULT now(), -- written at most once per 5 minutes per session
expires_at timestamptz NOT NULL, -- sliding expiry; lifetime per Section 11
absolute_expires_at timestamptz NOT NULL, -- hard cap; lifetime per Section 11
CONSTRAINT auth_sessions_token_check CHECK (octet_length(token_hash) = 32),
CONSTRAINT auth_sessions_expiry_check CHECK (expires_at <= absolute_expires_at),
CONSTRAINT auth_sessions_label_check CHECK (device_label IS NULL OR char_length(device_label) <= 200),
CONSTRAINT auth_sessions_previous_check CHECK ((previous_token_hash IS NULL) = (previous_valid_until IS NULL) AND (previous_token_hash IS NULL OR octet_length(previous_token_hash) = 32)),
CONSTRAINT auth_sessions_country_check CHECK (country IS NULL OR country ~ '^[A-Z]{2}$')
);
CREATE UNIQUE INDEX auth_sessions_token_hash_key ON auth_sessions (token_hash);
CREATE UNIQUE INDEX auth_sessions_previous_token_hash_key ON auth_sessions (previous_token_hash) WHERE previous_token_hash IS NOT NULL;
CREATE INDEX auth_sessions_user_idx ON auth_sessions (user_id);
CREATE INDEX auth_sessions_expires_idx ON auth_sessions (expires_at);5.3.4 email_tokens #
Purpose: single-use tokens for email verification, password reset and email change. Only the hash is stored; the raw token travels in the emailed link.
CREATE TABLE email_tokens (
id uuid PRIMARY KEY DEFAULT uuidv7(),
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
kind text NOT NULL,
token_hash bytea NOT NULL, -- HMAC-SHA-256 (key purpose email-token-hmac, Section 17.7) of 32 random bytes
new_email text, -- only for change_email; lowercase
expires_at timestamptz NOT NULL, -- lifetime per Section 11
used_at timestamptz,
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT email_tokens_kind_check CHECK (kind IN ('verify_email','password_reset','change_email')),
CONSTRAINT email_tokens_hash_check CHECK (octet_length(token_hash) = 32),
CONSTRAINT email_tokens_new_email_check CHECK ((kind = 'change_email') = (new_email IS NOT NULL) AND (new_email IS NULL OR new_email = lower(btrim(new_email))))
);
CREATE UNIQUE INDEX email_tokens_hash_key ON email_tokens (token_hash);
CREATE INDEX email_tokens_user_kind_idx ON email_tokens (user_id, kind, created_at DESC);Issuing a new token of the same kind for a user deletes that user's older unused tokens of that kind in the same transaction, so at most one live token per (user, kind) exists.
5.4 Teams: teams, team_memberships #
5.4.1 teams #
Purpose: the tenant. Every site belongs to exactly one team; every license belongs to a team. Signup creates one personal team (is_personal = true). Deletion is scheduled with a 7-day undo window (Section 11) and executed by team-purge.
CREATE TABLE teams (
id uuid PRIMARY KEY DEFAULT uuidv7(),
name text NOT NULL, -- 1..100 chars
is_personal boolean NOT NULL DEFAULT false,
created_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
require_2fa boolean NOT NULL DEFAULT false, -- owner-enforced 2FA for all members (Section 11)
stripe_customer_id text, -- hosted only; created lazily at first checkout
deleted_at timestamptz, -- deletion requested at (start of the 7-day undo window)
purge_after timestamptz, -- deleted_at + 7 days; team-purge hard-deletes the team after this instant
suspended_at timestamptz, -- set by the operator console (Section 15.17.4); suspended teams get ingestion drops and read-only access
suspended_reason text, -- 1..500 chars, required together with suspended_at
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT teams_name_check CHECK (char_length(name) BETWEEN 1 AND 100),
CONSTRAINT teams_deletion_check CHECK ((deleted_at IS NULL) = (purge_after IS NULL)),
CONSTRAINT teams_suspension_check CHECK ((suspended_at IS NULL) = (suspended_reason IS NULL) AND (suspended_reason IS NULL OR char_length(suspended_reason) BETWEEN 1 AND 500))
);
CREATE UNIQUE INDEX teams_personal_per_user_key ON teams (created_by_user_id) WHERE is_personal;
CREATE UNIQUE INDEX teams_stripe_customer_key ON teams (stripe_customer_id) WHERE stripe_customer_id IS NOT NULL;
CREATE INDEX teams_pending_deletion_idx ON teams (purge_after) WHERE purge_after IS NOT NULL;While purge_after IS NOT NULL the team is "pending deletion": members get 409 team_pending_deletion on mutations, API keys of the team are rejected as in Section 11.14.5, and ingestion treats every site of the team as an unknown domain and drops events (still 202, Section 8; there is no site "paused" state). Cancelling the undo clears both columns. The team owner is not a column: it is the single team_memberships row with role = 'owner' (5.4.2), which keeps the deferred one-owner trigger the only source of truth; Section 11 derives the owner from it.
5.4.2 team_memberships #
Purpose: user ↔ team with a role (permission matrix in Section 11).
CREATE TABLE team_memberships (
id uuid PRIMARY KEY DEFAULT uuidv7(),
team_id uuid NOT NULL REFERENCES teams(id) ON DELETE CASCADE,
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
role text NOT NULL,
invited_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT team_memberships_role_check CHECK (role IN ('owner','admin','editor','viewer'))
);
CREATE UNIQUE INDEX team_memberships_team_user_key ON team_memberships (team_id, user_id);
CREATE UNIQUE INDEX team_memberships_one_owner_key ON team_memberships (team_id) WHERE role = 'owner';
CREATE INDEX team_memberships_user_idx ON team_memberships (user_id);
-- "Exactly one owner" is verified at COMMIT so that ownership transfer (demote old owner, promote new owner) works inside one transaction.
CREATE FUNCTION app_assert_team_has_owner() RETURNS trigger LANGUAGE plpgsql AS $$
DECLARE tid uuid := COALESCE(NEW.team_id, OLD.team_id);
BEGIN
IF EXISTS (SELECT 1 FROM teams WHERE id = tid)
AND (SELECT count(*) FROM team_memberships WHERE team_id = tid AND role = 'owner') <> 1 THEN
RAISE EXCEPTION 'team % must have exactly one owner', tid USING ERRCODE = '23514';
END IF;
RETURN NULL;
END $$;
CREATE CONSTRAINT TRIGGER team_memberships_owner_check
AFTER INSERT OR UPDATE OR DELETE ON team_memberships
DEFERRABLE INITIALLY DEFERRED FOR EACH ROW EXECUTE FUNCTION app_assert_team_has_owner();The team and its owner membership are always inserted in one transaction. The service maps SQLSTATE 23514 from this trigger to 409 owner_required.
5.5 Sites and Access: sites, site_settings, site_guests, invitations, site_domain_reservations #
Decision on site settings: identity and lifecycle live on sites (including timezone, because every report, the realtime view and all date bucketing need it); every tunable toggle lives in the 1:1 table site_settings. "Site settings" elsewhere in this document means the union of the two rows. They are always created together in one transaction, and the ingestion cache (Section 8) loads sites ⨝ site_settings plus the site's exclusions in a single refresh. Sites are addressed by domain in the SPA and the dashboard API (/sites/:domain/..., /api/app/v1/sites/{domain}/..., Section 4); id is internal.
5.5.1 sites #
CREATE TABLE sites (
id uuid PRIMARY KEY DEFAULT uuidv7(), -- also the ClickHouse site_id
team_id uuid NOT NULL REFERENCES teams(id) ON DELETE RESTRICT,
domain text NOT NULL, -- public site identifier in all APIs; normalized (see notes)
script_id text NOT NULL, -- 12-char lowercase Crockford base32 id used in the per-site script URL (Section 7.2.1); generated at insert, immutable
proxy_key text, -- 32-char lowercase Crockford base32 (160 bits); NULL = no first-party proxy configured (Section 7.15.4)
timezone text NOT NULL DEFAULT 'UTC', -- IANA name, validated against the runtime tz database
first_event_at timestamptz, -- set once by ingestion on the first accepted event (drives the "waiting for first pageview" state)
created_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT sites_domain_check CHECK (domain = lower(domain) AND domain !~ '^www\.' AND char_length(domain) BETWEEN 1 AND 253
AND domain ~ '^([a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?\.)*[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$'),
CONSTRAINT sites_timezone_check CHECK (timezone ~ '^[A-Za-z][A-Za-z0-9_+/-]{0,63}$'),
CONSTRAINT sites_script_id_check CHECK (script_id ~ '^[0-9a-hjkmnp-tv-z]{12}$'),
CONSTRAINT sites_proxy_key_check CHECK (proxy_key IS NULL OR proxy_key ~ '^[0-9a-hjkmnp-tv-z]{32}$')
);
CREATE UNIQUE INDEX sites_domain_key ON sites (domain);
CREATE UNIQUE INDEX sites_script_id_key ON sites (script_id);
CREATE INDEX sites_team_idx ON sites (team_id, created_at);
CREATE FUNCTION app_sites_forbid_script_id_change() RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN
IF NEW.script_id IS DISTINCT FROM OLD.script_id THEN
RAISE EXCEPTION 'sites.script_id is immutable' USING ERRCODE = '42501';
END IF;
RETURN NEW;
END $$;
CREATE TRIGGER trg_sites_script_id_immutable BEFORE UPDATE ON sites FOR EACH ROW EXECUTE FUNCTION app_sites_forbid_script_id_change();Notes:
- Domain normalization (single rule for all sections).
domainis stored lowercase, in punycode for internationalized names, without scheme, port, path, trailing dot or a leadingwww.. The service strips one leadingwww.on create and update, and ingestion strips it (and any port) from the tracker-supplied domain before the lookup (Section 8.6), sowww.example.comandexample.comare the same site; other subdomains are different sites. Uniqueness is installation-wide on the normalized value, across all teams. There is no port: local development useslocalhost. Validation messages and thedomain_taken409 are in Section 4.4.3 and Section 9 (UI). timezonedefault isUTC; the create-site UI pre-fills the browser's IANA zone (Section 9). A value is valid only when it is a member of the runtime's supported IANA zone list (Intl.supportedValuesOf('timeZone')) orUTC, canonicalized to the zone's primary name; UTC-offset strings such as+05:00and unknown aliases are rejected, because ClickHouse rejects them intoStartOfDay(ts, tz)(theTimeZoneprimitive of Section 4.8 states the same rule). Changing the timezone re-buckets reports immediately; no stored data changes.script_idis generated from a CSPRNG at insert and never changes (a trigger rejects updates), so the tracker snippet stays valid across domain changes.proxy_keyauthorizes first-party proxies to pass the visitor's address inX-Quietstat-Client-IP(Section 8.7). Only a key that matches this value, compared in constant time, makes the header trusted; a request with an invalid key has the header ignored and is rate-limited as a normal client, so no elevated limit is ever granted without a valid key. The key is stored in plaintext by design (documented exception, 5.1: the settings page must be able to show it again), is never returned to roles below site-settings write, and is never logged. Generation and rotation: Section 7.15.4.- Changing
domainis allowed for owners and admins (actionsite.domain.change, Section 11.19.2); events and history stay attached because ClickHouse is keyed by the UUID. The old domain stays reserved to the same team for 30 days (5.5.5); the tracker snippet must be updated by the user, and events still sent for the old domain are dropped as an unknown domain. - Domain release rule. When a site is deleted or its domain is changed, the old domain is released to other teams exactly 30 days later (
reserved_until, 5.5.5); until then only the same team can claim it again. The ClickHouse purge job finishing earlier does not shorten the reservation. A deleted site's domain is therefore never claimable by another team within 30 days, and events for it are dropped as an unknown domain throughout. team_idisRESTRICTon purpose: deleting a team never cascades into sites silently. Theteam-purgejob deletes each site through the normal site-delete service (which enqueues the ClickHouse purge, 5.14 and Section 6.9) and only then deletes the team.- Site deletion (any path) runs in one transaction: call
freezeSiteUsage(team_id, site_id)(Section 15.9.4) so the team keeps the usage already counted, delete thesitesrow (children cascade), upsert the domain reservation (5.5.5) and enqueue the pg-boss jobsite-purgefor thatsite_id. Sites are hard-deleted at once; there is no paused or pending-deletion site state (a site whose team is pending deletion is treated as unknown by ingestion, 5.4.1).
5.5.2 site_settings #
CREATE TABLE site_settings (
site_id uuid PRIMARY KEY REFERENCES sites(id) ON DELETE CASCADE,
-- Tracker feature toggles. They drive the generated snippet (Section 7); ingestion never rejects an event because a toggle is off.
track_outbound_links boolean NOT NULL DEFAULT true,
track_file_downloads boolean NOT NULL DEFAULT true,
file_download_extensions text[], -- NULL = the default list of Section 7.5.6
track_form_submissions boolean NOT NULL DEFAULT false,
track_404_pages boolean NOT NULL DEFAULT false,
track_hash_routing boolean NOT NULL DEFAULT false,
-- Traffic quality
datacenter_filter_enabled boolean NOT NULL DEFAULT true,
allowed_hostnames text[] NOT NULL DEFAULT '{}', -- empty = the site domain and all its subdomains
-- Visibility
public_dashboard boolean NOT NULL DEFAULT false,
-- Revenue
reporting_currency char(3) NOT NULL DEFAULT 'USD',
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT site_settings_extensions_check CHECK (
file_download_extensions IS NULL OR (
cardinality(file_download_extensions) BETWEEN 1 AND 50
AND array_to_string(file_download_extensions, ',') ~ '^[a-z0-9]{1,10}(,[a-z0-9]{1,10})*$')),
CONSTRAINT site_settings_hostnames_check CHECK (
cardinality(allowed_hostnames) <= 50
AND (cardinality(allowed_hostnames) = 0
OR array_to_string(allowed_hostnames, ',') ~ '^([a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?\.)*[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?(,([a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?\.)*[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?)*$')),
CONSTRAINT site_settings_currency_check CHECK (reporting_currency ~ '^[A-Z]{3}$')
);| Column | Meaning and rules |
|---|---|
track_outbound_links |
Snippet enables outbound-link click events (Outbound Link: Click, property url; Section 7). |
track_file_downloads / file_download_extensions |
Snippet enables download events (File Download, property url) for links whose path ends in a listed extension. Extensions are stored lowercase without dots, 1–10 chars [a-z0-9], 1–50 entries, no duplicates (service dedupes before write). NULL means the default list defined in Section 7.5.6 (the generated snippet uses it); a stored array replaces it entirely. |
track_form_submissions |
Snippet enables Form: Submission events (property path; Section 7). |
track_404_pages |
Snippet enables 404 events (property path; Section 7). |
track_hash_routing |
Snippet reports hash-route navigation and sets h=1 on payloads; with it on, stored pathnames include the hash route (Section 6.3). |
| Scroll depth, engagement time, tagged events | Always on, no column. Every per-site script includes the engagement and tagged-event modules (Section 7). Automatic pageviews and honoring of privacy signals are initialization options of the tracker, not stored settings. |
datacenter_filter_enabled |
When false, events from datacenter CIDRs are accepted. The UA-based bot filter has no toggle (Section 8). |
allowed_hostnames |
Hostname allowlist, the defense against anyone posting events for your domain from other sites (and burning a hosted team's cap). Entries are lowercase hostnames without scheme, port, path or a leading www.; an entry matches that hostname and all its subdomains. Empty (default) = the site's own domain and all its subdomains. When non-empty it replaces the default, so it must list the site's own domain. An event whose URL hostname is not allowed is dropped with reason hostname_not_allowed (the response is still 202; the event is not billable). Max 50 entries (5.16). Edited in the tracking settings (Section 7), enforced at ingestion (Section 8). |
public_dashboard |
When true, the dashboard is readable without login at the public URL /public/<domain> (Section 12). It exposes the same data as an unrestricted shared link and is independent of shared_links rows. Default false. |
reporting_currency |
ISO 4217 code that revenue is converted into at ingestion (revenue_reporting_* columns, Section 6.3; rate rules in 5.9.4). Default USD. Changing it affects only events ingested afterwards; the UI warns about this (Section 10). |
The tracking-configuration API of Section 7 reads and writes exactly the tracker columns above (PATCH with any subset); there is no JSON configuration document. Defaults rationale: outbound links and downloads default on because they are the two lowest-risk, highest-value toggles; form submissions, 404 and hash routing change what is collected or how pages are keyed, so they default off.
5.5.3 site_guests #
Purpose: per-site access for people outside the team (agency clients). Roles guest_viewer (read dashboards) and guest_editor (read plus goals, funnels, segments, annotations; exact matrix in Section 11). A guest sees only the sites they are attached to.
CREATE TABLE site_guests (
id uuid PRIMARY KEY DEFAULT uuidv7(),
site_id uuid NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
role text NOT NULL,
invited_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT site_guests_role_check CHECK (role IN ('guest_viewer','guest_editor'))
);
CREATE UNIQUE INDEX site_guests_site_user_key ON site_guests (site_id, user_id);
CREATE INDEX site_guests_user_idx ON site_guests (user_id);A user who is already a member of the site's team cannot also be a guest of it (service check, 409 member_exists).
5.5.4 invitations #
Purpose: pending invitations of four kinds: team_member (join a team with a team role), site_guest (become a guest on one site), ownership_transfer (become the team owner) and site_transfer (take over a site and choose the destination team on acceptance). On acceptance the service performs the action of the kind (inserts the team_memberships or site_guests row, transfers ownership or the site) and sets accepted_at.
CREATE TABLE invitations (
id uuid PRIMARY KEY DEFAULT uuidv7(),
kind text NOT NULL,
team_id uuid NOT NULL REFERENCES teams(id) ON DELETE CASCADE, -- the inviting team (for site kinds: the site's current team)
site_id uuid REFERENCES sites(id) ON DELETE CASCADE,
email text NOT NULL, -- lowercase invitee address
role text, -- role granted; NULL for site_transfer
token_hash bytea NOT NULL, -- HMAC-SHA-256 (key purpose email-token-hmac, Section 17.7) of the 32-byte emailed token
invited_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
expires_at timestamptz NOT NULL DEFAULT now() + interval '7 days',
last_sent_at timestamptz NOT NULL DEFAULT now(), -- last time the email was (re)sent; drives the resend throttle
resend_count smallint NOT NULL DEFAULT 0,
accepted_at timestamptz,
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT invitations_kind_check CHECK (kind IN ('team_member','site_guest','ownership_transfer','site_transfer')),
CONSTRAINT invitations_email_check CHECK (email = lower(btrim(email)) AND char_length(email) BETWEEN 3 AND 254),
CONSTRAINT invitations_token_check CHECK (octet_length(token_hash) = 32),
CONSTRAINT invitations_resend_check CHECK (resend_count >= 0),
CONSTRAINT invitations_shape_check CHECK (
(kind = 'team_member' AND site_id IS NULL AND role IN ('admin','editor','viewer')) OR
(kind = 'site_guest' AND site_id IS NOT NULL AND role IN ('guest_viewer','guest_editor')) OR
(kind = 'ownership_transfer' AND site_id IS NULL AND role = 'owner') OR
(kind = 'site_transfer' AND site_id IS NOT NULL AND role IS NULL))
);
CREATE UNIQUE INDEX invitations_token_hash_key ON invitations (token_hash);
CREATE UNIQUE INDEX invitations_pending_team_key ON invitations (team_id, email) WHERE kind = 'team_member' AND accepted_at IS NULL;
CREATE UNIQUE INDEX invitations_pending_site_key ON invitations (site_id, email) WHERE kind IN ('site_guest','site_transfer') AND accepted_at IS NULL;
CREATE UNIQUE INDEX invitations_pending_owner_key ON invitations (team_id) WHERE kind = 'ownership_transfer' AND accepted_at IS NULL;
CREATE UNIQUE INDEX invitations_pending_site_transfer_key ON invitations (site_id) WHERE kind = 'site_transfer' AND accepted_at IS NULL;
CREATE INDEX invitations_email_idx ON invitations (email);team_id is always set (for guest and site-transfer invitations it is the site's team, so team admins can list and revoke them). The owner role is only carried by ownership_transfer invitations, so the owner role can never be invited directly; ownership changes only through transfer (Section 11). At most one pending ownership transfer exists per team and one pending site transfer per site. Re-inviting an email with a pending invitation replaces it (delete + insert) so the old link dies; resending keeps the row, issues a new token and bumps last_sent_at and resend_count (throttled per Section 17.6). Revoking deletes the row.
5.5.5 site_domain_reservations #
Purpose: stops a released domain from being claimed by another team while pages still carry the old tracker snippet (which would otherwise deliver the old owner's traffic to the new claimant).
CREATE TABLE site_domain_reservations (
domain text PRIMARY KEY, -- normalized domain, same rule as sites.domain
team_id uuid NOT NULL, -- the team the domain is reserved for; no FK so the record survives team deletion
released_at timestamptz NOT NULL DEFAULT now(),
reserved_until timestamptz NOT NULL, -- released_at + 30 days
CONSTRAINT site_domain_reservations_domain_check CHECK (domain = lower(domain) AND domain !~ '^www\.' AND char_length(domain) BETWEEN 1 AND 253),
CONSTRAINT site_domain_reservations_window_check CHECK (reserved_until > released_at)
);
CREATE INDEX site_domain_reservations_until_idx ON site_domain_reservations (reserved_until);Rules: when a site's domain changes, or the site is deleted, the previous domain is upserted here in the same transaction with reserved_until = now() + interval '30 days'. Creating a site or changing a domain to a value with an unexpired reservation held by a different team returns 409 domain_taken; the reserving team may take the domain again at any time (the reservation row is deleted in that transaction). Events for a reserved domain that has no site are dropped like any unknown domain. Expired rows are ignored by the check and deleted by site-orphan-reconcile (5.14).
5.6 Site Configuration: goals, funnels, funnel_steps, custom_property_allowlist, segments, annotations #
Semantics of goal matching, funnel evaluation and revenue belong to Sections 9, 10 and 13; this section fixes storage and integrity. Column names are the API field names of Section 10 (the 4.1 boundary rule forbids hand-renaming between column and field).
5.6.1 goals #
Purpose: a conversion definition. Three types: event (custom event by name, optionally with revenue), pageview (path pattern), scroll (path pattern plus minimum scroll depth). Up to three property conditions can narrow event and pageview goals (not scroll goals). Auto-collected events are ordinary custom event names: Outbound Link: Click and File Download carry the property url; 404 and Form: Submission carry the property path (Section 7).
CREATE TABLE goals (
id uuid PRIMARY KEY DEFAULT uuidv7(),
site_id uuid NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
type text NOT NULL,
display_name text NOT NULL, -- 1..120 chars, unique per site case-insensitively
event_name text, -- event goals: the custom event name
page_path text, -- pageview/scroll goals: path pattern, '*' = one segment, '**' = any depth; starts with '/' or '**'
scroll_threshold smallint, -- scroll goals: 1..100 (percent)
property_conditions jsonb NOT NULL DEFAULT '[]'::jsonb, -- [{"key":"plan","value":"pro"}, ...]; 0..3 entries, sorted by key; all values strings
revenue_currency char(3), -- event goals only: marks a revenue goal and sets its display currency
created_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT goals_type_check CHECK (type IN ('event','pageview','scroll')),
CONSTRAINT goals_name_check CHECK (char_length(display_name) BETWEEN 1 AND 120),
CONSTRAINT goals_shape_check CHECK (
(type = 'event' AND event_name IS NOT NULL AND page_path IS NULL AND scroll_threshold IS NULL) OR
(type = 'pageview' AND event_name IS NULL AND page_path IS NOT NULL AND scroll_threshold IS NULL AND revenue_currency IS NULL) OR
(type = 'scroll' AND event_name IS NULL AND page_path IS NOT NULL AND scroll_threshold BETWEEN 1 AND 100 AND revenue_currency IS NULL)),
CONSTRAINT goals_event_name_check CHECK (event_name IS NULL OR (char_length(event_name) BETWEEN 1 AND 120 AND event_name NOT IN ('pageview','engagement'))),
CONSTRAINT goals_page_path_check CHECK (page_path IS NULL OR ((page_path LIKE '/%' OR page_path LIKE '**%') AND char_length(page_path) <= 2000)),
CONSTRAINT goals_conditions_check CHECK (jsonb_typeof(property_conditions) = 'array' AND jsonb_array_length(property_conditions) <= 3),
CONSTRAINT goals_currency_check CHECK (revenue_currency IS NULL OR revenue_currency ~ '^[A-Z]{3}$')
);
CREATE UNIQUE INDEX goals_site_name_key ON goals (site_id, lower(display_name));
CREATE UNIQUE INDEX goals_identity_key ON goals (site_id, type, COALESCE(event_name,''), COALESCE(page_path,''), COALESCE(scroll_threshold,0), property_conditions);Notes: the identity index stops two goals from meaning the same thing (409 goal_exists); a duplicate display name returns 409 goal_name_taken. Because the index compares property_conditions as stored, the service sorts the array by key before every write, which gives the set semantics (order ignored) that Section 10.2.2 requires. Property keys are 1–300 chars and values 1–2,000 chars, validated by Zod in Section 10 (a CHECK on contents is not attempted). Names starting qs_, quietstat_ or $ are reserved (422 reserved_property_name; 422 reserved_event_name for event names). Path patterns are stored percent-normalized (see 5.8.3). Deleting a goal is blocked while a funnel step references it (funnel_steps.goal_id is RESTRICT; the service returns 409 goal_in_use with details.funnel_ids). Goals can be edited, but type is immutable (422 immutable_field). A path pattern contains at most 8 wildcard tokens (5.16).
5.6.2 funnels #
CREATE TABLE funnels (
id uuid PRIMARY KEY DEFAULT uuidv7(),
site_id uuid NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
name text NOT NULL, -- 1..120 chars
order_mode text NOT NULL DEFAULT 'ordered', -- 'strict': no other pageview/event may occur between consecutive steps
created_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT funnels_name_check CHECK (char_length(name) BETWEEN 1 AND 120),
CONSTRAINT funnels_order_mode_check CHECK (order_mode IN ('ordered','strict'))
);
CREATE UNIQUE INDEX funnels_site_name_key ON funnels (site_id, lower(name));A duplicate name returns 409 funnel_name_taken.
5.6.3 funnel_steps #
Purpose: ordered steps. A step is either a reference to an existing goal, or an inline page or event definition (so funnels can be built without pre-defining goals).
CREATE TABLE funnel_steps (
id uuid PRIMARY KEY DEFAULT uuidv7(),
funnel_id uuid NOT NULL REFERENCES funnels(id) ON DELETE CASCADE,
position smallint NOT NULL, -- 1-based, contiguous
type text NOT NULL,
goal_id uuid REFERENCES goals(id) ON DELETE RESTRICT,
page_path text, -- type = 'page': path pattern, same wildcard syntax as goals
event_name text, -- type = 'event'
property_conditions jsonb NOT NULL DEFAULT '[]'::jsonb, -- inline steps only; 0..3 entries, sorted by key
label text, -- optional display label, 1..120 chars
is_optional boolean NOT NULL DEFAULT false,
CONSTRAINT funnel_steps_position_check CHECK (position BETWEEN 1 AND 8),
CONSTRAINT funnel_steps_type_check CHECK (type IN ('goal','page','event')),
CONSTRAINT funnel_steps_shape_check CHECK (
(type = 'goal' AND goal_id IS NOT NULL AND page_path IS NULL AND event_name IS NULL AND property_conditions = '[]'::jsonb) OR
(type = 'page' AND goal_id IS NULL AND page_path IS NOT NULL AND event_name IS NULL) OR
(type = 'event' AND goal_id IS NULL AND page_path IS NULL AND event_name IS NOT NULL)),
CONSTRAINT funnel_steps_page_path_check CHECK (page_path IS NULL OR ((page_path LIKE '/%' OR page_path LIKE '**%') AND char_length(page_path) <= 2000)),
CONSTRAINT funnel_steps_event_name_check CHECK (event_name IS NULL OR (char_length(event_name) BETWEEN 1 AND 120 AND event_name NOT IN ('pageview','engagement'))),
CONSTRAINT funnel_steps_conditions_check CHECK (jsonb_typeof(property_conditions) = 'array' AND jsonb_array_length(property_conditions) <= 3),
CONSTRAINT funnel_steps_label_check CHECK (label IS NULL OR char_length(label) BETWEEN 1 AND 120)
);
CREATE UNIQUE INDEX funnel_steps_position_key ON funnel_steps (funnel_id, position);
CREATE INDEX funnel_steps_goal_idx ON funnel_steps (goal_id) WHERE goal_id IS NOT NULL;Service-enforced funnel invariants (Section 10 defines the API errors): 2 to 8 steps; positions contiguous from 1; the first and last step are never is_optional; a referenced goal_id must belong to the funnel's own site (a foreign goal returns 400 validation_failed on steps.N.goal_id); saving a funnel replaces all its steps in one transaction (delete + insert), so steps carry no updated_at.
5.6.4 custom_property_allowlist #
Purpose: the custom property names a site has registered. Decision: ingestion stores every property sent (within the payload limits of Section 8), so registering a name later makes historical data visible; the allowlist controls which property names appear in the dashboard property reports, the filter picker and the Stats API property dimensions (event:props:<name>, Section 13). A property that is not registered is not queryable.
CREATE TABLE custom_property_allowlist (
id uuid PRIMARY KEY DEFAULT uuidv7(),
site_id uuid NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
property_name text NOT NULL, -- 1..300 chars, no control characters
created_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT custom_property_allowlist_name_check CHECK (char_length(property_name) BETWEEN 1 AND 300 AND property_name !~ '[[:cntrl:]]')
);
CREATE UNIQUE INDEX custom_property_allowlist_site_name_key ON custom_property_allowlist (site_id, property_name);Property names are case-sensitive. Names starting qs_, quietstat_ or $ are reserved: the service rejects them (422 reserved_property_name) and ingestion drops such keys (Section 8.4). Removing a name hides the property but deletes no event data; a name still referenced by a goal, funnel step or segment cannot be removed (409 property_in_use with details.goal_ids, details.funnel_ids, details.segment_ids).
5.6.5 segments #
Purpose: saved filter sets. site segments are visible to everyone with access to the site (created by editors and above); personal segments are visible only to their owner (any role including viewers and guests). A site segment is also what a limited shared link is locked to.
CREATE TABLE segments (
id uuid PRIMARY KEY DEFAULT uuidv7(),
site_id uuid NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
segment_type text NOT NULL,
owner_user_id uuid REFERENCES users(id) ON DELETE CASCADE, -- personal segments only
created_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
name text NOT NULL, -- 1..120 chars
filters jsonb NOT NULL, -- array of filter clauses in the Stats API filter grammar (Section 13)
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT segments_type_check CHECK (segment_type IN ('personal','site')),
CONSTRAINT segments_owner_check CHECK ((segment_type = 'personal') = (owner_user_id IS NOT NULL)),
CONSTRAINT segments_name_check CHECK (char_length(name) BETWEEN 1 AND 120),
CONSTRAINT segments_filters_check CHECK (jsonb_typeof(filters) = 'array' AND jsonb_array_length(filters) BETWEEN 1 AND 10)
);
CREATE UNIQUE INDEX segments_id_site_key ON segments (id, site_id); -- target of the composite FK from shared_links
CREATE UNIQUE INDEX segments_site_name_key ON segments (site_id, lower(name)) WHERE segment_type = 'site';
CREATE UNIQUE INDEX segments_personal_name_key ON segments (site_id, owner_user_id, lower(name)) WHERE segment_type = 'personal';
CREATE INDEX segments_site_idx ON segments (site_id, segment_type);Filters are validated against the Stats API filter schema before write (Section 13); the database only checks shape (1 to 10 clauses). A duplicate name returns 409 segment_name_taken. A segment cannot be deleted while a shared link is locked to it (shared_links FK is RESTRICT; 409 segment_in_use with details.shared_link_ids). Limits per 5.16.
5.6.6 annotations #
Purpose: dated notes drawn on the time-series chart.
CREATE TABLE annotations (
id uuid PRIMARY KEY DEFAULT uuidv7(),
site_id uuid NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
created_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
visibility text NOT NULL DEFAULT 'personal', -- personal: creator only; shared: everyone with site access
occurred_on date NOT NULL, -- the site-local calendar day the note refers to (site timezone, Section 5.5.1); the chart snaps it to the active interval
title text NOT NULL, -- 1..120 chars
description text, -- 0..1000 chars
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT annotations_visibility_check CHECK (visibility IN ('personal','shared')),
CONSTRAINT annotations_title_check CHECK (char_length(title) BETWEEN 1 AND 120),
CONSTRAINT annotations_description_check CHECK (description IS NULL OR char_length(description) <= 1000)
);
CREATE INDEX annotations_site_day_idx ON annotations (site_id, occurred_on);Annotations are capped at 5,000 per site (5.16). Changing the site timezone does not move stored days.
5.7 Sharing and Notifications: shared_links, email_report_subscriptions, spike_notifications, spike_notification_recipients #
Behavior (URLs, password flow, embed headers, email content, spike detection) is owned by Section 12; this section fixes storage and carries exactly the attributes Section 12.13.1 requires.
5.7.1 shared_links #
Purpose: a read-only view of one site for people without an account. A link may be password-protected, locked to a fixed site segment (the viewer can never remove the segment's filters), embeddable under a per-link framing policy, and may include imported data.
CREATE TABLE shared_links (
id uuid PRIMARY KEY DEFAULT uuidv7(),
site_id uuid NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
name text NOT NULL, -- 1..60 chars; label in the settings list
token text NOT NULL, -- 32 chars [A-Za-z0-9_-], 192 random bits; stored in clear so the owner can copy the URL again (documented exception, 5.1)
password_hash text, -- argon2id PHC string; NULL = no password
password_updated_at timestamptz, -- invalidates previously unlocked viewer sessions when it changes
segment_id uuid, -- NULL = unrestricted; immutable after creation
embed_mode text NOT NULL DEFAULT 'disabled', -- who may render the link in an iframe (Section 12.4.3)
embed_origins text[] NOT NULL DEFAULT '{}', -- allowed framing origins; non-empty only when embed_mode = 'allowlist'
include_imports boolean NOT NULL DEFAULT true, -- true: queries through the link include imported data and the viewer may switch it off; false: the server forces imported data off for every request through the link (Section 12.2.1)
created_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
last_accessed_at timestamptz, -- written at most once per hour
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT shared_links_name_check CHECK (char_length(name) BETWEEN 1 AND 60),
CONSTRAINT shared_links_token_check CHECK (token ~ '^[A-Za-z0-9_-]{32}$'),
CONSTRAINT shared_links_password_check CHECK ((password_hash IS NULL) = (password_updated_at IS NULL)),
CONSTRAINT shared_links_embed_mode_check CHECK (embed_mode IN ('disabled','any','allowlist')),
CONSTRAINT shared_links_embed_origins_check CHECK ((embed_mode = 'allowlist') = (cardinality(embed_origins) BETWEEN 1 AND 10)),
CONSTRAINT shared_links_segment_fkey FOREIGN KEY (segment_id, site_id) REFERENCES segments (id, site_id) ON DELETE RESTRICT
);
CREATE UNIQUE INDEX shared_links_token_key ON shared_links (token);
CREATE INDEX shared_links_site_idx ON shared_links (site_id);Notes: the composite FK guarantees the segment belongs to the same site (with segment_id NULL the FK is not evaluated). The service additionally requires segments.segment_type = 'site' (a personal segment cannot back a link; 422 segment_not_shared). Because the FK is RESTRICT, a link can never silently widen to "unrestricted" when its segment is deleted: the service returns 409 segment_in_use with details.shared_link_ids. The origin format rules of embed_origins are enforced by the service (Section 12.4.3). Deleting the link hard-deletes the row and the URL stops working immediately.
5.7.2 email_report_subscriptions #
Purpose: scheduled email report recipients, one row per (site, frequency, recipient). A recipient is either a user (delivered to their current address) or an external email address (for example a client). Weekly reports send on Monday and monthly reports on the 1st, at the local send time of Section 12.5.2 in the site's timezone, covering the previous complete week (Monday–Sunday) or month.
CREATE TABLE email_report_subscriptions (
id uuid PRIMARY KEY DEFAULT uuidv7(),
site_id uuid NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
user_id uuid REFERENCES users(id) ON DELETE CASCADE,
email text, -- external recipient, lowercase
frequency text NOT NULL,
status text NOT NULL DEFAULT 'active',
unsubscribe_token text NOT NULL, -- 32 random bytes base64url (43 chars, 256 bits); the one-click unsubscribe/confirm link, resolved by one indexed lookup; stored in clear because it is placed in every email (documented exception, 5.1)
confirmed_at timestamptz, -- external recipients: when the recipient confirmed the subscription; NULL for user recipients and while pending
last_sent_period text, -- period key of the last report sent: 'YYYY-Www' (ISO week) or 'YYYY-MM'; makes a retried job idempotent
last_sent_at timestamptz,
consecutive_failures smallint NOT NULL DEFAULT 0, -- permanent delivery failures in a row (Section 12.9.4)
added_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT email_report_subscriptions_frequency_check CHECK (frequency IN ('weekly','monthly')),
CONSTRAINT email_report_subscriptions_status_check CHECK (status IN ('pending_confirmation','active','unsubscribed','bounced')),
CONSTRAINT email_report_subscriptions_recipient_check CHECK ((user_id IS NULL) <> (email IS NULL)),
CONSTRAINT email_report_subscriptions_pending_check CHECK (status <> 'pending_confirmation' OR email IS NOT NULL),
CONSTRAINT email_report_subscriptions_email_check CHECK (email IS NULL OR (email = lower(btrim(email)) AND char_length(email) BETWEEN 3 AND 254)),
CONSTRAINT email_report_subscriptions_token_check CHECK (unsubscribe_token ~ '^[A-Za-z0-9_-]{43}$'),
CONSTRAINT email_report_subscriptions_confirmed_check CHECK (status <> 'pending_confirmation' OR confirmed_at IS NULL),
CONSTRAINT email_report_subscriptions_period_check CHECK (last_sent_period IS NULL OR last_sent_period ~ '^[0-9]{4}-(W[0-9]{2}|[0-9]{2})$'),
CONSTRAINT email_report_subscriptions_failures_check CHECK (consecutive_failures >= 0)
);
CREATE UNIQUE INDEX email_report_subscriptions_token_key ON email_report_subscriptions (unsubscribe_token);
CREATE UNIQUE INDEX email_report_subscriptions_user_key ON email_report_subscriptions (site_id, frequency, user_id) WHERE user_id IS NOT NULL;
CREATE UNIQUE INDEX email_report_subscriptions_email_key ON email_report_subscriptions (site_id, frequency, email) WHERE email IS NOT NULL;
CREATE INDEX email_report_subscriptions_due_idx ON email_report_subscriptions (frequency) WHERE status = 'active';External recipients need consent. An external address is stored as pending_confirmation and receives only a one-time confirmation email with a Confirm button; no report is sent until the recipient confirms, which sets status = 'active' and confirmed_at. Unconfirmed rows are deleted 7 days after creation (5.14). Report emails render dimension values (pages, sources, properties) as plain text, never as links, because those values are visitor-supplied (Section 12). A user recipient who loses access to the site (removed from the team, guest removed) is skipped at send time and the row is deleted by the sender job (email-report-send, Section 16.6). A hard email bounce reported by the SMTP server sets status = 'bounced' for that address (Section 12).
5.7.3 spike_notifications and spike_notification_recipients #
Purpose: per-site traffic spike alert configuration (at most one row per site) and its recipients. Spike detection state (armed, last_triggered_at, recent_triggers) lives on the configuration row.
CREATE TABLE spike_notifications (
id uuid PRIMARY KEY DEFAULT uuidv7(),
site_id uuid NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
enabled boolean NOT NULL DEFAULT false,
threshold integer NOT NULL, -- visitor count that triggers the alert; unit and window per Section 12.6
armed boolean NOT NULL DEFAULT true, -- whether a new trigger is allowed
last_triggered_at timestamptz,
recent_triggers timestamptz[] NOT NULL DEFAULT '{}', -- triggers within the past 24 hours
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT spike_notifications_threshold_check CHECK (threshold BETWEEN 1 AND 1000000)
);
CREATE UNIQUE INDEX spike_notifications_site_key ON spike_notifications (site_id);
CREATE TABLE spike_notification_recipients (
id uuid PRIMARY KEY DEFAULT uuidv7(),
site_id uuid NOT NULL REFERENCES spike_notifications (site_id) ON DELETE CASCADE,
user_id uuid REFERENCES users(id) ON DELETE CASCADE,
email text, -- external recipient, lowercase
status text NOT NULL DEFAULT 'active',
confirmed_at timestamptz, -- same meaning as on email_report_subscriptions
unsubscribe_token text NOT NULL, -- same format and storage rule as email_report_subscriptions
consecutive_failures smallint NOT NULL DEFAULT 0,
added_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT spike_notification_recipients_status_check CHECK (status IN ('pending_confirmation','active','unsubscribed','bounced')),
CONSTRAINT spike_notification_recipients_recipient_check CHECK ((user_id IS NULL) <> (email IS NULL)),
CONSTRAINT spike_notification_recipients_pending_check CHECK (status <> 'pending_confirmation' OR email IS NOT NULL),
CONSTRAINT spike_notification_recipients_email_check CHECK (email IS NULL OR (email = lower(btrim(email)) AND char_length(email) BETWEEN 3 AND 254)),
CONSTRAINT spike_notification_recipients_token_check CHECK (unsubscribe_token ~ '^[A-Za-z0-9_-]{43}$'),
CONSTRAINT spike_notification_recipients_confirmed_check CHECK (status <> 'pending_confirmation' OR confirmed_at IS NULL),
CONSTRAINT spike_notification_recipients_failures_check CHECK (consecutive_failures >= 0)
);
CREATE UNIQUE INDEX spike_notification_recipients_token_key ON spike_notification_recipients (unsubscribe_token);
CREATE UNIQUE INDEX spike_notification_recipients_user_key ON spike_notification_recipients (site_id, user_id) WHERE user_id IS NOT NULL;
CREATE UNIQUE INDEX spike_notification_recipients_email_key ON spike_notification_recipients (site_id, email) WHERE email IS NOT NULL;Recipients are a table (not an array) so that GET /unsubscribe/:token resolves a token with one indexed lookup (Section 12.10). The limit of 10 recipients per site is enforced by the service (5.16). User recipients must have access to the site and are skipped at send time when access is gone; external recipients follow the same confirmation rule as report subscriptions.
5.8 Access Control and Filtering: api_keys, ip_exclusions, page_exclusions #
5.8.1 api_keys #
Purpose: team-scoped Bearer keys for the Stats API and Sites API (Section 13). The full key is shown once at creation.
CREATE TABLE api_keys (
id uuid PRIMARY KEY DEFAULT uuidv7(),
team_id uuid NOT NULL REFERENCES teams(id) ON DELETE CASCADE,
name text NOT NULL, -- 1..60 chars after trimming; unique per team case-insensitively (409 `api_key_name_taken`)
key_prefix text NOT NULL, -- first 12 characters of the key (`qs_live_` plus 4 more); shown in the UI to identify the key
key_hash bytea NOT NULL, -- SHA-256 digest of the full key (32 raw bytes)
scopes text[] NOT NULL DEFAULT ARRAY['stats:read'],
expires_at timestamptz, -- NULL = never
last_used_at timestamptz, -- written at most once per minute per key
created_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT api_keys_name_check CHECK (char_length(name) BETWEEN 1 AND 60),
CONSTRAINT api_keys_prefix_check CHECK (char_length(key_prefix) = 12),
CONSTRAINT api_keys_hash_check CHECK (octet_length(key_hash) = 32),
CONSTRAINT api_keys_scopes_check CHECK (cardinality(scopes) >= 1 AND scopes <@ ARRAY['stats:read','sites:provision'])
);
CREATE UNIQUE INDEX api_keys_hash_key ON api_keys (key_hash);
CREATE UNIQUE INDEX api_keys_team_name_key ON api_keys (team_id, lower(name));
CREATE INDEX api_keys_team_idx ON api_keys (team_id, created_at);Scopes: stats:read (POST /api/v2/query) and sites:provision (/api/v1/sites...). Revoking deletes the row. Keys work for all sites of the team; the sites:provision scope is the only way a key can create sites. Every key gets the fixed rate-limit policies of Section 17.6 (stats_api_key, sites_api_key); there is no per-key rate-limit column. When the user who created a key leaves the team or deletes their account, the keys they created are deleted at that moment (Section 11.16.5); created_by_user_id becomes NULL only through anonymization of a user whose keys were already deleted.
5.8.2 ip_exclusions #
Purpose: IP addresses or ranges whose events are dropped for one site. This table stores addresses the site owner typed in as configuration (their own office or monitoring IPs), not visitor data; visitor IPs are never stored (Section 17).
CREATE TABLE ip_exclusions (
id uuid PRIMARY KEY DEFAULT uuidv7(),
site_id uuid NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
address cidr NOT NULL, -- single IPs are /32 (IPv4) or /128 (IPv6)
note text, -- 0..200 chars
created_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT ip_exclusions_note_check CHECK (note IS NULL OR char_length(note) <= 200),
CONSTRAINT ip_exclusions_prefix_check CHECK (masklen(address) >= CASE family(address) WHEN 4 THEN 8 ELSE 32 END)
);
CREATE UNIQUE INDEX ip_exclusions_site_address_key ON ip_exclusions (site_id, address);The minimum prefix (IPv4 /8, IPv6 /32) prevents excluding huge ranges by accident. A range with host bits set (10.0.0.1/24) is normalized by the service to its network (10.0.0.0/24) before insert. The UI also accepts a trailing-wildcard form (203.0.113.*) and converts it to a CIDR.
5.8.3 page_exclusions #
Purpose: URL path patterns whose pageviews and events are dropped for one site.
CREATE TABLE page_exclusions (
id uuid PRIMARY KEY DEFAULT uuidv7(),
site_id uuid NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
pattern text NOT NULL, -- starts with '/', 1..2000 chars; '*' matches within one path segment, '**' matches across segments; at most 8 wildcard tokens; matched against the stored pathname
created_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT page_exclusions_pattern_check CHECK (pattern LIKE '/%' AND char_length(pattern) BETWEEN 1 AND 2000),
-- a run of consecutive '*' counts as one wildcard token
CONSTRAINT page_exclusions_wildcards_check CHECK (
char_length(regexp_replace(pattern, '\*+', '*', 'g')) - char_length(replace(regexp_replace(pattern, '\*+', '*', 'g'), '*', '')) <= 8)
);
CREATE UNIQUE INDEX page_exclusions_site_pattern_key ON page_exclusions (site_id, pattern);Matching is case-sensitive and is performed by Section 8 against the same normalized pathname that is stored in ClickHouse (Section 6.3), with a linear-time glob matcher (never a backtracking regular expression), which is why the pattern length and the 8-wildcard limit are fixed here; the same wildcard limit applies to goal and funnel path patterns (Section 10). The pattern length of 1 to 2,000 characters is the single value for all path patterns. Input paths are normalized with the WHATWG path percent-encode set (non-ASCII characters and spaces encoded; existing %XX sequences kept with upper-case hex) before validation and storage, so /café matches the stored /caf%C3%A9; the UI shows decoded values.
5.9 Data Movement and Platform: imports, exports, salts, exchange_rates #
5.9.1 imports #
Purpose: one row per data import (Plausible-compatible CSV or Google Analytics 4). Rows written to the imported_* ClickHouse tables carry import_id = this row's id (Section 6.5). Import mechanics, statuses and the API object are owned by Section 14; this table carries exactly the columns Section 14 requires.
CREATE TABLE imports (
id uuid PRIMARY KEY DEFAULT uuidv7(),
site_id uuid NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
kind text NOT NULL,
status text NOT NULL DEFAULT 'queued',
status_detail text, -- short human-readable progress or wait reason
resume_at timestamptz, -- GA4 quota waits: when the worker retries
source_label text, -- "CSV upload (<n> files)" or the GA4 property display name; <= 200 chars
date_from date, -- first day covered; set when the source range is known
date_to date,
progress_pct smallint NOT NULL DEFAULT 0,
rows_imported jsonb NOT NULL DEFAULT '{}'::jsonb, -- per-table row counts
warnings jsonb NOT NULL DEFAULT '[]'::jsonb,
error_code text,
error_message text,
error_details jsonb, -- CSV validation errors: [{file, line, column, message}]
checkpoint jsonb, -- resume cursor
oauth_credentials text, -- GA4 only; AES-256-GCM text format of Section 17.7.2, purpose oauth-token-encryption, associated data "imports.oauth_credentials:<id>"; set to NULL when the import completes, fails permanently or is cancelled
ga4_property_id text, -- GA4 only, digits
ga4_property_timezone text,
can_retry boolean NOT NULL DEFAULT false,
cancel_requested_at timestamptz,
created_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
created_at timestamptz NOT NULL DEFAULT now(),
started_at timestamptz,
finished_at timestamptz,
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT imports_kind_check CHECK (kind IN ('csv','ga4')),
CONSTRAINT imports_status_check CHECK (status IN ('configuring','queued','validating','importing','waiting_quota','completed','failed','cancelled','deleting')),
CONSTRAINT imports_progress_check CHECK (progress_pct BETWEEN 0 AND 100),
CONSTRAINT imports_range_check CHECK ((date_from IS NULL) = (date_to IS NULL) AND (date_from IS NULL OR date_from <= date_to)),
CONSTRAINT imports_ga4_check CHECK (kind = 'ga4' OR (ga4_property_id IS NULL AND ga4_property_timezone IS NULL AND oauth_credentials IS NULL)),
CONSTRAINT imports_label_check CHECK (source_label IS NULL OR char_length(source_label) <= 200),
CONSTRAINT imports_json_check CHECK (jsonb_typeof(rows_imported) = 'object' AND jsonb_typeof(warnings) = 'array'),
-- Imports of one site may not cover overlapping days. Checked as soon as the range is known (before data is loaded) and again at completion.
CONSTRAINT imports_no_overlap EXCLUDE USING gist (site_id WITH =, daterange(date_from, date_to, '[]') WITH &&)
WHERE (date_from IS NOT NULL AND status IN ('queued','validating','importing','waiting_quota','completed'))
);
CREATE UNIQUE INDEX imports_one_active_per_site_key ON imports (site_id) WHERE status IN ('queued','validating','importing','waiting_quota');
CREATE INDEX imports_site_idx ON imports (site_id, created_at DESC);Rules: at most one active import (queued, validating, importing, waiting_quota) per site (a second attempt gets 409 import_in_progress); an exclusion-constraint violation (SQLSTATE 23P01) maps to 409 import_overlap. A configuring GA4 import (waiting for property and dates) is not active and does not block. Deleting an import sets status = 'deleting' and enqueues import-purge, which deletes its ClickHouse rows by import_id (Section 6.9) and then hard-deletes the imports row only when its status is deleting; rows of failed or cancelled imports are kept for their error message until the user dismisses them, and the dismissal runs the same purge for any rows already written. When a site is deleted its imports rows cascade away with it, so import-purge is not used there: site-purge with scope all deletes the site's imported_* rows (5.14, Section 6.9). import-sweep fails or deletes imports stuck in configuring or stalled, revokes and clears oauth_credentials after the retry window, and removes orphan uploaded files (Section 14).
5.9.2 exports #
Purpose: asynchronous full-site CSV exports (a ZIP of Plausible-compatible CSV files, owned by Section 14). Small single-report CSV downloads from the dashboard stream synchronously and create no row.
CREATE TABLE exports (
id uuid PRIMARY KEY DEFAULT uuidv7(),
site_id uuid NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
requested_by_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
status text NOT NULL DEFAULT 'queued',
date_from date NOT NULL, -- inclusive, site timezone
date_to date NOT NULL,
filters jsonb NOT NULL DEFAULT '[]'::jsonb, -- Stats API filter clauses (Section 13.4.5)
include_imports boolean NOT NULL DEFAULT true, -- what the requester asked for
imports_included boolean NOT NULL DEFAULT false, -- what the file actually contains (imports merge only when no filters apply)
file_path text, -- file-store path of the generated ZIP: exports/{site_id}/{export_id}.zip
file_size_bytes bigint,
row_counts jsonb, -- rows per CSV file
error_code text,
error_message text,
created_at timestamptz NOT NULL DEFAULT now(),
started_at timestamptz,
completed_at timestamptz,
expires_at timestamptz, -- set when completed: completed_at + 7 days (Section 14.3.1)
CONSTRAINT exports_status_check CHECK (status IN ('queued','running','completed','failed','expired')),
CONSTRAINT exports_range_check CHECK (date_from <= date_to),
CONSTRAINT exports_filters_check CHECK (jsonb_typeof(filters) = 'array'),
CONSTRAINT exports_size_check CHECK (file_size_bytes IS NULL OR file_size_bytes >= 0),
CONSTRAINT exports_expiry_check CHECK (expires_at IS NULL OR completed_at IS NOT NULL)
);
CREATE UNIQUE INDEX exports_one_active_per_site_key ON exports (site_id) WHERE status IN ('queued','running');
CREATE INDEX exports_site_idx ON exports (site_id, created_at DESC);
CREATE INDEX exports_expiry_idx ON exports (expires_at) WHERE status = 'completed';At most one active export per site (409 export_in_progress). Files are deleted and status set to expired at expires_at by export-expire (hourly); the row itself is deleted 30 days later (5.14). The 7-day lifetime is owned by Section 14.3.1.
5.9.3 salts #
Purpose: the daily rotating salts of the visitor hash (algorithm and rotation window owned by Section 8). At most two rows exist at any time: the current salt and, between 00:00 and 00:30 UTC, the previous one (salt-rotate creates the new salt at 00:00 UTC, salt-expire deletes the previous one at 00:30 UTC).
CREATE UNLOGGED TABLE salts (
id uuid PRIMARY KEY DEFAULT uuidv7(),
salt bytea NOT NULL, -- 16 random bytes from the OS CSPRNG
valid_from date NOT NULL, -- UTC date the salt becomes current
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT salts_length_check CHECK (octet_length(salt) = 16)
);
CREATE UNIQUE INDEX salts_valid_from_key ON salts (valid_from);Salts never reach a backup. salts is a Postgres UNLOGGED table: its changes are not written to the WAL, so it is absent from point-in-time recovery archives and from physical replicas, and it is excluded from logical dumps (pg_dump --exclude-table-data=salts; the backup procedure in Section 16 uses that flag). After a crash or a restore the table is empty; the first event or the next salt-rotate generates a fresh salt (visitors may be counted twice on that one day; this is documented in Section 8 and is the accepted price of the 24.5-hour salt lifetime). The salt value is a secret and is never logged.
5.9.4 exchange_rates #
Purpose: currency rates used to fill revenue_reporting_amount at ingestion (rules in Sections 8.14 and 10.4). Source: the ECB euro foreign exchange reference rates, fetched by fx-rates-fetch. The base is EUR (EUR itself is implicit and not stored).
CREATE TABLE exchange_rates (
rate_date date NOT NULL,
currency char(3) NOT NULL,
rate_per_eur numeric(20,10) NOT NULL, -- units of `currency` per 1 EUR
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
PRIMARY KEY (rate_date, currency),
CONSTRAINT exchange_rates_currency_check CHECK (currency ~ '^[A-Z]{3}$' AND currency <> 'EUR'),
CONSTRAINT exchange_rates_rate_check CHECK (rate_per_eur > 0)
);Ingestion uses the latest row on or before the event date for the currency, looking back at most 10 days (the same rule as Section 8.14); when no rate exists in that window (unsupported currency, or an air-gapped install that never fetched), revenue_reporting_amount is stored as NULL and the event still counts everywhere except in converted revenue totals. Rows older than 400 days are deleted by fx-rates-fetch after each successful fetch.
5.10 Licensing and Usage: license_tiers, licenses, license_code_batches, license_codes, purchases, usage_monthly, usage_site_monthly #
These tables exist in every install (one migration set, file 0007_licensing.sql, 5.13), but are only written when EDITION=hosted. In self-hosted installs they stay empty except for the seeded license_tiers (5.12). Their DDL, constraints and indexes are defined once, in Section 15.19, so that the business rules and the schema cannot drift apart. They keep their place in the entity-relationship diagram (5.15) and in the migration order (5.13); Section 5 does not repeat the column lists.
| Table | Purpose | DDL and constraints |
|---|---|---|
license_tiers |
The three lifetime-deal tiers (solo, pro, agency): price in minor units, site limit and monthly event cap; operator-editable at runtime |
Section 15.19 |
licenses |
The entitlement of a team: at most one active row per team, never deleted, revocation changes status |
Section 15.19 |
license_code_batches |
Groups of marketplace codes generated together for one listing | Section 15.19 |
license_codes |
Individual single-use marketplace codes (one unit each), stored in plaintext by design | Section 15.19 |
purchases |
Every Stripe checkout (new or upgrade) with payment result, refunds, dispute, invoice and credit-note numbering | Section 15.19 |
usage_monthly |
Billable-event counters per team per UTC month, kept current by the usage-reconcile job |
Section 15.19 |
usage_site_monthly |
Per-(team, site, month) ledger so deleting or transferring a site never lowers the month's team total; site_id has no foreign key |
Section 15.19 |
Cross-table rules that Section 5 fixes because they concern other Section 5 tables: licenses.team_id, purchases.team_id and license_codes.redeemed_team_id are ON DELETE SET NULL (financial records survive the team purge, 5.14), usage_monthly.team_id and usage_site_monthly.team_id are ON DELETE CASCADE, and licenses, purchases and code records are never deleted by a time-based job. Usage counters are reconciled by the usage-reconcile job (Section 16.6). Deleting a site and transferring a site both call freezeSiteUsage(team_id, site_id) (Section 15.9.4) inside their transaction, before the purge is enqueued or the site changes team, so the old team keeps the usage already counted.
5.11 audit_log #
Purpose: append-only record of security- and administration-relevant actions (event catalogue and field semantics in Section 11.20; retention and access in Section 17). It carries identifiers only; it never contains raw IPs, user agents, tokens or passwords. team_id, site_id and actor_api_key_id deliberately have no foreign keys so history survives deletion of the team, site or key.
CREATE TABLE audit_log (
id uuid PRIMARY KEY DEFAULT uuidv7(), -- time-ordered; used as the pagination cursor
occurred_at timestamptz NOT NULL DEFAULT now(),
actor_type text NOT NULL,
actor_user_id uuid REFERENCES users(id) ON DELETE SET NULL,
actor_api_key_id uuid, -- Sites API mutations: the key id (the key prefix goes in metadata); no FK because keys are deleted
on_behalf_of_user_id uuid REFERENCES users(id) ON DELETE SET NULL, -- impersonation: the user being viewed
team_id uuid,
site_id uuid,
action text NOT NULL, -- dotted lowercase, for example site.deleted, member.role_changed, license.revoked
target_type text,
target_id text,
reason text, -- operator console actions: free-text reason, required there (Section 15.17)
metadata jsonb NOT NULL DEFAULT '{}'::jsonb, -- <= 8 KB, ids and non-sensitive values only
request_id text,
CONSTRAINT audit_log_actor_check CHECK (actor_type IN ('user','super_admin','impersonation','api_key','system')),
CONSTRAINT audit_log_api_key_check CHECK ((actor_type = 'api_key') = (actor_api_key_id IS NOT NULL)),
CONSTRAINT audit_log_action_check CHECK (action ~ '^[a-z][a-z0-9_]*(\.[a-z][a-z0-9_]*)+$' AND char_length(action) <= 100),
CONSTRAINT audit_log_reason_check CHECK (reason IS NULL OR char_length(reason) BETWEEN 1 AND 500),
CONSTRAINT audit_log_metadata_check CHECK (jsonb_typeof(metadata) = 'object' AND pg_column_size(metadata) <= 8192)
);
CREATE INDEX audit_log_team_idx ON audit_log (team_id, id DESC) WHERE team_id IS NOT NULL;
CREATE INDEX audit_log_actor_idx ON audit_log (actor_user_id, id DESC) WHERE actor_user_id IS NOT NULL;
CREATE INDEX audit_log_action_idx ON audit_log (action, id DESC);
CREATE TRIGGER trg_audit_log_no_update BEFORE UPDATE ON audit_log FOR EACH ROW EXECUTE FUNCTION app_forbid_update();Rows can only be removed by audit-prune (5.14) and, when a team is purged, by team-purge for that team's rows — except the actions license.*, purchase.*, code.*, admin.* and team.purged, which are kept as financial and dispute evidence until audit-prune (Section 11.14.5).
5.12 Seed Data and Initial Accounts #
| Seed | Mechanism | Content |
|---|---|---|
license_tiers |
Migration 0009, plain SQL, ON CONFLICT (code) DO NOTHING so a re-run (or an operator-edited value) is never overwritten |
See SQL below |
| Everything else | None | No default teams, users, sites or goals are seeded. |
INSERT INTO license_tiers (code, name, rank, price_minor, currency, max_sites, monthly_event_cap, sort_order) VALUES
('solo', 'Solo', 1, 5900, 'USD', 5, 100000, 1),
('pro', 'Pro', 2, 14900, 'USD', 25, 500000, 2),
('agency', 'Agency', 3, 34900, 'USD', 100, 2000000, 3)
ON CONFLICT (code) DO NOTHING;Prices are stored in minor units (US cents): 5900, 14900 and 34900. is_active takes its default true.
Initial account (self-hosted). There is no seeded account and no first-signup bypass. The first account is created only through POST /api/app/v1/setup with the one-time SETUP_TOKEN (printed once to the worker log on first start when the variable is unset, Section 16.3.5); that account owns its personal team. Registration rules of Section 11.2 apply only after setup has completed. SUPER_ADMIN_EMAILS is ignored on the self-hosted edition and there is no operator console.
Initial super admin (hosted). There is no seeded admin account and no admin column. The operator sets env SUPER_ADMIN_EMAILS (comma-separated, case-insensitive) and the first account is created through the setup flow above; later accounts whose address is in the list sign up normally. An account is a super admin only while all of these hold: its email_verified_at was set by consuming a verify_email token (never by a no-SMTP shortcut), totp_enabled_at IS NOT NULL (otherwise operator routes return 403 admin_2fa_required), and its address is in SUPER_ADMIN_EMAILS. /admin (Section 15) then becomes available. Removing an address from the variable takes effect after the web processes are restarted (the list is read at process start). A super admin has no team privileges merely from being a super admin.
5.13 Migration Order and Process #
Migrations are forward-only (no down migrations), applied by the migrate command of the single Docker image (runner behavior, locking and checks in Section 6.10, which covers both databases). Drizzle Kit generates the table DDL from the schema files (drizzle-kit generate); everything Drizzle cannot express (extensions, functions, triggers, constraint triggers, exclusion constraints, UNLOGGED tables) is a hand-written file created with drizzle-kit generate --custom. Drizzle records applied files in its own migrations table in schema drizzle. The runner then starts pg-boss once with schema migration enabled so the pgboss schema exists before any web or worker process starts; web and worker start pg-boss with migration disabled.
| File | Contents (in dependency order) |
|---|---|
0000_extensions_functions.sql |
btree_gist, app_set_updated_at, app_jsonb_key_count, app_forbid_update |
0001_identity.sql |
users, user_recovery_codes, auth_sessions, email_tokens |
0002_teams.sql |
teams, team_memberships, app_assert_team_has_owner and its constraint trigger |
0003_sites.sql |
sites (with the script_id immutability trigger), site_settings, site_guests, invitations, site_domain_reservations |
0004_site_config.sql |
goals, funnels, funnel_steps, custom_property_allowlist, segments, annotations |
0005_sharing_access.sql |
shared_links, email_report_subscriptions, spike_notifications, spike_notification_recipients, api_keys, ip_exclusions, page_exclusions |
0006_data_platform.sql |
imports (with the exclusion constraint), exports, salts (created UNLOGGED), exchange_rates |
0007_licensing.sql |
license_tiers, licenses, license_code_batches, license_codes, purchases, usage_monthly, usage_site_monthly |
0008_audit_and_triggers.sql |
audit_log and its trigger; the trg_<table>_updated_at triggers for the 20 tables listed in 5.2 |
0009_seed_license_tiers.sql |
The seed in 5.12 |
Later migrations continue the sequence (0010_...). Rules for every future migration: additive first (new columns nullable or defaulted, new tables, new indexes CREATE INDEX CONCURRENTLY in a migration of their own with no other statements), destructive changes (drop column, tighten constraint) only in the release after all code stopped using the column, so a rolling deploy never runs old code against a schema it cannot read. A migration that fails leaves the schema unchanged (each file runs in one transaction, except files that contain CONCURRENTLY).
5.14 Retention and Cleanup Matrix #
Jobs are registered and scheduled in Section 16.6 (the single source of job names and schedules); the "Cadence" column states how often this section requires the job to run. Every purge job is idempotent and deletes in batches of at most 5,000 rows per statement.
| Table | Retention rule | Job (cadence) |
|---|---|---|
users |
Soft-deleted users are anonymized in place 30 days after deleted_at; anonymized rows are kept indefinitely |
user-anonymize (daily) |
users (never verified) |
Accounts that never verified their email, have no sites and no membership in another team, and are older than 14 days are deleted (Section 11.3.3) | unverified-user-cleanup (Section 16.6) |
user_recovery_codes |
Deleted on regenerate and on user anonymization; used codes kept until then | none (event-driven) |
auth_sessions |
Delete rows where expires_at < now() - interval '1 day' or absolute_expires_at < now() |
session-cleanup (hourly) |
email_tokens |
Delete rows expired or used more than 7 days ago | session-cleanup (hourly) |
teams |
Hard-delete when purge_after < now(): first delete each site through the site-delete service, then the team (cascades to memberships, invitations, API keys, usage). Licenses, purchases and codes remain with team_id set NULL |
team-purge (hourly) |
team_memberships, site_guests |
Removed with their team/site/user; no time-based purge | none |
invitations |
Delete pending rows expired more than 30 days ago and accepted rows older than 30 days | invitation-expire (hourly) |
email_report_subscriptions, spike_notification_recipients |
External recipients still in pending_confirmation 7 days after created_at are deleted |
invitation-expire (hourly) |
site_domain_reservations |
Rows with reserved_until < now() are ignored by the claim check and deleted |
site-orphan-reconcile (daily) |
sites and all site children (site_settings, goals, funnels, funnel_steps, custom_property_allowlist, segments, annotations, shared_links, email_report_subscriptions, spike_notifications, spike_notification_recipients, ip_exclusions, page_exclusions, imports, exports) |
Deleted by cascade when the site is deleted; ClickHouse data purged by site-purge (below) |
site-purge (queue-driven, started in the delete transaction; Section 6.9) |
| ClickHouse rows of deleted sites missed by the queue | Detected by comparing ClickHouse site_ids with sites.id |
site-orphan-reconcile (daily) |
| Unlicensed teams' analytics data | ClickHouse data of all sites of a team whose license has purge_after < now() and purged_at IS NULL (licenses on dispute hold have purge_after IS NULL and are skipped); sites and settings are kept; sets licenses.purged_at; enqueues site-purge with scope native per site |
unlicensed-purge (daily) |
exports |
Files deleted and status set to expired at expires_at (7 days after completion); the row is deleted 30 days after expires_at; failed exports deleted after 7 days |
export-expire (hourly) |
imports |
Uploaded files and stored credentials removed per Section 14; imports stuck in configuring or stalled are failed or deleted by the sweep; a deleted import's ClickHouse rows are removed by import-purge, and the imported_* rows of a deleted site by site-purge scope all; rows themselves live until the user deletes them or the site is deleted |
import-sweep (hourly), import-purge (queue-driven) |
salts |
Never more than 2 rows; previous salt deleted at 00:30 UTC; the table is UNLOGGED and excluded from dumps (5.9.3) |
salt-rotate (00:00 UTC) and salt-expire (00:30 UTC), Section 8 |
exchange_rates |
Rows older than 400 days deleted after each successful fetch | fx-rates-fetch (daily) |
audit_log |
Hosted: delete rows older than AUDIT_RETENTION_MONTHS (default 24). Self-hosted: kept indefinitely (the job is not registered) |
audit-prune (weekly, hosted only) |
usage_monthly, usage_site_monthly, licenses, license_tiers, license_code_batches, license_codes, purchases |
Never purged by a time-based job (financial and entitlement records; usage rows disappear only with their team's hard delete) | none |
shared_links, api_keys, segments, goals, funnels, annotations, custom_property_allowlist |
No time-based purge; removed by user action or site deletion | none |
pgboss schema |
pg-boss maintenance deletes completed jobs after 7 days and failed jobs after 30 days (configured in Section 16) | managed by pg-boss |
5.15 Entity-Relationship Diagram #
erDiagram
users ||--o{ user_recovery_codes : has
users ||--o{ auth_sessions : has
users ||--o{ email_tokens : has
users ||--o{ team_memberships : joins
teams ||--o{ team_memberships : has
teams ||--o{ sites : owns
teams ||--o{ invitations : issues
teams ||--o{ api_keys : has
teams ||--o{ licenses : holds
teams ||--o{ usage_monthly : metered_by
teams ||--o{ usage_site_monthly : metered_per_site
teams ||--o{ purchases : pays
sites ||--|| site_settings : configured_by
sites ||--o{ site_guests : grants
users ||--o{ site_guests : is_guest
sites ||--o{ invitations : guest_invites
sites ||--o{ goals : defines
sites ||--o{ funnels : defines
funnels ||--o{ funnel_steps : ordered
goals ||--o{ funnel_steps : referenced_by
sites ||--o{ custom_property_allowlist : registers
sites ||--o{ segments : saves
users ||--o{ segments : personal
segments ||--o{ shared_links : locks
sites ||--o{ annotations : annotated_by
sites ||--o{ shared_links : shares
sites ||--o{ email_report_subscriptions : reports
users ||--o{ email_report_subscriptions : subscribes
sites ||--o| spike_notifications : alerts
spike_notifications ||--o{ spike_notification_recipients : notifies
sites ||--o{ ip_exclusions : excludes
sites ||--o{ page_exclusions : excludes
sites ||--o{ imports : imports
sites ||--o{ exports : exports
license_tiers ||--o{ licenses : grants
license_tiers ||--o{ license_code_batches : listed_as
license_tiers ||--o{ purchases : sells
license_code_batches ||--o{ license_codes : contains
licenses ||--o{ license_codes : funded_by
licenses ||--o{ purchases : paid_by
users ||--o{ audit_log : actssalts, exchange_rates, site_domain_reservations and audit_log (apart from its actor columns) have no foreign keys. usage_site_monthly.site_id is a plain value (no foreign key) so a deleted or transferred site keeps its usage history. ClickHouse tables reference sites.id by value only (Section 6).
5.16 Limits Enforced by Services #
Row-count limits cannot be expressed as constraints; the owning service enforces them in the same transaction as the insert and returns 409 limit_reached with details: { "limit_name", "limit", "current" } (error catalogue, Section 4.4.3). These values are binding for Sections 7 to 15 and 17; other sections cite this table instead of restating the numbers.
| Resource | Maximum | details.limit_name |
|---|---|---|
| Goals per site | 100 | goals |
| Funnels per site | 50 (each 2–8 steps) | funnels |
| Registered custom property names per site | 50 | custom_properties |
| Site segments per site | 100 | site_segments |
| Personal segments per user per site | 100 | personal_segments |
| Shared links per site | 50 | shared_links |
| Email report recipients per site | 25 per frequency (weekly and monthly are counted separately) | report_subscriptions |
| Spike notification recipients per site | 10 | spike_recipients |
| IP exclusions per site | 200 | ip_exclusions |
| Page exclusions per site | 200 | page_exclusions |
| Allowed hostnames per site | 50 | allowed_hostnames |
| File-download extensions per site | 50 | file_download_extensions |
| Wildcard tokens per path pattern (page exclusions, goals, funnel steps) | 8 | validation error (validation_failed), not a row count |
| API keys per team | 50 | api_keys |
| Pending invitations per team | 100 | pending_invitations |
| Annotations per site | 5,000 (personal and shared together) | annotations |
Pinned sites per user (users.preferences.pinned_sites) |
20 | pinned_sites |
| Teams owned per user | 20 | teams_owned |
| Guests per site | 50 | site_guests |
Imports per site (any status except deleting; at most 1 active at a time, Section 14) |
10 | imports |
| Sites per team | Self-hosted: unlimited. Hosted: the active license's max_sites, else 1 for an unlicensed team (409 site_limit_reached, Section 15) |
n/a |
6. ClickHouse Event Store #
ClickHouse stores every analytics event and session. It is append-only from the application's point of view: rows are inserted by the ingestion pipeline (Section 8) and the import jobs (Section 14), read by the query engine (Section 13), and removed only by TTL and by the purge jobs in 6.9. Accounts, settings and sites are in Postgres (Section 5); ClickHouse knows a site only by its UUID.
6.1 Design Decisions #
| Topic | Decision |
|---|---|
| Database | One database, name from env CLICKHOUSE_DATABASE (default quietstat), created by the migrate command if missing. Tables are not prefixed. |
| Topology | One ClickHouse server (single shard, no replication) is the supported topology; DDL uses plain MergeTree-family engines. Replicated clusters are out of scope. Hosted sizing is in 6.11 and Section 16. |
| Tables | events, sessions, nine imported_* tables, schema_migrations. Nothing else. |
| Denormalization | Session-level attributes (referrer, UTM, channel, browser, OS, device, screen size, geography) are stored on both events and sessions, with identical values for every event of one session (frozen at session start). This lets the query engine filter and group events by source, location or device without joining sessions. |
| Pre-aggregation | No materialized views and no projections in v1 (6.7). All reports read the raw tables, so results are exact and there is no second copy of the data to keep consistent. |
| Sampling | SAMPLE BY user_id is declared on events and sessions so sampled reads can be added later without rewriting data. The v1 query engine never samples: every result is exact. |
| Unknown values | Dimension columns use the empty string for "not available" (and 0 for numeric ids, NULL only for revenue). The UI labels empty values "(not set)" or "Direct / None" per Section 9. No Nullable strings. |
| Time | DateTime('UTC') everywhere, except events.timestamp which is DateTime64(3, 'UTC') (millisecond precision) so that a pageview and an event recorded in the same second keep a stable order for funnels and strict-order evaluation. Site time zones are applied only at query time (Section 13). |
| Identity | site_id is the Postgres sites.id UUID. user_id and session_id are UInt64 (visitor hash and random session id, Section 8). |
| Retention | Optional TTL on events and sessions from env DATA_RETENTION_MONTHS (default 0 = keep forever); imported tables have no TTL (6.6). |
| Access | The application connects over HTTP with two dedicated users, both limited to their own database. Read-write user (credentials in CLICKHOUSE_URL, Section 16.4): INSERT, ALTER ... DELETE and DDL; used only by ingestion, the purge jobs, the import and export writers and the migrate command. Read-only user (env CLICKHOUSE_RO_USER / CLICKHOUSE_RO_PASSWORD): SELECT only, with a settings profile that pins readonly = 2 and the server-side ceilings of Section 13.4.11 (max_execution_time, max_memory_usage, max_rows_to_read, max_result_rows); used by the query engine and therefore by every dashboard, share, public-dashboard and Stats API read. An injection bug in the query builder can therefore never reach DDL or deletes. The read-only user is required when EDITION=hosted; when both CLICKHOUSE_RO_* variables are unset in a self-hosted install the query engine falls back to the read-write user and sets readonly = 2 on every statement (environment table: Section 16.4). |
6.2 Conventions for the Column Definitions #
- Column order below is the physical order. Inserts name their columns explicitly (JSONEachRow), so reordering never matters.
LowCardinality(String)is used for any column with fewer than roughly 100,000 distinct values per instance; plainStringwithCODEC(ZSTD(1))for free-form or unbounded values (paths, referrers, UTM content/term, props).- Codecs:
Delta, ZSTD(1)on timestamps;ZSTD(1)on large strings and arrays; default (LZ4) elsewhere. - Strings never contain a query string, a fragment (unless hash routing, 6.3) or an IP address; those rules are enforced at ingestion (Section 8).
- Browser and OS versions are stored as
major.minoronly (for example140.0,15.1); a value with no minor part is stored asmajor.0(for example Chrome124→124.0, Windows10.0, macOS14.0). Section 8 applies the same rule when it parses user agents.
6.3 Table events #
One row per accepted pageview, custom event or engagement event. Engagement rows are non-billable and non-interactive (Section 9 defines the metrics built from them).
CREATE TABLE IF NOT EXISTS events
(
site_id UUID,
timestamp DateTime64(3, 'UTC') CODEC(Delta, ZSTD(1)),
name LowCardinality(String),
session_id UInt64,
user_id UInt64,
hostname LowCardinality(String),
pathname String CODEC(ZSTD(1)),
referrer String CODEC(ZSTD(1)),
referrer_source LowCardinality(String),
channel LowCardinality(String),
utm_medium LowCardinality(String),
utm_source LowCardinality(String),
utm_campaign LowCardinality(String),
utm_content String CODEC(ZSTD(1)),
utm_term String CODEC(ZSTD(1)),
browser LowCardinality(String),
browser_version LowCardinality(String),
os LowCardinality(String),
os_version LowCardinality(String),
device Enum8('' = 0, 'desktop' = 1, 'mobile' = 2, 'tablet' = 3),
screen_size Enum8('' = 0, 'mobile' = 1, 'tablet' = 2, 'laptop' = 3, 'desktop' = 4),
country_code LowCardinality(String),
subdivision1_code LowCardinality(String),
city_geoname_id UInt32,
`meta.key` Array(String) CODEC(ZSTD(1)),
`meta.value` Array(String) CODEC(ZSTD(1)),
revenue_amount Nullable(Decimal(18, 3)),
revenue_currency LowCardinality(String),
revenue_reporting_amount Nullable(Decimal(18, 3)),
revenue_reporting_currency LowCardinality(String),
scroll_depth UInt8,
engagement_time UInt32,
interactive Bool DEFAULT true
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(timestamp)
ORDER BY (site_id, toDate(timestamp), user_id, timestamp)
SAMPLE BY user_id
SETTINGS index_granularity = 8192, non_replicated_deduplication_window = 1000;| Column | Meaning and rules |
|---|---|
site_id |
Site UUID. First sorting-key column; every query filters on it. |
timestamp |
Server receipt time (UTC, ms). Clients never supply it. |
name |
pageview, engagement, or the custom event name (1–120 chars as sent). Built-in event names produced by tracker extensions (for example outbound link or file download events) are ordinary custom names defined in Section 7. |
session_id |
Random UInt64 assigned at session start (Section 8). |
user_id |
UInt64 SipHash-2-4 daily visitor hash (Section 8). Meaningful only within a day, never joinable across days or sites. |
hostname |
Hostname of the page URL (u), lowercase, no www. stripping, no port. |
pathname |
Path of the page URL starting with /, ≤ 2,000 chars, query string removed. Trailing slash is preserved except for the root. With hash routing on (h=1), the hash route is appended (/app#/settings); the fragment's own query string is dropped; without hash routing the fragment is dropped. Custom-event rows store the path of the page the event happened on. |
referrer |
Referring URL normalized as host + path (no scheme, no www., no query), ≤ 500 chars; empty for direct visits and for self-referrals (same hostname). Session-level. |
referrer_source |
Classified source name (for example Google, Hacker News, ChatGPT) or the referrer hostname if unclassified; set from utm_source/ref/source when present; empty = direct. Session-level; classification rules in Section 8. |
channel |
One of the 18 channel names in Section 8: Direct, Organic Search, Paid Search, Organic Social, Paid Social, Email, Affiliates, Display, Organic Video, Paid Video, Cross-network, Organic Shopping, Paid Shopping, Audio, SMS, Mobile Push Notifications, Referral, AI Assistants. Never empty on events and sessions rows (Direct is the fallback). Session-level. |
utm_medium, utm_source, utm_campaign, utm_content, utm_term |
Values of the UTM parameters captured at session start, each trimmed and capped at 300 chars; empty when absent. utm_medium is lowercased; the other four keep the case as sent (Section 8.10.5). Session-level. |
browser, browser_version |
Parsed user agent (Section 8); version is major.minor. Empty if unparseable. Session-level. |
os, os_version |
Operating system name and major.minor version. Session-level. |
device |
Device class from the user agent: desktop, mobile, tablet; '' unknown. Smart TVs, game consoles, wearables and embedded devices are stored as ''. Session-level. The dashboard "Devices > Device" report (visit:device, displayed as Desktop, Mobile, Tablet) groups by this column. |
screen_size |
Class derived from the tracker-reported screen width w: mobile, tablet, laptop, desktop (thresholds in Section 8); '' when no width was sent. The width itself is not stored. Session-level. The dashboard "Devices > Screen size" report and the visit:screen_size dimension use this column, never device. |
country_code |
ISO 3166-1 alpha-2, uppercase; '' unknown. Session-level. |
subdivision1_code |
ISO 3166-2 code with country prefix (US-CA, DE-BE); '' unknown. Session-level. |
city_geoname_id |
GeoNames id of the city; 0 unknown. City names are resolved at query time from the bundled GeoNames data (packages/shared/data/cities.json and subdivisions.json, Section 8.10.3). Session-level. |
meta.key, meta.value |
Custom properties as parallel arrays. Invariant: length(meta.key) = length(meta.value), ≤ 30 pairs, keys ≤ 300 chars, values ≤ 2,000 chars, keys unique within a row, order as sent. Read with meta.value[indexOf(meta.key, 'plan')] (empty string when absent) and has(meta.key, 'plan'). All properties are stored whether or not registered (Section 5.6.4); in the hosted edition the combined key and value length per event is additionally capped at ingestion (Section 8.4). |
revenue_amount, revenue_currency |
Amount as sent (exact decimal, rounded half up to 3 fractional digits) and its ISO 4217 code; NULL / '' when the event carries no revenue. Range 0 ≤ amount ≤ 999,999,999,999.999; an out-of-range or malformed amount rejects the whole event at ingestion (Section 8.14). |
revenue_reporting_amount, revenue_reporting_currency |
revenue_amount converted into the site's reporting_currency (Section 5.5.2) with the rate for the event date, looking back at most 10 days (Section 5.9.4, Section 8.14); rounded half up to 3 decimals. Equal to the source amount when the currencies match; NULL / '' when no rate was available. Frozen at ingestion. Goal-currency totals are derived from these columns or converted at query time (Section 10.4.3). |
scroll_depth |
engagement rows: maximum scroll percentage 0–100 reached on the page when the event was sent. 0 on all other rows. |
engagement_time |
engagement rows: milliseconds the page was actively engaged since the previous engagement event for the same page view (Section 7), capped at 3,600,000. 0 on all other rows. |
interactive |
true for pageviews and custom events by default; false for engagement and for custom events sent with i:false. Interactive custom events prevent a session from being a bounce (Section 9). |
Design notes:
- Partitioning by month of
timestampkeeps TTL and purges cheap and bounds the parts a query touches. A session that crosses a month boundary has events in two partitions; this is harmless forevents(sessions have their own table). - ORDER BY
(site_id, toDate(timestamp), user_id, timestamp): reports always restrict by site and a date range (prefix pruning), and rows of one visitor on one day are contiguous, which makes funnel, journey and per-visitor aggregations cheap.user_idalso serves as the sampling key. No secondary indexes in v1; goal and event-name filters read only thenamecolumn inside the pruned range. - Deduplication:
non_replicated_deduplication_window = 1000makes a retried insert of a byte-identical batch a no-op. Section 8 therefore retries a failed batch unchanged and never re-splits it. timestamphas no TTL clause in the migration; TTL is applied by the runner (6.6).
6.4 Table sessions #
One logical row per session. The engine is VersionedCollapsingMergeTree(sign, version): a session's state is updated by inserting a cancel row (the previous state with sign = -1 and the same version) and a new state row (sign = 1, version + 1).
CREATE TABLE IF NOT EXISTS sessions
(
site_id UUID,
session_id UInt64,
user_id UInt64,
started_at DateTime('UTC') CODEC(Delta, ZSTD(1)),
ended_at DateTime('UTC') CODEC(Delta, ZSTD(1)),
duration UInt32,
pageviews UInt32,
events UInt32,
interactive_events UInt32,
is_bounce UInt8,
entry_page String CODEC(ZSTD(1)),
entry_hostname LowCardinality(String),
exit_page String CODEC(ZSTD(1)),
exit_hostname LowCardinality(String),
referrer String CODEC(ZSTD(1)),
referrer_source LowCardinality(String),
channel LowCardinality(String),
utm_medium LowCardinality(String),
utm_source LowCardinality(String),
utm_campaign LowCardinality(String),
utm_content String CODEC(ZSTD(1)),
utm_term String CODEC(ZSTD(1)),
browser LowCardinality(String),
browser_version LowCardinality(String),
os LowCardinality(String),
os_version LowCardinality(String),
device Enum8('' = 0, 'desktop' = 1, 'mobile' = 2, 'tablet' = 3),
screen_size Enum8('' = 0, 'mobile' = 1, 'tablet' = 2, 'laptop' = 3, 'desktop' = 4),
country_code LowCardinality(String),
subdivision1_code LowCardinality(String),
city_geoname_id UInt32,
sign Int8,
version UInt32
)
ENGINE = VersionedCollapsingMergeTree(sign, version)
PARTITION BY toYYYYMM(started_at)
ORDER BY (site_id, toDate(started_at), user_id, session_id)
SAMPLE BY user_id
SETTINGS index_granularity = 8192, non_replicated_deduplication_window = 1000;| Column | Meaning and rules |
|---|---|
site_id, session_id, user_id |
As in events. (site_id, session_id) identifies the session; started_at never changes, so every version of a session sorts under the same key and lands in the same partition (required for collapsing to work). |
started_at |
Timestamp (seconds) of the first event of the session. |
ended_at |
Timestamp (seconds) of the latest event of the session, including engagement events. |
duration |
Seconds, always ended_at − started_at (stored so that sums need no arithmetic). 0 for single-event sessions. |
pageviews |
Number of pageview events. |
events |
Number of billable events: pageviews plus custom events; engagement events are excluded. This is what the events metric sums (Section 9). |
interactive_events |
Number of custom events (not pageviews) with interactive = true. |
is_bounce |
1 iff pageviews = 1 AND interactive_events = 0, otherwise 0. Recomputed on every state write. A session without any pageview is never a bounce. |
entry_page, entry_hostname |
Path and hostname of the first event of the session (normally a pageview). Never changes after session start. |
exit_page, exit_hostname |
Path and hostname of the most recent pageview (equal to the entry values until a later pageview arrives). Custom and engagement events do not change them. |
referrer … city_geoname_id |
Same columns, types, normalization and values as in events (6.3); frozen at session start, identical across all versions. |
sign |
1 = state row, -1 = cancel row. |
version |
Starts at 1; each state update uses the previous version plus one. A cancel row carries the version of the row it cancels. |
Write protocol (executed by Section 8; stated here because the table depends on it):
- New session: insert one row with
sign = 1,version = 1. - Any later event of the session: insert two rows in the same INSERT block (same batch): the previous state with
sign = -1(sameversion), and the new state withsign = 1,version = previous + 1. The ingestion buffer never splits the pair across batches. The previous state is held in Valkey under the session key, so no read of ClickHouse is needed. The cancel row is a copy of the last state row confirmed inserted into ClickHouse (theflushedstate and thepending_bymarker kept next to the live state in Valkey; the cancel-row rule is defined in Section 8.12.4), not of the latest in-memory state, so a version that was never written (crash, dropped batch) can never leave an uncancelled predecessor behind. - If a session's previous state is lost from Valkey (eviction, restart) or no confirmed
flushedstate exists, the next event starts a new session (newsession_id); a half-written pair is never produced and orphaned versions cannot occur. The sign-weighted read contract below therefore never has to tolerate an orphan cancel row. - Cancel and state rows are physically merged away by background merges; until then both exist.
Read contract (used by Section 13): never read sessions without accounting for sign. Two equivalent patterns are allowed: (a) sign-weighted aggregates over the raw table, for example sum(sign) (visits), sum(sign * pageviews), sum(sign * is_bounce), sum(sign * duration); or (b) FINAL / an inner GROUP BY site_id, session_id HAVING sum(sign) > 0 with argMax(col, version) for small ranges and for columns that change between versions (ended_at, duration, pageviews, events, interactive_events, is_bounce, exit_page, exit_hostname). Distinct visitor counts grouped by a session-invariant column (everything except those eight) may use uniqIf(user_id, sign > 0); grouping visitors by exit_page must use pattern (b). The invariants above guarantee that a cancel row always equals the row it cancels in every column except sign, so weighted sums cancel exactly.
6.5 Imported Tables #
Pre-aggregated daily rows loaded by CSV and GA4 imports (Section 14). Every table has the shared leading columns site_id UUID, import_id UUID (= imports.id, Section 5.9.1) and date Date (a day in the site's timezone as supplied by the source), partitioned by toYYYYMM(date), ordered by (site_id, date, <dimension columns>), plain MergeTree, no TTL. Counters are UInt64. Dimension value rules match the native tables (empty = unknown; major.minor versions; the device enum as in events). Import loaders normalize source values into these shapes, including channel via the classifier in Section 8.
CREATE TABLE IF NOT EXISTS imported_visitors (
site_id UUID, import_id UUID, date Date,
visitors UInt64, pageviews UInt64, bounces UInt64, visits UInt64, visit_duration UInt64
) ENGINE = MergeTree PARTITION BY toYYYYMM(date) ORDER BY (site_id, date);
CREATE TABLE IF NOT EXISTS imported_sources (
site_id UUID, import_id UUID, date Date,
source LowCardinality(String), referrer String CODEC(ZSTD(1)), channel LowCardinality(String),
utm_source LowCardinality(String), utm_medium LowCardinality(String), utm_campaign LowCardinality(String),
utm_content String CODEC(ZSTD(1)), utm_term String CODEC(ZSTD(1)),
visitors UInt64, visits UInt64, pageviews UInt64, bounces UInt64, visit_duration UInt64
) ENGINE = MergeTree PARTITION BY toYYYYMM(date) ORDER BY (site_id, date, source, channel);
CREATE TABLE IF NOT EXISTS imported_pages (
site_id UUID, import_id UUID, date Date,
hostname LowCardinality(String), page String CODEC(ZSTD(1)),
visitors UInt64, pageviews UInt64, exits UInt64,
total_scroll_depth UInt64, total_scroll_depth_visits UInt64,
total_time_on_page UInt64, total_time_on_page_visits UInt64
) ENGINE = MergeTree PARTITION BY toYYYYMM(date) ORDER BY (site_id, date, hostname, page);
CREATE TABLE IF NOT EXISTS imported_entry_pages (
site_id UUID, import_id UUID, date Date,
entry_page String CODEC(ZSTD(1)),
visitors UInt64, entrances UInt64, pageviews UInt64, bounces UInt64, visit_duration UInt64
) ENGINE = MergeTree PARTITION BY toYYYYMM(date) ORDER BY (site_id, date, entry_page);
CREATE TABLE IF NOT EXISTS imported_exit_pages (
site_id UUID, import_id UUID, date Date,
exit_page String CODEC(ZSTD(1)),
visitors UInt64, exits UInt64, pageviews UInt64
) ENGINE = MergeTree PARTITION BY toYYYYMM(date) ORDER BY (site_id, date, exit_page);
CREATE TABLE IF NOT EXISTS imported_locations (
site_id UUID, import_id UUID, date Date,
country_code LowCardinality(String), subdivision1_code LowCardinality(String), city_geoname_id UInt32,
visitors UInt64, visits UInt64, pageviews UInt64, bounces UInt64, visit_duration UInt64
) ENGINE = MergeTree PARTITION BY toYYYYMM(date) ORDER BY (site_id, date, country_code, subdivision1_code, city_geoname_id);
CREATE TABLE IF NOT EXISTS imported_devices (
site_id UUID, import_id UUID, date Date,
device Enum8('' = 0, 'desktop' = 1, 'mobile' = 2, 'tablet' = 3),
visitors UInt64, visits UInt64, pageviews UInt64, bounces UInt64, visit_duration UInt64
) ENGINE = MergeTree PARTITION BY toYYYYMM(date) ORDER BY (site_id, date, device);
CREATE TABLE IF NOT EXISTS imported_browsers (
site_id UUID, import_id UUID, date Date,
browser LowCardinality(String), browser_version LowCardinality(String),
visitors UInt64, visits UInt64, pageviews UInt64, bounces UInt64, visit_duration UInt64
) ENGINE = MergeTree PARTITION BY toYYYYMM(date) ORDER BY (site_id, date, browser, browser_version);
CREATE TABLE IF NOT EXISTS imported_operating_systems (
site_id UUID, import_id UUID, date Date,
os LowCardinality(String), os_version LowCardinality(String),
visitors UInt64, visits UInt64, pageviews UInt64, bounces UInt64, visit_duration UInt64
) ENGINE = MergeTree PARTITION BY toYYYYMM(date) ORDER BY (site_id, date, os, os_version);
CREATE TABLE IF NOT EXISTS imported_custom_events (
site_id UUID, import_id UUID, date Date,
name LowCardinality(String), link_url String CODEC(ZSTD(1)), path String CODEC(ZSTD(1)),
visitors UInt64, events UInt64
) ENGINE = MergeTree PARTITION BY toYYYYMM(date) ORDER BY (site_id, date, name, link_url, path);| Table | Feeds report(s) | Notes |
|---|---|---|
imported_visitors |
Top-level metrics and time series | One row per day. |
imported_sources |
Sources, channels, UTM campaigns | channel is derived at import time. GA4 rows without UTM data leave the utm_* columns empty. |
imported_pages |
Pages, time on page, scroll depth | exits feeds exit rate. The total_* columns are numerators; the *_visits columns are the matching denominators (averages = total ÷ visits). |
imported_entry_pages, imported_exit_pages |
Entry pages, exit pages | entrances/exits are session counts. |
imported_locations |
Countries, regions, cities | Unresolvable GA4 regions/cities are stored as empty and appear as "(not set)". |
imported_devices, imported_browsers, imported_operating_systems |
Devices, browsers, OS | Screen-size classes are not imported (no source provides them in a compatible form), so the screen-size report shows native data only. |
imported_custom_events |
Goal conversions for custom event goals | Matched to goals by name only; imported rows have no custom properties, so goals narrowed by properties and revenue goals exclude imported data. link_url and path come from outbound-link and file-download events. |
How imported rows are combined with native events/sessions rows in each report (date coverage, overlap, which metrics and filters cannot include imported data, the "include imported data" toggle) is owned by Section 13.4.10 (engine) and Section 9.21 (UI). In short, imported rows are used only for dates strictly earlier than the site's first native day. This section guarantees only that imported tables are additive daily rows keyed by (site_id, import_id, date) and that Section 5.9.1 prevents two imports of one site from covering the same day.
6.6 Retention (TTL) #
Env DATA_RETENTION_MONTHS (integer ≥ 0; default 0; canonical entry in Section 16.4) controls how long native analytics rows are kept. 0 means keep forever, which is the default for both editions (a lifetime deal has no retention tier). Any other value must be an integer from 1 to 240, otherwise the migrate command and the web/worker boot checks fail with invalid DATA_RETENTION_MONTHS.
| Table | TTL expression when DATA_RETENTION_MONTHS = N > 0 |
|---|---|
events |
toDateTime(timestamp) + toIntervalMonth(N) |
sessions |
started_at + toIntervalMonth(N) (all versions of a session share started_at, so a session's rows expire together) |
imported_* |
None. Imported history is the owner's own data and is exempt, otherwise any import older than N months would be deleted immediately. It is removed only by deleting the import or the site. |
The migration files contain no TTL clause. After applying migrations, the runner executes a reconcile step on every run, which makes the live TTL match the env value:
-- N > 0
ALTER TABLE events MODIFY TTL toDateTime(timestamp) + toIntervalMonth({N});
ALTER TABLE sessions MODIFY TTL started_at + toIntervalMonth({N});
-- N = 0
ALTER TABLE events REMOVE TTL;
ALTER TABLE sessions REMOVE TTL;The runner reads SHOW CREATE TABLE first and issues ALTER only if the current TTL differs, so repeated runs are no-ops. Both tables keep the default ttl_only_drop_parts = 0, so expiry is exact per row (not per month); expired rows are physically removed by background merges (merge_with_ttl_timeout default) and, after lowering N, by the part rewrite that MODIFY TTL triggers (materialize_ttl_after_modify default). Raising N never resurrects rows that are already deleted. Queries never assume that data older than N months is gone: rows past the TTL may remain visible for up to a day until the next merge. The imported-table exemption is deliberate and final: Section 16.4 states that only events and sessions carry the TTL.
6.7 Materialized Views and Projections #
Decision: none in v1. Rationale: (1) the hosted caps (at most 2,000,000 billable events per month per team) and typical self-hosted volumes make scans over a site's date range, pruned by (site_id, date), fast enough on the raw tables; (2) sessions already is the pre-aggregated per-visit table that session-level reports use; (3) a second aggregate copy would need its own backfill, purge and correction paths (site deletion, TTL, imports) and would complicate exactness. The only structures beyond the base tables are the sorting keys above. If a future release adds one, it ships as a new numbered migration that creates the view and backfills it, and the purge in 6.9 must be extended to include it in the same release.
6.8 Access Rules for the Query Engine #
These are properties of the schema that Section 13 relies on; the engine itself is defined there.
- Always filter by
site_idand a time range (timestampforevents,started_atforsessions,dateforimported_*) so the sorting key and monthly partitions prune. A query without asite_idpredicate must be rejected by the query layer. Every read runs as the read-only ClickHouse user (6.1). - Time ranges are converted from the site's IANA timezone to UTC bounds by the query engine; they are half-open
[from, to). Bucketing (hour/day/week/month) uses the site timezone. - Events of one visitor with identical millisecond timestamps have no defined relative order; Section 10 defines how funnels and strict-order evaluation treat such ties.
sessionsmust be read withsignhandling (6.4).eventsandimported_*rows are plain facts and are summed or counted directly.- Property reads use the parallel arrays (6.3); filters on a property compare
meta.value[indexOf(meta.key, :name)]and requirehas(meta.key, :name)for the positive case. - Parameters are always passed as ClickHouse query parameters (
{name:Type}), never interpolated into SQL text. - The realtime view reads the last 30 minutes of
eventsfor one site with the predicatetimestamp >= now64(3) - INTERVAL 30 MINUTE(session-level realtime figures readsessionsby the version-aware pattern of 6.4, as defined in Section 9.14). Shortly after midnight UTC the 30-minute window spans two dates (and at a month boundary two monthly partitions); the time predicate, not a "today only" date filter, bounds the scan, and the sorting-key prefix(site_id, toDate(timestamp))still prunes to those one or two dates. The current-visitors figure is not read from ClickHouse: it is the 5-minute window of the Valkey setqs:rt:{site_id}maintained at ingestion (Section 9.14, Section 8).
6.9 Site Deletion and Data Purge #
ClickHouse has no foreign keys, so Postgres drives every purge through pg-boss queues. Two jobs run the same mutation procedure below: site-purge (whole-site and native-data purges) and import-purge (one import's rows). Both appear in the jobs table of Section 16.6.
site-purge payload: { "site_id": "<uuid>", "scope": "all" | "native" }. import-purge payload: { "site_id": "<uuid>", "import_id": "<uuid>" }.
| Job and scope | Tables affected | Trigger |
|---|---|---|
site-purge, all |
events, sessions, all nine imported_* tables (every import of the site) |
Site deleted (user, Sites API, team purge) |
site-purge, native |
events, sessions |
unlicensed-purge (sites remain, imports are kept because they are user-supplied) |
import-purge |
the nine imported_* tables, filtered by import_id |
Import deleted, cancelled or failed (Section 14.6) |
Procedure (jobs site-purge and import-purge; site-purge batches up to 50 jobs per run):
- The job is enqueued in the same Postgres transaction that deletes the site (or marks the import
deleting), withstartAfter = 300 seconds. The delay lets ingestion caches (refreshed within 30 s, Section 8.6) and the 2-second insert buffer drain, so no row for the deleted site is written after the purge starts. - For every affected table the worker issues one mutation covering all sites in the batch:
ALTER TABLE <t> DELETE WHERE site_id IN ({ids:Array(UUID)}) SETTINGS mutations_sync = 0(forimport-purge:WHERE site_id = {site:UUID} AND import_id = {import:UUID}).ALTER ... DELETEphysically rewrites parts, so the data is truly removed rather than hidden. - The worker records the mutation ids and re-queues itself every 30 seconds until
system.mutationsreportsis_done = 1for all of them. A mutation not finished after 24 hours marks the pg-boss jobfailed, raises an error-level log line and an operator-visible alert (Section 16); the job is retried manually.import-purgethen hard-deletes the Postgresimportsrow only when its status isdeleting; rows offailedorcancelledimports are kept until the user dismisses them (Section 5.9.1).site-purgenever touchesimportsrows (they cascade away with the site). - The job is idempotent (a repeated delete of an absent site is a no-op) and may be retried at any time.
site-orphan-reconcile(daily, Section 16.6) runsSELECT DISTINCT site_idover the last 40 days ofeventsandsessionsand overimported_*, compares against Postgressites.id, and enqueuessite-purge(scopeall) for any site that no longer exists. It also deletes expired rows ofsite_domain_reservations(Section 5.5.5). This repairs loss of a queue message and the rare late write.
A deleted site's domain becomes unreachable for events at once (the UUID is gone from sites), and the domain stays reserved to the same team for 30 days (site_domain_reservations, Section 5.5.5) so another team cannot claim it and receive traffic from pages that still carry the old snippet. ClickHouse rows of the old site are unreachable from the first moment because the UUID is gone from sites and no query can address it.
6.10 Migration Runner and schema_migrations #
ClickHouse DDL is applied by the in-repo SQL runner invoked by the migrate command (the same command that applies Postgres migrations, Section 5.13). Migration files live in packages/shared/migrations/clickhouse/ and are named NNNN_description.sql with a four-digit, gap-free, ascending prefix.
| File | Contents |
|---|---|
0001_events.sql |
events |
0002_sessions.sql |
sessions |
0003_imported_tables.sql |
the nine imported_* tables |
schema_migrations is created by the runner itself (it cannot be created by a migration file because it tracks them):
CREATE TABLE IF NOT EXISTS schema_migrations
(
version UInt32,
name String,
checksum FixedString(64), -- lowercase hex SHA-256 of the file bytes
applied_at DateTime('UTC') DEFAULT now(),
duration_ms UInt32
)
ENGINE = MergeTree
ORDER BY version;Runner algorithm:
- Acquire the global migration lock: the Postgres advisory lock
pg_advisory_lock(7426001)on a dedicated connection (this serializes concurrentmigrateruns across replicas and also protects the Postgres side). Wait up to 120 seconds, then exit with code 1. CREATE DATABASE IF NOT EXISTS, then createschema_migrationsif missing.- List migration files sorted by version. Reject duplicated or non-contiguous prefixes (exit code 1, message
invalid migration sequence). - For each version already recorded: compare the stored checksum with the file's. A mismatch aborts with
migration_checksum_mismatch <file>; applied files are immutable. - For each unapplied version in ascending order: split the file into statements on lines that contain exactly
-- statement-breakpoint; execute each statement sequentially over the HTTP interface; on success insert theschema_migrationsrow. ClickHouse DDL is not transactional, so every statement must be idempotent (CREATE TABLE IF NOT EXISTS,ADD COLUMN IF NOT EXISTS,DROP ... IF EXISTS); a partially applied file is simply re-run in full. - Run the TTL reconcile step (6.6).
- Release the lock; exit 0 and print the applied file names. Errors print the failing statement's first 200 characters and the server error, and exit 1.
web and worker never migrate. At boot each compares the highest recorded version in schema_migrations (and Drizzle's applied list in Postgres) against the migration files bundled in the image; if the database is behind, the process exits with code 1 and the message database schema is behind: run the migrate command; if the database is ahead of the image (a rollback after a newer release applied migrations), the process starts and logs a warning. GET /readyz additionally reports ClickHouse readiness only when SELECT 1 succeeds and schema_migrations is readable.
Migration policy: forward-only; additive changes first (new columns need a default or Nullable); renames and type changes use a new column plus a backfill (INSERT ... SELECT or ALTER ... UPDATE) in a later release; changing the sorting key, partition key or engine of events/sessions requires creating a *_new table, copying data, and an atomic EXCHANGE TABLES, shipped as an explicit, documented upgrade step (Section 16).
6.11 Sizing Estimates #
Planning figures, not guarantees; measure with real traffic. They assume the schemas above, ZSTD/LZ4 codecs and typical pageview-heavy traffic.
| Quantity | Estimate |
|---|---|
Compressed bytes per events row (all columns, incl. properties on ~10% of rows) |
40–100 B, plan with 100 B |
Compressed bytes per sessions row |
60–120 B per logical session; state and cancel rows coexist until merged, so plan with 2 rows per session update until merges complete |
| Sessions per billable event | roughly 1 session per 2.5 events |
| Storage per 1,000,000 events/month | ≈ 100 MB events + ≈ 50 MB sessions ≈ 150 MB per month |
| Hosted per-team maximum (Agency cap 2,000,000/month plus 10% grace) | ≈ 330 MB/month, ≈ 4 GB/year |
| Hosted fleet example: 2,000 teams averaging 300,000 events/month | 600 M events/month ≈ 90 GB/month ≈ 1.1 TB/year before any TTL |
| Deployment size | Events per month | Suggested ClickHouse node |
|---|---|---|
| Small self-hosted | up to 10 M | 2 vCPU, 4 GB RAM, 20 GB SSD per year of data |
| Medium | up to 100 M | 4 vCPU, 16 GB RAM, ~200 GB SSD per year |
| Large (hosted) | up to 1 B | 16 vCPU, 64 GB RAM, ~2 TB NVMe per year |
Insert load: each web process flushes a batch every 2 s or at 5,000 rows, so one process issues at most 0.5 inserts/s to events (plus the same to sessions). Even 40 processes stay far below the part-creation limits (the default parts_to_delay_insert and parts_to_throw_insert thresholds are in the hundreds to thousands of active parts per partition, and background merges keep up at these rates). Query load is dominated by dashboard loads of 8–12 queries each, every one pruned to one site and one date range; typical latency target is under 500 ms per query at up to 10 M events per site-period (verified by the performance tests in Section 18).
7. Tracking Script & Integrations #
This section specifies everything that runs in the visitor's browser or on the customer's side of the integration boundary: the tracker script, how it is delivered, its behavior, its public JavaScript API, the @quietstat/tracker npm package, the WordPress plugin, the Google Tag Manager template, reverse-proxy recipes, the framework setup guides shown inside the dashboard, and the install/verify flow. Server-side processing of what the tracker sends is owned by Section 8. The wire format is defined once, in Section 7.6, and validated by the rules in Section 8.4.
Source lives in packages/tracker (Section 3). The tracker, the npm package and the generic script are MIT licensed; the WordPress plugin is GPL-2.0-or-later; the GTM template is Apache-2.0 (Section 3).
7.1 Design Principles #
| # | Principle | Consequence |
|---|---|---|
| 1 | No persistent client state | The tracker never sets cookies and never writes localStorage, sessionStorage or IndexedDB. It only reads one localStorage key (the site owner's opt-out flag, Section 7.5.12). That read stores nothing, identifies nobody and exists solely so a site owner can exclude their own visits; Section 17.9.2 records it as a condition of the privacy position. |
| 2 | No client-side identifiers | The tracker sends no visitor id, session id or timestamp. The server derives visitor and session (Section 8.11, 8.12) and stamps the event with its own receive time. |
| 3 | Never break the host page | Every tracker entry point is wrapped so that an exception is swallowed (Section 7.6.5). The tracker never throws into page code, never blocks rendering, never calls preventDefault except in the one documented legacy fallback (Section 7.5.5). |
| 4 | Tiny core | The core file is at most 1,500 bytes gzipped; optional behavior ships as separate modules assembled per site (Section 7.2.3, 7.8). |
| 5 | Silent by default | The tracker logs to the console only when logging is true (Section 7.4). |
| 6 | One wire format | Browser tracker, npm package, WordPress proxy, GTM template and the public Events API (Section 13) all post the same payload to POST /api/event. |
7.2 Script Delivery #
7.2.1 Script URLs #
The web process (Section 3) serves tracker files. All paths below are relative to the instance's public origin (BASE_URL, Section 16.4).
| Route | Purpose | Notes |
|---|---|---|
GET /js/q-{script_id}.js |
Per-site script. Core plus only the modules enabled in the site's tracking configuration, with the site domain and module flags baked in. This is the recommended script. |
{script_id} is the site's script_id: 12 characters, lowercase Crockford base32 (0-9a-hjkmnp-tv-z), generated randomly at site creation, unique, immutable, never reused. Stored on the sites row (Section 5). The script_id is not a secret; it only selects configuration. |
GET /js/script.js |
Generic script. Core plus every module. Configured through data-* attributes on its own <script> tag (Section 7.3.2) or through quietstat.init(...). |
Used for the plain-HTML variant, for sites without a dashboard-created site, and for the GTM template when no script_id is given. |
GET /js/v1/script.js and GET /js/v1/q-{script_id}.js |
Major-pinned aliases of the two routes above. | Serve identical bytes while the tracker API major is 1. See Section 7.2.4. |
GET /js/script.min.js.map |
Source map for the generic script. | Served with Cache-Control: public, max-age=86400; not referenced by a //# sourceMappingURL comment in production output (the comment is stripped to keep the file small). |
Unknown script_id, or a script_id whose site was deleted: 404 with Content-Type: text/plain; charset=utf-8, body unknown script, and Cache-Control: public, max-age=60. A syntactically invalid script_id (not 12 characters of the alphabet) returns the same 404.
HEAD and conditional GET are supported. Methods other than GET/HEAD return 405 with Allow: GET, HEAD.
7.2.2 Response headers #
Every successful tracker response carries exactly these headers:
| Header | Value |
|---|---|
Content-Type |
application/javascript; charset=utf-8 |
Cache-Control |
Per-site and generic scripts: public, max-age=600, stale-while-revalidate=86400, stale-if-error=604800. Major-pinned aliases: identical. |
ETag |
Strong ETag: the first 16 hex characters of the SHA-256 of the response body, quoted. |
Access-Control-Allow-Origin |
* |
Cross-Origin-Resource-Policy |
cross-origin |
Timing-Allow-Origin |
* |
X-Content-Type-Options |
nosniff |
Vary |
Accept-Encoding |
Content-Encoding |
br or gzip when the client accepts it; otherwise identity. Pre-compressed at build/assembly time (brotli quality 11, gzip level 9); never compressed per request. |
No Set-Cookie, no Content-Security-Policy, no Server-Timing, no per-request variance other than Accept-Encoding. The 10-minute max-age is a deliberate decision: it bounds how long a changed tracking configuration (Section 7.15.3) takes to reach visitors while keeping origin load negligible; the stale-while-revalidate window keeps the script available if the origin is briefly down.
7.2.3 Per-site assembly #
The build (Section 7.8) produces, in packages/tracker/dist/:
core.js: the core IIFE source template. It contains exactly two placeholder tokens,__CFG__and__EXT__.- One file per module (Section 7.8.1), each a single function expression
function(q){...}whereqis the internal core API object. script.js: the generic script (core with__CFG__replaced by{}and__EXT__by an array of all modules).
On the first request for a given script_id (and again whenever the site's tracking configuration changes), the server assembles the file:
- Load the site's
domainand itssite_settingstracker columns (Section 5.5.2; mapped in Section 7.15.3). - Replace
__CFG__with a JSON object literal{"d":"<domain>","h":0|1,"fe":["<ext>",...]|0}(domain;track_hash_routingflag;file_download_extensionsoverride list, or0when the column is NULL and the default list applies). Only keys that differ from the defaults in Section 7.4 are emitted. - Replace
__EXT__with[+ the enabled modules joined by,+], in the fixed order:tagged,outbound,downloads,forms,notfound,engagement. Thetaggedandengagementmodules are always included (they have nosite_settingscolumn; Section 5.5.2 states engagement is always on).outboundis included whentrack_outbound_linksis true,downloadswhentrack_file_downloadsis true,formswhentrack_form_submissionsis true,notfoundwhentrack_404_pagesis true. - Minify nothing further (modules are already minified), compress, compute the ETag, and cache the result in process memory keyed by
script_idplus a configuration hash (LRU, 5,000 entries). The cache entry is invalidated when the site's configuration changes (the site-configuration invalidation message on the Valkey channelqs:site-configowned by Section 8.6, and a 30-second maximum entry age as a backstop, matching the site-configuration cache lifetime).
The server never evaluates user-controlled text as code: domain is inserted via JSON.stringify and </script / U+2028 / U+2029 are escaped in the literal.
7.2.4 Versioning #
- The tracker has an API major version (currently 1). The major changes only when the wire format (Section 7.6) or a documented init option changes incompatibly.
/js/script.jsand/js/q-{script_id}.jsalways serve the latest tracker of the current major./js/v{N}/...paths serve the latest tracker of major N. When a major 2 ships,/js/v1/...keeps serving the last 1.x tracker for at least 24 months after the major-2 release, and the unversioned paths switch to major 2.- Snippets generated by the dashboard use the unversioned per-site URL (Section 7.3.1). The generic-script snippets generated for the plain-HTML guide use
/js/v1/script.jsso a customer's snippet never silently changes major. - Each script begins with the banner comment
/*! quietstat tracker v1 | MIT */(major only). - The
POST /api/eventpayload is additive within a major: the server ignores unknown fields (Section 8.4) and the tracker never relies on a response body.
7.3 Snippet Formats #
The dashboard (Section 7.15) generates one of the following formats. In every example https://analytics.example.com stands for the instance's BASE_URL, k3x9d2m4q7ab for the site's script_id, and example.com for the site's domain; the dashboard substitutes real values.
7.3.1 Format A: per-site script with queue stub (recommended) #
<script async src="https://analytics.example.com/js/q-k3x9d2m4q7ab.js"></script>
<script>
window.quietstat = window.quietstat || function () { (quietstat.q = quietstat.q || []).push(arguments) };
quietstat.init = quietstat.init || function (i) { quietstat.o = i || {} };
quietstat.init();
</script>Properties: loads asynchronously, never blocks parsing; the stub makes quietstat('Signup') callable immediately, before the script has loaded (Section 7.7.3); quietstat.init(options) accepts init options (Section 7.4), which override the options baked into the per-site script. The domain is baked into the script, so quietstat.init() takes no arguments in the default case.
Attribute decisions: async (not defer) because the script is independent of document order; the second tag has no src, so it executes immediately and defines the stub even when the async script has not yet downloaded. If the async script wins the race and runs first, it defines the real quietstat function; the stub lines then keep the real function (||), and the real init receives the call. The real script defers its start until DOMContentLoaded (or immediately if the document is already past loading) so an init(options) in the same snippet is always honored (Section 7.5.1).
7.3.2 Format B: generic script with data attributes #
<script defer data-domain="example.com" src="https://analytics.example.com/js/v1/script.js"></script>Optional attributes (all booleans accept true/false, or an empty value meaning true):
| Attribute | Equivalent init option | Default |
|---|---|---|
data-domain |
domain |
location.hostname with one leading www. removed |
data-api |
endpoint |
<script origin>/api/event |
data-auto-pageviews |
autoCapturePageviews |
true |
data-hash-routing |
hashBasedRouting |
false |
data-outbound-links |
outboundLinks |
false |
data-file-downloads |
fileDownloads: true, or a comma-separated extension list (replaces the default list) |
false |
data-form-submissions |
formSubmissions |
false |
data-track-404 |
track404 |
false |
data-tagged-events |
taggedEvents |
true |
data-engagement |
engagement |
true |
data-capture-on-localhost |
captureOnLocalhost |
false |
data-respect-privacy-signals |
respectPrivacySignals |
false |
data-logging |
logging |
false |
defer is used here (not async) so that document.currentScript data attributes are read from a tag that executes in document order. The generic script reads attributes from document.currentScript; when currentScript is null (the script was injected by a loader), it falls back to the last <script> element whose src path ends in /js/script.js or /js/v1/script.js.
7.3.3 Format C: manual pageviews (SPA frameworks or custom routers) #
Format A with quietstat.init({ autoCapturePageviews: false }) and explicit quietstat('pageview') calls from the application's router (Section 7.7.2).
7.3.4 Format D: first-party proxy #
Format A with an explicit endpoint and a script URL on the customer's own domain (proxy recipes in Section 7.13):
<script async src="/m/script.js"></script>
<script>
window.quietstat = window.quietstat || function () { (quietstat.q = quietstat.q || []).push(arguments) };
quietstat.init = quietstat.init || function (i) { quietstat.o = i || {} };
quietstat.init({ endpoint: "/m/event" });
</script>The default proxy path prefix used throughout this section is /m/ (script at /m/script.js, events at /m/event). The prefix is a placeholder the customer may change; the dashboard's proxy tab (Section 7.15.4) substitutes the chosen prefix into every recipe.
7.3.5 Format E: npm / bundler #
import { init } from "@quietstat/tracker";
init({ domain: "example.com", endpoint: "https://analytics.example.com/api/event" });See Section 7.10.
7.4 Init Options #
Options are resolved in this order, later sources overriding earlier ones: built-in defaults, options baked into the per-site script (Section 7.2.3), data-* attributes (generic script), the object passed to quietstat.init(options) (pre-load stub or post-load call; the first call wins and later calls are ignored with a log line when logging is true).
| Option | Type | Default | Meaning |
|---|---|---|---|
domain |
string |
baked domain, else location.hostname minus one leading www. |
Site identifier sent as d. A comma-separated list of up to 10 domains enables multi-site rollup (Section 8.6). The value is sent verbatim; the server normalizes it. |
endpoint |
string |
<origin of the script URL>/api/event; for the npm package there is no default and init throws if absent (Section 7.10) |
Absolute URL or same-origin path where events are POSTed. |
autoCapturePageviews |
boolean |
true |
Send a pageview on load and on every SPA navigation. When false no automatic pageview or engagement baseline is created until quietstat('pageview') is called. |
hashBasedRouting |
boolean |
false |
Treat location.hash as part of the path (Section 7.5.3). Sends h: 1. |
outboundLinks |
boolean |
false |
Section 7.5.5. Needs the outbound module. |
fileDownloads |
boolean or { fileExtensions: string[] } |
false |
Section 7.5.6. { fileExtensions } replaces the default extension list. Needs the downloads module. |
formSubmissions |
boolean |
false |
Section 7.5.7. Needs the forms module. |
track404 |
boolean |
false |
Section 7.5.8. Needs the notfound module. |
taggedEvents |
boolean |
true |
CSS-class tagged events (Section 7.5.10). Needs the tagged module. |
engagement |
boolean |
true |
Scroll depth and engaged time pings (Section 7.5.9). Needs the engagement module. |
customProperties |
object or (eventName: string) => object |
undefined |
Properties merged into every event (Section 7.6.2). Function form receives the event name; its return value must be a flat object. |
transformRequest |
(payload: EventPayload) => EventPayload | null |
undefined |
Last chance to modify or drop an event (Section 7.5.14). |
captureOnLocalhost |
boolean |
false |
Send events when the page host is a loopback/local host (Section 7.5.13). |
respectPrivacySignals |
boolean |
false |
Skip tracking when the browser sends Do Not Track or Global Privacy Control (Section 7.5.15). |
allowAutomation |
boolean |
false |
Send events from automated browsers (navigator.webdriver). Used by the product's own end-to-end tests (Section 18). |
logging |
boolean |
false |
Log ignored events and send errors to the console (console.warn). |
Options that name a module the assembled script does not contain (for example outboundLinks: true on a per-site script whose site has track_outbound_links = false) are accepted, ignored, and logged when logging is true. The generic script and the npm package contain every module, so this applies only to per-site scripts. The dashboard always shows the per-site configuration as the source of truth (Section 7.15.3). The options autoCapturePageviews and respectPrivacySignals are init-only: they are not stored per site and are never baked into the per-site script.
Type coercion for data-* attributes: the string "false" and "0" are false; the empty string, "true" and "1" are true; any other value for a boolean attribute is true; for data-file-downloads any value other than true/false/1/0/empty is parsed as a comma-separated extension list.
7.5 Tracker Behavior #
Terminology: a tracked page is the pair (pathname, hash path when hashBasedRouting is on) most recently reported by a pageview. "Send" means construct the payload (Section 7.6.2), apply transformRequest, and transmit (Section 7.6.3).
7.5.1 Startup #
- If the script has already started on this window (
window.quietstat.l === 1), return without effect. Otherwise setwindow.quietstat.l = 1. - Determine options (Section 7.4). If
document.readyState === "loading", wait forDOMContentLoadedbefore continuing; otherwise continue immediately. - Run the ignore checks (Section 7.5.11 to 7.5.15): file protocol, localhost, automation, opt-out flag, privacy signals. If any applies, the tracker still installs
window.quietstat(so calls to it are harmless no-ops), logs the reason whenloggingis true, installs no listeners and sends nothing. - Replace the stub: define the real
quietstatfunction, copyinit,oandl, then drain the queuequietstat.qin order (entries areargumentsobjects or arrays; each entry is converted withArray.prototype.slice.call). Theinitentry is not queued;initoptions are read fromquietstat.o. After draining, the core setsquietstat.q = { push: function (entry) { apply(entry) } }, so entries pushed later by code that still holds the stub's queue convention (for example the GTM template, Section 7.12) are applied immediately. - Install the History wrappers and listeners (Section 7.5.2, 7.5.3), module listeners, and visibility/pagehide listeners (Section 7.5.9).
- If
autoCapturePageviewsistrue, send the initial pageview, subject to the prerender rule below.
Prerender and background tabs. If document.prerendering is true, the initial pageview waits for the prerenderingchange event. If document.visibilityState === "hidden" at startup (a tab opened in the background), the initial pageview waits for the first visibilitychange to visible. In both cases the tracked page is determined at the moment the pageview is sent, not at startup. A tab that never becomes visible never reports a pageview.
bfcache. On pageshow with event.persisted === true (page restored from the back/forward cache), the tracker resets engagement counters (Section 7.5.9) and sends a new pageview, because the visitor is viewing the page again.
7.5.2 Automatic pageviews and SPA navigation #
Pageview payload: n: "pageview", u set per Section 7.6.2.
The tracker wraps history.pushState and history.replaceState once (guarded by a flag on the wrapper so multiple tracker instances cannot double-wrap), calls the original first, and then evaluates a navigation. It also listens to popstate. A navigation sends a pageview if and only if all of the following hold:
autoCapturePageviewsistrue.- The page key differs from that of the last tracked page. The page key is
location.pathname, plus, whenhashBasedRoutingistrue, the hash with its query part removed (Section 7.5.3). Changes that alter only the query string, or only the hash while hash routing is off, are not navigations. - The call is not made while the initial pageview is deferred (Section 7.5.1); in that window only the page key is updated.
Both pushState and replaceState are handled identically: the dedupe rule in item 2 makes a replaceState that merely rewrites the query string, strips tracking parameters, or re-sets the same path a no-op, while a replaceState that changes the path (as client routers do for redirects) is reported as a navigation.
Before sending a navigation pageview the tracker first flushes an engagement ping for the page being left (Section 7.5.9) and resets the engagement counters for the new page.
popstate triggered by hash anchor clicks on pages without hash routing changes only the hash, so by rule 2 it is not a navigation.
Manual pageviews (quietstat("pageview", options)) ignore the dedupe rule and always send, update the tracked page to options.url's path (or the current location if absent), and also flush and reset engagement.
7.5.3 Hash-based routing #
When hashBasedRouting is true:
h: 1is sent on every event.uretains the fragment.- The tracker also listens to
hashchangeand treats it as a navigation under the rules of Section 7.5.2. - The hash part of the page key is the fragment with everything from the first
?removed, so#/products?sort=ascand#/products?sort=descare the same page. - An empty fragment or a bare
#contributes the empty string.
The server stores pathname + "#" + hash path as the page path only when h is 1 (Section 8.10.1). When hashBasedRouting is false, the tracker removes the fragment from u before sending.
7.5.4 Event delivery order #
Events are sent in the order they are generated within a page. The tracker keeps no queue after startup other than the pre-load stub queue; each send is independent and unordered on the wire. Server-side ordering assumptions are documented in Section 8.12.
7.5.5 Outbound link tracking #
Enabled by outboundLinks. Event name Outbound Link: Click, property url, interactive true.
Listeners: click and auxclick on document, capture phase, passive. Only click events (any button=0) and auxclick events with button === 1 (middle click) are processed. Right-click (contextmenu) and keyboard activation are covered because Enter on a focused link dispatches click.
Finding the link: starting at event.target, walk up through parentElement at most 5 levels to the nearest <a> or <area> element with a non-empty href. If none, ignore.
Outbound test: let u = new URL(link.href, location.href). The link is outbound when u.protocol is http: or https: and u.hostname (lowercased, one leading www. removed) differs from location.hostname (lowercased, one leading www. removed). Subdomains are therefore outbound. mailto:, tel:, javascript:, sms: and in-page # links are never outbound. Links that match the file-download rule (Section 7.5.6) while fileDownloads is enabled are reported as downloads only, never as both.
Property value: url is u.origin + u.pathname. The query string and fragment are removed before sending.
Delayed navigation rules. The browser may unload the page before an in-flight request completes, so delivery relies on navigator.sendBeacon first and fetch with keepalive: true second (Section 7.6.3), which survive unload in every supported browser. No navigation is ever delayed on supported browsers. The delay mechanism exists only as a legacy fallback: if neither sendBeacon nor fetch is available (only possible outside the supported matrix of Section 7.9), then for a click that would navigate the current tab, the tracker calls preventDefault, sends via XMLHttpRequest, and navigates with location.assign(href) when the request completes or after 500 ms, whichever comes first. The fallback applies only when all of these hold: button === 0; none of ctrlKey, metaKey, shiftKey, altKey is pressed; link.target is empty, _self, _parent, _top or equal to window.name; the link has no download attribute; event.defaultPrevented is false. In every other case (new tab via target=_blank, modifier keys, middle click, download attribute) the event is sent without any delay or preventDefault.
7.5.6 File download tracking #
Enabled by fileDownloads. Event name File Download, property url, interactive true. Uses the same listeners, link search and delayed-navigation rules as Section 7.5.5.
Match rule: the link is a download when its download attribute is present (any value), or when the last path segment of new URL(link.href).pathname, lowercased and with any trailing / removed, ends in . followed by one of the configured extensions. Matching ignores the query string and the fragment and applies to links on any host (same site or external). Only http: and https: URLs are considered.
Default extension list (replaced, not extended, by a custom list):
pdf, xls, xlsx, doc, docx, ppt, pptx, pps, odt, ods, odp, txt, rtf, csv, epub, mobi, key, pages, numbers, zip, rar, 7z, gz, tgz, bz2, xz, tar, iso, dmg, pkg, exe, msi, apk, deb, rpm, mp3, wav, wma, midi, flac, ogg, m4a, mp4, mov, avi, wmv, mpeg, mkv, webm
Custom list: fileDownloads: { fileExtensions: ["pdf", "stl"] } (or data-file-downloads="pdf,stl"). Each entry is lowercased, a single leading . is removed, and entries that are not 1 to 10 characters of a-z0-9 are discarded. An empty resulting list disables file-download tracking and logs a message when logging is true. A custom list is persisted in the per-site configuration as site_settings.file_download_extensions (Section 7.15.3), with the same normalization and a maximum of 50 entries (Section 5.16); a NULL column means the default list above.
Property value: url is origin + pathname (no query, no fragment).
7.5.7 Form submission tracking #
Enabled by formSubmissions. Event name Form: Submission, properties path (the current location.pathname) and, when the form has a non-empty id attribute, form_id (the id truncated to 120 characters). Interactive true.
Listener: submit on document, bubble phase, passive. Native HTML validation failures do not dispatch submit, so invalid forms are not reported. The tracker does not check defaultPrevented, so AJAX-handled forms (which call preventDefault to submit via fetch) are reported.
Which forms are tracked: every <form> element except these, which are ignored:
- forms with
method="dialog"; - forms carrying the attribute
data-quietstat-ignore(any value); - forms that carry a
quietstat-event-name=...class (they are handled by tagged events, Section 7.5.10, so a submission is not reported twice).
De-duplication: a second submit for the same form element within 1,000 ms is ignored.
Field names, field values and the action URL are never read.
7.5.8 404 tracking #
Enabled by track404. Sends, in addition to the normal pageview, a custom event named 404 with property path (location.pathname) and i: false (non-interactive, so a 404 view does not stop the session from counting as a bounce, Section 8.12).
Detection is evaluated once per tracked page, after the initial pageview (or navigation pageview) is sent, in this order; the first applicable rule decides:
- A
<meta name="quietstat:status" content="404">element exists in the document, orwindow.quietstat404 === true: the page is a 404. performance.getEntriesByType("navigation")[0].responseStatusis an integer between 100 and 599 (supported browsers expose it; it is only evaluated for the initial load, not SPA navigations): the page is a 404 if it equals404, and is not a 404 otherwise, and no further rules are consulted. A value of0(reported for cross-origin or unknown navigations), a missing value or any other value is not decisive: continue with rule 3.- Otherwise heuristic: lowercase
document.titlecontains404or the phrasepage not foundornot found.
For SPA navigations only rule 1 and rule 3 are evaluated, 50 ms after the navigation pageview is sent (to allow the router to update document.title). Platform integrations that know the response status (the WordPress plugin, Section 7.11) emit the meta tag from rule 1, which is the most reliable signal.
7.5.9 Engagement: scroll depth and engaged time #
Enabled by engagement (default on). Active only for tracked pages that sent a pageview. Event name engagement, i: false, with sd (scroll depth) and e (engaged time delta). Engagement events are non-interactive and not billable (Section 15).
Scroll depth algorithm.
- State:
maxDepth(integer 0 to 100, initially 0) for the current tracked page;lastSentDepth(initially -1). - Sample function
measure():docHeight = Math.max(document.body ? document.body.scrollHeight : 0, document.documentElement.scrollHeight, document.body ? document.body.offsetHeight : 0, document.documentElement.offsetHeight)viewport = window.innerHeightscrolled = (window.scrollY || document.documentElement.scrollTop || 0) + viewport- If
docHeight <= viewport, depth is100(the whole page fits in the viewport). Otherwise depth isMath.min(100, Math.floor(scrolled / docHeight * 100)). maxDepth = Math.max(maxDepth, depth).
measure()runs: once when the pageview is sent; onscroll(passive listener, throttled withrequestAnimationFrameso it runs at most once per frame and at most every 100 ms); and immediately before every engagement send. The document height is re-read on every sample, so lazy-loaded or infinitely growing pages produce a depth relative to the height at the time of the sample, andmaxDepthnever decreases.- Reset to
maxDepth = 0,lastSentDepth = -1when a new pageview is sent (navigation, manual pageview, bfcache restore).
Engaged time algorithm.
- State:
engagedMsaccumulated for the current page since the last send (initially 0);resumeAt(monotonic timestamp fromperformance.now(), ornullwhile the page is not engaged). - The page is engaged while
document.visibilityState === "visible". Window focus is not required (visitors commonly have two windows side by side). There is no idle timeout. - When a pageview is sent and the document is visible, set
resumeAt = performance.now(). Onvisibilitychangetohidden:engagedMs += performance.now() - resumeAt; resumeAt = null. Onvisibilitychangetovisible:resumeAt = performance.now(). - The value to send is
delta = engagedMs + (resumeAt === null ? 0 : performance.now() - resumeAt), rounded to an integer. TheengagedMsaccumulator is reset to 0 andresumeAttoperformance.now()(if visible) after each send, so every ping carries only the time since the previous ping for the same pageview. The server sums the deltas (Section 8.13).
When pings are sent (the triggers, exhaustive):
| Trigger | Notes |
|---|---|
visibilitychange to hidden |
Covers tab switch, minimize, mobile app switch, and most page closes. |
pagehide |
Covers closes where visibilitychange did not fire first, and bfcache entry. |
| Start of an SPA navigation or manual pageview | Sent before the new pageview, carrying the old page's URL (u is the old page's location.href captured when its pageview was sent). |
There are no timers and no periodic pings.
Send condition: a ping is sent only if delta >= 1000 or maxDepth > lastSentDepth. Otherwise nothing is sent. After sending, lastSentDepth = maxDepth. e is clamped to 3600000 (one hour) per ping. The payload carries e: delta and sd: maxDepth.
Transport: engagement pings on visibilitychange/pagehide use navigator.sendBeacon when available, otherwise fetch with keepalive (Section 7.6.3). A pagehide immediately following a visibilitychange ping finds delta below 1,000 ms and maxDepth === lastSentDepth, so it sends nothing (no duplicate).
Disabled cases: engagement state is not created for a page whose pageview was ignored (Section 7.5.11 to 7.5.15), whose pageview was dropped by transformRequest, or when engagement is false.
7.5.10 CSS-class tagged events #
Enabled by taggedEvents (default on). Any element whose class attribute contains the token quietstat-event-name=<name> produces a custom event when the visitor interacts with it.
Class grammar. Tokens are separated by whitespace. The recognized tokens are:
| Token | Meaning |
|---|---|
quietstat-event-name=<value> |
Required. The event name. |
quietstat-event-<key>=<value> |
Optional. Sets custom property <key> to <value>. <key> is everything between quietstat-event- and the first =, and must not be name. |
quietstat-revenue-amount=<value> |
Optional. Revenue amount (decimal string). |
quietstat-revenue-currency=<value> |
Optional. Three-letter currency code. |
Values are decoded by replacing + with a space and then applying decodeURIComponent (a malformed escape leaves the raw value). Revenue is attached only when both quietstat-revenue-amount and quietstat-revenue-currency are present.
Interaction rules.
- For ordinary elements and
<a>/<area>elements:click(button 0) andauxclick(button 1) on the tagged element or on a descendant of it. The tagged element is found by walking up fromevent.targetat most 5 levels. - For
<form>elements:submitinstead of click. - For tagged
<a>links that would navigate the current tab, the same delayed-navigation rules and fallback apply as in Section 7.5.5. - A tagged element that is also an outbound link or file download produces both events (the tagged event is an explicit customer instruction); the tagged event is sent first.
Event is interactive true.
Example: <button class="btn quietstat-event-name=Signup+Click quietstat-event-plan=Pro">Sign up</button> sends Signup Click with p: {"plan": "Pro"}.
7.5.11 file:// pages #
If location.protocol === "file:" the tracker is inert (Section 7.5.1 step 3), regardless of captureOnLocalhost, because the server accepts only http, https and app URLs (Section 8.4).
7.5.12 Opt-out #
The tracker is inert when either of the following is true; it is evaluated at startup and again at the moment of every send:
localStorage.getItem("quietstat_ignore") === "true"(access tolocalStorageis wrapped intry; a thrownSecurityErrorcounts as "not set").- The document element has the attribute
data-quietstat-ignorewith any value other than the stringfalse(for example<html data-quietstat-ignore>). Sites use this to opt out server-rendered pages for logged-in staff.
The opt-out flag lives on the tracked site's origin; a dashboard on another origin cannot set it. The dashboard's "Exclude my own visits" help (Section 7.15.5) therefore shows the bookmarklet javascript:localStorage.setItem('quietstat_ignore','true');alert('Quietstat: your visits on this site are now ignored.') plus the instruction to run it on the tracked site, and links to IP exclusions (Section 8.8.4).
7.5.13 Localhost handling #
The tracker is inert (unless captureOnLocalhost is true) when location.hostname is localhost, ends with .localhost, is 127.0.0.1, [::1], or ::1, or is 0.0.0.0. When captureOnLocalhost is true, events are sent with the page's real localhost URL while d is the configured domain; the pollution this causes in the production site's data is the caller's explicit choice, and the dashboard warns about it in the init-options help. The server additionally drops such events with reason hostname_not_allowed unless the site's allowed_hostnames list includes localhost (Section 8.6 rule 5), so a production site is not polluted by accident.
7.5.14 transformRequest #
transformRequest(payload) is called synchronously after the payload is built (Section 7.6.2) and before transmission, for every event including engagement pings. It receives a mutable copy of the wire-format payload and must return the payload to send (the same object or a new one) or null/undefined/false to drop the event. Rules:
- A returned value that is not a plain object drops the event.
- If the function throws, the event is dropped (fail closed: transformers are typically used to scrub personal data), and the error is logged when
loggingis true. - The returned object is sent as-is, without re-validation; invalid output is rejected by the server with
400(Section 8.4). - Dropped events invoke the event's
callback(if any) with{ ignored: true, reason: "transform" }.
7.5.15 Do Not Track and Global Privacy Control #
Decision: by default the tracker ignores the Do Not Track header and Global Privacy Control signal, because the product collects no personal data, sets no identifiers and cannot be used for cross-site or cross-day tracking (Section 17). Customers whose policy is to honor the signals set respectPrivacySignals: true. When it is true the tracker is inert if navigator.doNotTrack === "1", window.doNotTrack === "1", navigator.msDoNotTrack === "1", or navigator.globalPrivacyControl === true. The signals are evaluated at startup.
7.5.16 Automated browser self-check #
Unless allowAutomation is true, the tracker is inert when navigator.webdriver === true or any of window._phantom, window.__nightmare, window.callPhantom, window.Cypress, window.__playwright__binding__ is truthy. This is an optimization only; the authoritative bot filter is server-side (Section 8.8).
7.6 Payload, Transport and Errors #
7.6.1 Wire format #
POST to endpoint, Content-Type: text/plain;charset=UTF-8 (a CORS-safelisted type, so no preflight request is made), body a single JSON object. Field semantics and server validation are defined in Section 8.4; the table lists what the tracker sends.
| Field | Type | Sent when | Source |
|---|---|---|---|
n |
string | always | "pageview", "engagement", or the custom/auto event name. |
u |
string | always | Absolute URL of the page the event belongs to: location.href (fragment removed unless hash routing is on), or options.url for manual events. |
d |
string | always | domain option. |
r |
string | pageviews only | See Section 7.6.2. |
w |
integer | always | window.innerWidth rounded. Omitted when 0 or unavailable. |
h |
integer | only when hash routing is on | 1. |
p |
object | only when non-empty | Merged properties (Section 7.6.2), at most 30 keys; key at most 300 characters; value coerced to a string and truncated to 2,000 characters. |
$ |
object | only when revenue is provided | { "amount": "<decimal string>", "currency": "<ISO 4217>" }. |
i |
boolean | only when false |
Omitted means true. Always false for engagement and 404, and when the caller passes interactive: false. |
sd |
integer | engagement only |
maxDepth (0 to 100). |
e |
integer | engagement only |
delta milliseconds. |
7.6.2 Construction rules #
u: for automatic eventsnew URL(location.href); if hash routing is off,url.hash = "". For engagement events,uis the stored URL of the page the ping belongs to (not the current location). For manual events withoptions.url: an absolutehttp/httpsURL is used as-is (fragment removed unless hash routing); a string beginning with/is resolved againstlocation.origin; anything else is ignored with a log line and the current location is used.r(pageviews only): the first pageview of a page load sendsdocument.referrer(omitted if empty). Every later pageview in the same page load (SPA navigation, manual pageview) sends the URL of the page being left (the storeduof the previous tracked page), so the server's self-referral rule (Section 8.10.4) suppresses it and the original external referrer is never attributed twice. After a bfcache restore the referrer is omitted.pmerge order, later overriding earlier:customProperties(object, or the function's return value for the event name), then event-specific properties (options.props, module-supplied properties, or tagged-class properties). Non-object or throwingcustomPropertiesfunctions contribute nothing. Values:stringas is;numberandbooleancoerced withString();null/undefinedskipped; objects and arrays skipped. Keys are trimmed; empty keys are skipped. After merging, if more than 30 keys remain, the first 30 in insertion order are kept.- Revenue:
options.revenue = { amount, currency }.amountmay be a number or a numeric string; it is converted withString(amount).currencyis upper-cased. Ifamountis not a finite non-negative decimal of at most 999,999,999,999.999 (the server rounds extra fractional digits to 3), orcurrencyis not three ASCII letters, the revenue is dropped on the client and a log line written; the event is still sent without$. The tracker therefore never sends the invalid revenue that the server would reject with400 $ is invalid(Section 8.14); hand-written server-side callers receive that400. - Interactive:
options.interactive === falsesetsi: false. - The event name is trimmed; an empty or non-string name drops the call (log line, callback with
{ ignored: true, reason: "invalid_name" }). Names longer than 120 characters are truncated to 120.
7.6.3 Transmission #
For each event, after transformRequest:
- Serialize with
JSON.stringify. - Unload-sensitive events (engagement pings on
visibilitychange/pagehide, and clicks that may navigate away: outbound links, downloads, tagged links): trynavigator.sendBeacon(endpoint, new Blob([body], { type: "text/plain" })). If it returnsfalseor throws, fall through to step 3. - All other events and fallbacks:
fetch(endpoint, { method: "POST", body, headers: { "Content-Type": "text/plain" }, keepalive: true, credentials: "omit", mode: "cors", cache: "no-store" }). Payload bounding happens before serialization: if the serialized body would exceed 30,000 bytes (possible only with very large property values, since 30 properties of 2,000 characters can reach about 60,000 bytes), every property value is truncated to 500 characters; if the body is still above 30,000 bytes,pis omitted entirely. This keeps every request below both the browser'skeepalivebody limit (64 KiB) and the server's 32 KB limit (Section 8.2). - If
fetchis undefined, useXMLHttpRequestwith the same method, body and header. credentials: "omit"guarantees no cookies are attached to the request.
The tracker never reads the response body. A 202 is success. The request is not retried: events are best-effort, and retries on unload would produce duplicates.
7.6.4 Callbacks #
If options.callback is a function it is called exactly once per quietstat(...) call: callback({ status }) on any HTTP response (including non-2xx), callback({ error }) on a network failure, callback({ ignored: true, reason }) when nothing was sent. reason is one of opted_out, localhost, file_protocol, automation, privacy_signal, transform, invalid_name, module_disabled. For beacon-sent events, which give no response, the callback is called with { status: 0, beacon: true } immediately after sendBeacon returns true. An exception in a callback is swallowed.
7.6.5 Error handling #
- Each public function and each listener body runs inside
try/catch; exceptions are swallowed and logged whenloggingis true. - A rejected
fetchpromise is caught; nothing is thrown or reported to the page's unhandled-rejection handlers. - Non-2xx responses are logged when
loggingis true (the400reason text is included, Section 8.3); they are never retried. - If
endpointcannot be resolved to a URL, the tracker is inert and logs the reason. - If the script is blocked (content blocker, CSP, network failure), the stub from Section 7.3.1 keeps accepting calls into an array that is never drained. The array holds only the arguments of calls the page itself made, so no cap is needed.
7.7 Public JavaScript API #
7.7.1 Global function #
After the script (or the stub) is present, window.quietstat is a function with this signature:
quietstat(eventName: string, options?: EventOptions): void;
interface EventOptions {
/** Custom properties for this event. Flat object; values are coerced to strings (Section 7.6.2). */
props?: Record<string, string | number | boolean | null | undefined>;
/** Revenue attached to the event. Meaningful on custom events only (Section 8.14). */
revenue?: { amount: number | string; currency: string };
/** Override the page URL. Absolute http(s) URL, or a path starting with "/". */
url?: string;
/** false = does not count against bounce rate. Default true. */
interactive?: boolean;
/** Called once with the send result (Section 7.6.4). */
callback?: (result: { status?: number; error?: unknown; ignored?: boolean; reason?: string; beacon?: boolean }) => void;
}Additional members on the function: quietstat.init(options) (Section 7.4), quietstat.q (the pre-load queue array), quietstat.o (pre-load options), quietstat.l (set to 1 once the real script has started).
7.7.2 Reserved names #
| Name | Behavior |
|---|---|
"pageview" |
Sends a pageview (manual pageview). options.url and options.props are honored; revenue is ignored. Always sent regardless of the SPA dedupe rule. |
"engagement" |
Reserved. Calls with this name from page code are ignored with a log line, because only the tracker may send engagement pings. |
| Any other string | Custom event. Length 1 to 120 characters after trimming. |
7.7.3 Calls before the script has loaded #
Calls made while only the stub exists are appended to quietstat.q as arguments objects. At startup (Section 7.5.1 step 4) the real script processes the queue in order, applying every entry as though it had just been called; pageviews and custom events from the queue are sent after the initial automatic pageview. After the queue is drained, quietstat.q.push(entry) applies the entry immediately (Section 7.5.1 step 4). Because the server stamps events with its own receive time, queued events are not backdated.
7.7.4 Examples #
quietstat("Signup", { props: { plan: "Pro", method: "email" } });
quietstat("Purchase", { revenue: { amount: "29.90", currency: "EUR" }, props: { sku: "A-100" } });
quietstat("Video Played", { interactive: false });
quietstat("pageview", { url: "/virtual/checkout/step-2" });
quietstat("Download", { callback: (r) => { if (r.status === 202) location.assign("/file.zip"); } });7.8 Build, Size Budget and CI Enforcement #
7.8.1 Modules and byte budgets #
The build is esbuild (version per Section 3) with minify: true, target: "es2018", format: "iife" for the core and function-expression output for modules, legalComments: "none", property mangling for names beginning with _, and no runtime dependencies. Sizes are measured as gzip -9 of the minified file, in bytes.
| Artifact | Contents | Budget (gzip bytes) |
|---|---|---|
core.js |
Startup, option resolution, ignore checks, History wrapping and SPA/hash navigation, pageview sending, custom events API, queue drain, transformRequest, payload construction, transport (beacon, fetch, XHR) |
1,500 |
tagged |
CSS-class tagged events | 650 |
outbound |
Outbound link tracking and shared link-walk helper | 450 |
downloads |
File download tracking | 400 |
forms |
Form submission tracking | 300 |
notfound |
404 tracking | 260 |
engagement |
Scroll depth and engaged time | 800 |
script.js (generic: core + all modules) |
Everything | 4,800 |
7.8.2 CI gate #
A script packages/tracker/build/check-size.ts runs in CI (Section 18) after the tracker build. It computes the gzip -9 size of core.js and every module and of the assembled generic script, compares each against the table above, and fails the build (exit code 1, printing the artifact, actual size and budget) when any exceeds its budget. It also writes packages/tracker/dist/sizes.json and prints a comparison against the previous main-branch value; a growth of more than 100 bytes in any artifact without a corresponding budget change in the same pull request fails the check. Budgets are constants in packages/tracker/build/size-budget.json. Changing a budget requires editing that file in the same pull request as the code change that needs it.
7.8.3 Other build outputs #
dist/npm/for the npm package (Section 7.10): ESM, CJS and type declarations built from the same source modules.- A test page set in
packages/tracker/test/pages/used by the Playwright suite (Section 18): hash routing, SPA, tagged events, forms, downloads, outbound links, 404, scrolling and visibility scenarios. - Every release of the tracker runs a golden test that asserts the exact JSON payloads produced for a fixed set of interactions.
7.9 Browser Support #
Supported: browsers released in the last four years at the time of each release, meaning: Chrome and Edge 90 and later, Firefox 90 and later, Safari 14 and later (macOS and iOS), Samsung Internet 15 and later, Opera 76 and later. Required platform features: URL, JSON, addEventListener, history.pushState. Optional features with fallbacks: fetch with keepalive (Firefox before 133 lacks keepalive support; the tracker then uses sendBeacon), navigator.sendBeacon, performance.now, visibilityState, requestAnimationFrame, document.prerendering.
Unsupported browsers (Internet Explorer 11 and older, Opera Mini): if URL or JSON or addEventListener is missing, the script exits at startup without error and without installing any listener, and window.quietstat remains the stub.
7.10 npm Package @quietstat/tracker #
7.10.1 Package contract #
| Field | Value |
|---|---|
| Name | @quietstat/tracker |
| License | MIT |
| Module formats | ESM (import) and CJS (require) via the exports map; types for TypeScript |
sideEffects |
false |
| Runtime dependencies | none |
| SSR | Importing in Node, edge runtimes or during server rendering is safe: no code touches window at import time. All functions are no-ops when typeof window === "undefined". |
exports map entries:
| Subpath | Content |
|---|---|
. |
Full tracker (all modules). |
./lite |
Core only (pageviews, SPA navigation, custom events, transformRequest); the module-dependent options are accepted but ignored. |
./package.json |
Package metadata. |
The package is published with npm provenance enabled from the release workflow (Section 4.12.2).
7.10.2 API and types #
export interface RevenueOptions { amount: number | string; currency: string }
export interface EventResult {
status?: number;
error?: unknown;
ignored?: boolean;
reason?:
| "opted_out" | "localhost" | "file_protocol" | "automation"
| "privacy_signal" | "transform" | "invalid_name" | "module_disabled" | "not_initialized";
beacon?: boolean;
}
export interface EventOptions {
props?: Record<string, string | number | boolean | null | undefined>;
revenue?: RevenueOptions;
url?: string;
interactive?: boolean;
callback?: (result: EventResult) => void;
}
/** The wire-format payload (Section 7.6.1), passed to transformRequest. */
export interface EventPayload {
n: string;
u: string;
d: string;
r?: string;
w?: number;
h?: 1;
p?: Record<string, string>;
$?: { amount: string; currency: string };
i?: false;
sd?: number;
e?: number;
}
export interface FileDownloadsOptions { fileExtensions: string[] }
export interface InitOptions {
/** Required: the site domain, or comma-separated list of up to 10. */
domain: string;
/** Required: absolute URL, or same-origin path, of the event endpoint. */
endpoint: string;
autoCapturePageviews?: boolean; // default true
hashBasedRouting?: boolean; // default false
outboundLinks?: boolean; // default false
fileDownloads?: boolean | FileDownloadsOptions; // default false
formSubmissions?: boolean; // default false
track404?: boolean; // default false
taggedEvents?: boolean; // default true
engagement?: boolean; // default true
customProperties?:
| Record<string, string | number | boolean | null | undefined>
| ((eventName: string) => Record<string, string | number | boolean | null | undefined>);
transformRequest?: (payload: EventPayload) => EventPayload | null | undefined | false;
captureOnLocalhost?: boolean; // default false
respectPrivacySignals?: boolean; // default false
allowAutomation?: boolean; // default false
logging?: boolean; // default false
}
/** Starts the tracker. Returns a function that removes every listener and History wrapper installed. */
export function init(options: InitOptions): () => void;
/** Sends a custom event. Before init() or on the server it returns without effect and calls back with reason "not_initialized". */
export function track(eventName: string, options?: EventOptions): void;
/** Sends a pageview manually. Equivalent to track("pageview", options). */
export function pageview(options?: Pick<EventOptions, "props" | "url" | "callback">): void;
/** True after init() has been called in this window and not yet destroyed. */
export function isInitialized(): boolean;
/** Sets the localStorage opt-out flag (Section 7.5.12). Returns false if storage is unavailable. */
export function optOut(): boolean;
/** Removes the localStorage opt-out flag. */
export function optIn(): boolean;7.10.3 Behavior differences from the script #
endpointhas no default in the package, because there is no script URL to derive it from.initthrows aTypeErrorsynchronously whendomainorendpointis missing or not a string (the only place in the tracker that throws; a build-time or first-render failure is preferable to silently collecting nothing).- Calling
inittwice returns the same teardown function and the second call's options are ignored (logged whenloggingis true). Calling the returned teardown function restores the originalhistory.pushState/replaceState(only if the tracker's wrapper is still the outermost wrapper; otherwise the wrapper becomes a pass-through), removes all listeners, and allowsinitagain. This is required for hot-module-replacement and React strict mode double-mounting. - The package registers
window.quietstatfor interoperability with the CSS-class and form features and for GTM coexistence only wheninitruns in a browser; it uses the same function, queue andlflag as the script. If the script is also present on the page, whichever starts first wins and the other becomes a no-op (thelflag). track/pageviewbeforeinitin a browser are queued inwindow.quietstat.qand drained byinit.
7.10.4 Packaging and tests #
Source in packages/tracker/src; declarations generated with the TypeScript compiler (version per Section 3); a type-test file (packages/tracker/test/types.test-d.ts) asserts the exported surface compiles under strict. Release is manual-approved from CI on a tag tracker-v* (Section 4.12.2). The README documents install, init for Next.js, Nuxt, SvelteKit, Astro and plain bundlers, proxy usage and the opt-out helpers.
7.11 WordPress Plugin #
Directory: integrations/wordpress. License GPL-2.0-or-later. Requires PHP 8.1 or later; tested up to the current WordPress release at release time (Section 3.1); minimum WordPress 6.3 (needed for script strategy support).
7.11.1 Identity #
| Item | Value |
|---|---|
| Plugin name | Quietstat Analytics (the visible brand string comes from the compile-time constant mirroring PRODUCT_NAME, Section 16.4, in the plugin's includes/brand.php) |
| Slug / text domain | quietstat |
| Main file | quietstat.php |
| Options | One option row quietstat_options (serialized array, autoload yes); version row quietstat_db_version |
| REST namespace | quietstat/v1 |
| Uninstall | uninstall.php deletes both option rows and the transients quietstat_script_cache and quietstat_site_status |
7.11.2 Settings page #
Location: Settings → Quietstat (options-general.php?page=quietstat), capability manage_options, saved through the WordPress Settings API (nonce-protected, register_setting( 'quietstat_group', 'quietstat_options', [ 'sanitize_callback' => ... ] )).
| Field (option key) | Type | Default | Validation (server-side, on save) |
|---|---|---|---|
Instance URL (instance_url) |
text | empty | Required for tracking. Must parse as https:// URL (or http:// only when the host is localhost); trailing slash stripped; no path, query or credentials. On failure: settings error "Enter the https:// address of your Quietstat instance." |
Site domain (domain) |
text | host of home_url() with one leading www. removed |
Lowercased; 1 to 253 characters; letters, digits, ., -; at least one dot. |
Script ID (script_id) |
text | empty | Empty, or exactly 12 characters of [0-9a-hjkmnp-tv-z]. Empty means: use the generic script with data-domain. |
Enable first-party proxy (proxy_enabled) |
checkbox | off | When on, requires proxy_key to be non-empty. |
Proxy key (proxy_key) |
password text | empty | Exactly 32 characters of [0-9a-hjkmnp-tv-z] (Section 7.15.4). Stored in the option row; displayed masked; editing replaces it. |
Outbound links (outbound_links) |
checkbox | on | |
File downloads (file_downloads) |
checkbox | on | |
Form submissions (form_submissions) |
checkbox | off | |
404 tracking (track_404) |
checkbox | on | |
Tagged events (tagged_events) |
checkbox | on | |
Engagement (engagement) |
checkbox | on | |
Hash-based routing (hash_routing) |
checkbox | off | |
Exclude logged-in roles (excluded_roles) |
multi-checkbox of wp_roles() |
administrator |
Subset of existing role slugs. |
Track logged-out previews (track_previews) |
checkbox | off | When off, requests with is_preview() or is_customize_preview() true never receive the snippet. |
Trusted client-IP header (ip_header) |
select | REMOTE_ADDR |
One of REMOTE_ADDR, HTTP_X_FORWARDED_FOR (first entry), HTTP_CF_CONNECTING_IP, HTTP_X_REAL_IP. Used only by the proxy route. |
Show dashboard menu item (show_dashboard) |
checkbox | off | Section 7.11.6. |
Embedded dashboard URL (embed_url) |
text | empty | Empty, or an https:// URL of a shared-link embed (Section 12). |
The page includes a Test connection button (admin-AJAX, nonce and manage_options checked) that performs GET {instance_url}/readyz through wp_remote_get with a 5-second timeout and, when script_id is set, GET {instance_url}/js/q-{script_id}.js, and prints success/failure for each. It never sends the proxy key. It also shows a status line "Last event forwarded: " read from the transient quietstat_site_status, which the proxy route updates every time it forwards an event (Section 7.11.4); it reads "No events forwarded yet" when the transient is absent and "Proxy disabled" when proxy_enabled is off, in which case the page links to the dashboard's Verify installation flow (Section 7.15.2). The plugin never calls the dashboard API (it holds no API credentials).
7.11.3 Snippet injection #
Hooks: wp_enqueue_scripts at priority 1 (front end only; never in wp-admin, never for REST, XML-RPC, feeds, or AJAX). The snippet is injected only if all of the following hold:
instance_urlanddomainare valid per Section 7.11.2.- The request is not
is_admin(),is_feed(), a REST request, awp-login.phprequest, and (unlesstrack_previews) not a preview or Customizer request. - The current user is not logged in with a role listed in
excluded_roles. A user with several roles is excluded if any role is excluded. The check runs at render time, so page caches that cache logged-out pages work; pages served to excluded users are not cached by well-behaved cache plugins because they are served to logged-in users, and as a second safeguard the plugin also addsdata-quietstat-ignoreto the<html>element through thelanguage_attributesfilter for excluded users (Section 7.5.12).
Injection method: wp_register_script( 'quietstat', $src, [], null, [ 'strategy' => 'async', 'in_footer' => false ] ), then wp_add_inline_script( 'quietstat', $stub, 'before' ) where $stub is the stub of Section 7.3.1 with an init call containing, as applicable, endpoint (proxy) and the option overrides for toggles the user disabled in WordPress that differ from the baked configuration. $src is:
- Proxy disabled and
script_idset:{instance_url}/js/q-{script_id}.js. - Proxy disabled and
script_idempty:{instance_url}/js/v1/script.jsand the inline stub passes all options throughinit(the generic script has no baked configuration). - Proxy enabled:
{site_url}/wp-json/quietstat/v1/script.js.
A script_loader_tag filter adds data-cfasync="false", data-no-optimize="1" and data-no-defer="1" to the tag (so Cloudflare Rocket Loader and popular optimization plugins do not delay or combine it).
404 integration: on wp_head priority 1, when is_404() is true, the plugin prints <meta name="quietstat:status" content="404"> (Section 7.5.8 rule 1).
7.11.4 Proxy through the WordPress REST API #
Registered with register_rest_route( 'quietstat/v1', ... ), permission_callback => '__return_true' (public by design: visitors are anonymous).
| Route | Method | Behavior |
|---|---|---|
/wp-json/quietstat/v1/script.js |
GET | Returns the instance script body fetched with wp_remote_get from the same URL the non-proxy mode would use, cached in the transient quietstat_script_cache for 1 hour (stale copy served if the instance is unreachable; 502 with body script unavailable if there is no cached copy). Sends Content-Type: application/javascript; charset=utf-8 and Cache-Control: public, max-age=3600. |
/wp-json/quietstat/v1/event |
POST | Reads the raw request body (rejected with 413 above 32,768 bytes and 400 if empty). Forwards to {instance_url}/api/event using wp_remote_post with timeout 3 seconds, blocking false, and these headers: Content-Type copied from the request (default text/plain), User-Agent copied from the visitor's User-Agent header, X-Forwarded-For set to the visitor IP, X-Quietstat-Client-IP set to the visitor IP, X-Quietstat-Proxy-Key set to proxy_key (the server-side contract is Section 8.7.2). Responds 202 with an empty body immediately, never waiting for the instance. |
The visitor IP is read from the server variable selected in ip_header; if the value is not a valid IPv4/IPv6 literal (for HTTP_X_FORWARDED_FOR the first comma-separated entry is taken and trimmed) the plugin falls back to REMOTE_ADDR. The plugin applies a per-IP limit of 60 requests per 10 seconds keeping the counter in wp_cache_* when a persistent object cache exists, otherwise in a transient with a 10-second expiry, keyed by substr( hash_hmac( 'sha256', $ip, wp_salt( 'nonce' ) ), 0, 16 ) (never the address itself), and answers 429 beyond it. In proxy mode the inline stub sets endpoint: "{site_url}/wp-json/quietstat/v1/event". Because the instance's own limits apply as well (Section 8.5), the plugin's limit is a courtesy protecting the WordPress host.
7.11.5 Shortcode and block for the embedded dashboard #
[quietstat_dashboard] renders an <iframe> pointing to the embed_url option (Section 12 defines embeddable dashboards and their parameters). Attributes:
| Attribute | Default | Rule |
|---|---|---|
url |
the embed_url option |
https:// URL only; otherwise the shortcode renders nothing (and an HTML comment for administrators). |
height |
600 |
Integer 200 to 3000, pixels. |
width |
100% |
100% or an integer pixel value 200 to 3000. |
The iframe has loading="lazy", referrerpolicy="no-referrer", sandbox="allow-scripts allow-same-origin allow-popups", and title="Analytics dashboard". A Gutenberg block quietstat/dashboard with the same three attributes ships using the block API v3 and renders through the same PHP function. Both require the embed_url option or url attribute; capability to insert them is the standard edit_posts.
7.11.6 Admin dashboard menu item #
When show_dashboard is on and embed_url is set, a top-level menu "Analytics" (capability manage_options, icon dashicons-chart-area) renders the same iframe full-width. Otherwise no menu is registered.
7.11.7 Security and privacy requirements #
All admin output is escaped (esc_html, esc_attr, esc_url), all admin writes verify nonces and current_user_can( 'manage_options' ), all input is sanitized on save per the table in Section 7.11.2, and no data is sent by the plugin to any host other than the configured instance_url. The plugin stores no visitor data except short-lived (10-second) keyed-hash rate-limit counters, which contain no address, sets no cookies, adds nothing durable to the database beyond the two options, and makes no outbound requests at install or activation time.
7.11.8 WordPress.org packaging #
| Item | Content |
|---|---|
readme.txt |
Standard sections: header (Contributors, Tags: analytics, privacy, cookie-free, statistics, gdpr; Requires at least: 6.3; Tested up to: set from the current WordPress release at release time, Section 3.1; Requires PHP: 8.1; Stable tag; License: GPLv2 or later; License URI), Description, Installation, FAQ (does it need a cookie banner; how to exclude admins; how to use the proxy; what data leaves the site), Screenshots (settings page, test connection, shortcode), Changelog, Upgrade Notice, and a External services section naming the configured Quietstat instance, what is sent (page URL, referrer, screen width, user-agent and IP forwarded only in proxy mode), when, and linking to the instance's privacy policy (the service the plugin talks to is the customer's own instance). |
| Assets | .wordpress-org/ folder: icon-128x128.png, icon-256x256.png, banner-772x250.png, banner-1544x500.png, screenshot-1.png to screenshot-3.png. |
| i18n | All strings wrapped in __()/esc_html__() with text domain quietstat; languages/quietstat.pot generated by wp i18n make-pot. |
| Lint and tests | PHPCS with the WordPress Coding Standards ruleset and PHPCompatibility for 8.1+, PHPUnit with wp-env; CI fails on any PHPCS error. The WordPress.org Plugin Check tool must report no errors. |
| Release | A zip built from integrations/wordpress by scripts/build-wp-zip.sh (excludes tests and dev files, includes LICENSE), and an SVN deploy performed by the CI release job (Section 4.12.2) on tag wordpress-v*. |
| Guideline compliance | No obfuscated code, no external loading of non-service assets, no tracking of users without consent (the plugin collects nothing itself), no "phone home", no upsell notices beyond one dismissible link, GPL-compatible assets only. |
7.12 Google Tag Manager Template #
Directory: integrations/gtm. License Apache-2.0 (required for the Community Template Gallery). Files: template.tpl (the exported template: INFO, TEMPLATE_PARAMETERS, SANDBOXED_JS_FOR_WEB_TEMPLATE, WEB_PERMISSIONS, TESTS sections), metadata.yaml, LICENSE, README.md.
7.12.1 Template identity #
| Item | Value |
|---|---|
| Type | Tag |
| Display name | Quietstat Analytics |
| Brand | Quietstat (compile-time constant mirroring PRODUCT_NAME, Section 16.4) |
| Categories | ANALYTICS, STATISTICS |
| Container contexts | WEB |
7.12.2 Fields #
| Field (parameter name) | Type | Required | Default | Validation / notes |
|---|---|---|---|---|
tagType |
Select: init ("Initialize and track pageviews"), event ("Track custom event") |
yes | init |
Controls which of the groups below are shown. |
siteDomain |
Text | yes | {{Page Hostname}} hint in help text |
Non-empty; regex ^[a-zA-Z0-9.-]+(,[a-zA-Z0-9.-]+){0,9}$. Group init. |
instanceUrl |
Text | yes | empty | Regex ^https://[a-zA-Z0-9.-]+(:[0-9]{1,5})?$; no trailing slash. Group init. |
scriptId |
Text | no | empty | Empty, or regex ^[0-9a-hjkmnp-tv-z]{12}$. Empty selects the generic script. Group init. |
endpointOverride |
Text | no | empty | Empty, or an absolute https:// URL, or a path starting with /. Used for first-party proxy. Group init. |
autoCapturePageviews |
Checkbox | no | checked | Group init. |
hashBasedRouting |
Checkbox | no | unchecked | Group init. |
outboundLinks |
Checkbox | no | unchecked | Group init. Meaningful on the generic script and on per-site scripts with the module enabled. |
fileDownloads |
Checkbox | no | unchecked | Group init. |
fileExtensions |
Text | no | empty | Comma-separated list; shown when fileDownloads is checked; empty means default list. |
formSubmissions |
Checkbox | no | unchecked | Group init. |
track404 |
Checkbox | no | unchecked | Group init. |
taggedEvents |
Checkbox | no | checked | Group init. |
engagement |
Checkbox | no | checked | Group init. |
respectPrivacySignals |
Checkbox | no | unchecked | Group init. |
customProperties |
Simple table (name, value) | no | empty | Up to 30 rows; applied to every event of this tag's script instance. Group init. |
eventName |
Text | yes (event) | empty | 1 to 120 characters. Group event. |
eventProps |
Simple table (name, value) | no | empty | Up to 30 rows. Group event. |
revenueAmount |
Text | no | empty | Decimal number or GTM variable. Group event. If set, revenueCurrency is required. |
revenueCurrency |
Text | no | empty | Three letters. Group event. |
interactive |
Checkbox | no | checked | Group event. |
eventUrl |
Text | no | empty | Optional URL override. Group event. |
7.12.3 Sandboxed JavaScript behavior #
init tag.
- Validate
instanceUrl,siteDomainandscriptIdagainst the patterns in Section 7.12.2. On failure calldata.gtmOnFailure()and return without injecting. - If
copyFromWindow("quietstat.l")is truthy, or the template's owntemplateStoragekeyinjectedistrue, calldata.gtmOnSuccess()and return (a secondinittag on the same page is a no-op). - Build the options object from the fields (keys as in Section 7.4;
domainfromsiteDomain;endpointfromendpointOverridewhen set;fileDownloadsastrueor{ fileExtensions: [...] };customPropertiesfrom the table). - Create the stub with
setInWindow("quietstat", ...)only through the permitted globals:const queue = createQueue("quietstat.q")provides the queue function, andsetInWindow("quietstat.o", options, true)stores the options. The real script drainsquietstat.qentries whether they areargumentsobjects or arrays (Section 7.5.1 step 4). - Call
injectScript(url, data.gtmOnSuccess, data.gtmOnFailure, "quietstat-script"), whereurlisinstanceUrl + "/js/q-" + scriptId + ".js"whenscriptIdis set andinstanceUrl + "/js/v1/script.js"otherwise. The cache token"quietstat-script"makes GTM load the script at most once per page. SettemplateStoragekeyinjectedtotrue.
event tag. Validate eventName (1 to 120 characters) and, when revenueAmount is set, a decimal amount and a three-letter revenueCurrency; on failure call data.gtmOnFailure(). Build options = { props, revenue, url, interactive } (omitting empty members) and push [eventName, options] through createQueue("quietstat.q"), then call data.gtmOnSuccess(). If the real script has already started it processes new queue entries synchronously, because after startup quietstat.q.push is replaced by a function that applies the entry immediately (Section 7.5.1 step 4 and Section 7.7.3; covered by the golden tests in Section 7.8.3).
SPA navigations on pages that load GTM are handled by the tracker's History wrapping (Section 7.5.2), so no "History Change" trigger is needed for pageviews; the template's help text states this.
7.12.4 Permissions #
| Permission | Configuration |
|---|---|
inject_script |
Allowed URL match patterns: https://*/js/*. (Instances are on customer-chosen domains, so the host component is a wildcard; the path is restricted to /js/.) |
access_globals |
Keys quietstat (read, write, execute), quietstat.q (read, write, execute), quietstat.o (read, write), quietstat.l (read). |
logging |
Debug and test environments only. |
read_title, get_url, read_referrer |
Not requested; the tracker reads these itself. |
No cookie, local storage or data-layer permission is requested; the template therefore cannot read or write cookies or storage.
7.12.5 Template tests #
The TESTS section contains scenarios (run in the Template Editor) asserting: init with only the domain injects https://<instance>/js/v1/script.js; init with a scriptId injects the per-site URL; autoCapturePageviews unchecked produces an options object containing autoCapturePageviews: false; event tag with revenue pushes { revenue: { amount, currency } }; failure of injectScript calls gtmOnFailure; invalid instanceUrl calls gtmOnFailure without injecting.
7.12.6 Gallery submission #
- A public GitHub repository (
quietstat/gtm-template, or the owner's chosen organization) mirrorsintegrations/gtm:template.tpl,metadata.yaml,LICENSE(Apache-2.0 text),README.md. metadata.yamlcontainshomepage(the product's public home URL),documentation(the URL of the install documentation),versions(a list whose newest entry hasshaequal to the full 40-character commit SHA of the release commit andchangeNotes), andbrand/displayNamefields consistent withtemplate.tpl.- A release is a Git tag
gtm-v*; CI validates thattemplate.tplimports cleanly (an import-parser script checks section headers and permission JSON), that tests pass in a headless template-test runner where available, and that theshainmetadata.yamlrefers to an existing commit. - Submission is performed by the maintainer through the Template Editor's "Submit to gallery" flow (or the equivalent gallery contribution flow) pointing at the repository; updates are new entries in
versions. The owner must complete this once manually; it cannot be automated. Until the gallery accepts the template, the dashboard's GTM guide (Section 7.14) instructs importingtemplate.tplfrom the repository's release asset (Templates → New → Import).
7.13 Proxy Guides #
Purpose: serve the script and receive events on the customer's own domain so content blockers that match third-party hosts do not interfere and so visitors see a first-party request.
7.13.1 Common contract #
- Two proxied paths per customer site, using the prefix
PREFIX(default/m):PREFIX/script.jsforwardsGETtoINSTANCE/js/q-SCRIPT_ID.js(orINSTANCE/js/v1/script.jswhen no script id), andPREFIX/eventforwardsPOSTtoINSTANCE/api/event. - The event request must carry the visitor's real
User-Agent(default behavior of proxies) and these two headers:X-Quietstat-Client-IP: <visitor ip>andX-Quietstat-Proxy-Key: <site proxy key>. The instance honorsX-Quietstat-Client-IPonly for a site whose proxy key matches (Section 8.7.2). The proxy must overwrite, never append to or forward, any client-supplied value of these headers. - The proxy key is created in the dashboard (Section 7.15.4). It is a secret and is configured in the proxy's environment or config, never in client-side code.
- The proxy must pass the request body unmodified, limit it to 32 KB, accept only
POSTon the event path, and not cache the event response. - The script response may be cached by the proxy for up to 1 hour.
- In every recipe, replace
INSTANCE(for examplehttps://analytics.example.com),SCRIPT_IDandPROXY_KEYwith the values the dashboard shows.
7.13.2 Nginx #
# inside the server { } block of the customer's site
location = /m/script.js {
proxy_pass https://analytics.example.com/js/q-SCRIPT_ID.js;
proxy_ssl_server_name on;
proxy_set_header Host analytics.example.com;
proxy_set_header Accept-Encoding "";
proxy_cache_valid 200 1h;
add_header Cache-Control "public, max-age=3600";
}
location = /m/event {
limit_except POST { deny all; }
client_max_body_size 32k;
proxy_pass https://analytics.example.com/api/event;
proxy_ssl_server_name on;
proxy_set_header Host analytics.example.com;
proxy_set_header X-Quietstat-Client-IP $remote_addr;
proxy_set_header X-Quietstat-Proxy-Key "PROXY_KEY";
proxy_set_header X-Forwarded-For $remote_addr;
proxy_pass_request_headers on;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_request_buffering on;
}Notes: $remote_addr is the TCP peer. If this Nginx sits behind another proxy or CDN, configure real_ip_header and set_real_ip_from so $remote_addr is the visitor. proxy_cache_valid requires a proxy_cache_path/proxy_cache pair in the http and location context; if the customer does not use a cache zone, delete that line.
7.13.3 Caddy #
example.com {
handle /m/script.js {
rewrite * /js/q-SCRIPT_ID.js
reverse_proxy https://analytics.example.com {
header_up Host analytics.example.com
}
}
handle /m/event {
@notpost not method POST
respond @notpost 405
request_body {
max_size 32KB
}
rewrite * /api/event
reverse_proxy https://analytics.example.com {
header_up Host analytics.example.com
header_up X-Quietstat-Client-IP {client_ip}
header_up X-Quietstat-Proxy-Key "PROXY_KEY"
header_up X-Forwarded-For {client_ip}
}
}
}Notes: {client_ip} honors Caddy's trusted_proxies global option, so set it when Caddy is behind a CDN.
7.13.4 Cloudflare Workers #
wrangler.toml:
name = "quietstat-proxy"
main = "src/index.js"
compatibility_date = "2026-01-01"
routes = [{ pattern = "example.com/m/*", zone_name = "example.com" }]
[vars]
QS_ORIGIN = "https://analytics.example.com"
QS_SCRIPT_ID = "SCRIPT_ID"The key is stored as a secret: wrangler secret put QS_PROXY_KEY.
src/index.js:
export default {
async fetch(request, env) {
const url = new URL(request.url);
if (url.pathname === "/m/script.js" && request.method === "GET") {
const upstream = await fetch(`${env.QS_ORIGIN}/js/q-${env.QS_SCRIPT_ID}.js`, {
cf: { cacheEverything: true, cacheTtl: 3600 },
});
return new Response(upstream.body, {
status: upstream.status,
headers: {
"content-type": "application/javascript; charset=utf-8",
"cache-control": "public, max-age=3600",
},
});
}
if (url.pathname === "/m/event" && request.method === "POST") {
const length = Number(request.headers.get("content-length") || 0);
if (length > 32768) return new Response("payload too large", { status: 413 });
const headers = new Headers();
headers.set("content-type", request.headers.get("content-type") || "text/plain");
headers.set("user-agent", request.headers.get("user-agent") || "");
headers.set("x-quietstat-client-ip", request.headers.get("cf-connecting-ip") || "");
headers.set("x-quietstat-proxy-key", env.QS_PROXY_KEY);
const upstream = await fetch(`${env.QS_ORIGIN}/api/event`, {
method: "POST",
headers,
body: request.body,
});
return new Response(null, { status: upstream.status });
}
return new Response("not found", { status: 404 });
},
};7.13.5 Vercel and Next.js #
Script (rewrite). The script is a static, cacheable asset, so a rewrite is sufficient. next.config.js:
module.exports = {
async rewrites() {
return [
{ source: "/m/script.js", destination: "https://analytics.example.com/js/q-SCRIPT_ID.js" },
];
},
};Equivalent vercel.json:
{
"rewrites": [
{ "source": "/m/script.js", "destination": "https://analytics.example.com/js/q-SCRIPT_ID.js" }
]
}Events (route handler, required for correct visitor attribution). A plain rewrite cannot overwrite request headers, so the instance would attribute every event to the hosting platform's egress address. Use a Route Handler: app/m/event/route.ts:
export const runtime = "edge";
export async function POST(request: Request): Promise<Response> {
const forwardedFor = request.headers.get("x-forwarded-for") ?? "";
const clientIp = forwardedFor.split(",")[0].trim();
const body = await request.text();
if (body.length > 32768) return new Response("payload too large", { status: 413 });
const upstream = await fetch(`${process.env.QS_ORIGIN}/api/event`, {
method: "POST",
headers: {
"content-type": request.headers.get("content-type") ?? "text/plain",
"user-agent": request.headers.get("user-agent") ?? "",
"x-quietstat-client-ip": clientIp,
"x-quietstat-proxy-key": process.env.QS_PROXY_KEY ?? "",
},
body,
});
return new Response(null, { status: upstream.status });
}Environment variables QS_ORIGIN and QS_PROXY_KEY are set in the Vercel project (server-side, not NEXT_PUBLIC_). Vercel sets the first entry of x-forwarded-for to the client address. Pages Router equivalent: pages/api/m/event.ts with export const config = { runtime: "edge" } and the same body; the snippet's endpoint is then /api/m/event.
7.13.6 Netlify #
netlify.toml (script proxy):
[[redirects]]
from = "/m/script.js"
to = "https://analytics.example.com/js/q-SCRIPT_ID.js"
status = 200
force = trueEvents use an Edge Function so the visitor IP can be passed: netlify/edge-functions/quietstat-event.ts:
import type { Context } from "https://edge.netlify.com";
export default async (request: Request, context: Context): Promise<Response> => {
if (request.method !== "POST") return new Response("method not allowed", { status: 405 });
const body = await request.text();
if (body.length > 32768) return new Response("payload too large", { status: 413 });
const upstream = await fetch(`${Netlify.env.get("QS_ORIGIN")}/api/event`, {
method: "POST",
headers: {
"content-type": request.headers.get("content-type") ?? "text/plain",
"user-agent": request.headers.get("user-agent") ?? "",
"x-quietstat-client-ip": context.ip,
"x-quietstat-proxy-key": Netlify.env.get("QS_PROXY_KEY") ?? "",
},
body,
});
return new Response(null, { status: upstream.status });
};
export const config = { path: "/m/event" };QS_ORIGIN and QS_PROXY_KEY are set under Site configuration → Environment variables.
7.13.7 After setting up a proxy #
The snippet must use Format D (Section 7.3.4) with the matching endpoint. The dashboard's Verify installation flow (Section 7.15.2) detects a script served from the customer's domain and checks that endpoint is same-origin.
7.14 Framework and Platform Setup Guides (in-app) #
7.14.1 Delivery and structure #
Guides are Markdown documents stored in apps/web/src/content/guides/<slug>.md, rendered inside the dashboard at /sites/<domain>/setup/<slug> (the :guide segment of the setup route, Section 7.15.1). English only. They contain the placeholders {{INSTANCE_URL}}, {{SCRIPT_ID}}, {{DOMAIN}}, {{SNIPPET}} (Format A for the site), {{GENERIC_SNIPPET}} and {{PROXY_PREFIX}}; the app substitutes the site's real values at render time and the copy button copies the substituted text. Every guide has exactly these headings, in order: Before you start, Add the snippet, Track single-page navigation (or a statement that it is automatic), Optional: custom events, Verify, Troubleshooting. Every guide ends with a "Verify installation" button that opens the flow in Section 7.15.2. Every troubleshooting list includes: ad blocker/content blocker, Content-Security-Policy (script-src and connect-src must allow {{INSTANCE_URL}}, or the proxy path), and caching (clear the platform cache after editing templates).
The guide list, in display order, with required content beyond the common headings:
| Slug | Platform | Required content |
|---|---|---|
html |
Plain HTML | Paste Format A inside <head> on every page; where to put it in common templates; static-site generator note. |
nextjs |
Next.js | App Router: app/layout.tsx using next/script with strategy="afterInteractive" and an inline stub (<Script id="quietstat-stub" strategy="afterInteractive">{...stub...}</Script> followed by the src script); Pages Router: pages/_app.tsx / _document.tsx; npm alternative (init inside a client component useEffect with teardown); route changes are tracked automatically; proxy via Section 7.13.5. |
nuxt |
Nuxt | nuxt.config.ts app.head.script array with src, async: true, and an inline stub using innerHTML and tagPosition: 'head'; plugin alternative (plugins/quietstat.client.ts) using the npm package; automatic SPA tracking. |
sveltekit |
SvelteKit | Add Format A to src/app.html inside <head>; npm alternative in +layout.svelte with onMount; note about data-sveltekit-reload not being required. |
astro |
Astro | Add Format A to the shared layout's <head>; use is:inline on both <script> tags so Astro does not bundle them; View Transitions: pages loaded by the router are tracked via History API automatically, add data-astro-rerun only if the script is inline and must re-run. |
gatsby |
Gatsby | gatsby-ssr.js onRenderBody with setHeadComponents containing the async script tag and the inline stub (dangerouslySetInnerHTML); route changes tracked automatically; do not use gatsby-browser.js onRouteUpdate for pageviews. |
hugo |
Hugo | Add the snippet to layouts/partials/head.html (or the theme's equivalent), guarded by {{ if hugo.IsProduction }} so local hugo server builds do not track; set the domain in hugo.toml under [params] and read it with .Site.Params.quietstat.domain. |
ghost |
Ghost | Admin → Settings → Code injection → Site Header: paste Format A; note that Ghost(Pro) and self-hosted both support it; members portal pages are tracked. |
webflow |
Webflow | Project settings → Custom code → Head code: paste Format A; publish the site (custom code does not run in the Designer); a paid Webflow site plan is required for custom code. |
shopify |
Shopify | Online Store → Themes → Edit code → layout/theme.liquid: paste Format A before </head>. Checkout and thank-you pages do not run theme scripts: add a Customer events → Custom pixel (Settings → Customer events) that sends the purchase via fetch to {{INSTANCE_URL}}/api/event: see the code block below. |
wix |
Wix | Settings → Custom code → Add custom code: paste Format A, place in Head, apply to All pages, load once; a premium plan with a connected domain is required. |
squarespace |
Squarespace | Settings → Developer tools → Code injection → Header: paste Format A; a Business plan or higher is required. |
wordpress |
WordPress | Install the plugin (Section 7.11) and fill three fields, or paste Format A in the theme's header; exclude roles; shortcode for embedding. |
gtm |
Google Tag Manager | Import or add the template (Section 7.12), create an init tag fired on All Pages (Initialization trigger), optional event tags with Custom Event triggers; publish the container. |
npm |
npm / bundler | npm install @quietstat/tracker, init(...) call, TypeScript types, teardown, SSR-safety (Section 7.10). |
proxy |
First-party proxy | The five recipes of Section 7.13 as tabs (Nginx, Caddy, Cloudflare Workers, Vercel/Next.js, Netlify), the proxy key panel and Format D. |
Shopify custom pixel code block shown in the Shopify guide:
analytics.subscribe("checkout_completed", (event) => {
const checkout = event.data.checkout;
fetch("{{INSTANCE_URL}}/api/event", {
method: "POST",
keepalive: true,
headers: { "Content-Type": "text/plain" },
body: JSON.stringify({
n: "Purchase",
u: event.context.document.location.href,
d: "{{DOMAIN}}",
r: event.context.document.referrer || null,
w: event.context.window.innerWidth,
p: { order_id: String(checkout.order && checkout.order.id) },
$: { amount: String(checkout.totalPrice.amount), currency: checkout.totalPrice.currencyCode },
}),
});
});The guide tells the user to create a revenue goal named Purchase (Section 10). The Shopify guide also states that the pageview of the thank-you page is not tracked by the theme snippet.
Guides for Hugo, Ghost, Webflow, Wix and Squarespace explicitly state that single-page navigation is not applicable (full page loads) and that the tracker still detects it if the platform uses client-side routing.
7.14.2 Guide index and search #
The setup page (Section 7.15.1) shows the guides as a grid of cards with name and logo-less text labels, filterable by a search field that matches slug and name (case-insensitive substring). The page remembers the last selected guide per site in sessionStorage key qs:install_guide:{domain} (dashboard state, unrelated to the tracker).
7.15 Install Snippet and Verify Installation Flow (in-app) #
Visibility and permissions follow the permission matrix in Section 11.19.2: any role holding site.view can open the install page and see the snippet; changing tracking configuration and managing the proxy key requires site.settings.update (owner, admin, editor, guest_editor); running verification requires the same.
7.15.1 Install page #
Route (SPA): /sites/:domain/setup, with the optional guide segment /sites/:domain/setup/:guide (Section 4.13.3). The page is called the install page in the UI. It is shown automatically after site creation and is reachable later from Site Settings → Tracking.
Layout, top to bottom:
- Status banner: "Waiting for your first pageview" (neutral, animated), "Receiving data. Last event 12 seconds ago" (success), or "No data in the last 24 hours" (warning). Driven by the status poll (Section 7.15.2).
- Snippet panel: the code block for Format A with a Copy button and a segmented control to switch between Format A (default), Format B, and Format D (Format D is selectable only when a proxy key exists). Selecting a format never changes saved configuration.
- Tracking configuration panel (Section 7.15.3): checkboxes with live effect on the snippet explanation.
- Guides grid (Section 7.14).
- Verify installation button.
7.15.2 Status polling and verification #
Status poll. GET /api/app/v1/sites/{domain}/tracking/status, polled by the install page every 3 seconds while the banner is not "Receiving data" and the page is visible, and stopped when the status is "Receiving data" or the page is hidden.
{
"data": {
"receiving": true,
"first_event_at": "2026-11-03T10:15:02Z",
"last_event_at": "2026-11-03T10:15:41Z",
"events_last_24h": 12,
"filtered_last_24h": { "bot_user_agent": 0, "datacenter_ip": 0, "excluded_ip": 0, "excluded_page": 0, "spam_referrer": 0, "hostname_not_allowed": 0 }
}
}receiving is true when at least one accepted event exists for the site with timestamp in the last 24 hours; first_event_at/last_event_at are null when the site has never received an event. Counts come from a ClickHouse query on events (Section 6) and the filtered counters of Section 8.8.6. filtered_last_24h.bot_user_agent is the sum of the three bot reasons bot_user_agent, bot_missing_ua and bot_headless (Section 8.8.1); the other keys are the counters of the same name. Response time target below 300 ms; the query is cached in Valkey for 2 seconds per site.
Active verification. POST /api/app/v1/sites/{domain}/tracking/verify (CSRF header required, Section 4). Rate limit: policy tracking_verify (Section 17.6; 10 requests per minute per user). Permission: site.settings.update. Behavior:
- Fetch
https://{domain}/only (no plain-http fallback); redirects are followed up to 3 hops, each re-validated (Section 17.4). - SSRF protection: all server fetches go through the
guardedFetchclient of Section 17.4. Resolve the hostname before connecting; refuse (and return the diagnosticblocked_target) if any resolved address is loopback, private (RFC 1918), link-local, unique-local, carrier-grade NAT, multicast, or unspecified; or falls in any of these additional ranges: IPv4-mapped or IPv4-compatible IPv6 (::ffff:0:0/96, evaluated as the embedded IPv4 address), NAT6464:ff9b::/96,0.0.0.0/8,192.0.0.0/24,198.18.0.0/15,240.0.0.0/4,255.255.255.255, and the documentation ranges (192.0.2.0/24,198.51.100.0/24,203.0.113.0/24,2001:db8::/32). Connect to the resolved address directly so DNS rebinding cannot change it. The same checks apply to the resolved address of every redirect hop. TLS certificates are verified (certificate verification is never disabled). The request sends no cookies, no credentials and noAuthorizationheader. Onlyhttpson port 443 is allowed. 10-second total timeout; response body read capped at 2 MB;User-Agent: QuietstatVerifier/1. - Parse the HTML (no JavaScript execution) and search for a
<script>whosesrcresolves to a path ending in/js/q-{script_id}.jsor/js/script.jsor/js/v1/script.jsor/js/v1/q-{script_id}.json the instance host or on{domain}(proxy), or any<script>whosesrcends in/script.json{domain}together with an inline script containingquietstat.init. - Evaluate and return diagnostics:
{
"data": {
"verified": false,
"fetched_url": "https://example.com/",
"diagnostics": [
{ "code": "script_not_found", "severity": "error", "message": "No Quietstat script tag found on the homepage." }
]
}
}code |
Severity | Condition |
|---|---|---|
script_found |
info | A matching script tag is present. verified is true when this is present and no error diagnostic is present. |
script_not_found |
error | No matching script tag in the HTML. |
script_not_found_static |
warning | No script tag, but the page references googletagmanager.com, so the snippet may be injected by GTM; the message tells the user to rely on the status poll. |
wrong_script_id |
error | A Quietstat script tag is present but its script_id belongs to a different site. |
wrong_domain |
error | The generic script's data-domain differs (after normalization, Section 8.6) from the site's domain. |
multiple_scripts |
warning | More than one matching script tag (double counting risk; the tracker's l flag prevents the second from running, so severity is warning). |
www_mismatch |
info | The homepage redirected between www. and apex; informational because both are matched (Section 8.6). |
proxy_endpoint_cross_origin |
error | A proxy-style script source on {domain} but endpoint points to another origin. |
csp_blocks_script |
warning | A Content-Security-Policy response header exists whose script-src (or default-src fallback) does not allow the script host. |
fetch_failed |
error | Connection error, TLS failure, timeout or non-HTML response; message includes the failure kind. |
http_error |
error | The final response status is 400 or above; message includes the status. |
blocked_target |
error | SSRF rule triggered (step 2). |
Verification never sends events to the site's data and never requires a pageview. A site is considered installed for onboarding purposes (hiding the "Install" prompt on the sites list, Section 9) when receiving is true, regardless of the verification result.
7.15.3 Tracking configuration #
Tracking configuration is the set of site_settings columns track_outbound_links, track_file_downloads, file_download_extensions, track_form_submissions, track_404_pages and track_hash_routing (Section 5.5.2 owns the columns and their defaults; this section owns the API and the mapping to tracker behavior). There is no separate configuration document.
| API field (same name as the column) | Type | Default (Section 5.5.2) | Effect on the per-site script (Section 7.2.3) | Init option it controls |
|---|---|---|---|---|
track_outbound_links |
boolean | true |
Includes the outbound module |
outboundLinks |
track_file_downloads |
boolean | true |
Includes the downloads module |
fileDownloads |
file_download_extensions |
array of 1 to 50 strings, or null |
null (default list of Section 7.5.6) |
Baked as fe |
fileDownloads.fileExtensions |
track_form_submissions |
boolean | false |
Includes the forms module |
formSubmissions |
track_404_pages |
boolean | false |
Includes the notfound module |
track404 |
track_hash_routing |
boolean | false |
Bakes h |
hashBasedRouting |
The tagged and engagement modules are always included in per-site scripts and have no setting. autoCapturePageviews and respectPrivacySignals are init-only options (Section 7.4): they are not stored per site, so the dashboard shows them only as code in the snippet help. Options not representable as columns (endpoint, customProperties, transformRequest, captureOnLocalhost, allowAutomation, logging) are set only in code via init. The hostname allowlist allowed_hostnames (Section 5.5.2) is edited in the same panel; it is enforced by the server (Section 8.6 rule 5) and has no effect on the script. It accepts up to 50 entries, each a hostname (lowercased, punycode, one leading www. removed, no port or path); an entry matches that hostname and all its subdomains. An empty list (the default) means the site domain and all its subdomains.
Validation (Zod schema in packages/shared): the request body is a strict object (unknown keys rejected with 400 validation_failed); booleans are booleans; file_download_extensions is null or an array of 1 to 50 strings, each lowercase [a-z0-9]{1,10} (the limit is the one in Section 5.16).
Routes:
| Route | Permission | Behavior |
|---|---|---|
GET /api/app/v1/sites/{domain}/tracking-config |
site.view |
Returns the six fields above plus allowed_hostnames in the data envelope. |
PATCH /api/app/v1/sites/{domain}/tracking-config |
site.settings.update |
Updates any subset of the fields (strict object, at least one field) and returns the full set. Publishes the site-configuration invalidation on the Valkey channel qs:site-config (Section 8.6), which also invalidates the assembled-script cache (Section 7.2.3). The UI states that changes reach visitors within 10 minutes because of script caching (Section 7.2.2). Audited as site.updated with the changed field names (Section 11.20). |
7.15.4 Proxy key management #
Panel "First-party proxy" on the install page.
| Route | Permission | Behavior |
|---|---|---|
POST /api/app/v1/sites/{domain}/tracking/proxy-key |
site.settings.update |
Creates the proxy key if none exists, or rotates it if one exists. Returns { "data": { "proxy_key": "<value>" } }. The key is 32 characters, base32 (lowercase Crockford alphabet, 160 bits of randomness), generated with a cryptographically secure RNG. Rotation takes effect on the instance within 30 seconds (the site-configuration cache lifetime, Section 8.6); proxies configured with the old key stop having their forwarded client IP honored from then on (events are still accepted, attributed to the proxy's own address and rate-limited as a normal client, Section 8.5). |
GET /api/app/v1/sites/{domain}/tracking/proxy-key |
site.settings.update |
Returns the current key or 404 not_found if none exists. |
DELETE /api/app/v1/sites/{domain}/tracking/proxy-key |
site.settings.update |
Removes the key. |
The key is stored in plaintext in sites.proxy_key (Section 5.5.1; one of the documented exceptions to hashed secrets, Section 4.2: it must be re-displayed to the site owner and compared in constant time against a header, and is not a password: its worst-case abuse is attributing fake client IPs to that single site's events and unlocking the proxy rate tier of Section 8.5 for that site's key). It is displayed to holders of site.settings.update only, never to viewers or in shared dashboards, never logged. The panel offers the five recipes of Section 7.13 with the key, instance URL and script id pre-filled and a prefix field (default /m, validated ^/[a-z0-9_-]{1,32}$).
7.15.5 Exclude my own visits help #
A collapsible panel on the install page listing three methods with copyable instructions: the opt-out bookmarklet (Section 7.5.12), the data-quietstat-ignore attribute for server-rendered pages, and IP exclusions with a button "Add my current IP" that opens the exclusion form in Site Settings pre-filled with the IP the dashboard request came from (resolved by the same rules as Section 8.7.1).
7.15.6 Empty and error states #
| State | UI |
|---|---|
| Status poll fails (non-2xx or network) | Banner "Could not check status. Retrying." and the poll continues with exponential backoff to a maximum of 30 seconds. |
Verification returns fetch_failed |
Diagnostics list shown with the message; the "Verify installation" button is re-enabled after 10 seconds. |
| User lacks permission to verify | Button disabled with tooltip "You do not have permission to verify this site." |
8. Ingestion Pipeline & Privacy Engine #
This section owns everything that happens between the arrival of POST /api/event and the rows written to ClickHouse: parsing and validation, rate limiting, site resolution, traffic filtering, enrichment, the privacy-preserving visitor hash, sessions, revenue normalization, buffering and failure behavior. The tracker side of the contract is Section 7. The Events API documentation for third parties (Section 13) restates this contract; if the two ever differ, this section is authoritative. Table and column definitions are owned by Section 6; this section names the columns it populates.
8.1 Pipeline Overview #
The endpoint lives in apps/server (Section 3). The handler is a straight-line pipeline; each step either continues or terminates the request. Every termination after step 3 that is not an error returns 202 with an empty body, so tracker-side code never sees the reason.
| # | Step | Terminates with | Section |
|---|---|---|---|
| 1 | CORS and method handling | 204 for OPTIONS; 405 for other methods |
8.2 |
| 2 | Per-IP rate limit | 429 |
8.5 |
| 3 | Body read with 32 KB limit, parse | 413, 400 |
8.2, 8.3 |
| 4 | Schema validation and normalization | 400 |
8.4 |
| 5 | Client IP and User-Agent acquisition | none | 8.7 |
| 6 | For each domain in d: site resolution, including the hostname allowlist |
skip that domain (all skipped: 202); hostname_not_allowed counted |
8.6 |
| 7 | UA bot filter | 202, counted |
8.8.1 |
| 8 | IP exclusion | 202, counted |
8.8.4 |
| 9 | Datacenter-IP filter (per-site toggle) | 202, counted |
8.8.2 |
| 10 | URL normalization, page exclusion | 202, counted |
8.10.1, 8.8.5 |
| 11 | Spam-referrer filter | 202, counted |
8.8.3 |
| 12 | License and cap check hook | 202, counted by Section 15 |
8.9 |
| 13 | Enrichment: UA parse, GeoIP, referrer, UTM, channel | none | 8.10 |
| 14 | Visitor hash | none | 8.11 |
| 15 | Session lookup and update | 503 if Valkey is unavailable |
8.12 |
| 16 | Engagement handling (engagement events only; replaces 13 to 15 for them) | 202 |
8.13 |
| 17 | Revenue normalization | none | 8.14 |
| 18 | Append to the in-process buffers | 202 |
8.15 |
| 19 | Respond 202 |
The response is sent after step 18 (the append is synchronous and in-memory), never after the ClickHouse insert. Processing for each domain in a multi-domain request (step 6) runs steps 7 to 18 independently, so one request can yield events for several sites. The handler latency target is the one in Section 3.7 (p50 below 3 ms, p99 below 25 ms at 2,000 events per second per process), excluding network time.
8.2 HTTP Handling #
| Aspect | Rule |
|---|---|
| Route | POST /api/event. No authentication, no CSRF, no cookies read or set. |
| CORS | Every response (including errors and the OPTIONS response) carries Access-Control-Allow-Origin: *. OPTIONS /api/event returns 204 with Access-Control-Allow-Methods: POST, OPTIONS, Access-Control-Allow-Headers: Content-Type, Access-Control-Max-Age: 86400. The tracker's text/plain POST does not trigger a preflight; the OPTIONS handler exists for server-side callers and application/json browser callers. Access-Control-Allow-Credentials is never sent. |
| Other response headers | Cache-Control: no-store, Cross-Origin-Resource-Policy: cross-origin, X-Content-Type-Options: nosniff. |
Accepted Content-Type |
text/plain (any charset parameter), application/json, or no Content-Type header. Any other value gets 400 with reason unsupported content type. |
| Body size limit | 32,768 bytes, enforced while streaming: the connection is cut and 413 returned as soon as the limit is exceeded; the Content-Length header is checked first when present. Reason text: payload too large. |
| Body read timeout | 5 seconds for the full body; on timeout 408 with reason request timeout. |
| Success | 202 Accepted, Content-Length: 0. |
| Logging | The request is not access-logged at info level. Only warn/error events about the pipeline itself are logged, and never include the IP, User-Agent, URL, referrer, properties or body (redaction rules, Section 17). |
8.3 Parsing and Error Responses #
The body is decoded as UTF-8 (invalid sequences replaced with U+FFFD) and parsed as JSON. Errors from this endpoint are plain text with Content-Type: text/plain; charset=utf-8; they do not use the JSON error envelope of Section 4 (the single documented exception). The reason is a short lowercase phrase from the table below.
| Status | Reason text | Condition |
|---|---|---|
| 400 | empty body |
Zero-length body or whitespace only. |
| 400 | invalid json |
Body is not valid JSON, or the top level is not an object. |
| 400 | unsupported content type |
Section 8.2. |
| 400 | <field> <problem> |
A validation failure from Section 8.4, for example n is required, u is not a valid url, d has too many domains, p has too many keys. Exactly one reason (the first failure in field order n, u, d, r, w, h, p, $, i, sd, e). |
| 405 | method not allowed |
With Allow: POST, OPTIONS. |
| 408 | request timeout |
Section 8.2. |
| 413 | payload too large |
Section 8.2. |
| 429 | rate limited |
Section 8.5, with Retry-After in seconds. |
| 503 | unavailable |
Valkey or Postgres required for the decision is unreachable (Section 8.15.4), with Retry-After: 5. |
8.4 Validation and Normalization #
All fields come from the JSON object. Unknown fields are ignored (never stored). The shared Zod schema lives in packages/shared (ingestPayloadSchema) and is used by the server and by the tracker's test-suite.
| Field | Type | Required | Rules | Failure |
|---|---|---|---|---|
n |
string | yes | Trimmed; 1 to 120 characters (Unicode code points); no control characters (U+0000 to U+001F, U+007F). Case preserved. Two names are reserved: pageview and engagement. |
400 n is required / n is invalid / n is too long |
u |
string | yes | At most 2,000 characters. Parses with the WHATWG URL parser. Scheme must be http:, https: or app: (the last for mobile/desktop apps using the Events API). For http(s) the host must be non-empty. |
400 u is required / u is not a valid url / u is too long |
d |
string | yes | Comma-separated list of 1 to 10 domains. Each entry: trimmed, lowercased, a trailing :<port> (digits only) removed, IDN converted to punycode, 1 to 253 characters, characters a-z0-9.- only after conversion, at least one dot or equal to localhost. Duplicates after normalization (including the port removal and the www. strip, Section 8.6) are collapsed. |
400 d is required / d is invalid / d has too many domains |
r |
string or null | no | Absolute URL of at most 2,000 characters. An invalid or over-long value is ignored (treated as no referrer); it never causes 400. |
none |
w |
integer or null | no | Integer 1 to 100,000. A number outside the range, a non-integer or a non-number is ignored (treated as unknown). | none |
h |
1, true, 0, false or absent |
no | 1/true means hash-based routing (Section 8.10.1); anything else is false. |
none |
p |
object or null | no | At most 30 keys. Each key: trimmed, 1 to 300 characters, no control characters, duplicates after trim keep the last. Each value: string of at most 2,000 characters; numbers and booleans are accepted and converted with String() (true/false, JSON number text); null drops the key; objects and arrays are rejected. An empty-string value drops the key, and a key starting with qs_, quietstat_ or $ is dropped (neither is an error). In the hosted edition the sum of key and value lengths per event is capped at 4,096 characters: entries beyond the cap are dropped from the end and counted under the metric quietstat_ingest_props_truncated_total. |
400 p has too many keys / p key is invalid / p value is invalid / p value is too long |
$ |
object or null | no | Revenue; rules in Section 8.14. Ignored (not an error) for pageview and engagement. |
400 $ is invalid |
i |
boolean | no | Default true. Forced to false for engagement. For pageview the value is ignored (a pageview is always interactive for bounce purposes). |
none (non-boolean ignored) |
sd |
integer | engagement | Integer 0 to 100. Ignored for non-engagement events. For engagement, at least one of sd, e must be present and valid. |
400 sd is invalid |
e |
integer | engagement | Integer 0 to 86,400,000 milliseconds; values above 3,600,000 are clamped to 3,600,000 (Section 8.13). Ignored for non-engagement events. | 400 e is invalid / engagement requires sd or e |
// packages/shared/src/ingest.ts (shape only; implementation follows the rules above)
export const ingestPayloadSchema = z.object({
n: z.string(), u: z.string(), d: z.string(),
r: z.string().nullish(), w: z.number().nullish(),
h: z.union([z.literal(1), z.literal(0), z.boolean()]).nullish(),
p: z.record(z.string(), z.unknown()).nullish(),
$: z.object({ amount: z.union([z.string(), z.number()]), currency: z.string() }).nullish(),
i: z.boolean().nullish(), sd: z.number().nullish(), e: z.number().nullish(),
}).loose();Normalization performed here and used by all later steps: n trimmed; d entries normalized (Section 8.6); p values stringified. The event timestamp is the server receive time in UTC with millisecond precision (DateTime64(3), Section 6.3, so that same-second events keep a stable order for funnels); sessions.started_at and ended_at are that value truncated to the second. Client-supplied times are not accepted. Request headers other than User-Agent, X-Forwarded-For (Section 8.7), X-Quietstat-Client-IP, X-Quietstat-Proxy-Key and Content-Type are never read.
Events whose page, referrer or properties contain what looks like personal data (emails, tokens) are not inspected or rewritten by the server; scrubbing is the integrator's responsibility through transformRequest (Section 7.5.14). Section 17 documents this boundary.
8.5 Rate Limiting #
The ingestion limits are the policies event_ingest and event_ingest_proxy of the canonical rate-limit table in Section 17.6; the table below restates their key and tiers for the implementer.
| Tier (policy) | Key | Limit | Window | Applies to |
|---|---|---|---|---|
Client (event_ingest) |
qs:rl:ev:<h> where <h> is the first 16 hex characters of HMAC-SHA-256(key = a key derived from SECRET_KEY_BASE with purpose "ratelimit", message = client IP text) (Section 17.6) |
100 requests | fixed 10 seconds | Every request except those that qualify for the Proxy tier |
Proxy (event_ingest_proxy) |
qs:rl:evp:<h> with <h> computed from the socket peer address in the same way |
2,000 requests | fixed 10 seconds | Requests whose X-Quietstat-Proxy-Key header equals a currently configured site proxy key |
Proxy-tier qualification. The header value is hashed with SHA-256 and looked up in an in-process set of SHA-256 digests of all configured sites.proxy_key values, refreshed together with the site cache (Section 8.6); a hit is confirmed with a constant-time comparison. The check happens before the body is read and involves no per-site lookup. A request whose header is absent, malformed or not in the set is an ordinary client: the header is ignored (Section 8.7.2), the request is limited by the Client tier, and no elevated limit is ever granted without a valid key.
Mechanics: INCR the key; on the first increment set EXPIRE 10 in the same MULTI. The client IP used for the Client tier is resolved by Section 8.7.1 (socket peer or trusted X-Forwarded-For); per-site validation of the proxy key and acceptance of X-Quietstat-Client-IP is a separate step (Section 8.7.2). The counter key contains a keyed hash derived from SECRET_KEY_BASE, never the IP itself and never the daily salt (so a salt leak or backup cannot be used to confirm addresses), and expires in 10 seconds, so rate-limit state is not a store of addresses. The limit counts requests, not events. When exceeded: 429, Retry-After: <seconds until the window ends, minimum 1>, reason rate limited, and the metric in Section 8.17 is incremented. The limiter runs before the body is read. On Valkey errors the limiter follows Section 17.6: it fails open for 60 seconds and then applies an in-process limit of 10 requests per second per IP-HMAC; session handling then fails separately with 503 (Section 8.15.4). The thresholds are constants in packages/shared, not configuration.
8.6 Site Resolution by Domain #
- Normalization. For each entry of
d: trim; lowercase; remove a trailing:<port>; convert IDN to ASCII (punycode); strip one leadingwww.. The result is the lookup key. Site domains are stored lowercase, without scheme, port, path or a leadingwww., and are unique installation-wide under this same normalization (Section 5.5.1), sowww.example.comandexample.comalways resolve to one site, whichever form the site was created with and whichever form the tracker sends, and a site onlocalhostreceives events sent withd=localhost:3000. Other subdomains (for exampleblog.example.com) are different keys and different sites. - Lookup. An in-process LRU (10,000 entries) maps the key to the resolved site configuration: site id, team id, timezone,
domain, the exclusion rules (Section 8.8.4, 8.8.5),site_settings.datacenter_filter_enabled,site_settings.allowed_hostnames, the proxy key (sites.proxy_key), andsite_settings.reporting_currency(Section 8.14). Entries live 30 seconds. A miss reads Postgres (sites ⨝ site_settingsand the exclusion tables, Section 5). Unknown domains are negatively cached for 30 seconds. The cache is invalidated early by a Valkey pub/sub message on channelqs:site-configpublished by every site-settings write (the message carries the site id); every web process subscribes. If Postgres is unreachable and a cached entry exists, the stale entry is served for up to 10 minutes past its expiry; with no cached entry the request fails with503. - Multiple domains. A request whose
dlists several domains is processed once per distinct resolved site, each with its own exclusion rules, caps, visitor hash and session. A site's events from a multi-domain request count against that site's team cap (Section 8.9) separately. Unknown entries are skipped silently. If no entry resolves to a site, the request returns202and the metricquietstat_ingest_events_dropped_total{reason="unknown_domain"}is incremented once. - A site whose team has
purge_after IS NOT NULL(pending deletion, Section 5.4.1) is treated like an unknown domain. Sites have no pause state, and a deleted site no longer exists in Postgres, so it too resolves as an unknown domain once the cache entry expires. A domain that is only held as a 30-day reservation after its site was deleted (Section 5.5.1) is likewise an unknown domain: the event is dropped with202and counted asunknown_domain. - Hostname allowlist.
site_settings.allowed_hostnames(text array, default empty, at most 50 entries; edited in the tracking settings, Section 7.15.3) is the defence against third parties naming a victim's domain indto exhaust a hosted team's allowance. Theuhost is lowercased, has its port removed and one leadingwww.stripped. When the list is empty, the host must equal the sitedomainor be a subdomain of it; when the list is non-empty, the host must equal an entry or be a subdomain of one. An event whose host is not allowed is dropped for that site with reasonhostname_not_allowed(the response is still202, the event is not billable, and the drop happens before the cap check of Section 8.9). Events with anapp:URL are checked the same way: the site owner adds the app host (conventionallylocalhost) toallowed_hostnames, because the empty-list default accepts it only when the site domain is itselflocalhost. Theuhost is stored ashostname(Section 8.10.1).
8.7 Client IP and User-Agent #
8.7.1 Client IP resolution #
- Let
peerbe the socket's remote address (IPv4-mapped IPv6 converted to IPv4). - If
peeris inside any CIDR inTRUSTED_PROXIES(Section 16.4), take theX-Forwarded-Forheader value, split on commas, trim, and walk the list from right to left, skipping entries that are insideTRUSTED_PROXIES; the first entry that is not trusted (and is a valid IP literal) is the client IP. If the list is empty, contains only trusted entries, or the first non-trusted entry is invalid, usepeer. - If
peeris not insideTRUSTED_PROXIES,X-Forwarded-Foris ignored entirely. - IPv4-mapped IPv6 results are converted to IPv4; zone identifiers are stripped; the result is a canonical textual address. Private, loopback and link-local client addresses are valid (they are hashed and stored as-is in the hash only) but produce no GeoIP result.
The client IP lives only in the request-handling variables of the current event. It is never written to ClickHouse, Postgres, Valkey (outside the keyed-hash rate-limit key and the one-way visitor hash), logs or metrics.
8.7.2 Proxy key override #
A first-party proxy (Section 7.13) cannot be in TRUSTED_PROXIES for a shared hosted instance, so it identifies the visitor explicitly. For each resolved site (Section 8.6) independently: if the request carries both X-Quietstat-Client-IP and X-Quietstat-Proxy-Key, the site has a proxy key (Section 7.15.4), the header value equals it under a constant-time comparison, and X-Quietstat-Client-IP is a valid IPv4/IPv6 literal, then that address replaces the client IP for that site's events (visitor hash, GeoIP, IP exclusions, datacenter filter). Otherwise both headers are ignored, the address from Section 8.7.1 is used, and the request has already been rate-limited as an ordinary client (Section 8.5). Only a per-site proxy key validated in constant time ever unlocks trust in X-Quietstat-Client-IP. A mismatching or malformed proxy key never causes an error response. The User-Agent of a proxied event is the proxy-forwarded User-Agent header, which the recipes preserve.
8.7.3 User-Agent #
The User-Agent request header is read as a string, truncated to its first 1,024 bytes, and used for the bot filter (Section 8.8.1), UA parsing (Section 8.10.2) and the visitor hash (Section 8.11). A missing or empty header is treated as a bot (Section 8.8.1). The raw string is never stored or logged.
8.8 Traffic Filtering #
All filters drop the event after responding 202; each drop increments the per-site counters of Section 8.8.6 and the metric quietstat_ingest_events_dropped_total{reason} (Section 8.17).
8.8.1 User-Agent bots (always on, no toggle) #
An event is dropped when any of these hold, with the reason named in each rule (bot_missing_ua for rule 1, bot_user_agent for rule 2, bot_headless for rule 3; these are three separate counters, Section 8.8.6):
- The User-Agent is missing, empty, or shorter than 8 characters (reason
bot_missing_ua). isbot(userAgent)is true (isbot per Section 3, with its bundled pattern list) (reasonbot_user_agent).- The User-Agent contains (case-insensitive) any of the headless/automation markers:
HeadlessChrome,Headless,PhantomJS,SlimerJS,Selenium,Playwright,Puppeteer,WebDriver,Lighthouse,PageSpeed,GTmetrix,Pingdom,UptimeRobot,python-requests,python-urllib,curl/,wget/,libwww,Go-http-client,okhttp,axios/,node-fetch,Java/,Apache-HttpClient(reasonbot_headless).
The filter applies to the Events API as well: server-side callers must forward the end user's real User-Agent (Section 13), and mobile or desktop apps must send a real device User-Agent. Events whose u scheme is app: skip the headless-marker list of rule 3 (so default HTTP-library User-Agents of native apps are not discarded) but are still subject to rules 1 and 2. The filter cannot be disabled per site.
8.8.2 Datacenter IPs (default on, per-site toggle) #
The site setting site_settings.datacenter_filter_enabled (Section 5.5.2; default true) enables dropping events whose client IP falls inside a known datacenter range (reason datacenter_ip). The list is a set of CIDRs for the large cloud and hosting providers (Amazon Web Services, Google Cloud, Microsoft Azure, Oracle Cloud, DigitalOcean, Linode/Akamai Connected Cloud, Vultr, Hetzner, OVHcloud), IPv4 and IPv6. In memory it is two sorted arrays of non-overlapping [start, end] ranges (IPv4 as 32-bit integers, IPv6 as BigInt), merged and deduplicated at load, searched with binary search (O(log n)). Residential, mobile, corporate-VPN and consumer-VPN ranges are not on the list. Addresses that do not parse are not matched.
8.8.3 Known spam referrers (always on) #
If the referrer host (Section 8.10.4) equals or is a subdomain of any entry in the spam-referrer domain list (the copy in DATA_DIR/lists/ refreshed by lists-refresh, else the bundled packages/shared/data/spam-referrers.txt; one lowercase domain per line; Section 8.8.7), the event is dropped with reason spam_referrer. The list is not user-editable in the product.
8.8.4 IP exclusions #
Rules come from the site's ip_exclusions rows (Section 5): each rule is a single IPv4/IPv6 address or a CIDR block (IPv4 prefix 8 to 32, IPv6 prefix 32 to 128). A site has at most 200 IP rules. The client IP (after Section 8.7) is matched against the compiled rule set (sorted ranges, binary search). A match drops the event with reason excluded_ip.
8.8.5 Page exclusions #
Rules come from the site's page_exclusions rows (Section 5): each is a path pattern of 1 to 2,000 characters beginning with / (Section 5.8.3), containing at most 8 wildcard tokens (* or **; a pattern beyond that is rejected with 400 validation_failed when it is saved). A site has at most 200 page rules. Patterns entered by users are percent-encoded with the WHATWG path percent-encode set (non-ASCII characters and spaces encoded, existing %XX kept with uppercase hex) before validation and storage, so /café matches the stored /caf%C3%A9; the dashboard displays decoded values. Pattern syntax: * matches any sequence of characters except / (including the empty sequence); ** matches any sequence including /; every other character matches itself literally and case-sensitively; no other metacharacters exist. Examples: /admin/** matches /admin/ and /admin/users/12; /blog/*/draft matches /blog/a/draft but not /blog/a/b/draft; /*.pdf matches /a.pdf but not /docs/a.pdf. Matching is done against the normalized stored page path (Section 8.10.1) and, to make /admin match /admin/ and the reverse, also against the same path with a single trailing / added or removed (the root path / is never modified). Patterns are matched by a linear-time glob matcher (a two-pointer algorithm with backtracking points at * and **, O(pattern length × path length)) compiled at load time and cached with the site configuration; they are never converted to backtracking regular expressions, so a hostile pattern cannot cause catastrophic backtracking on the ingestion hot path. A match drops the event, whatever its name (pageview, custom, or engagement), with reason excluded_page.
8.8.6 Filtered-traffic counters #
Every drop in Sections 8.8.1 to 8.8.5 and every hostname_not_allowed drop (Section 8.6 rule 5) increments the Valkey hash qs:filt:<site_id>:<yyyymmdd> (UTC date), field = reason, by 1, and refreshes EXPIRE to 35 days. Reasons: bot_user_agent, bot_missing_ua, bot_headless, datacenter_ip, spam_referrer, excluded_ip, excluded_page, hostname_not_allowed. Cap and licensing drops are counted by Section 15. The "Filtered traffic" statistic in site settings reads these hashes for the last 30 days; the tracking status endpoint (Section 7.15.2) reads the last 24 hours (today and yesterday, UTC). Failed counter increments are ignored.
8.8.7 List sources and refresh #
| List | Source | Refresh |
|---|---|---|
| Bot patterns | isbot (Section 3) bundled with the release, plus the explicit markers of Section 8.8.1 | With each release only |
| Datacenter ranges | Provider-published machine-readable range feeds for AWS, Google Cloud and Oracle Cloud, fetched by the job lists-refresh; ranges for the remaining providers shipped as a static file in the release (packages/shared/data/datacenter-ranges.txt) |
Weekly lists-refresh (schedule in Section 16.6) for the fetched feeds; every release for the static file |
| Spam referrers | packages/shared/data/spam-referrers.txt as shipped, and the refreshed copy written by lists-refresh |
Every release and weekly lists-refresh |
The lists-refresh job (both editions; a no-op when LISTS_AUTO_UPDATE=false, Section 16.4) fetches each feed with a 30-second timeout and a 20 MB size cap, parses to CIDRs or domain lines, validates (every line must parse; a feed that yields fewer than 50% of the previous entry count for that provider is rejected as suspect), merges with the shipped file, and writes the merged lists atomically (temporary file plus rename) to DATA_DIR/lists/ (datacenter-ranges.txt, spam-referrers.txt). It does not store the lists in Valkey. Each web process checks the modification time of those files every 60 seconds and reloads a list when the mtime changes. If a file under DATA_DIR/lists/ is missing or unparseable, or all feeds fail, the process uses the file bundled in the image. A failed refresh keeps the previous lists, logs a warning and increments quietstat_ingest_datacenter_refresh_failures_total. Lists are read-only at runtime for the ingestion path and require no database.
8.9 License and Cap Check Hook #
On the hosted edition (EDITION=hosted, Section 3) the pipeline calls the licensing component after all filters have passed, so filtered events never consume the allowance. The component's state, counters, thresholds (80%, 100%, 110% with the 10% grace), unlicensed handling, and emails are owned by Section 15. This section defines only the hook:
// apps/server/src/ingestion/cap-gate.ts: thin adapter over apps/server/src/modules/licensing/entitlement.ts (Section 15.3.4)
import { decideIngest, type IngestDecision } from "../modules/licensing/entitlement";
/** 'accept' | 'drop_unlicensed' | 'drop_cap' | 'drop_suspended' (Section 15.9.3). */
export type CapDecision = IngestDecision;
export interface CapGate {
/** Called once per event per resolved site, after filters, before enrichment. Wraps decideIngest(teamId, nowMs). */
check(teamId: string, nowMs: number): Promise<CapDecision>;
/** Called after a billable event was appended to the buffer: increments the in-memory usage accumulator of Section 15.9.3. */
recordAccepted(teamId: string): void;
}- Billable events are
pageviewand custom events;engagementevents are never counted. Engagement events are nevertheless dropped whencheckwould drop billable events for the team (drop_unlicensed,drop_capordrop_suspended), so a capped team produces no partial data. - The call must complete within a 50 ms budget; on timeout or error the decision is
accept(fail-open; the counter drift is corrected by the reconciliation of Section 15.9.4). - On
EDITION=selfhostedthe implementation is a constantacceptandrecordAcceptedis a no-op. - A non-
acceptdecision responds202and the drop is counted by Section 15 under the reasonsunlicensed,cap_exceededandsuspended; the metricquietstat_ingest_events_dropped_total{reason}receives the same three reason values. - Events dropped earlier (filters,
hostname_not_allowed) never reach this hook, so they never consume the allowance.
8.10 Enrichment #
Enrichment applies to pageview and custom events. Engagement events skip it (Section 8.13).
8.10.1 URL normalization #
Parse u with the WHATWG URL parser, then:
hostname= lowercased host without port (punycode form).hostnameis stored ashostname.pathname= the URL's percent-encoded pathname, unchanged otherwise (no decoding, no case folding, trailing slash preserved), truncated to 2,000 characters. An empty pathname becomes/.- If
his true (hash routing): take the fragment (without the leading#), remove everything from the first?; if the remainder is non-empty,pathname = pathname + "#" + remainder. Otherwise the fragment is discarded. - The stored page path is
pathname. The query string is never stored. Only the parameters in Sections 8.10.4 and 8.10.5 are read from it, into dedicated columns, before it is discarded. - For
app:URLs the same rules apply to the URL's path; the host (conventionallylocalhost) is stored inhostname.
8.10.2 User-Agent parsing and device classification #
ua-parser-js (Section 3) parses the User-Agent. Results are stored as follows; an unknown value is stored as an empty string.
| Column | Rule |
|---|---|
browser |
Parsed browser name, normalized by the table below. |
browser_version |
Parsed version reduced to major.minor (for example 124.0.6367.60 becomes 124.0; a version with only a major component is stored as 124.0). Never the full version. |
os |
Parsed OS name; Mac OS becomes macOS; Windows stays Windows; Chrome OS becomes ChromeOS; iOS, Android, Linux, Ubuntu, Fedora, Debian and others as parsed. |
os_version |
major.minor in the same way (a major-only version is stored as major.0; Windows 10.0, macOS 14.0). |
device |
See classification below. |
screen_size |
From w (Section 8.4): mobile if w < 576; tablet if w < 992; laptop if w < 1440; desktop otherwise; '' if w is absent. |
Browser name normalization (parsed name to stored name): Mobile Safari and Safari to Safari; Chrome, Chrome Headless (dropped earlier as bot), Chrome WebView, Chromium to Chrome (Chromium is kept as Chromium when the User-Agent contains Chromium/); Mobile Firefox and Firefox to Firefox; Edge, Edge WebView2 to Edge; Opera, Opera Mini, Opera Mobi, Opera Touch to Opera; Samsung Browser to Samsung Internet; UCBrowser to UC Browser; Android Browser to Android Browser; Facebook and Instagram in-app browsers to Facebook and Instagram; every other parsed name is kept as parsed (truncated to 100 characters).
Device classification (first match wins, based on the parser's device type, then the User-Agent):
- Parsed type
tablet(this includes iPad User-Agents; an iPad that reports a desktop Mac User-Agent cannot be distinguished server-side and is classifieddesktop) →tablet. - Parsed type
mobile→mobile. - No parsed device type, or parsed type
desktop, →desktop. - Any other parsed type (
smarttv,console,wearable,embedded,xr) →''(unknown). Smart TVs, consoles, wearables and embedded devices therefore have no device class and appear as "(none)" in the Devices report.
device therefore takes exactly desktop, mobile, tablet or ''; screen_size exactly mobile, tablet, laptop, desktop or '' (the Devices report "Screen size" tab groups by screen_size, Section 9). The parse is memoized in a per-process LRU (20,000 entries) keyed by the User-Agent string; the cache holds only parsed results and the key is held in memory only.
8.10.3 GeoIP lookup #
The client IP (Section 8.7) is looked up with the maxmind reader in the MMDB file chosen in this order (Section 16.2.2): GEOIP_DB_PATH if set; otherwise DATA_DIR/geoip/current if the geoip-update job has installed a file; otherwise the DB-IP City Lite database bundled at image build (Section 3). The reader is opened once per process; the file's modification time is checked every 60 seconds and the reader is swapped atomically when it changes (so a monthly database update needs no restart). The job geoip-update (both editions, schedule in Section 16.6; a no-op when GEOIP_AUTO_UPDATE=false or when the current month's file is already installed) downloads the new monthly database to a temporary file, validates it by opening it and looking up a fixed test address, and replaces the file atomically. An invalid download leaves the old file in place.
| Column | Rule |
|---|---|
country_code |
country.iso_code (two uppercase letters). The codes XX, ZZ, T1, A1, A2 and O1 are stored as empty. |
subdivision1_code |
ISO 3166-2 code <country>-<subdivision> built from the first subdivision's iso_code (for example US-CA). If the database lacks iso_code for subdivisions, the English subdivision name is mapped through the bundled table packages/shared/data/subdivisions.json (country plus name to ISO 3166-2 code); an unmapped name leaves the region empty. |
city_geoname_id |
city.geoname_id when present. If the database carries only city names, (country_code, subdivision1_code, English city name) is mapped through the bundled table packages/shared/data/cities.json (derived from GeoNames cities with population above 15,000); no match stores 0. |
Private, loopback, link-local and unlisted addresses return no record: all three columns are empty/0. Nothing finer than city is read: latitude, longitude, postal code, accuracy radius, ASN and time zone from the database are discarded. The dashboard (Section 9) resolves city_geoname_id and subdivision1_code to display names using the same bundled tables (packages/shared/data/cities.json, subdivisions.json).
8.10.4 Referrer parsing #
Inputs: r (Section 8.4), the page's query parameters, and the site domain. Output columns: referrer (host plus path of the referrer URL, www. stripped, query and fragment removed, at most 500 characters), referrer_source (normalized source name), plus utm_* columns (Section 8.10.5).
- Parse
r.http/httpsURLs: host lowercased, one leadingwww.removed.android-app://<package>URLs: mapped by the tablecom.google.android.gm→ Gmail,com.google.android.googlequicksearchbox→ Google,com.facebook.katanaandcom.facebook.lite→ Facebook,com.instagram.android→ Instagram,com.twitter.android→ Twitter,com.linkedin.android→ LinkedIn,com.reddit.frontpage→ Reddit,org.telegram.messenger→ Telegram,com.whatsapp→ WhatsApp,com.slack→ Slack,com.pinterest→ Pinterest,com.google.android.youtube→ YouTube; other packages are ignored. Any other scheme is ignored. - Self-referral suppression. If the referrer host equals the site's normalized domain or is a subdomain of it (host ends with
.+ domain), or equals any host in the same request'sdlist, the referrer is discarded (referrerandreferrer_sourcebecome empty) and parsing continues with query-parameter overrides only. - Source override by query parameters. The first present, non-empty value among
utm_source,source,ref(in this order; trimmed, truncated to 300 characters) replaces the source derived from the referrer host. Values that match a host in the source table (Section 8.10.4 table) or a source name are normalized to the canonical name; others are kept as typed (original case). Thereferrercolumn still holds the referrer host and path if one exists. - Click-ID inference. If no source was set by step 3 and the page has
gclid,gbraidorwbraid, the source isGoogleand, ifutm_mediumis absent, the medium used for channel classification iscpc. If it hasmsclkidthe source isBingwith mediumcpc. These parameters are read and discarded; they are never stored. - Source from host. With no override, the source is looked up from the referrer host in the table below (exact host or suffix match on a registered domain; longest match wins). A host not in the table becomes the source named by its host with
www.removed (for examplenews.example.org), categoryother. - If there is no referrer, no override and no click ID,
referrer_sourceis empty (the event is direct).
Source normalization and category table. Hosts match exactly or as a suffix with a preceding dot (l.facebook.com matches facebook.com). Category drives channel classification (Section 8.10.6).
| Source name | Host patterns | Category |
|---|---|---|
google.* (any country TLD), google.com, googleusercontent.com, com.google.android.googlequicksearchbox |
search | |
| Bing | bing.com, cn.bing.com |
search |
| DuckDuckGo | duckduckgo.com |
search |
| Yahoo | search.yahoo.com, *.search.yahoo.com, yahoo.com, yahoo.co.jp |
search |
| Baidu | baidu.com |
search |
| Yandex | yandex.*, ya.ru |
search |
| Ecosia | ecosia.org |
search |
| Brave Search | search.brave.com |
search |
| Startpage | startpage.com |
search |
| Qwant | qwant.com |
search |
| Kagi | kagi.com |
search |
| Naver | naver.com, search.naver.com |
search |
| Seznam | seznam.cz |
search |
| Sogou | sogou.com |
search |
facebook.com, fb.com, fb.me, l.facebook.com, m.facebook.com, lm.facebook.com, web.facebook.com |
social | |
instagram.com, l.instagram.com |
social | |
twitter.com, x.com, t.co, mobile.twitter.com |
social | |
linkedin.com, lnkd.in, www.linkedin.com |
social | |
reddit.com, old.reddit.com, out.reddit.com, redd.it |
social | |
pinterest.*, pin.it |
social | |
| TikTok | tiktok.com, vm.tiktok.com |
social |
| Threads | threads.net, threads.com |
social |
| Bluesky | bsky.app |
social |
| Mastodon | mastodon.social, mastodon.online, fosstodon.org, hachyderm.io |
social |
| Telegram | t.me, web.telegram.org |
social |
wa.me, web.whatsapp.com, api.whatsapp.com |
social | |
| Snapchat | snapchat.com |
social |
| Tumblr | tumblr.com |
social |
| VK | vk.com, away.vk.com |
social |
| Quora | quora.com |
social |
| Hacker News | news.ycombinator.com |
social |
| Product Hunt | producthunt.com |
social |
| Discord | discord.com, discordapp.com |
social |
| Slack | slack.com, app.slack.com |
social |
| YouTube | youtube.com, m.youtube.com, youtu.be |
video |
| Vimeo | vimeo.com |
video |
| Twitch | twitch.tv |
video |
| Dailymotion | dailymotion.com |
video |
| Amazon | amazon.* |
shopping |
| eBay | ebay.* |
shopping |
| Etsy | etsy.com |
shopping |
| Shopify | shopify.com, shop.app |
shopping |
| Walmart | walmart.com |
shopping |
| AliExpress | aliexpress.com |
shopping |
| Gmail | mail.google.com, com.google.android.gm |
|
| Outlook | outlook.live.com, outlook.office.com, outlook.office365.com |
|
| Yahoo Mail | mail.yahoo.com |
|
| Proton Mail | mail.proton.me, proton.me |
|
| ChatGPT | chatgpt.com, chat.openai.com, openai.com |
ai |
| Claude | claude.ai |
ai |
| Perplexity | perplexity.ai, www.perplexity.ai |
ai |
| Gemini | gemini.google.com, bard.google.com |
ai |
| Microsoft Copilot | copilot.microsoft.com |
ai |
| DeepSeek | chat.deepseek.com, deepseek.com |
ai |
| Grok | grok.com, x.ai |
ai |
| Meta AI | meta.ai |
ai |
| Mistral Le Chat | chat.mistral.ai |
ai |
| Phind | phind.com |
ai |
| You.com | you.com |
ai |
Precedence inside the table: a specific host pattern beats a generic one (gemini.google.com is Gemini, not Google; mail.google.com is Gmail; chat.deepseek.com before deepseek.com). Tags in utm_source or source such as chatgpt.com, chatgpt, claude.ai, perplexity, gemini are matched case-insensitively against the source names and host patterns, so utm_source=chatgpt.com normalizes to ChatGPT with category ai. A utm_source that matches no table entry has category other and may still be classified by its medium (Section 8.10.6). Hosts of the form *.googleusercontent.com map to Google only when the referrer path starts with /search; otherwise they use the generic host rule.
8.10.5 UTM handling #
Read from the page URL's query string, then discarded:
| Parameter | Column | Normalization |
|---|---|---|
utm_medium |
utm_medium |
Trimmed, lowercased, truncated to 300 characters. |
utm_source |
utm_source |
Stored as the normalized source name when it matches the table in Section 8.10.4 (for example chatgpt.com becomes ChatGPT), otherwise trimmed as typed, truncated to 300 characters. |
utm_campaign |
utm_campaign |
Trimmed, truncated to 300 characters, case preserved. |
utm_content |
utm_content |
Same as campaign. |
utm_term |
utm_term |
Same as campaign. |
Parameter names are matched case-insensitively; the first occurrence wins; values are percent-decoded once, with + read as a space. Empty values are stored as empty. Only these five utm_* parameters, plus source, ref, gclid, gbraid, wbraid, msclkid (read for Section 8.10.4 and discarded) are interpreted; no other query parameter is read or stored.
8.10.6 Channel classification #
The channel is computed for every pageview and custom event and stored as channel; a session takes the channel of its entry event (Section 8.12). Inputs: src (normalized source name, possibly empty), cat (category of src from the table: search, social, video, shopping, email, ai, other, or none when there is no source), medium (utm_medium, or the click-ID-inferred medium, lowercased, with the placeholder values (none), (not set), none and not set treated as empty), campaign (utm_campaign lowercased).
Definitions: PAID = medium matches the regular expression ^(.*cp.*|ppc|retargeting|paid.*)$. SHOP_CAMPAIGN = campaign matches ^(.*(([^a-df-z]|^)shop|shopping).*)$.
Rules are evaluated in this order; the first match decides (18 channels, one rule each, plus the fallback):
| # | Channel | Rule |
|---|---|---|
| 1 | Cross-network | campaign contains cross-network, or medium equals cross-network. |
| 2 | Paid Shopping | (cat = shopping or SHOP_CAMPAIGN) and PAID. |
| 3 | Paid Search | (cat = search and PAID), or (src is Google or Bing and the medium was inferred from a click ID). |
| 4 | Paid Social | cat = social and PAID. |
| 5 | Paid Video | cat = video and PAID. |
| 6 | AI Assistants | cat = ai. (Applies for any medium that did not match rules 1 to 5; AI sources never match rules 2 to 5 because their category is ai.) |
| 7 | Display | medium in {display, banner, expandable, interstitial, cpm}, or (PAID and cat is other or none, that is, a paid medium from an unclassified source). |
| 8 | Audio | medium equals audio. |
| 9 | SMS | src or medium equals sms (case-insensitive). |
| 10 | Mobile Push Notifications | medium ends with push, or contains mobile or notification, or src equals firebase. |
| 11 | cat = email, or src lowercased in {email, e-mail, e_mail, e mail, newsletter}, or medium in {email, e-mail, e_mail, e mail}. |
|
| 12 | Affiliates | medium in {affiliate, affiliates}. |
| 13 | Organic Shopping | cat = shopping or SHOP_CAMPAIGN. |
| 14 | Organic Social | cat = social, or medium in {social, social-network, social-media, sm, social network, social media}. |
| 15 | Organic Video | cat = video, or medium matches ^(.*video.*)$. |
| 16 | Organic Search | cat = search, or medium equals organic. |
| 17 | Referral | medium in {referral, app, link}, or src is non-empty (any other source, including every unrecognized referrer host). |
| 18 | Direct | Otherwise (no source and no recognized medium). |
Worked examples: no referrer, no UTM → Direct. Referrer www.google.com → Organic Search. utm_source=google&utm_medium=cpc → Paid Search. gclid with referrer empty → Paid Search. Referrer l.facebook.com → Organic Social; utm_source=facebook&utm_medium=paid-social → Paid Social (paid-social matches PAID through the paid.* alternative). Referrer chatgpt.com or utm_source=chatgpt.com → AI Assistants. Referrer mail.google.com → Email. utm_medium=email → Email. Referrer news.ycombinator.com → Organic Social. Referrer example.org → Referral. utm_source=partner&utm_medium=cpc (unclassified source, paid medium) → Display. Self-referral → treated as no referrer → Direct for a new session.
8.11 Visitor Hash and Salt Lifecycle #
8.11.1 Algorithm #
user_id = SipHash-2-4( key = daily_salt[16 bytes],
message = site_id_bytes[16] || ip_family_byte || ip_bytes || ua_bytes )site_id_bytes: the 16 raw bytes of the site UUID (the ClickHousesite_id), not the domain, so the hash differs per site and per rotation even for a visitor on several of the customer's sites.ip_family_byte:0x04for IPv4,0x06for IPv6.ip_bytes: the 4 address bytes for IPv4; for IPv6 the first 8 bytes (the /64 prefix), because IPv6 privacy extensions rotate the lower 64 bits of a device's address within a day and would otherwise inflate unique visitors.ua_bytes: the User-Agent truncated to 1,024 bytes, UTF-8, exactly as received (Section 8.7.3).- SipHash-2-4 returns a 64-bit value, stored as
UInt64user_id. SipHash is implemented inpackages/sharedwith a test against the reference vectors of the algorithm's authors. - The salt is
bytea(16), generated with a cryptographically secure RNG.
8.11.2 Salts #
Salts live in the Postgres salts table (id, salt bytea(16), valid_from date in UTC; Section 5). The table is UNLOGGED: it is excluded from the write-ahead log, from point-in-time recovery and from streaming replicas, and it is excluded from logical dumps (pg_dump --exclude-table-data=salts, Section 16). No backup of any kind therefore contains a salt. After a database crash or a restore the table is empty and rule 6 generates a fresh salt; visitors may be counted twice on that one day (a deliberate trade-off, documented in Section 17). Rules, exactly:
- The current salt is the row with the greatest
valid_fromnot later than today's UTC date. - At 00:00 UTC the job
salt-rotate(schedule in Section 16.6) inserts the row for the new date (INSERT ... ON CONFLICT (valid_from) DO NOTHING, so a retry is harmless). - From 00:00:00 to 00:29:59 UTC the previous salt (the row before the current one) is also valid. Ingestion computes the hash with both salts during this window. If a live session exists under the previous-salt hash (Section 8.12), the event continues that session and keeps the previous-salt
user_id. Otherwise the current-salt hash is used. - At 00:30 UTC the job
salt-expire(Section 16.6) deletes the previous salt row. At no time do more than two salt rows exist after the purge job has run, and no salt older than the previous one is ever retained. A salt is never recoverable once deleted, which makes the previous day's hashes irreversible in practice. - Each web process caches both salts in memory and refreshes them every 60 seconds. The previous salt is used only when
now_utc < current.valid_from + 30 minutes, regardless of whether its row still exists in the cache, so the up-to-60-second lag of cache refresh cannot extend the window. - If no row exists for today (the rotation job is late or failed, or the table was emptied by a crash or restore), the first ingesting process creates it with the idempotent insert of rule 2, so ingestion never stalls or reuses yesterday's salt.
- Salts are never logged, never exported (the
saltstable is excluded from every export in Section 14) and never backed up (above). A test in Section 18 asserts that a database backup archive contains nosaltsrows.
Consequences, stated for implementers: the same visitor has a different user_id every UTC day; there is no cross-day, cross-site or cross-device identification; "returning visitor" metrics do not exist.
8.12 Sessions #
8.12.1 Definition #
A session is the sequence of non-engagement events with the same user_id on the same site where consecutive events are less than 30 minutes apart (strictly: a new session starts when event_time - last_event_time >= 1800 seconds). Engagement events keep a session alive (Section 8.13). session_id is a random UInt64 (cryptographically secure RNG, never 0), generated when the session starts.
8.12.2 Valkey state #
| Key | Value | TTL |
|---|---|---|
qs:s:<site_id>:<user_id as 16 lowercase hex characters> |
JSON object with three fields: state, the complete current sessions row (every column defined in Section 6, including interactive_events); flushed, the last state row of this session that a flush has confirmed as inserted into ClickHouse (null until the first confirmed flush); and pending_by, the instance id of the process whose buffer holds rows of this session that are not yet confirmed (null when none) |
1,800 s, reset on every write |
qs:ing:<instance_id> |
Liveness marker of an ingesting process, refreshed every 3 seconds | 10 s |
qs:rt:<site_id> (sorted set) |
member = user_id hex, score = last event epoch seconds |
600 s on the key; members older than 300 s removed on each write with ZREMRANGEBYSCORE. Read by the current-visitors count (Section 9.14) |
Session state contains no IP, User-Agent or raw URL beyond the already-normalized entry_page/exit_page paths.
8.12.3 Session columns maintained #
Every field below corresponds to the same-named column of the sessions table (Section 6).
| Field | On session start | On each later pageview or custom event | On engagement event |
|---|---|---|---|
session_id, site_id, user_id |
set | unchanged | unchanged |
started_at |
event time (seconds) | unchanged | unchanged |
ended_at (last activity) |
event time (seconds) | event time | event time |
duration (seconds) |
0 | ended_at - started_at |
ended_at - started_at |
entry_page |
event pathname |
unchanged | unchanged |
exit_page |
event pathname |
event pathname (pageviews only; custom events do not change it) |
unchanged |
entry_hostname, exit_hostname |
hostname of the starting event for both |
exit_hostname updated with exit_page; entry_hostname unchanged |
unchanged |
pageviews |
1 for a pageview, 0 otherwise | +1 per pageview | unchanged |
events |
1 | +1 per non-engagement event (pageviews included) | unchanged |
interactive_events |
1 if the starting event is a custom event with i true, else 0 |
+1 per custom event with i true |
unchanged |
is_bounce |
1 if pageviews = 1 and interactive_events = 0 |
recomputed with the same rule after the update | unchanged |
referrer, referrer_source, channel, utm_* |
from the starting event | unchanged (first-touch within the session) | unchanged |
country_code, subdivision1_code, city_geoname_id, browser, browser_version, os, os_version, device, screen_size |
from the starting event | unchanged | unchanged |
sign / version |
1 / 1 |
see Section 8.12.4 | see Section 8.12.4 |
A session whose starting event is a custom event without a preceding pageview has pageviews = 0 and is not a bounce (is_bounce = 0). is_bounce can only go from 1 to 0 within a session. The channel and referrer are fixed at session start, so mid-session landings with different UTM tags do not change the session's source; the individual event rows still carry their own parsed values.
8.12.4 Table updates: sign and version #
sessions is a VersionedCollapsingMergeTree (Section 6.4). The cancel row of an update must equal, in every column except sign, a state row that really exists in ClickHouse (or will be inserted by a live process); a cancel row for a state that never arrived leaves sign-weighted sums permanently wrong. The Valkey object of Section 8.12.2 therefore carries flushed and pending_by. Per processed event:
- First event of a session: emit one row,
sign = 1,version = 1. Setstateto that row,flushed = null,pending_by = <this instance>. - Subsequent event: the new state row has
sign = 1andversion = state.version + 1. The cancel row is chosen as follows, then emitted immediately before the new state row (both go into the same insert block, Section 6.4):- If no rows of this session are pending (
pending_by = null), the cancel row is a byte-for-byte copy offlushedwithsign = -1(sameversion). - If rows are pending in another process (
pending_bynames an instance whoseqs:ing:<instance_id>liveness marker exists), the cancel row is a copy ofstate(the pending row will be inserted by that process, so the pair chains onto it). - If rows are pending but the named instance is no longer alive (it crashed with unflushed rows), those rows are lost: the cancel row is a copy of
flushedif it is notnull; ifflushedisnull, nothing of this session reached ClickHouse and no cancel row is emitted (the new state row, which carries the cumulative counts, stands alone). The version still continues fromstate.version + 1. - If the Valkey object is gone (flush, eviction, expiry), the event starts a new session (rule 1 and Section 6.4 write protocol step 3).
After computing the pair, set
stateto the new state row andpending_by = <this instance>.
- If no rows of this session are pending (
- Coalescing: the in-process buffer (Section 8.15) keeps, per
session_id, at most one pending pair: the first cancel row and only the latest state row. A later event of the same session handled by the same process replaces the buffered state row and leaves the buffered cancel row unchanged (intermediate states are never emitted). Coalescing applies only while the Valkeystate.versionequals the buffer's latest state version for that session; if another process advanced the session in between, the buffer emits its pending rows as they are and starts a new pending pair (rule 2). - Confirmation: after a batch insert is acknowledged, the process confirms every state row of the batch in Valkey with one pipelined script per session: if the key still exists and
flushed.versionis lower than the confirmed row's version, setflushedto the confirmed row; ifstate.versionequals the confirmed version, also setpending_by = null. A confirmation never recreates a missing key. A batch that is dropped asinvalid_row(Section 8.15.2) triggersDELof the affected session keys instead, so the next event starts a new session. - Dashboard queries aggregate with sign-aware sums or collapse by version as defined in Section 6; nothing in this section deletes rows.
Residual risk, stated plainly: a process that crashes in the milliseconds between a ClickHouse acknowledgement and the confirmation of rule 4 leaves flushed one flush behind for the sessions of that single batch, so one later cancel row per such session cancels a state that was already cancelled. The same applies if the process holding rows that another process chained onto crashes before flushing. Both windows are bounded by the number of crashes and by one batch of sessions, and are an accepted trade-off.
8.12.5 Lookup and concurrency algorithm #
For an event with candidate hashes (current, and previous during the 00:00 to 00:30 window, Section 8.11.2):
- Read the Valkey key for
current; if absent and aprevioushash exists, read the key forprevious. If a state is found andevent_time - state.ended_at < 1800, it is the live session; otherwise a new session starts under the current hash (an expired but still-present state, which can occur within the TTL jitter, is overwritten). - For a live session, the update is applied with optimistic locking:
WATCH <key>, re-read, compute, thenMULTI/SET key <json> EX 1800/ZADDreal-time set (non-engagement events only, Section 8.13 step 4) /EXEC. IfEXECfails because the key changed, re-read and retry, up to 5 attempts. - After 5 failed attempts the event is still inserted into
eventswith the last-readsession_id, the session update is skipped, andquietstat_ingest_session_cas_conflicts_totalis incremented. - For a new session:
SET key <json> EX 1800 NX. IfNXfails (another process created the session concurrently), re-read and treat it as a live session (step 2). - A visitor who has the
previoushash session keeps it (key under the previous hash is rewritten with TTL reset); the new events'user_idis the previous-salt hash for the remainder of that session, as required by Section 8.11.2. - Event ordering: the server's receive time is the event time. When concurrent events from one visitor reach different processes, a late-arriving earlier event is applied with its own time, but
ended_atnever moves backwards (ended_at = max(old, event_time)), andentry_page/started_atare never changed after creation.
8.12.6 Edge cases #
| Case | Behavior |
|---|---|
| Valkey flushed or evicted mid-session | The next event starts a new session. The previous session's last confirmed row remains valid (its sign-1 row has no cancel row), so it is counted as a complete session. |
| Clock skew between processes | Event time is taken from the processing process; NTP-synchronized hosts are required (Section 16). Negative gaps are treated as 0. |
| Session spanning midnight UTC | Continues across the salt change under the previous-salt hash for as long as events keep arriving less than 30 minutes apart and the 00:30 purge has not occurred; from 00:30 UTC the next event for a visitor whose session is keyed by the previous hash no longer finds the previous salt and starts a new session. This is accepted behavior: the session is split once at that boundary. |
| Events for a deleted site, or a site of a team pending deletion | Dropped at Section 8.6 before sessions. |
8.13 Engagement Events #
An engagement event (n = "engagement") never creates a session and is not enriched. Processing:
- Validation (Section 8.4); site resolution; bot, IP-exclusion and datacenter filters; page exclusion (against the event's
upath); cap hook withbillable = false(Section 8.9). - Compute the
user_idhash(es) (Section 8.11) and look up the live session (Section 8.12.5 step 1). If there is no live session, the event is dropped (202, counted under reasonengagement_no_sessionin the metrics only, not in Section 8.8.6). - Insert one row into
events:name = "engagement",pathnamefromu(Section 8.10.1), the session'ssession_idanduser_id,hostname,scroll_depth = sd(when present),engagement_time = e(when present, clamped to 3,600,000),interactive = 0; all enrichment columns are copied from the session state so the row can be filtered like any other event. - Update the session as in Section 8.12.3 (engagement column): only
ended_atanddurationchange, via the sign/version mechanism of Section 8.12.4, and the Valkey TTL is reset. No other session field changes, sopageviews,events,is_bounceandexit_pageare unaffected; the realtime setqs:rt:{site_id}is not updated by engagement events. - Metric definitions that consume these rows (scroll depth, time on page) are owned by Section 9: the server stores
sdper ping as the maximum depth reached on that page up to the ping, andeas the delta since the previous ping, which Section 9 sums per pageview.
8.14 Revenue Normalization and Currency Conversion #
Applies to custom events with a $ object (ignored for pageview and engagement). This section owns the validation; the columns it writes are revenue_amount, revenue_currency, revenue_reporting_amount and revenue_reporting_currency (Section 6.3).
Validation. $ must be an object with amount and currency. currency is trimmed, uppercased, and must be an ISO 4217 code present in the bundled table packages/shared/data/currencies.json (code and number of minor-unit digits, for example JPY 0, USD 2, KWD 3); otherwise 400 $ is invalid. amount is a JSON number or a decimal string matching ^[0-9]+(\.[0-9]+)?$ of at most 32 characters; negative, scientific-notation, non-finite and empty values are rejected with 400 $ is invalid. After rounding to 3 fractional digits the amount must satisfy 0 <= amount <= 999,999,999,999.999 (12 integer digits); larger values are rejected with 400 $ is invalid. The whole event is rejected in every one of these cases (this is the single rule for the platform; Section 10.4.1 and the Events API of Section 13 restate it). Arithmetic never uses binary floating point: strings are parsed to scaled integers (BigInt) and JSON numbers are first converted to their shortest decimal string.
Source amount. revenue_amount = the exact decimal amount rounded half up to 3 fractional digits (Decimal(18,3), major units, not minor units); revenue_currency = the validated currency code. Both are NULL / empty when the event carries no revenue.
Reporting amount. revenue_reporting_currency = the site's site_settings.reporting_currency (Section 5.5.2) for every revenue event, whether or not a revenue goal exists; revenue_reporting_amount = revenue_amount converted into that currency with rules 1 to 5 below and rounded half up to 3 decimals. Both are NULL / empty when no rate exists. Revenue for a goal that is reported in a different currency than the site's reporting currency is converted at query time by Section 10.4.3 using the same rate table and the same rate rules (rule 2 below); ingestion does not depend on any goal.
- Rate source: the Postgres table
exchange_rates(rate_date date,currency char(3),rate_per_eur numeric(20,10), primary key(rate_date, currency); owned by Section 5), filled by the jobfx-rates-fetch(schedule in Section 16.6) from the European Central Bank daily euro foreign-exchange reference rates (theeurofxref-dailyfeed, with the 90-day history feed used for the first run and for gap filling).EURhas an implicit rate of 1. The set of convertible currencies is whatever the feed currently publishes. - Date used: the event's UTC date. If no rate exists for that date for a currency (weekends, TARGET holidays, publication lag), use the most recent earlier date with a rate, searching back at most 10 days; if none is found, no rate exists and rule 4 applies. This is the only lookback rule used anywhere (ingestion and Section 10.4.3).
- Conversion:
reporting = round_half_up_3( revenue_amount × rate_per_eur[R] ÷ rate_per_eur[source] ), whereRis the reporting currency, with the exact decimal source amount and rate arithmetic in decimal (scale 8), the division performed once at the end. Whensource = R, no rate is used and the amounts are equal. - If either currency has no rate at all, the reporting columns stay NULL / empty, the source columns are still stored, and the metric
quietstat_ingest_revenue_conversion_failed_totalis incremented. The event is accepted. - In-process rate cache: today's and the previous 10 days' rates, reloaded every 15 minutes.
Changing a site's reporting currency affects only events ingested afterwards (Section 5.5.2).
8.15 Buffering, Batch Insert, Backpressure and Failure Modes #
8.15.1 Buffers #
Each web process holds two in-memory append buffers: eventsBuffer and sessionsBuffer. The event handler appends and returns. A flush is triggered when either buffer reaches INGEST_BATCH_ROWS rows (default 5,000) or INGEST_FLUSH_INTERVAL_MS (default 2,000 ms) has passed since the last flush started (the timer is always armed). Both variables, and INGEST_BUFFER_MAX_ROWS below, are defined in Section 16.4. Flushes insert with the ClickHouse client's JSONEachRow format into events and sessions, with async_insert disabled, date_time_input_format = 'best_effort', a 30-second request timeout, and compression enabled. Only one flush runs at a time per process; rows arriving during a flush wait in the buffer for the next one. The events batch is inserted first, then sessions; if the first fails the second is not attempted, and both retry together.
8.15.2 Failure and retry #
- A failed insert (connection error, timeout, ClickHouse
5xx) keeps the rows in the buffer and retries with exponential backoff: 1 s, 2 s, 4 s, 8 s, 16 s, then every 30 s, with ±20% jitter, until success. A batch that failed with a connection error, timeout or5xxis always retried unchanged, so that the insert deduplication window of Section 6.3 makes a retry of an already-stored batch a no-op. Only a ClickHouse4xxrejection (nothing was stored) caused by a malformed single row is handled by splitting the batch in halves recursively to isolate the row, which is dropped with the metricquietstat_ingest_events_dropped_total{reason="invalid_row"}and oneerrorlog line without row content; the Valkey session keys of the dropped row's session are deleted (Section 8.12.4 rule 4). - Delivery is at least once: an insert whose acknowledgement is lost is retried, so a very small number of duplicated rows is possible after network timeouts. This is an accepted trade-off.
8.15.3 Backpressure and bounded memory #
The buffers are bounded per process at INGEST_BUFFER_MAX_ROWS event rows (default 100,000) and twice that many session rows (each update can add a cancel row and a state row). Before a handler touches session state (Section 8.12), it checks both buffers: if either is at its bound, the event is dropped with 202 and quietstat_ingest_events_dropped_total{reason="buffer_full"} is incremented; no Valkey session state is advanced for a dropped event, so sessions stay consistent. There is no disk spill. At 2,000 events per second per process the bound is reached after about 50 seconds of ClickHouse outage. Beyond the bound new events are dropped (never the oldest buffered rows) and a warning is logged; a dropped event is acknowledged with 202 so that trackers do not retry. A warning log line (rate-limited to one per 30 seconds) is emitted while the buffer is full.
8.15.4 Dependency failures #
| Dependency down | Behavior |
|---|---|
| ClickHouse | Ingestion keeps accepting and buffering up to the bounds in Section 8.15.3 (an outage of up to about 45 seconds at 2,000 events per second per process loses nothing), then drops new events with 202 and the buffer_full metric. /readyz reports ClickHouse unreachable (Section 3) but ingestion does not stop on its own. |
| Valkey | The rate limiter follows Section 17.6 (fail open for 60 seconds, then an in-process limit of 10 requests per second per IP-HMAC). Session state cannot be read, so the event is not processed: the response is 503 unavailable with Retry-After: 5 and quietstat_ingest_events_dropped_total{reason="valkey_unavailable"}. Valkey commands use a 200 ms timeout. |
| Postgres | Cached site configuration is served stale for up to 10 minutes (Section 8.6); salts come from the in-memory cache and keep working; beyond those bounds the response is 503. |
| GeoIP file missing or unreadable | Events are accepted with empty location columns; a warning is logged once per minute; the metric quietstat_ingest_geoip_lookups_total{result="error"} increments. |
| Cap hook slow | Section 8.9 (fail open at 50 ms). |
8.15.5 Shutdown #
On SIGTERM the process stops accepting new connections, reports not-ready on /readyz, and flushes both buffers (retrying as above) for up to 10 seconds before exiting. Rows still unflushed after the deadline are lost (counted by quietstat_ingest_events_dropped_total{reason="shutdown"}). An abrupt crash (kill signal, out-of-memory) loses every row still in the process buffer: normally at most one flush interval (up to INGEST_BATCH_ROWS events and their session rows), and up to INGEST_BUFFER_MAX_ROWS events if ClickHouse was unreachable at the time. The flushed and pending_by fields of Section 8.12.2 keep the session chains of the affected sessions consistent: the next event of such a session cancels the last confirmed row rather than a row that never arrived (Section 8.12.4 rule 2), with the residual window stated there.
8.16 Event Row Assembly #
For an accepted pageview or custom event the following events columns are populated (types per Section 6): site_id (site UUID), timestamp, name (pageview or the custom name), user_id, session_id, hostname, pathname, referrer, referrer_source, channel, utm_medium, utm_source, utm_campaign, utm_content, utm_term, country_code, subdivision1_code, city_geoname_id, browser, browser_version, os, os_version, device, screen_size, interactive (1 or 0), meta.key / meta.value (parallel arrays built from p, in the order sent), revenue_amount, revenue_currency, revenue_reporting_amount, revenue_reporting_currency (NULL / empty when the event carries no revenue, Section 8.14); scroll_depth and engagement_time are 0 for these events. Properties are stored for every key sent; ingestion does not filter by the per-site property allowlist (Section 10 uses the allowlist for what the dashboard lists and offers in filters).
8.17 Metrics Emitted #
Exposed on the Prometheus endpoint defined in Section 16. Labels never include site ids, domains, IPs or user agents.
| Metric | Type | Labels |
|---|---|---|
quietstat_ingest_requests_total |
counter | status (202, 204, 400, 405, 408, 413, 429, 503) |
quietstat_ingest_request_duration_seconds |
histogram | none |
quietstat_ingest_events_accepted_total |
counter | type (pageview, custom, engagement) |
quietstat_ingest_events_dropped_total |
counter | reason (bot_user_agent, bot_missing_ua, bot_headless, datacenter_ip, spam_referrer, excluded_ip, excluded_page, hostname_not_allowed, unknown_domain, unlicensed, cap_exceeded, suspended, engagement_no_session, buffer_full, valkey_unavailable, invalid_row, shutdown) |
quietstat_ingest_buffer_rows |
gauge | table (events, sessions) |
quietstat_ingest_flush_duration_seconds |
histogram | none |
quietstat_ingest_flush_rows_total |
counter | table |
quietstat_ingest_flush_failures_total |
counter | none |
quietstat_ingest_session_cas_conflicts_total |
counter | none |
quietstat_ingest_site_cache_total |
counter | result (hit, miss, negative_hit, stale) |
quietstat_ingest_geoip_lookups_total |
counter | result (hit, miss, error) |
quietstat_ingest_revenue_conversion_failed_total |
counter | none |
quietstat_ingest_props_truncated_total |
counter | none |
quietstat_ingest_datacenter_refresh_failures_total |
counter | none |
quietstat_ingest_datacenter_ranges |
gauge | family (ipv4, ipv6) |
quietstat_ingest_salt_rows |
gauge | none (alert if greater than 2 after 00:35 UTC) |
8.18 Privacy Guarantees and Never-Stored Data #
8.18.1 Guarantees #
- No cookies are set, read or required by ingestion; the endpoint ignores the
Cookieheader and sends noSet-Cookie. - A visitor identifier exists only as
user_id, a one-way keyed hash that changes every UTC day; the key (salt) is deleted 30 minutes after the next rotation, so it exists for at most 24.5 hours (Section 8.11.2); it lives in anUNLOGGEDtable that is excluded from write-ahead log, point-in-time recovery, replicas and logical dumps, so no backup ever contains a salt. - There is no cross-site, cross-day or cross-device linkage; the site id is inside the hash input.
- The raw IP address and raw User-Agent are used in memory for the duration of the request only.
- Location is limited to country, region (ISO 3166-2) and city (GeoNames id); no coordinates.
- Browser and OS versions are stored at
major.minorprecision at most. - The URL query string and the hash fragment (unless hash routing is on) are discarded; only the whitelisted parameters of Sections 8.10.4 and 8.10.5 are interpreted.
- The server never fingerprints: no canvas, font, language-header, timezone or screen-height signals are read; the only viewport-derived value is the coarse
screen_sizebucket. - The ingestion path writes no per-request log containing network or content data (Section 8.2).
8.18.2 Never stored (anywhere: ClickHouse, Postgres, Valkey beyond the stated keys, logs, metrics, backups, exports) #
The raw client IP address; the raw User-Agent string; any hash that persists beyond the salt retention window in a recoverable form (the stored user_id is not reversible without the deleted salt and the deleted inputs); the Accept-Language, Cookie, Authorization, Referer (header) and any other request headers; URL query strings other than the parsed UTM/ref values; hash fragments (unless hash routing); full outbound-link or download URLs beyond origin and path (Section 7.5.5); latitude, longitude, postal code or ASN; form field names or values; client-supplied timestamps; screen heights; any identifier set by the tracker; any salt, in any backup, dump or replica; any email address, name or other personal data (unless the integrator sends it in a custom property or URL, which the product neither inspects nor prevents, Section 8.4). Page paths and referrer paths are stored as sent (query strings and, without hash routing, fragments removed); integrators must keep personal data out of URL paths or scrub it with transformRequest (Section 7.5.14, Section 17.9.2).
9. Stats Dashboard & Reports #
This section defines the single-page stats dashboard: the canonical metric definitions, every widget and report panel, the time range and comparison model, realtime, annotations, filters, segments, the sites list, the consolidated all-sites view, imported-data behaviour, empty states and performance rules. Conversion-specific panels (Goals, Properties, Funnels, User Journeys) are placed here and fully specified in Section 10. The query engine that computes every number is owned by Section 13; the dashboard calls its internal equivalent described in 9.1.
9.1 Scope, Data Flow and Routes #
9.1.1 Principles #
- One page, many independent panels. The dashboard is one SPA route (
/sites/:domain). Each widget (top stats, main chart, every report panel) issues its own query so a slow panel never blocks the others. - Every number comes from the query engine of Section 13. The dashboard never writes its own SQL and never reads ClickHouse directly. The browser calls the dashboard API; the dashboard API calls the engine in-process (
packages/query, Section 3) with the authenticated site context. - All date logic (today, week boundaries, month boundaries, comparison ranges, chart buckets) uses the site's IANA
timezone(Section 4). The browser's timezone is never used for bucketing. - The dashboard resolves every period preset into an absolute range before querying: calendar-based presets send two ISO dates (
["2026-11-01","2026-11-03"]), rolling presets (Last 24 hours, Realtime) send two ISO 8601 timestamps with the site timezone offset, rounded down to the minute. The engine therefore never interprets relative tokens for dashboard calls, and cache keys (9.23) are deterministic. - Browser storage. The authenticated dashboard keeps UI preferences only in the browser (
localStoragekeys prefixedqs:: panel tabs, remembered period and comparison, dismissed banners, sort choices, the full-URL toggle). Shared-link, public-dashboard and embed views read and write no browser storage and set no cookies: they always start from the defaults in this section, and their theme comes from thethemequery parameter (Section 12).
9.1.2 Dashboard API routes (cookie session + CSRF per Section 4) #
All routes are under /api/app/v1. {domain} is the site's normalized domain (Section 5.5.1), URL-encoded; the dashboard never addresses a site by its internal id. Reading routes require the site.view action (the viewer role, or guest_viewer, and above), per the matrix in Section 11. Mutating routes list their minimum role. Shared-link and embed viewers use the read-only mirror under /api/share/v1 defined in Section 12.2.3: the mirrored set is listed there, accepts the same request bodies as the routes below, and every route outside it returns 404 not_found.
| Method and path | Purpose | Min role |
|---|---|---|
POST /sites/{domain}/query |
Run one stats query (body is a Section 13 query object without site_id). Response has the Section 13 shape {results, meta, query} (no data envelope). |
viewer |
POST /sites/{domain}/query/batch |
Run 1 to 20 stats queries in one request. Body {"queries": [<query>, ...]}; response {"data": [<{results, meta, query}> or <{error}>, ...]} in request order, each element succeeding or failing on its own (an {error} element carries the error object of Section 4.4). Used by the tiles and panel previews on first load. Each query counts separately toward the rate limit (9.23.3). |
viewer |
GET /sites/{domain}/current-visitors |
Live visitor count (9.14). Returns {"data": {"visitors": 12}}. |
viewer |
GET /sites/{domain}/segments |
List segments the caller may use (9.18). | viewer |
POST /sites/{domain}/segments |
Create a segment. | viewer (personal) / editor (site) |
PATCH /sites/{domain}/segments/{id} |
Rename, replace filters, or promote personal to site. | owner of personal / editor |
DELETE /sites/{domain}/segments/{id} |
Delete a segment. | owner of personal / editor |
GET /sites/{domain}/annotations?from=&to= |
List annotations visible to the caller within a date range (9.15). | viewer |
POST /sites/{domain}/annotations |
Create an annotation. | viewer (personal) / editor (shared) |
PATCH /sites/{domain}/annotations/{id} |
Edit an annotation. | author of personal / editor |
DELETE /sites/{domain}/annotations/{id} |
Delete an annotation. | author of personal / editor |
GET /sites-overview |
Sites list with stats and sparklines (9.19). | any authenticated user |
POST /me/pinned-sites |
Pin a site for the caller. Body {"domain": "example.com"} (9.19.4). |
viewer on the site |
DELETE /me/pinned-sites/{domain} |
Unpin a site (9.19.4). | any authenticated user |
POST /consolidated/query |
Consolidated all-sites aggregate (9.20). | any authenticated user |
GET /labels?dimension=&ids= |
Resolve display names for coded values: region ISO 3166-2 codes and city GeoNames IDs (country names come from the browser's Intl.DisplayNames). Up to 200 ids per call. |
any authenticated user |
Goal, property, funnel and journey routes are in Section 10. Every list or error response follows the envelopes in Section 4.4.
Read-only POST routes. POST /sites/{domain}/query, POST /sites/{domain}/query/batch, POST /sites/{domain}/funnels/{id}/results, POST /sites/{domain}/funnels/preview and POST /sites/{domain}/journeys use POST only because their bodies do not fit in a URL. They change no state and carry the route-registry flag readOnly (Section 11), so read-only operator impersonation (Section 15) can call them while every other POST, PATCH and DELETE stays blocked.
9.1.3 Query body used by the dashboard #
The dashboard builds Section 13 query objects with these members (field names as defined in Section 13): metrics, date_range, dimensions, filters, order_by, include (imports, time_labels, total_rows, comparisons), and pagination. The server fills site_id from {domain}, applies the caller's scope restrictions (a shared link's fixed segment is AND-ed into filters server-side and cannot be removed by the client, Section 12), and forces pagination.limit <= 10000.
The dashboard requests include.comparisons (the comparison range per 9.13) instead of issuing two queries, so both periods are computed in one round trip and cached as one entry.
Shared-link and embed viewers cannot send the segment pseudo-dimension: the /api/share/v1 mirror rejects any filter on it with 400 validation_failed, so a viewer cannot expand a segment id they guessed. The link's locked segment filters apply to every mirrored endpoint, including filter-value suggestions, realtime, funnels and journeys, and they count as active filters for the imported-data rules (9.21).
9.1.4 Internal dimension identifiers #
The dashboard uses the dimension identifiers of Section 13: time, time:minute, time:hour, time:day, time:week, time:month, visit:source, visit:channel, visit:referrer, visit:utm_medium, visit:utm_source, visit:utm_campaign, visit:utm_term, visit:utm_content, visit:entry_page, visit:exit_page, visit:country, visit:region, visit:city, visit:browser, visit:browser_version, visit:os, visit:os_version, visit:device (the user-agent device class: desktop, mobile, tablet), visit:screen_size (viewport class: mobile, tablet, laptop, desktop), event:page, event:hostname, event:goal, and event:props:<key>.
9.2 Metric Definitions (Canonical) #
This table is the single source of truth for what each metric means. Section 13 returns exactly these values; Section 10 and Section 12 reuse them.
9.2.1 Scoping rules #
- Event-scoped metrics are computed from the
eventstable (Section 6) over events whosetimestampfalls in the selected range, after filters. They are:pageviews,events,scroll_depth,time_on_page. - Session-scoped metrics are computed from the
sessionstable (Section 6) over sessions whose start time falls in the selected range. They are:visits,bounce_rate,visit_duration,views_per_visit,exit_rate. A session that starts at 23:50 on the last day of the range belongs to that day even if it ends the next day. visitorsis computed from theeventstable as the distinct count ofuser_idover events in range. When a filter references a session-level dimension (source, country, entry page and so on) the filter is evaluated on the session that owns the event.- When a filter references an event-level dimension (
event:page,event:goal,event:props:*,event:hostname) and the metric is session-scoped, the session is included when at least one of its events in range matches the filter (a session "contains" a matching event). The metric value is then computed over the whole session, not only the matching events. - The engine excludes
engagementevents frompageviews,eventsandvisitors(anengagementevent alone never makes a visitor; Section 8 guarantees one is always preceded by a pageview). The single exception: when the active goal filter contains a scroll goal (Section 10.2.4), the qualifyingengagementevents are the conversions and are counted byvisitorsandeventsfor that goal. - Visitors are daily. Because the visitor hash rotates daily (Section 8), a person who returns on two different days has two different
user_idvalues and is counted twice in a multi-day range.visitorsover a range is the exact distinct count ofuser_idvalues in that range, i.e. the number of distinct visitor-days. This is intended and disclosed; the tooltip on the Visitors tile reads: "Unique visitors are counted per day. A returning person on a different day counts again."
9.2.2 Metric table #
| Key | UI label | Formula | Unit | API rounding | Display format | Shown in |
|---|---|---|---|---|---|---|
visitors |
Unique visitors | uniqExact(user_id) over non-engagement events in range. In conversion mode (a goal filter is active) the label becomes "Unique conversions" and the value counts distinct visitors with at least one matching goal event. |
count | integer | Compact number (9.2.4) | Top stats, chart, every report, funnels, journeys (as visits), sites list, consolidated view |
visits |
Total visits | count() of sessions with start time in range (sum(sign) over the collapsing sessions table). |
count | integer | Compact number | Top stats, chart, Entry/Exit pages, Sources and Campaigns details, sites list, consolidated view |
pageviews |
Total pageviews | countIf(name = 'pageview'). |
count | integer | Compact number | Top stats, chart, Pages, details modals, consolidated view |
views_per_visit |
Views per visit | pageviews / visits where both are over the same session set. NULL when visits = 0. |
ratio | 2 decimals | 1 decimal (2.4) |
Top stats, chart |
bounce_rate |
Bounce rate | bounced_sessions / visits * 100. A session is a bounce when it has exactly one pageview event and no interactive custom event (interactive = 1, Section 6.3); engagement events never un-bounce a session. NULL when visits = 0. |
percent | 1 decimal | Integer with % (47%); tooltip shows 1 decimal |
Top stats, chart, Pages, Entry pages, Sources and Campaigns details, consolidated view |
visit_duration |
Visit duration | sum(duration) / visits where duration is the sessions.duration column in seconds (ended_at - started_at, Section 6.4); engagement events extend ended_at, so a one-page visit with engagement has a non-zero duration. NULL when visits = 0. |
seconds | integer (round half up) | 45s, 3m 12s, 1h 5m |
Top stats, chart, Entry pages, Sources and Campaigns details, consolidated view |
events |
Total conversions (conversion mode) / Events | countIf(name != 'engagement') (pageviews plus custom events). In conversion mode this is the number of events matching the goal filter. |
count | integer | Compact number | Goals, Properties, conversion-mode panels, top stats in conversion mode |
scroll_depth |
Scroll depth | Mean over page visits of the maximum scroll_depth (Section 6.3) seen on that page visit. A page visit is the set of events of one session on one pathname. Only page visits with at least one engagement event are included; page visits without one are excluded from numerator and denominator. NULL when no page visit qualifies. |
percent | integer (round half up) | Integer with % |
Top stats when a page filter is active, Pages panel |
time_on_page |
Time on page | Mean over the same set of page visits of the summed engagement_time (milliseconds, Section 6.3) converted to seconds. NULL when no page visit qualifies. |
seconds | integer (round half up) | Duration format | Top stats when a page filter is active, Pages panel |
conversion_rate |
Conversion rate | converting_visitors / total_visitors * 100, where converting_visitors is the distinct visitors with at least one event matching the active goal filter, and total_visitors is the distinct visitors over the whole query with the goal filter removed (all other filters kept). For a breakdown the denominator is the whole-query total, not the row. NULL when total_visitors = 0. |
percent | 1 decimal | 12.3% (one decimal) |
Top stats and chart in conversion mode, Goals report |
group_conversion_rate |
Conversion rate (per row) | converting_visitors_in_row / total_visitors_in_row * 100 where both counts use the row's dimension value and the denominator is computed with the goal filter removed. Equals conversion_rate when there is no dimension. NULL when the row denominator is 0. |
percent | 1 decimal | CR column, 1 decimal |
Sources, Channels, Campaigns, Pages, Locations, Devices, Properties reports in conversion mode |
total_revenue |
Total revenue | Sum, over events matching the active revenue goal(s), of the event revenue converted to the goal currency (Section 10.4). NULL when no revenue goal is in the filter. |
money | rounded half away from zero to the currency's ISO 4217 exponent at output | Short money ($1.2K); tooltip long ($1,234.56) |
Top stats and chart in conversion mode, Goals report, conversion-mode reports, funnels |
average_revenue |
Average revenue | total_revenue / count of matching goal events that carry a revenue value. NULL when that count is 0. |
money | rounded half away from zero to the currency's ISO 4217 exponent at output | Short money; tooltip long | Same places as total_revenue |
percentage |
Percentage (%) |
row_visitors / total_visitors * 100 where total_visitors is the distinct visitors of the entire query with the same filters (before pagination). Rows of a single-valued dimension sum to 100. |
percent | 1 decimal | Whole percent (35%; <1% for a value above zero that rounds to zero) |
Locations, Devices, Properties, AI Assistants |
exit_rate |
Exit rate | For a page P: exits_from_P / visits_that_included_P * 100, where exits_from_P is sessions whose exit page is P and visits_that_included_P is sessions with at least one pageview of P. |
percent | 1 decimal | Integer with %; tooltip 1 decimal |
Exit pages |
9.2.3 Metric details and edge cases #
- Bounce rate in the Pages report is computed only over sessions that entered on that page (it equals the bounce rate shown for the same path in the Entry pages tab). Pages that are never entry pages show
-. - Bounce rate with a page filter on the top stats row uses the session set containing a matching pageview (rule 4 in 9.2.1).
- Visit duration for single-event sessions is 0 seconds and is included in the average (not excluded).
- Zero-fill. Count metrics (
visitors,visits,pageviews,events, revenue totals) are 0 for empty buckets. Ratio and average metrics (views_per_visit,bounce_rate,visit_duration,scroll_depth,time_on_page,average_revenue,conversion_rate) arenullfor buckets without denominator and are drawn as gaps in the chart and shown as-in tables. eventsand goals.eventscounts apageviewas an event, matching the "Total conversions" semantic for pageview goals (Section 10.2).- Revenue is never summed across currencies. If the active goal filter contains revenue goals of more than one currency,
total_revenueandaverage_revenueare not requested and the revenue columns are hidden with the tooltip "Select revenue goals that share one currency." - Imported data contributes only to
visitors,visits,pageviews,bounce_rate,visit_duration,views_per_visit,percentage,events(custom events) and, on the Pages report,time_on_page; see 9.21. - Exact vs approximate. All distinct counts are exact (
uniqExact); no approximate counting is used anywhere in the product.
9.2.4 Number formatting (shared by all widgets) #
Every widget uses the formatting helpers and rules of Section 4.7.2 (en-US, the product's single UI locale): counts in tables and tooltips in full with thousands separators; counts in tiles and chart axes in full below 10,000 and compact from 10,000 (12.3K, 1.2M, 1.2B, one decimal at most, trailing .0 removed); bounce rate and exit rate as whole percents; conversion rates with one decimal; views_per_visit with one decimal; durations as 42s, 1m 24s, 1h 5m (zero is 0s); a value above zero that rounds to 0% shows <1%; a missing (null) value is an em dash — with the tooltip "No data for this period." Tooltips show the exact integer, and the CSV export writes raw numbers (9.6.5). Money uses the short form {symbol}{compact} in tiles and tables and the long form with exact minor-unit precision in the tooltip; symbols and grouping follow the goal currency in en-US, and currency exponents follow ISO 4217 (0 for JPY, 3 for KWD).
9.3 Dashboard Layout (Top to Bottom) #
The route /sites/:domain renders the following regions in order. The app shell (navigation, team switcher) is owned by Section 4; this section defines the content area.
| Order | Region | Behaviour |
|---|---|---|
| 1 | Header bar (sticky) | Left: site switcher dropdown (searchable list of the user's sites, current site first), site favicon-less avatar (first letter on a deterministic color), site domain. Center: Add filter button (shortcut F) and the Segment menu (9.18). Right: Current visitors pill (green dot plus count, 9.14), Imported data toggle (only when the site has completed imports, 9.21), Comparison select (9.13), Date range button (9.12), overflow menu (Settings, Export all data link to Section 14, Copy shared link shortcut to Section 12 for editors). On viewports below 768 px the header splits into two rows: row 1 site switcher and current visitors, row 2 filter, segment, comparison and date controls with horizontal scroll. |
| 2 | Active filters row | Chips for every active filter and segment (9.16). Hidden when there are none. Sticky below the header. Contains "Clear all" when two or more chips exist. |
| 3 | Status banners | Zero or more full-width banners, stacked: unlicensed/over-cap (hosted, Section 15), first-data waiting (9.22), imported data notice (9.21), query degraded notice. Dismissible banners remember dismissal in localStorage per site for 7 days (authenticated dashboard only; shared and embed views show them again on every load). |
| 4 | Top stats row + main chart (one card) | Tiles (9.4) above the chart (9.5). |
| 5 | Sources panel (left half) and Campaigns panel (right half) | Two-column grid from 1024 px; stacked below. |
| 6 | Pages panel (left) and Locations panel (right) | Same grid. |
| 7 | Devices panel (left) and Conversions area (right) | Devices panel has tabs Screen size, Device, Browser, Operating system. The Conversions area is the Goals panel (Section 10.2.8). |
| 8 | Properties panel | Full width; shown only when the site has at least one allowlisted custom property (Section 10.3). |
| 9 | Funnels panel | Full width; shown only when the site has at least one saved funnel, otherwise shows the "Create a funnel" empty state for editors and is hidden for viewers. |
| 10 | User Journeys panel | Full width; always shown (Section 10.6). |
| 11 | Footer | The footer of the app shell (Section 4.13.1: product name and version, and the geoip_attribution text from the runtime config, which defaults to "IP geolocation by DB-IP"), plus a "Keyboard shortcuts" link that opens the list in 9.12.3. |
Grid rules: panels have a fixed minimum height of 420 px so layout does not jump while loading; the dashboard max content width is 1280 px, centered; panels use skeleton rows while loading (9 gray rows). Panels never reorder; a user cannot customise the layout.
Page title: {domain} · {PRODUCT_NAME}. The browser tab title does not include visitor counts.
9.4 Top Stats Row #
9.4.1 Tiles #
The row is a horizontal list of selectable tiles above the main chart. Exactly one tile is selected (it drives the chart, 9.5.3); the selected tile has a 2 px accent underline and aria-selected="true".
| Mode (when active) | Tiles (left to right) |
|---|---|
| Default (no goal filter, no page filter) | Unique visitors, Total visits, Total pageviews, Views per visit, Bounce rate, Visit duration |
Page filter (any event:page filter active, no goal filter) |
Unique visitors, Total visits, Total pageviews, Bounce rate, Scroll depth, Time on page |
Conversion mode (any event:goal filter active) |
Unique conversions, Total conversions, Conversion rate; plus Total revenue and Average revenue when exactly one revenue currency applies (9.2.3) |
| Realtime (period is Realtime) | Unique visitors (last 30 minutes), Total pageviews (last 30 minutes); the large live counter is shown in the header (9.14) |
| Both page filter and goal filter | Conversion mode tiles |
9.4.2 Tile content #
Each tile shows: label (with an info icon whose tooltip gives the definition from 9.2), the value, and (when comparison is on) the delta row.
- Value: formatted per 9.2.4; full exact value in the
title/tooltip. - Loading: skeleton bar. Error:
-plus a retry icon button; an error in one tile does not affect others. - Tiles use one aggregate query with all tile metrics and
include.comparisonswhen comparison is on.
9.4.3 Comparison deltas #
When comparison is active (9.13) each tile shows the previous value ("vs 1.2K") and a delta badge:
| Case | Display |
|---|---|
| Count, ratio, duration, money metrics | Relative change (current - previous) / previous * 100, rounded half away from zero to an integer percent, with an up or down arrow: +12%, -4%. |
bounce_rate, conversion_rate, scroll_depth |
Absolute change in percentage points: +3 pp, -1.5 pp (one decimal, trailing .0 dropped). |
previous = 0 and current > 0 |
Text New with an up arrow (no percent). |
previous = 0 and current = 0 |
0%, neutral gray, no arrow. |
current or previous is null |
-, neutral gray. |
| Relative change above zero but under 1% in magnitude | <1% (with its sign in text for assistive tech), neutral gray, no arrow (Section 4.7.2). An absolute change under 0.05 pp is rendered 0 pp, neutral. |
Color semantics: an increase is green for every metric except bounce_rate, where a decrease is green and an increase is red. A zero or null change is gray. Color is never the only signal (arrow and sign are always present).
Tooltip on the delta: "{current} vs {previous} ({comparison range start} - {comparison range end})".
9.5 Main Chart #
9.5.1 Series and rendering #
A time-series chart (library per Section 3) directly under the tiles, height 288 px desktop, 224 px below 768 px. The primary series is a solid line with a light area fill in the accent color. The comparison series (when on) is a dashed gray line, no fill. The chart draws no gridlines other than three horizontal guides, a compact y-axis, and an x-axis with at most 8 evenly spaced labels formatted by interval (9.5.2).
- Buckets are zero-filled for count metrics and gap-filled (
null) for ratio metrics (9.2.3). - In-progress bucket: the bucket containing "now" is rendered with a dashed segment from the previous point and its tooltip says "In progress". Buckets after "now" are never drawn.
- Week buckets start on Monday (ISO). If the selected range starts or ends mid-week, the first and last week buckets are clipped to the selected range and the tooltip shows the clipped dates.
- Month buckets start on the first day of the month, clipped likewise.
- Tooltip (hover and keyboard focus): bucket label in the site timezone, metric value, and the comparison value with its own date label when comparison is on; annotations inside the bucket are listed under the values.
- Accessibility: the chart has
role="img"with anaria-labelsummarising metric, range and total; a visually hidden table with the same data is available to screen readers; keyboard users move through points with the arrow keys.
9.5.2 Intervals #
Supported intervals: minute, hour, day, week, month. The interval selector is a dropdown at the chart's top right showing only the intervals allowed for the current range.
Let D = the number of calendar days the selected range spans in the site timezone (inclusive; Last 24 hours counts as D = 1 for every rule below).
| Condition | Allowed intervals | Auto (default) interval |
|---|---|---|
| Realtime | minute only |
minute |
D <= 1 (Today, Yesterday, a single custom day, Last 24 hours) |
minute, hour |
hour |
2 <= D <= 7 |
hour, day |
day |
8 <= D <= 31 |
day, week |
day |
32 <= D <= 91 |
day, week, month |
day |
92 <= D <= 183 |
day, week, month |
week |
184 <= D <= 366 |
day, week, month |
month |
D > 366 (including All time) |
week (only while D <= 10,000), month |
month |
Rules:
- The chosen interval is part of the URL state (
interval=, 9.17). If the URL carries an interval that is not allowed for the range, it is ignored and the auto interval is used (no error). - Changing the range resets the interval to auto unless the previously chosen interval is still allowed for the new range, in which case it is kept.
minuteandhourintervals are not available when imported data is included (imports are daily). If the user selects them, imported data is excluded for that view (9.21).- Auto interval for All time: if the site's first event is within 91 days the auto interval is
day; within 183 daysweek; otherwisemonth. - Bucket count never exceeds 1,500 (guaranteed by the table above;
weekis withheld whenD > 10,000).
X-axis label formats (en-US, 12-hour clock, Section 4.7.2): minute 2:07 PM; hour 2 PM (with a date label when the range spans days); day Nov 3; week Nov 3 (week start); month Nov 2026.
9.5.3 Metric switching #
Clicking (or pressing Enter/Space on) a tile selects its metric for the chart. Selectable chart metrics: visitors, visits, pageviews, views_per_visit, bounce_rate, visit_duration; in conversion mode visitors (Unique conversions), events (Total conversions), conversion_rate, total_revenue, average_revenue. scroll_depth and time_on_page tiles are not selectable (clicking does nothing; the cursor is default). The selected metric is stored in the URL as metric=<key>. If the selected metric is unavailable after a filter change (for example views_per_visit while a goal filter is added) the chart falls back to visitors.
9.5.4 Comparison overlay #
With comparison on, the query returns both series. The comparison series is aligned by bucket index, not by date: bucket n of the comparison range is drawn at bucket n of the main range. Both ranges contain the same number of buckets by construction (9.13.2). Tooltip rows are labelled with each series' own date.
9.5.5 Annotation markers #
Annotations (9.15) appear as small circular markers on the x-axis baseline at the bucket containing the annotation date (for minute and hour intervals the marker sits at the first bucket of that date). Several annotations in one bucket collapse into one marker with a count badge. Hover or focus shows a popover listing title, optional description, author and visibility (Personal or Shared); the popover contains Edit and Delete actions when the caller is permitted (9.15.3). Clicking an empty chart position does nothing; creating an annotation is done from the Add annotation item in the chart's overflow menu (also the keyboard route) or by clicking a bucket's tooltip action "Annotate this day".
9.5.6 Empty and error states #
No events in range: the chart renders a flat zero line and the message "No visitors in this period." Query error: inline error with Retry; the tiles remain independent.
9.6 Report Panel Framework #
Every report panel (Sources, Campaigns, Pages, Locations, Devices, Goals, Properties) follows the framework below. Panel-specific tabs and columns are in 9.7 to 9.11.
9.6.1 Anatomy #
| Part | Behaviour |
|---|---|
| Title and tabs | A panel title on the left and a tab strip (role="tablist"). The active tab is stored in localStorage per site and panel (qs:{domain}:tab:{panel}), not in the URL; shared-link, public and embed views always open the first tab and store nothing. Keyboard: Left/Right arrows move between tabs. |
| Column header | First column is the dimension label; remaining columns are metrics. The header shows the metric names for the active mode (9.6.2). The panel preview is not sortable; it is always ordered by the default metric descending. |
| Rows | Top 9 rows (fixed). Each row has a label cell with a horizontal bar behind it whose width is row_value / max_row_value of the first metric column (the bar uses the accent color at 12% opacity). Long labels truncate with an ellipsis and a full-text tooltip. |
| Row click | Clicking a row label adds a filter on that value (is) for the dimension the active tab represents and refreshes the whole dashboard. Rows that link to an external URL (Referrers, Pages when full URLs are on) also show a small external-link icon that opens the URL in a new tab with rel="noopener noreferrer"; the row click itself still filters. |
| Details link | Footer link "Details" (icon plus text) opens the details modal (9.6.4). Hidden when the tab returned fewer than 9 rows and total_rows <= 9. |
| Metric header tooltips | Each metric header has an info tooltip with the definition from 9.2. |
9.6.2 Column modes #
A panel is in exactly one of three modes, derived from the active filters:
| Mode | Condition | Preview columns |
|---|---|---|
| Default | No event:goal filter |
The panel's default metrics (9.7 to 9.11). |
| Conversion | At least one event:goal filter |
Dimension label, Conversions (visitors with the goal applied, header "Conversions"), CR (group_conversion_rate). |
| Revenue | Conversion mode and exactly one revenue currency applies (9.2.3) | Dimension label, Conversions, Revenue (total_revenue), CR. The details modal adds Total conversions (events), Average revenue and sorts by total_revenue first (9.6.4). |
When the goal filter changes the mode, the active tab stays selected.
9.6.3 Preview ordering and ties #
Default metric descending; ties broken by the dimension label ascending using a locale-insensitive binary comparison so results are deterministic across requests. The preview query uses order_by: [[<first metric>, "desc"], [<dimension>, "asc"]] and pagination: {limit: 9, offset: 0} with include.total_rows: true.
9.6.4 Details modal #
The modal opens from "Details" and is routable (/sites/:domain/<report-slug>, for example /sites/example.com/pages); closing it returns to /sites/:domain preserving the query string. Escape closes it; focus is trapped inside and returned to the Details link on close.
- Header: panel title, tab strip (same tabs as the panel), search input, "Export CSV" button, close button.
- Table: all columns of the report's details set (9.7 to 9.11), one row per dimension value.
- Sorting: clicking a metric header cycles
desc,asc, then back to the default order. The dimension header sorts alphabeticallyasc/desc. Sorting is server-side (order_by), and always followed by the dimension ascending as the tie-breaker. The active sort is shown with an arrow andaria-sort. Sort state is kept only while the modal is open. - Pagination: page size 100. The modal uses a "Load more" button (not infinite scroll) which requests the next page with
pagination.offset. A counter shows "Showing 100 of 2,345". Maximum reachable rows is 10,000 (the engine limit, Section 13); whentotal_rowsexceeds 10,000 the counter reads "Showing 10,000 of 12,345 (limit reached)" and the button is hidden. - Search: a text input filtering the dimension label server-side with a case-insensitive
containscondition added to the query (the engine escapes%,_and\and binds the value as a parameter, Section 13.5.1; this is a transient filter, not shown as a chip). Debounce 300 ms; minimum 1 character; maximum 100 characters. For coded dimensions (country, region, city) the search is applied client-side against the resolved names over the loaded page set instead, because the engine only knows codes; for those tabs the modal loads up to 10,000 rows on open. Clearing the input restores the first page. - Empty search result: "No results for ''".
- Modal state:
Esc, clicking the backdrop or the close button closes. The modal never changes filters unless a row is clicked; a row click adds the filter, closes the modal and refreshes the dashboard. - Loading and error: skeleton rows; an error shows a retry button and keeps already loaded rows.
9.6.5 CSV export of a panel #
The details modal's Export CSV button exports the active tab with the current filters and date range (comparison is never included). It is generated in the browser from a fresh query with pagination: {limit: 10000, offset: 0}, the current sort, and the modal's search text.
| Aspect | Rule |
|---|---|
| Columns | The details-set columns in their displayed order. Header cells use the English column labels (not localized), for example Visitors, Bounce rate. |
| Values | Machine-readable: counts as integers, bounce_rate/percentage/conversion_rate as plain numbers with up to 1 decimal (47.3, no %), visit_duration and time_on_page as integer seconds, scroll_depth as integer, revenue as a decimal number with the minor-unit precision in one column and the ISO 4217 code in an adjacent <column> currency column. null is an empty cell. Dimension labels are the same display names as the UI (country names in English, (none) kept literally). |
| Format | UTF-8 without BOM, comma separator, RFC 4180 quoting (quote cells containing comma, quote, CR or LF; double embedded quotes), \r\n line endings, header row always present. |
| Formula injection | A cell whose first character is =, +, -, @, tab or carriage return is prefixed with a single quote '. |
| File name | {domain}_{panel-slug}-{tab-slug}_{start}_{end}.csv with dates in the site timezone, for example example.com_pages-top-pages_2026-10-01_2026-10-31.csv. all period uses the first event date and today. |
| Limits | 10,000 rows; if truncated the final row is not added, a toast says "Exported the first 10,000 rows." |
| Permissions | Any user who can view the panel, including shared-link viewers. The whole-site export in Section 14 is separate and role-restricted. |
9.6.6 Panel states #
| State | UI |
|---|---|
| Loading | 9 skeleton rows (animated, reduced when prefers-reduced-motion). |
| No data (no filters) | Centered text "No data yet for this period." |
| No data (filters active) | "No data matches the current filters." with a "Clear filters" link. |
| Error | "Could not load this report." and a Retry button; code query_timeout shows "This report took too long. Try a shorter date range or fewer filters." |
| Unsupported combination | Shown instead of data when the engine returns unsupported_combination (Section 13): "This report can't be combined with the current filters." with Clear filters. |
9.7 Sources and Campaigns Panels #
9.7.1 Sources panel #
Tabs, in order: Channels, Sources, Referrers, AI Assistants.
| Tab | Dimension | Default preview columns | Details modal columns |
|---|---|---|---|
| Channels | visit:channel |
Visitors | Visitors, Visits, Percentage, Bounce rate, Visit duration |
| Sources | visit:source |
Visitors | Visitors, Visits, Percentage, Bounce rate, Visit duration |
| Referrers | visit:referrer (full referrer URL, query string already removed at ingestion, Section 8) |
Visitors | Visitors, Visits, Percentage, Bounce rate, Visit duration |
| AI Assistants | visit:source with an implicit visit:channel is "AI Assistants" |
Visitors | Visitors, Visits, Percentage, Bounce rate, Visit duration |
Details modal columns in Conversion and Revenue mode follow 9.6.2 plus the two columns Total visitors (visitors of the row with the goal removed) and Total conversions (events). Revenue mode sorts by Revenue by default; this is how revenue by source is surfaced.
Behaviour:
- Channels lists the 18 channels defined in Section 8 in value order of the metric. Channels with zero visitors in range are not listed. Row click filters
visit:channel is <channel>. - Sources lists the normalized source name from Section 8 (for example
Google,ChatGPT). Direct traffic appears as the labelDirect / None(stored as an empty source). Row click filtersvisit:source is <source>; forDirect / Nonethe filter value is(none)(9.16.5). - Referrers shows hostname plus path of the referring URL; when a source filter is active the tab shows only referrers of that source. Rows link out (9.6.1). Direct visits have no referrer and are omitted from this tab.
- AI Assistants tab: an implicit, non-removable constraint
visit:channel is "AI Assistants"is added to the query for this tab only (it is not shown as a chip). Rows are the AI sources: ChatGPT, Claude, Perplexity, Gemini, Microsoft Copilot, DeepSeek, Grok, Meta AI, Mistral Le Chat, Phind, You.com, and any other source Section 8 classifies as AI Assistants, includingutm_sourcevalues such aschatgpt.com. Row click adds the source filter and also the channel filter, so every other panel (Pages, Locations and so on) then answers "what do visitors from this assistant do". The tab has a header line "AI Assistants: {n} visitors ({p}% of all visitors)" computed from one extra aggregate query (visitorswith the channel constraint overvisitorswithout it). If no AI traffic exists in range the tab shows "No visits from AI assistants in this period."
9.7.2 Campaigns panel #
Tabs, in order: UTM Medium, UTM Source, UTM Campaign, UTM Term, UTM Content.
| Tab | Dimension | Preview columns | Details modal columns |
|---|---|---|---|
| UTM Medium | visit:utm_medium |
Visitors | Visitors, Visits, Percentage, Bounce rate, Visit duration |
| UTM Source | visit:utm_source |
Visitors | same |
| UTM Campaign | visit:utm_campaign |
Visitors | same |
| UTM Term | visit:utm_term |
Visitors | same |
| UTM Content | visit:utm_content |
Visitors | same |
Rules: sessions without the parameter group into a single row labelled (none) which is listed last in the preview even when large (it is the only row exempt from the ordering rule, to keep campaign values visible); in the details modal (none) sorts normally. Row click filters is <value>; for (none) the filter value is (none). utm_medium is stored lower-cased and the other UTM values keep the case that was sent (Section 8.10.5); labels are shown as stored. Conversion and Revenue modes apply as in 9.6.2, giving campaign-level revenue.
9.8 Pages Panel #
Tabs: Top Pages, Entry Pages, Exit Pages.
| Tab | Dimension | Preview columns | Details modal columns |
|---|---|---|---|
| Top Pages | event:page (pathname), or hostname plus pathname when Full URL is on (9.8.1) |
Visitors | Visitors, Pageviews, Bounce rate, Time on page, Scroll depth |
| Entry Pages | visit:entry_page |
Unique entrances (visitors) |
Unique entrances (visitors), Total entrances (visits), Bounce rate, Visit duration |
| Exit Pages | visit:exit_page |
Unique exits (visitors) |
Unique exits (visitors), Total exits (visits), Exit rate |
Notes:
- Row click filters
event:page is <path>(Top Pages),visit:entry_page is <path>(Entry),visit:exit_page is <path>(Exit). - Labels are paths exactly as stored: query strings are absent (Section 8); the hash fragment is present only for sites with hash routing enabled.
- A page filter and an entry-page filter are different filters: "Page is /pricing" means "sessions that viewed /pricing", while "Entry page is /pricing" means "sessions that started on /pricing".
- Conversion and Revenue modes replace the columns per 9.6.2 for all three tabs;
Exit rateis not shown in conversion mode. Scroll depthandTime on pageshow-for pages without engagement data (9.2.3).
9.8.1 Full-URL page reports option #
The Top Pages tab has a toggle "Show full URLs" (switch in the tab's toolbar and in the modal header). Off (default): rows group by pathname. On: rows group by the pair (event:hostname, event:page) and are labelled hostname/path (no scheme). The link-out opens https://hostname/path.
- Applies to the Top Pages tab only; the Entry and Exit tabs group by path (sessions carry
entry_hostnameandexit_hostname, which are not shown in v1). - The state is stored in the URL as
full_urls=true(so shared dashboards can carry it) and persisted per user per site inlocalStorage. - Row click with the toggle on adds two filters:
event:hostname is <hostname>andevent:page is <path>. - CSV export (9.6.5) writes the same
hostname/pathlabels with an additional leadingHostnamecolumn andPathcolumn instead of one label column. - Which hostnames are accepted at all is controlled by the site's
allowed_hostnamessetting (column in Section 5.5.2, edited in the tracking settings of Section 7, enforced at ingestion per Section 8; events from other hostnames are dropped with reasonhostname_not_allowed); this toggle only changes grouping.
9.9 Locations Panel #
Tabs: Map, Countries, Regions, Cities. The default tab is Map.
| Tab | Dimension | Preview columns | Details modal columns |
|---|---|---|---|
| Map | visit:country |
Choropleth by visitors; the tab footer lists the top 3 countries |
n/a (the Countries tab provides Details) |
| Countries | visit:country |
Visitors | Visitors, Percentage |
| Regions | visit:region |
Visitors | Visitors, Percentage |
| Cities | visit:city |
Visitors | Visitors, Percentage |
9.9.1 Map tab #
An SVG choropleth rendered from a bundled simplified world boundary dataset (public domain) with one shape per ISO 3166-1 country. Color scale: five sequential steps based on quantiles of the non-zero visitors values, with a legend underneath; countries with no visitors are neutral gray. Hover or focus shows a tooltip "{country}: {visitors} visitors ({percentage}%)". Clicking a country filters visit:country is <code> and moves the panel to the Regions tab. Zoom: mouse wheel and the +/- buttons zoom up to 8x with panning by drag; the Reset button restores the default view. For keyboard and screen-reader users the Countries tab contains the same data. The map never loads third-party tile services (privacy; no external requests).
9.9.2 Labels and unknowns #
- Country labels come from
Intl.DisplayNamesforen-US, with the English name from the bundled location dataset (Section 8.10.3) as the fallback. - Region labels and city labels are resolved through
GET /labels(9.1.2) in one batched call per page of results, cached in the browser for the session. Region label format:{subdivision name}, {country code}when the same region name appears in more than one country in the result; otherwise the plain name. City label format:{city name}; an unresolvable GeoNames ID showsCity {id}. - Visitors whose location could not be resolved are grouped under the label
Unknownand are listed last in the preview and the modal's default order. ClickingUnknownfiltersis (none). - The Regions and Cities tabs list all values in range regardless of country; when a country filter is active they are restricted to that country automatically (as any filter would).
9.10 Devices Panel #
Tabs: Screen Size, Device, Browser, Operating System.
| Tab | Dimension | Preview columns | Details modal columns |
|---|---|---|---|
| Screen Size | visit:screen_size with values Mobile, Tablet, Laptop, Desktop (the stored values mobile, tablet, laptop, desktop with the first letter capitalized; classification by viewport width is owned by Section 8.10.2) |
Visitors | Visitors, Percentage |
| Device | visit:device with values Desktop, Mobile, Tablet (stored lowercase; the first letter is capitalized for display) |
Visitors | Visitors, Percentage |
| Browser | visit:browser |
Visitors | Visitors, Percentage |
| Operating System | visit:os |
Visitors | Visitors, Percentage |
9.10.1 Versions drilldown #
Clicking a row in the Browser (or Operating System) tab adds the is <name> filter for that browser (or OS) and switches the tab to a version view titled "{Name} versions" with a "Back to browsers" link above the rows. The version view breaks down visit:browser_version (or visit:os_version) restricted by that filter and shows the same columns. Versions are major.minor strings as stored (126.0, 14.5); sessions without a version group under Unknown. Clicking a version row adds the is <version> filter on the version dimension. "Back to browsers" removes the version filter if present and the browser filter, and returns to the list. A version filter can exist only together with a browser (or OS) is filter; removing the parent filter chip removes the version chip too. If a browser filter is active without a version filter, the tab opens directly in the version view.
Unknown browser, OS, device or screen-size values are labelled Unknown; clicking filters is (none).
9.11 Conversions Area, Properties, Funnels and Journeys (Placement) #
These panels use the framework in 9.6; their content is defined in Section 10:
| Panel | Placement (9.3) | Definition |
|---|---|---|
| Goals | Right of Devices, tab-less | Section 10.2.8 (columns, conversion and revenue behaviour, goal detail) |
| Properties | Full-width, only if a property is allowlisted | Section 10.3.5 |
| Funnels | Full-width, only if a funnel exists | Section 10.5.10 |
| User Journeys | Full-width, always | Section 10.6 |
Interaction contract shared with the rest of the dashboard:
- Clicking a goal row adds
event:goal is <display name>and switches every panel to Conversion or Revenue mode (9.6.2). - Clicking a property value row adds
event:props:<key> is <value>. - Funnels and Journeys obey the active filters, date range and comparison as specified in Section 10; they ignore the imported-data toggle (imports do not contain step-level event data).
9.12 Time Range Picker #
9.12.1 Presets #
The date button in the header opens a menu listing the presets below, then "Custom range". Each preset has a URL token (period=), a range definition in the site timezone (evaluated at request time), and a keyboard shortcut.
| Preset (UI label) | period token |
Range | Shortcut |
|---|---|---|---|
| Realtime | realtime |
The last 30 minutes (9.14) | R |
| Today | day (no date) |
00:00 today to now | D |
| Yesterday | day with date=<yesterday> |
00:00 to 23:59:59 yesterday | E |
| Last 24 hours | 24h |
now - 24h to now (rolling, minute-rounded) |
H |
| Last 7 days | 7d |
The 7 calendar days ending today (today included) | 7 |
| Last 28 days | 28d |
28 calendar days ending today | 4 |
| Last 30 days | 30d |
30 calendar days ending today | 3 |
| Last 91 days | 91d |
91 calendar days ending today | 9 |
| Month to date | month (no date) |
First day of the current month to today | M |
| Last month | month with date=<first day of previous month> |
Whole previous calendar month | L |
| Year to date | year (no date) |
January 1 of the current year to today | Y |
| Last 12 months | 12mo |
The 12 calendar months ending with the current month (rule 3) | T |
| All time | all |
From the site's first recorded day (sites.created_at or the first imported day, whichever is earlier) to today |
A |
| Custom range | custom with from and to |
Two inclusive dates chosen in a calendar | C |
Rules:
- Default period for a site visited without a
periodparameter is30d. The last explicit non-realtime choice is remembered per user per site inlocalStorage(qs:{domain}:period) and restored when noperiodparameter is present. Realtime is never remembered. - A day, month or year selection with a
dateparameter is a navigable period: the left and right arrow buttons beside the date button, and the keyboard keysArrowLeft/ArrowRight, step it by one day, one month, or one year respectively. The right arrow is disabled when the next step would start in the future. Rolling presets (7d,28d,30d,91d,24h,12mo) step back and forward by their own length;allandrealtimeare not navigable (arrows hidden). - Last 12 months (
12mo) is the 12 calendar months ending with the current month (current month to date included), starting on the first day of the month 11 months earlier. All other*dpresets end today, inclusive. - A range never extends before 1970-01-01 and never beyond today; picking
toafter today clamps it to today.fromaftertois swapped silently in the calendar and rejected withinvalid_date_range(422) when it arrives that way in a URL, in which case the dashboard falls back to the default period and shows a toast "That date range was invalid, showing the last 30 days." - Custom range maximum length: 3,660 days (10 years). Longer ranges are rejected the same way.
- The calendar picker shows days before the site's first recorded day (its
created_ator first imported day) in a muted style; they remain selectable and simply return empty results. - Rolling presets re-evaluate "now" on each refresh; the displayed label keeps the preset name and shows the resolved dates under it ("Oct 5 - Nov 3, 2026").
- The site timezone is shown at the bottom of the menu: "Times shown in Europe/Berlin." (the actual
timezonevalue).
9.12.2 Realtime transitions #
Selecting Realtime keeps filters, drops the comparison (disabled in the menu), hides the imported toggle, and switches to the layout in 9.14. Leaving Realtime restores the previously remembered period.
9.12.3 Keyboard shortcuts #
The dashboard registers a single global keydown listener. Shortcuts are ignored when: focus is inside an input, textarea, select or contenteditable element; any of Ctrl, Meta or Alt is pressed; or a modal dialog other than the shortcuts list is open. Shift is allowed only for ?.
| Key | Action |
|---|---|
R |
Realtime |
D |
Today |
E |
Yesterday |
H |
Last 24 hours |
7 |
Last 7 days |
4 |
Last 28 days |
3 |
Last 30 days |
9 |
Last 91 days |
M |
Month to date |
L |
Last month |
Y |
Year to date |
T |
Last 12 months |
A |
All time |
C |
Open the custom range calendar |
ArrowLeft / ArrowRight |
Previous / next day, month or year (navigable periods) |
F |
Open the Add filter menu |
S |
Open the Segment menu |
X |
Toggle comparison on or off (restores the last comparison mode, default previous_period) |
Escape |
Close the open menu or modal |
? (Shift+/) |
Show the keyboard shortcuts list |
Shortcuts are listed in the "Keyboard shortcuts" dialog (footer link and ?); each date menu item also shows its key on the right. Shortcuts are disabled for shared-link viewers only where the action is unavailable there (Segment menu keys are inert if the link is segment-restricted, Section 12).
9.13 Comparison #
9.13.1 Modes #
The comparison select in the header offers:
comparison token |
Label | Comparison range |
|---|---|---|
off (default) |
No comparison | none |
previous_period |
Previous period | The range of equal length immediately before the selected range (rolling presets and custom ranges), or the previous calendar unit for calendar presets (rules in 9.13.2) |
year_over_year |
Year over year | The same calendar dates one year earlier |
custom |
Custom period | Two inclusive dates chosen by the user via compare_from and compare_to |
A checkbox Match day of week (match_day_of_week=true) is available for previous_period and year_over_year; it is hidden for off and custom. Comparison is unavailable (disabled) for Realtime and for all time. The mode is remembered per user per site in localStorage (qs:{domain}:comparison), including off. A URL comparison parameter always wins over the remembered value.
9.13.2 Exact range computation (site timezone) #
Let the selected range be [S, E] in whole days (or minute-rounded timestamps for 24h), with length L days.
- Previous period, rolling or custom range:
[S - L days, S - 1 day](for24h: the preceding 24 hours,[S - 24h, S]). - Previous period, calendar presets (
month,year, andday):month(month to date or a past month): the previous calendar month, truncated to the same number of days as the selected range (for an in-progress month, the firstNdays whereNis the days elapsed; for a completed month the whole previous month, truncated to the length of the shorter month).year: the previous calendar year, truncated to the same number of days.day(today): yesterday, truncated at the same time of day as now (00:00 to the current minute); for any completed day: the whole previous day.
- Year over year: the same dates shifted back one year (February 29 maps to February 28). For
daytoday: the same date last year up to the same time of day. - Match day of week: the comparison range is shifted so that its first day has the same weekday as
S:previous_period: the start becomesS - 7 * ceil(L / 7)days (the nearest earlier alignment), and the end isstart + L - 1days. This may leave a gap between the comparison range and the selected range.year_over_year: the unaligned start isS - 1 year; letd = (weekday(S) - weekday(start)) mod 7; ifd > 3used - 7; addddays to the start. The end isstart + L - 1days.- For
day(single-day) selections, "match day of week" is a no-op under previous period when the weekday already matches; otherwise the same rule applies withL = 1.
- Custom comparison:
compare_fromtocompare_to, inclusive. If its length differs fromL, the comparison series is padded withnullor truncated at the end so both series have the same bucket count, and a note under the chart reads "Comparison range has a different length." - The comparison range always has
Ldays (except the truncation cases above) so bucket counts are identical (9.5.4). The comparison range may not start before 1970-01-01; if it would, comparison shows as "No comparison data available." - Comparison for ranges in which the site had no events before the range shows deltas of
New(9.4.3).
The label under the date button reads: "Compared with Oct 5 - Nov 3, 2025". For weekday matching the label appends "(matching weekday)".
9.13.3 Where comparison applies #
Top stats (deltas), the main chart (overlay), and the Goals report totals line. Report panels do not show comparison columns in this release (their rows show only the selected period). Funnels apply comparison per Section 10.5.8. Imported data follows the toggle in both periods.
9.14 Realtime View #
- Range: the last 30 minutes ending now, bucketed per minute (30 buckets, the last one partial). Computed as
start = floor_to_minute(now) - 29 minutes,end = now. - Current visitors: the distinct
user_idcount over non-engagement events in the last 5 minutes (a sliding window evaluated at request time from the Valkey realtime keys of Section 8). Shown as a pill in the header on every period ("● 12 current visitors"; singular "1 current visitor";0 current visitorsis rendered gray). On the dashboard (non-realtime) the pill refreshes every 30 seconds while the tab is visible. Clicking the pill selects Realtime. - Layout in Realtime: header, filters row, a stats row with two tiles (Unique visitors and Total pageviews over the 30-minute range), the main chart (minute interval, metric visitors, fixed), then the Sources (Channels, Sources, Referrers, AI Assistants tabs), Pages (Top Pages only; Entry and Exit tabs hidden), Locations (Map, Countries, Regions, Cities), Devices, and Goals panels with the same behaviour as 9.6. The Campaigns, Properties, Funnels and Journeys panels are hidden.
- Polling: the whole realtime view refreshes every 10 seconds while the tab is visible (polling pauses when
document.hidden; one immediate refresh happens when the tab becomes visible again). Failed polls retry on the next tick without error toasts; after 3 consecutive failures a banner "Live data is delayed" appears and is removed on the next success. - Freshness: events become visible within about 3 seconds of acceptance (ingestion buffer flush,
INGEST_FLUSH_INTERVAL_MSdefault 2 seconds, Section 8.15). Server-side realtime responses are cached for 5 seconds (9.23). - Filters work in Realtime exactly as elsewhere. Comparison, imported data and the interval selector are unavailable.
- Empty: "No visitors in the last 30 minutes." on the chart, and
0tiles. - The live visitor count is computed by
GET /sites/{domain}/current-visitors(9.1.2) from the Valkey realtime keys (qs:rt:{site_id}, a 5-minute window maintained at ingestion, Section 8), not from ClickHouse; it uses no filters and never includes imported data.
9.15 Annotations #
9.15.1 Model #
An annotation is a note attached to a calendar date of the site, rendered on the main chart (9.5.5). Persisted in annotations (Section 5). Fields:
| Field | Type | Rules |
|---|---|---|
id |
uuid | Server generated. |
site_id |
uuid | From {domain}. |
created_by_user_id |
uuid | The creating user; set from the session; immutable. |
visibility |
enum personal, shared |
Default personal. |
occurred_on |
date (YYYY-MM-DD, site timezone) |
Required; not in the future more than 1 day ahead of the site's today; not before 2000-01-01. |
title |
string | Required, trimmed, 1 to 120 characters; no control characters. |
description |
string or null | Optional, trimmed, up to 1,000 characters; plain text (rendered escaped; URLs are not auto-linked). |
created_at, updated_at |
timestamptz | Server managed. |
Limit (Section 5.16): 5,000 annotations per site, personal and shared together. Exceeding returns 409 limit_reached with details.limit_name annotations, details.limit and details.current.
9.15.2 Visibility #
personal: visible only to its author. Never visible to other members, to shared-link viewers, or in embeds.shared: visible to every member and guest of the site in the authenticated dashboard. Shared annotations are not shown on shared links and embedded dashboards (Section 12), because notes are written for the internal team.- Imported-data days show annotations normally.
9.15.3 Permissions (summary; the authoritative matrix is in Section 11) #
| Action | Who |
|---|---|
| Create, edit, delete a personal annotation | The author, for roles viewer, editor, admin, owner, guest_viewer, guest_editor (the author only touches their own). |
| Create a shared annotation | editor, admin, owner, guest_editor. |
| Edit or delete a shared annotation | editor, admin, owner, guest_editor (any shared annotation, not only their own). |
| Change visibility between personal and shared | The author, only when also allowed to create shared (promotion). Demoting shared to personal is allowed to the author, and the annotation then disappears for others. |
| Read | Per the visibility rules above. |
If an author loses access to the site, their personal annotations are retained but invisible; shared ones stay. Deleting the user's account anonymizes the author (Section 11) but keeps shared annotations.
9.15.4 UI and API behaviour #
- Create: overflow menu "Add annotation" or the tooltip action. Modal fields: Date (defaults to the bucket date or today), Title, Description, Visibility (radio Personal/Shared; Shared disabled for roles that cannot create it).
- List:
GET /annotations?from=YYYY-MM-DD&to=YYYY-MM-DDreturns{"data": [...], "meta": {"next_cursor": null, "limit": 200}}for annotations the caller can see, filtered onoccurred_onand ordered byoccurred_onascending thencreated_at. The dashboard sends the visible chart range (limit up to 200 per call; ranges with more than 200 annotations page via the standard cursor). - Validation failures use
validation_failedwithdetails.fields. An unknown or invisible annotation id returns404 not_found. - Any create, edit or delete is recorded in
audit_logonly for shared annotations (action namesannotation.created,annotation.updated,annotation.deleted).
9.16 Filters #
9.16.1 Filter model #
A filter is a triple [operator, dimension, values] where values is a non-empty array. Active filters form a list. Rules:
- Across filters: AND. A row/session/event must satisfy every filter.
- Within a filter: OR across
valuesforisandcontains; for the negations (is not,does not contain) the row must satisfy the negation for every value (that is, none of the values may match). - Several filters may use the same dimension (for example
Page contains /blogANDPage does not contain draft). The UI merges a new filter into an existing chip only when dimension and operator are identical; otherwise it creates another chip. - Limits: at most 10 filters and at most 25 values per filter; each value at most 500 characters. Exceeding is rejected in the UI (the Apply button is disabled with an inline message) and server-side with
422 invalid_filteranddetails.fields.
9.16.2 Operators #
| UI label | Query operator (Section 13) | Semantics |
|---|---|---|
| is | is (or matches when a value contains *) |
Exact, case-sensitive equality. For path-like dimensions (event:page, visit:entry_page, visit:exit_page) a * in a value turns the match into a wildcard match: * matches any characters except /; ** matches any characters including /; the match is anchored to the whole value. Before a path value is sent it is normalized with the WHATWG path percent-encode set (non-ASCII characters and spaces are percent-encoded, existing %XX sequences are kept with uppercase hex), so /café matches the stored /caf%C3%A9; the UI displays decoded values. |
| is not | is_not (or matches_not) |
Negation of the above. |
| contains | contains |
Case-insensitive substring match (the dashboard sends case_sensitive: false; the engine escapes %, _ and \ and binds the value as a parameter, Section 13.5.1). Wildcards are not interpreted. |
| does not contain | contains_not |
Negation of contains. |
9.16.3 Dimension catalogue #
| Filter label | Dimension | Operators allowed | Value input | Absent value label |
|---|---|---|---|---|
| Page | event:page |
is, is not, contains, does not contain | Autocomplete from top pages in range | n/a |
| Entry page | visit:entry_page |
all four | Autocomplete | n/a |
| Exit page | visit:exit_page |
all four | Autocomplete | n/a |
| Hostname | event:hostname |
all four | Autocomplete | n/a |
| Source | visit:source |
all four | Autocomplete | Direct / None |
| Channel | visit:channel |
is, is not | Select from the 18 channels | n/a |
| Referrer | visit:referrer |
all four | Autocomplete | Direct / None |
| UTM medium, source, campaign, term, content | visit:utm_* |
all four | Autocomplete | (none) |
| Country | visit:country |
is, is not | Searchable list of countries | Unknown |
| Region | visit:region |
is, is not | Autocomplete (names, stored as codes) | Unknown |
| City | visit:city |
is, is not | Autocomplete (names, stored as GeoNames IDs) | Unknown |
| Browser | visit:browser |
all four | Autocomplete | Unknown |
| Browser version | visit:browser_version |
is, is not | Autocomplete (requires a Browser is filter, auto-added) |
Unknown |
| Operating system | visit:os |
all four | Autocomplete | Unknown |
| OS version | visit:os_version |
is, is not | Autocomplete (requires an OS is filter, auto-added) |
Unknown |
| Screen size | visit:screen_size |
is, is not | Select Mobile, Tablet, Laptop, Desktop | Unknown |
| Device | visit:device |
is, is not | Select Desktop, Mobile, Tablet | Unknown |
| Goal | event:goal |
is, is not | Select from the site's goals by display name | n/a |
| Property | event:props:<key> |
all four | Step 1 choose an allowlisted key, step 2 autocomplete of values | (none) |
| Segment | segment |
is | Select a segment (9.18); no contains; not available to shared-link and embed viewers |
n/a |
Operators not allowed for a dimension are not offered by the UI and are rejected by the API with 422 unsupported_filter_operator.
9.16.4 Add filter UI #
- The Add filter button (or
F) opens a menu listing dimension groups: Page, Source, Location, Screen (Screen size and Device), Browser, Operating system, Goal, Property, UTM, Segment (each group expands to its dimensions). - After choosing a dimension the modal shows: an operator select (only allowed operators), a multi-value combobox, and Apply. Autocomplete requests
POST /querywith the dimension, the current date range and the current filters excluding filters of the same dimension, limit 100, search text as acontainsconstraint, debounce 250 ms, minimum 0 characters (shows the top values on open). - Free-text values (not in the suggestions) are accepted for
is,containsand wildcard use; selecting a suggestion stores its raw value. - Apply updates the URL (9.17) and refreshes every widget. Escape cancels.
- A chip reads
Page is /pricing,Country is Germany or France,Source is not Google. Chips have an edit action (reopens the modal) and a remove action. A removed filter that had dependent filters (browser version) removes them too. - Filters reset when switching sites; the filter list is part of the URL, not remembered.
9.16.5 Special values #
(none)is the token for an empty or missing value in the UI, URLs, segments and engine filters; the engine treats it as the reserved clause "absent" (Section 13.4.5). A stored value that is literally the text(none)(for example a campaign sent asutm_campaign=(none)) cannot be told apart from an absent value and is reported in the(none)row; this limitation is accepted and documented in Section 13.4.5.- Goal filters use the goal display name (unique per site, Section 10.2.2).
- Property filter values are matched as strings. Empty-string property values do not exist: ingestion drops properties with empty values (Section 10.3.2).
9.16.6 Filter validation #
The server validates each filter against the catalogue: unknown dimension, operator not allowed, empty values, too many filters or values, a version filter without its parent, a property key not on the allowlist (422 property_not_allowed), or a segment id that does not exist on this site (404 not_found, the same response for a segment of another site or one the caller may not use). A malformed filter in a URL is ignored by the dashboard (not sent) and a toast says "Some filters in this link were invalid and were removed." Invalid filters in direct API calls get 422 invalid_filter.
9.17 URL State Encoding #
The dashboard state lives in the query string of /sites/:domain (and of each report modal route) so every view is linkable and the browser Back button works (each state change is a pushState except typing/search which uses replaceState).
| Parameter | Values | Default / omitted when |
|---|---|---|
period |
realtime, day, 24h, 7d, 28d, 30d, 91d, month, year, 12mo, all, custom |
Omitted when equal to the remembered or default period (30d) |
date |
YYYY-MM-DD (day, month or year anchor; for month and year only the first day of the unit is needed) |
Omitted for "current" |
from, to |
YYYY-MM-DD |
Only with period=custom |
comparison |
previous_period, year_over_year, custom |
Omitted when off |
compare_from, compare_to |
YYYY-MM-DD |
Only with comparison=custom |
match_day_of_week |
true |
Omitted when false |
interval |
minute, hour, day, week, month |
Omitted when auto |
metric |
visitors, visits, pageviews, views_per_visit, bounce_rate, visit_duration, events, conversion_rate, total_revenue, average_revenue |
Omitted when visitors |
f |
Repeated parameter, one per filter | Omitted when none |
with_imported |
true, false |
Omitted when true (default on) |
full_urls |
true |
Omitted when false |
Filter encoding: each f value is operator,dimension,value1|value2|... where operator is one of is, is_not, contains, contains_not; dimension is the short key (the identifier without the scope prefix: page, entry_page, exit_page, hostname, source, channel, referrer, utm_medium, utm_source, utm_campaign, utm_term, utm_content, country, region, city, browser, browser_version, os, os_version, device, screen_size, goal, segment, or props:<key> with the key percent-encoded). Each value, and the property key, is encoded with encodeURIComponent, so commas, pipes and the percent sign inside values are escaped; the whole f value is then percent-encoded once more by the URL serializer. Example: ?period=7d&f=is,country,DE|FR&f=contains,page,%2Fblog&f=is,props:plan,Pro.
Rules:
- Parameter order is canonical:
period,date,from,to,comparison,compare_from,compare_to,match_day_of_week,interval,metric,with_imported,full_urls, thenfin the order shown in the chips. Unknown parameters are preserved but ignored. - A modal route keeps the same parameters;
/sites/:domain/<report-slug>slugs aresources,campaigns,pages,locations,devices,goals,properties. - Invalid values for any parameter are dropped individually (period falls back to the default). Duplicate non-
fparameters keep the first occurrence. - Maximum URL length the dashboard generates is 8,000 characters; filters beyond that are not added (the Add filter dialog shows "Too many filter values for one link."). With the 10-filter and 25-value limits this cannot occur for ordinary values.
- Shared-link URLs carry the same parameters minus anything the link forbids: the
/api/share/v1mirror rejects everysegmentfilter, so the SPA dropsfvalues with dimensionsegmenton shared and embed pages, and a segment-restricted link's fixed segment is applied server-side (Section 12).
9.18 Segments #
9.18.1 Model #
A segment is a named, saved set of filters (never a date range). Persisted in segments (Section 5).
| Field | Type | Rules |
|---|---|---|
id |
uuid | Server generated. |
site_id |
uuid | From {domain}. |
owner_user_id |
uuid or null | The owning user of a personal segment; NULL for site segments (Section 5.6.5). |
segment_type |
enum personal, site |
Default personal. |
name |
string | Required, trimmed, 1 to 120 characters, unique case-insensitively among the owner's personal segments of the site, and unique case-insensitively among the site's site-type segments. |
filters |
array of [operator, dimension, values] |
1 to 10 filters, validated exactly as in 9.16.6; a segment filter is not allowed inside a segment (no nesting). |
created_at, updated_at |
timestamptz | Server managed. |
Limits (Section 5.16): 100 personal segments per user per site and 100 site segments per site. Exceeding returns 409 limit_reached with details.limit_name personal_segments or site_segments. A name conflict returns 409 segment_name_taken.
9.18.2 Personal vs site segments and permissions (summary; authoritative matrix in Section 11) #
| Action | Personal segment | Site segment |
|---|---|---|
| See and apply | Owner only | Every member and guest of the site |
| Create | Any role with view access (viewer, guest_viewer and above) |
editor, admin, owner, guest_editor |
| Edit, rename, delete | Owner | editor, admin, owner, guest_editor |
| Promote personal to site | The owner of the segment, when also allowed to create site segments | n/a |
| Demote site to personal | Not allowed | Not allowed |
9.18.3 Behaviour #
- Segment menu (button in the header, key
S): lists "Site segments" and "Personal segments" groups, each item with the number of filters and an overflow (Edit, Rename, Delete, Promote). Items apply by adding asegmentfilter chip (Segment is Paid campaigns). Only one segment filter may be active at a time; picking another replaces the first. The menu also offers Save current filters as segment, enabled when at least one non-segment filter is active, which opens a modal with Name, Type (Personal/Site, Site disabled for roles that cannot create it). - Resolution: the server resolves a
segmentfilter into the segment's filters at query time and ANDs them with the other filters. If the segment's filters reference a property no longer allowlisted or a goal that no longer exists, the query still runs (the missing goal matches nothing) and the chip shows a warning icon "This segment references a missing goal or property." - Editing a segment replaces its filters for all users and all uses, including shared links restricted to it. When editing a site segment used by shared links the confirm dialog states "N shared links use this segment."
- Deleting a segment referenced by any shared link is rejected with
409 segment_in_useanddetails.shared_link_ids; the UI points to the Shared links list (Section 12). Deleting an unreferenced segment removes it; URLs referencing it then show "This segment no longer exists." and the filter is dropped. - Goal rename cascade: renaming a goal rewrites matching
event:goalvalues inside segments of the same site in the same transaction (Section 10.2.6). - Segment-restricted shared links: a shared link may be restricted to one site segment; personal segments cannot be used. Section 12 defines creation, enforcement and what the viewer sees; this section guarantees that the engine ANDs the segment filters on the server for every
/api/share/v1call. The locked filters apply to every mirrored endpoint (queries, filter-value suggestions, realtime, funnels, journeys) and count as active filters for imported data (9.21); viewers cannot send asegmentfilter (9.1.3). - Operator view: super admins can read segments only through impersonation (Section 15).
9.19 Sites List #
9.19.1 Page #
Route /sites is the post-login landing page (the app shell in Section 4 redirects / here). It lists every site the user can see across all teams where the user is a member, plus sites where the user is a guest. A team filter dropdown ("All teams", then each team by name) narrows the list; the selection is remembered in localStorage.
Each site renders as a card in a responsive grid (1 column under 640 px, 2 under 1024 px, 3 above):
| Element | Content |
|---|---|
| Avatar and domain | First-letter avatar on a deterministic color derived from the domain (no external favicon requests), the domain as the link to /sites/:domain. A small team name badge appears under the domain when the user belongs to more than one team or is a guest. |
| Visitors | Unique visitors in the last 24 hours (rolling), formatted per 9.2.4, with a delta badge versus the preceding 24 hours (rules of 9.4.3, relative percent). |
| Pageviews | Total pageviews in the last 24 hours. |
| Sparkline | Hourly unique visitors for the last 24 hours (24 points), an 8 px high unlabeled line with a light fill, aria-hidden; the card's text alternative gives the values. |
| Status | Waiting for first pageview (no events ever, shown as a gray pill) or Receiving data (green) based on the most recent accepted event within 24 hours; a site with older data but none in the last 24 hours shows No data in 24h (neutral). |
| Pin button | Star toggle (9.19.4). |
| Overflow | Open dashboard, Settings (roles that can), Copy snippet. |
The card as a whole is a link; the pin and overflow controls stop propagation.
9.19.2 Sorting and search #
| Control | Behaviour |
|---|---|
| Sort | Select with Traffic (default: visitors in the last 24 hours descending, ties by domain ascending) and Name (domain ascending, case-insensitive). Remembered per user in localStorage. Pinned sites are always listed first, in the same sort, in a "Pinned" section; unpinned sites follow in an "All sites" section. |
| Search | Text input filtering by case-insensitive substring of the domain, debounced 200 ms, applied client-side to the loaded page and, when more pages exist, server-side with ?q= (1 to 100 characters). The / key focuses the search box. |
| Pagination | Cursor, default limit 50, max 200, per Section 4. The page loads 50 cards then a "Load more" button. |
9.19.3 API #
GET /api/app/v1/sites-overview?limit=&cursor=&q=&sort=traffic|name&team_id=
Response:
{
"data": [
{
"domain": "example.com",
"team": {"id": "uuid", "name": "Acme's team"},
"role": "editor",
"pinned": true,
"status": "receiving",
"last_event_at": "2026-11-03T09:41:12Z",
"visitors_24h": 1342,
"visitors_prev_24h": 1210,
"pageviews_24h": 3820,
"sparkline": [12, 8, 9, 20, 31, 40, 55, 61, 70, 72, 80, 90, 88, 77, 65, 60, 51, 45, 40, 35, 30, 25, 20, 18]
}
],
"meta": {"next_cursor": null, "limit": 50}
}statusis one ofwaiting,receiving,idle(mapping above).sparklinehas exactly 24 integers, oldest first, hour buckets in the site's timezone ending with the current partial hour.sort=trafficpagination is stable althoughvisitors_24hchanges live: the first request computes the full ordering and stores it in the 60-second cache entry (below); the cursor (Section 4.5) encodes the sort tuple of the last returned row (pinned flag,visitors_24h, domain), and pages are cut from that snapshot, so a live change never reorders or duplicates rows between pages. After the snapshot expires, paging continues by keyset on freshly computed values (a site may then move between pages, but no request fails).- The stats for the whole page are computed with one batched query grouping by
site_id(not one per site), executed with the user's visible site ids. The result is cached in Valkey for 60 seconds per (user, parameters) key. - Sites of a team in its 7-day deletion window are not listed: they return
404to everyone but the owner, who finds the restore screen under the team settings (Section 11.14.5). Deleted sites no longer exist (site deletion is immediate, Section 5.5.1).
9.19.4 Pinning #
Pins are per user. They are stored in the user's users.preferences jsonb under the key pinned_sites, an array of site ids of at most 20 entries (enforced by a CHECK, Section 5); the list returns them as pinned. A pin whose site was deleted is ignored and removed on the next write. POST /me/pinned-sites with {"domain": "example.com"} pins the site (idempotent: always 200 with {"data": {"domain": "example.com", "pinned": true}}, also when it was already pinned); DELETE /me/pinned-sites/{domain} unpins (idempotent, 204). Limit (Section 5.16): 20 pinned sites per user; the 21st returns 409 limit_reached with details.limit_name pinned_sites. A pin for a site the user lost access to is ignored and removed lazily on the next write.
9.19.5 Empty states #
| Condition | UI |
|---|---|
| User has no sites and can create one | A card titled "Add your first site" with an Add site button that opens the site creation flow (Section 7). |
| User has no sites and cannot create (guest only with none shared, or at the plan site limit on hosted) | "You don't have access to any sites yet." plus, on hosted at the site limit, the 409 site_limit_reached explanation and a link to /teams/:teamId/billing (Section 15). |
| Search returns nothing | "No sites match ''." with a Clear search link. |
| Team filter returns nothing | "No sites in this team." |
9.20 Consolidated All-Sites View #
9.20.1 Scope #
The consolidated view aggregates headline metrics across every site the user can view, optionally restricted to one team. Route: /all-sites (linked from the sites list header as "All sites overview" and from the site switcher). Shared links and embeds have no consolidated view.
9.20.2 Layout #
- Header: team scope select ("All teams" default), date range button, comparison select, imported toggle. Presets and comparison follow 9.12 and 9.13, except that Realtime and Last 24 hours are not offered, and every preset is resolved per site in that site's own timezone (rule 3 in 9.20.3).
- Top stats row: Unique visitors, Total visits, Total pageviews, Views per visit, Bounce rate, Visit duration (same tile behaviour as 9.4, deltas included).
- Main chart: the selected tile's metric summed (or weighted) over all included sites, with the comparison overlay of 9.5 and the intervals
day,weekandmonth(minute and hour buckets are not offered because the included sites' timezones do not share hour boundaries); for a one-day range the chart is replaced by the note "Select a longer range to see a trend." No annotations. - Sites table: one row per site with columns Site, Visitors, Visits, Pageviews, Bounce rate, Visit duration, plus a sparkline of the selected metric, each with a delta when comparison is on. Sortable by every column (default Visitors descending, ties by domain), searchable by domain, paged by 50 with "Load more", row click opens
/sites/:domainwith the same period.
No filters, segments, goals, funnels or report panels exist in this view; the filter button is absent.
9.20.3 Aggregation rules #
| Metric | Aggregate |
|---|---|
visitors |
Sum of per-site visitors. Visitor IDs are salted per site (Section 8), so the same person on two sites counts twice. The tile tooltip reads "Sum of unique visitors per site; a person visiting several of your sites is counted once per site." |
visits, pageviews |
Sum. |
views_per_visit |
sum(pageviews) / sum(visits). |
bounce_rate |
sum(bounced_sessions) / sum(visits) * 100 (visit-weighted, not the mean of per-site rates). |
visit_duration |
sum(duration) / sum(visits) (visit-weighted, duration as in 9.2.2). |
Rules:
- Only sites on which the caller has at least
viewer(or guest viewer) access are included, re-evaluated on every request. Sites of a team in unlicensed state (hosted) are included (dashboards stay readable, Section 15); sites of a team in its 7-day deletion window are not (Section 11.14.5). - Maximum 100 sites per request (the limit of
executeConsolidatedQuery, Section 13.5.1). A user with more than 100 visible sites must pick a team scope; if the chosen scope still exceeds 100, the view aggregates the 100 sites with the highestvisitors_24hand shows the banner "Showing the 100 busiest sites. Narrow by team to see the rest." - The query is executed by
executeConsolidatedQuery(Section 13.5.1): per-site statements grouped bysite_id, each resolved with that site's own IANAtimezone, and summed in the application by local calendar date (a day, week or month bucket). "Today" therefore means each site's own today, and the bucket labelled Nov 3 contains every included site's Nov 3. The per-site rows of the sites table equal the numbers on each site's own dashboard for the same period. - The range and comparison rules of 9.12 and 9.13 apply per site; the interval rules of 9.5.2 apply with the restriction to
day,weekandmonth(any otherintervalreturns422 invalid_interval). - Imported data is added per site according to 9.21 when the toggle is on.
POST /api/app/v1/consolidated/querybody:{"team_id": uuid|null, "metrics": [...], "date_range": ..., "interval": "day", "group_by_site": false, "include": {"comparisons": ..., "imports": true}, "pagination": {"limit": 50, "offset": 0}, "order_by": [["visitors","desc"]]};group_by_site: truereturns one result row per site withdimensions: [domain]. Response shape equals Section 13{results, meta, query}. Errors:404 not_foundifteam_idis not a team the user belongs to;422 invalid_date_range.- Cached for 60 seconds per (user, normalized body) in Valkey, 10 minutes when the whole range is in the past.
9.21 Imported Data Toggle #
Imported data is the data loaded by CSV import or GA4 import (Section 14) into the imported_* tables (Section 6). The decision whether imported rows are merged into a result belongs to the query engine (Section 13.4.10); this section defines what the dashboard shows.
- Visibility: the toggle "Include imported data" appears in the header only when the site has at least one import with status
completed; otherwise it is absent and nowith_importedparameter is generated. - Default: on. State is in the URL (
with_imported=falsewhen off). - What imports cover:
visitors,visits,pageviews,bounce_rate,visit_duration,views_per_visitin the main chart and tiles; breakdowns of sources (channel, source, referrer, UTM), pages (includingtime_on_page), entry pages, exit pages, locations (country, region, city), device class, browsers, operating systems, and custom events as goals (event goals without property conditions only). Never imported: screen size,scroll_depth,exit_rate, revenue metrics, properties, funnels, journeys, the realtime view and current visitors. - Merge rule: imported rows are used only for dates strictly earlier than the site's first native day, which is the site-timezone date of its earliest native event (Section 13.4.10 rule 2); on and after that day only native data is used. The same rule applies to the comparison range. A site with no native events at all uses imported rows for every imported date.
- When imports are not merged: the engine merges imports only when all of these hold (Section 13.4.10): the site has a completed import; the interval is
day,weekormonth; the query's dimensions and filters all map to one imported table; every requested metric is supported by that table; and at least one goal in scope can be matched. The locked segment filters of a segment-restricted shared link count as filters. Otherwise the query is answered from native data only, the badge below is not shown, and the toggle's tooltip explains why using the reason the engine returns inmeta.imports_skip_reason(no_imports,no_imported_data_in_range,unsupported_granularity,incompatible_dimensions,unsupported_metric,unsupported_goal). In the Realtime view the toggle is hidden (9.12.2); funnels and journeys never use imports. - Partial goals: when some goals in scope can contribute only native data the response carries the warning
imports_partial_goalsand the Goals panel shows an info icon "Imported data covers event goals without property conditions only." - Indicator: when merged results include imported rows, the chart header shows a small badge "Includes imported data (, )"; the engine reports it in
meta.imports_included: true. - Shared links and embeds: these views include imported data only when the link's
include_importsoption is on (default on, Section 12); when it is off their requests sendinclude.imports = falseand the toggle is not shown. - Annotations: show a dashed marker for the first and last imported day on the chart baseline only when the toggle is on and imports were merged.
9.22 Empty States #
| Situation | UI behaviour |
|---|---|
| No events ever received (site just created) | Banner "Waiting for your first pageview." plus the tracking snippet panel and framework guides entry point from Section 7. The banner polls GET /sites/{domain}/tracking/status (Section 7.15.2) every 3 seconds (Section 7.15.2) until receiving is true; on the first event the banner flips to "We're receiving data!" for 5 seconds and the dashboard reloads its queries. Tiles show 0; panels show their "No data yet" state. |
| Events exist, none in the selected range | Tiles 0 (ratio tiles -), chart "No visitors in this period.", panels "No data yet for this period." The date menu shows a hint "Latest data: " computed from last_event_at. |
| No data with filters | Panels per 9.6.6; the chart reads "No visitors match these filters." and the filter chips row shows "Clear all". |
| Imported-only site (imports completed, no native events) | The dashboard behaves normally using imported data (9.21); the waiting banner is not shown. |
| Unlicensed or over-cap team (hosted) | Dashboard readable; banner "Tracking is paused for this team. Activate a license." linking to Section 15's activation flow, or the over-cap banner defined there. |
| Site not found / no access | 404 page "This site doesn't exist or you don't have access to it." (identical text for both, to avoid leaking existence) with a link to /sites. |
| Site of a team in its 7-day deletion window | The 404 page above, identical to an unknown site; the team owner finds the restore screen under the team settings (Section 11.14.5). |
Query engine unavailable (503 unavailable) |
Each widget shows its own error with Retry; a banner "Reports are temporarily unavailable." appears when at least three widgets failed within 10 seconds. |
9.23 Performance, Caching and Timeouts #
9.23.1 Targets #
p95 server time for a dashboard query: 300 ms for a 30d range on a site with up to 5 million events in range; p99 under 2 seconds. First dashboard paint with skeletons under 1 second on a cached SPA bundle.
9.23.2 Server cache (Valkey) #
- Key:
qs:stats:{site_id}:{epoch}:{sha256(normalized_query_json + scope)}wherenormalized_query_jsonhas sorted keys, resolved absolute ranges (9.1.1) and deduplicated filter values;scopeis the caller's restriction (segment id for restricted links, else empty) and the import toggle outcome;epochis the site's configuration epoch. - Epoch: an integer at
qs:epoch:{site_id}incremented on any change that alters query results: goal create/update/delete, property allowlist change, segment edit (for segment-scoped keys), import completion or deletion, exclusions change,allowed_hostnameschange, site data purge, site timezone change, funnel change. Incrementing the epoch invalidates all of the site's cached entries without scanning. - TTL:
| Query kind | TTL |
|---|---|
Realtime period and current-visitors |
5 seconds |
Any range whose end is today or later (includes 24h, today, rolling presets) |
60 seconds |
| Range entirely before the start of today in the site timezone | 10 minutes |
| Autocomplete queries (9.16.4) | 60 seconds |
sites-overview and consolidated |
60 seconds (10 minutes when the range is entirely in the past) |
| Label lookups | 24 hours |
- Single flight: concurrent identical cache-miss requests share one execution using a Valkey lock
qs:lock:{key}with a 10 second expiry; waiters poll the cache every 100 ms for up to 10 seconds and then run the query themselves. - Cached payloads are stored gzip-compressed JSON; payloads above 5 MB are not cached.
- The cache never stores shared-link results under a user's key; shared-link keys include the link id in
scope.
9.23.3 Timeouts and limits #
| Limit | Value |
|---|---|
| ClickHouse per-query limits | The constants of Section 13.4.11 (max_execution_time 30 seconds, max_memory_usage 4 GB, group-by and result-row limits); they are not operator settings |
API request timeout for POST /query |
35 seconds; on exceed 503 with code query_timeout |
| Browser abort | 40 seconds client side; shows the timeout message in 9.6.6 |
| Rows returned per query | 10,000 maximum (Section 13) |
| Dashboard query rate limit | Policy dashboard_query (Section 17.6): 240 requests per minute per authenticated user, each query inside a batch counted separately, answered with 429 rate_limited and Retry-After. Shared-link and public-dashboard viewers are limited by policy share_view (Section 17.6), and each query in a batch counts as one request |
| Concurrent in-flight dashboard queries per browser tab | 6 (the rest queue in FIFO order; TanStack Query networkMode and a client semaphore) |
| Autocomplete | 100 values max per response |
9.23.4 Client caching and refresh #
- TanStack Query keys include the full normalized state (route, period, filters, comparison, interval, metric, imports flag).
staleTime: 30 seconds for ranges including the present, 5 minutes for ranges entirely in the past.gcTime: 5 minutes.keepPreviousDatais used on range/interval changes so charts do not flash. - No background auto-refresh except Realtime (10 s, 9.14) and the current-visitors pill (30 s). A manual reload is available through the header overflow menu item "Refresh data", which invalidates the dashboard's queries.
- Refetch on window focus is enabled only when the cached data is older than 5 minutes for present-including ranges.
- Details modal pages are cached per (tab, sort, search, offset) key.
9.23.5 Degradation #
If a query exceeds 10 seconds of server time the response includes meta.slow: true (informational); the dashboard keeps the previous data visible with a small spinner. The engine rejects a query that exceeds the memory or group-by limits of Section 13.4.11 with 422 query_too_large ("Narrow the date range or add filters."); no dashboard widget issues such a query for ranges up to 10 years unless filters are extremely broad.
9.24 Dashboard Permission and Error Summary #
| Scenario | Result |
|---|---|
| Unauthenticated request | 401 unauthenticated; the SPA redirects to login preserving the target URL. |
| Authenticated but no access to the site, or unknown site | 404 not_found (never 403, to avoid leaking site existence). |
| Authenticated with access but the action needs a higher role | 403 forbidden with details.required_role. |
| Shared-link viewer calls a route outside the mirrored set of Section 12.2.3 | 404 not_found (never 403). |
| Invalid filter or query body | 422 with the codes below, or 400 validation_failed for malformed JSON/types. |
Dashboard-specific error codes:
| Code | Status | When |
|---|---|---|
invalid_date_range |
422 | Start after end, range longer than 3,660 days, before 1970-01-01, unparsable dates, comparison before 1970. |
invalid_interval |
422 | Interval not allowed for the range (9.5.2). The browser never sends one; this protects direct callers. |
invalid_comparison |
422 | Unknown mode, custom without both dates, match_day_of_week with off or custom, or comparison used with realtime or all. |
invalid_filter |
422 | Unknown dimension, empty values, too many filters or values, over-long value, nested segment, parent filter missing. |
unsupported_filter_operator |
422 | Operator not allowed for the dimension. |
property_not_allowed |
422 | Property key not on the site's allowlist (Section 10.3). |
not_found |
404 | Unknown, invisible or inaccessible segment or annotation id (one response for "does not exist" and "not yours"). |
segment_name_taken |
409 | Duplicate name (case-insensitive). |
segment_in_use |
409 | Deleting a segment used by shared links. |
limit_reached |
409 | A limit of Section 5.16 (annotations, personal or site segments, pinned sites), with details.limit_name, details.limit and details.current (9.15.1, 9.18.1, 9.19.4). |
unsupported_combination |
422 | The engine rejects the metric, dimension and filter combination (9.6.6). |
query_timeout |
503 | Execution exceeded 35 seconds. |
query_too_large |
422 | Scan guard exceeded. |
Audit log entries (catalogue in Section 11.20, table in Section 5) are written for segment create/update/delete of site segments (segment.created, segment.updated, segment.deleted) and shared annotations (9.15.4). Personal segments and personal annotations are not audited.
10. Goals, Custom Properties, Funnels, User Journeys & Revenue #
This section defines everything that turns raw events into conversions: goals, custom properties, revenue, funnels and user journeys, including their configuration, validation, calculation semantics, reports and APIs. Metric formulas (conversion_rate, group_conversion_rate, total_revenue, average_revenue, events, visitors) are canonical in Section 9.2; this section applies them. Table definitions are owned by Section 5 (goals, funnels, funnel_steps, custom_property_allowlist); the event columns are owned by Section 6; event ingestion and currency rates are owned by Section 8; the generic query engine is owned by Section 13.
10.1 Overview, Routes and Ownership #
10.1.1 Concepts #
| Concept | Definition |
|---|---|
| Goal | A named rule that marks certain events as conversions: a custom event, a pageview matching a path pattern, or a scroll-depth threshold on pages matching a pattern. |
| Custom property | A key/value pair (string to string) attached to a pageview or custom event. Only allowlisted keys appear in reports and can be used in filters, goal conditions and funnel conditions. |
| Revenue goal | A custom-event goal with a currency; matching events that carry a revenue value contribute to revenue metrics. |
| Funnel | An ordered list of 2 to 8 steps evaluated per session, showing how many visitors reach each step. |
| User journey | An aggregated visual of the sequences of pages visitors view within sessions, starting from or ending at a chosen page. |
10.1.2 Dashboard API routes #
All routes are under /api/app/v1/sites/{domain}, use the cookie session and CSRF rules of Section 4, return the envelopes of Section 4.4, and are subject to the role requirements in 10.7.
| Method and path | Purpose |
|---|---|
GET /goals |
List goals (cursor pagination, default limit 50, max 200; ordered by created_at ascending). |
POST /goals |
Create a goal. |
GET /goals/{id} |
Read one goal with usage counts. |
PATCH /goals/{id} |
Update a goal (editable fields in 10.2.6). |
DELETE /goals/{id} |
Delete a goal. |
GET /goals/suggestions |
Discovered events and pages for the goal creation UI (10.2.7). |
GET /properties |
List allowlisted properties. |
POST /properties |
Add a property key to the allowlist. |
DELETE /properties/{key} |
Remove a key (key percent-encoded in the path). |
GET /properties/discovered |
Keys seen in the last 30 days that are not allowlisted. |
GET /funnels, POST /funnels, GET /funnels/{id}, PATCH /funnels/{id}, DELETE /funnels/{id} |
Funnel management. |
POST /funnels/{id}/results |
Evaluate a saved funnel (10.5.13). |
POST /funnels/preview |
Evaluate an unsaved funnel definition from the editor. |
POST /journeys |
Compute a user-journey diagram (10.6.9). |
Goal-driven reports (Goals panel, Properties panel) use POST /sites/{domain}/query (Section 9.1.2); they are not separate routes. An unknown goal id, funnel id or property key, and any request for a site the caller cannot access, returns 404 not_found. POST /funnels/{id}/results, POST /funnels/preview and POST /journeys change no state and carry the route-registry flag readOnly (Section 9.1.2).
10.2 Goals #
10.2.1 Goal types #
type |
Marks as a conversion | Required fields |
|---|---|---|
event |
Every accepted custom event whose name equals event_name (case-sensitive) and, if conditions are set, whose properties match. |
event_name |
pageview |
Every pageview event whose pathname matches page_path (10.2.3) and, if conditions are set, whose properties match. |
page_path |
scroll |
A page visit (one session on one pathname) whose maximum recorded scroll depth reaches scroll_threshold percent, on a pathname matching page_path. |
page_path, scroll_threshold |
10.2.2 Fields and validation #
| Field | Type | Applies to | Rules |
|---|---|---|---|
id |
uuid | all | Server generated (uuidv7()). |
type |
enum event, pageview, scroll |
all | Required on create; immutable. |
display_name |
string | all | Optional on create; trimmed; 1 to 120 characters; no control characters; unique per site, case-insensitive, across all goal types. This is the value used by the event:goal filter and dimension. When omitted the default is generated (below) and, if that collides, a (2), (3) suffix is appended. Default names: event goals use event_name; pageview goals use Visit {page_path}; scroll goals use Scroll {threshold}% on {page_path}. The default is truncated to 120 characters. |
event_name |
string | event |
Required for event; trimmed; 1 to 120 characters; case-sensitive; must not equal the reserved names pageview or engagement. Must be null for other types. |
page_path |
string | pageview, scroll |
Required; 1 to 2,000 characters; must start with / or **; exact grammar in 10.2.3. Must be null for event. |
scroll_threshold |
integer | scroll |
Required for scroll; 1 to 100 inclusive; the UI preselects 75 and offers quick picks 25, 50, 75, 90, 100. Null for other types. |
property_conditions |
array of {key, value} |
event, pageview |
0 to 3 items; see 10.2.5. Stored sorted by key (the service sorts before writing), so the uniqueness check below compares arrays. Always [] for scroll. |
revenue_currency |
ISO 4217 code or null | event only |
Null means "not a revenue goal". When set: uppercase 3 letters, must be in the supported currency list of Section 8; immutable after creation. |
created_at, updated_at |
timestamptz | all | Server managed. |
Cross-field uniqueness (409 goal_exists): a site cannot have two goals with the same definition tuple, defined as (type, event_name, page_path compared after normalization, scroll_threshold, the set of property_conditions compared as a set of key/value pairs ignoring order). Two goals for the same event name with different conditions are allowed. A second goal with the same display name returns 409 goal_name_taken.
Limit (Section 5.16): 100 goals per site; the 101st returns 409 limit_reached with details.limit_name goals, details.limit 100 and details.current.
10.2.3 Pageview and scroll path patterns #
page_path is a pattern matched against the stored pathname of the event (Section 8: query string removed, hash removed unless hash routing is enabled).
Grammar and normalization:
- After trimming, the pattern must begin with
/or with**(the pattern**alone matches every page). Any other first character is rejected withvalidation_failedonpage_path. - Characters: any characters valid in a URL path;
*is the wildcard; no other character is special. Spaces and non-ASCII characters are percent-encoded by the normalization in rule 8 before validation, so they never cause a rejection. *matches any sequence of characters except/(including the empty sequence).**matches any sequence including/. Three or more consecutive asterisks are rejected. Matching is anchored to the entire pathname and is case-sensitive.- Trailing slashes are significant:
/pricingdoes not match/pricing/. The UI shows a hint "Use/pricing*to match both." - Equivalent regular expression (reference implementation in
@quietstat/shared): escape regex metacharacters, replace**with.*and*with[^/]*, wrap in^...$. - Patterns without any
*are compared by equality and use the fast path in the engine. - Patterns are stored trimmed and percent-encoded per rule 8; no other normalization is applied, and uniqueness checks (10.2.2) compare the stored strings.
- Percent-encoding. Pathnames are stored percent-encoded (Section 8.10.1), so a pattern is normalized with the WHATWG path percent-encode set before validation and storage: non-ASCII characters and spaces are percent-encoded, existing
%XXsequences are kept with uppercase hex./caféis stored as/caf%C3%A9and matches the stored pathname; the UI displays decoded values. - Wildcard budget. A pattern may contain at most 8 wildcard tokens (each run of
*or**counts as one); more is rejected withvalidation_failedonpage_path. The limit keeps pattern evaluation (RE2 in ClickHouse) linear and cheap.
Examples (informative): /blog/* matches /blog/post-1 but not /blog/2026/post-1; /blog/** matches both; /docs/*/intro matches /docs/v2/intro.
10.2.4 Matching and counting semantics #
| Type | Candidate rows | Match test | One conversion is |
|---|---|---|---|
event |
events rows with name = event_name |
Conditions all satisfied (10.2.5). | One event row. |
pageview |
events rows with name = 'pageview' |
Pathname matches page_path; conditions all satisfied. |
One pageview event. |
scroll |
events rows with name = 'engagement' |
Pathname matches page_path and scroll_depth >= scroll_threshold |
One distinct page visit, identified by (session_id, pathname); the earliest qualifying engagement event of that page visit represents it, and its timestamp is the conversion time. |
For all goals, per Section 9.2.2: visitors is the count of distinct visitors with at least one conversion ("Unique conversions"), events is the number of conversions ("Total conversions"), and conversion_rate is converting visitors over total visitors of the whole query with the goal filter removed.
Additional rules:
- A goal filter turns every session-level metric into the metric of sessions containing at least one conversion (rule 4 of 9.2.1).
engagementevents never count as visits, pageviews or interactive events, but they do count as conversions of scroll goals; they do not count toward billable usage (Section 15).- Matching uses the event row exactly as stored; a goal created today applies to all historical events, because definitions are evaluated at query time. Deleting a goal removes its numbers from reports; the events remain.
- Goal definitions are evaluated at query time against ClickHouse; no per-event goal flag is stored.
- Imported custom events can be reported by goal when the imported event name equals an
eventgoal'sevent_nameand the goal has no property conditions (Section 14, 9.21).
10.2.5 Property conditions #
- A condition is an exact string-equality test
{key, value}: the event's propertykeymust exist and equalvalue(case-sensitive). - Up to 3 conditions per goal, all combined with AND. Two conditions with the same
keyare rejected (duplicate_property_condition); to match several values, create several goals. keymust be an allowlisted key of the site (10.3.3) or one of the reserved auto-collected keys (10.3.4); otherwise422 property_not_allowed.value: 1 to 2,000 characters, trimmed. An empty value is rejected (the "no value" case cannot be expressed in goals).- Conditions are not allowed on
scrollgoals (422conditions_not_supported). - Removing a key from the allowlist is blocked while any goal condition uses it (10.3.3).
10.2.6 Updating, renaming and deleting #
| Operation | Rule |
|---|---|
Editable fields (PATCH) |
display_name, event_name (event goals), page_path and scroll_threshold (pageview and scroll goals), property_conditions. Not editable: type, revenue_currency. Sending an immutable field with a different value returns 422 immutable_field. Setting revenue_currency on a goal that has none is also rejected (create a new goal instead). |
| Same-definition check | Applied after the patch; conflicts return 409 goal_exists. |
| Rename | Changing display_name rewrites, in the same Postgres transaction, every event:goal filter value that equals the old name inside segments of the same site, so segments keep working. Stored bookmark URLs, shared-link URLs with f=...goal..., and API clients using the old name are not rewritten and will match nothing. Email report subscriptions store no goal names (Section 12). |
| Delete | Hard delete (Section 4). Blocked with 409 goal_in_use and details.funnel_ids when a funnel step references the goal. Segments referencing the goal by name are left unchanged and show the missing-goal warning (Section 9.18.3). Deleting increments the site's cache epoch (Section 9.23.2). |
| Audit | Create, update and delete are written to audit_log (catalogue in Section 11.20) as goal.created, goal.updated, goal.deleted with the goal id and display name. |
10.2.7 Creating goals from discovered events #
GET /goals/suggestions?kind=event|page&q=&limit= returns up to 50 suggestions (default 20), from ClickHouse, over the last 30 days in the site timezone:
kind=event: distinct custom event names excludingpageviewandengagement, excluding names for which aneventgoal without conditions already exists. Each item:{"event_name": "Signup", "events": 412, "visitors": 380, "last_seen": "2026-11-03T08:12:00Z", "properties": ["plan", "coupon"]}wherepropertieslists up to 10 property keys seen on that event (allowlisted or not, with a flag in the itemallowlisted: boolper key:[{"key":"plan","allowlisted":true}]). Ordered byeventsdescending, ties by name.kind=page: top pathnames by pageviews, excluding paths exactly equal to thepage_pathof an existing unconditionedpageviewgoal. Each item:{"page_path": "/thank-you", "pageviews": 55}. Ordered by pageviews descending.qfilters by case-insensitive substring (1 to 120 characters).- Response uses the list envelope with
next_cursor: null(no paging beyond the limit).
UI flow: the Add goal modal has a type selector. For event, the event-name field is a combobox populated from the suggestions; picking a suggestion fills event_name, proposes the same string as the display name, and shows the discovered property keys as chips that, when clicked, add a condition row (enabled only for allowlisted keys; non-allowlisted keys show an "Add to properties first" action which calls POST /properties). Free typing of unseen event names is allowed. The auto-collected events (Outbound Link: Click, File Download, Form: Submission, 404) appear in suggestions once seen and are labeled "Auto-collected". For pageview and scroll, the path field is a combobox of top pages that also accepts wildcard patterns. A live preview line shows "Matched N events by M visitors in the last 30 days" by calling the query endpoint with a temporary definition (editor+ only, debounced 500 ms).
The Goals panel empty state also lists the top 5 undeclared event names with an Add goal button each.
10.2.8 Goals report panel #
Placement and framework: Section 9.11 and 9.6. The panel has no tabs.
| Aspect | Specification |
|---|---|
| Dimension | event:goal (display names). |
| Default columns (preview) | Goal, Uniques (visitors), Total (events), CR (conversion_rate). |
| Revenue columns | When at least one revenue goal is present among rows: Revenue (total_revenue) and Avg revenue (average_revenue) are added to the details modal; the preview shows Revenue instead of Total when all listed goals share one revenue currency. A row for a non-revenue goal shows - in revenue cells. Each money cell is formatted in that goal's own currency (Section 9.2.4); revenue is never summed across rows. |
| Ordering | visitors descending then name ascending. |
| Row click | Adds event:goal is <display name> (Section 9.11). |
| Filters | Honors all active non-goal filters and the date range; with a goal filter active the panel still lists every goal, and each row's numbers are restricted to sessions that also converted on the filtered goal(s). |
| Zero rows | Goals with zero conversions in range are still listed (with 0 and CR 0%), after goals with conversions, sorted by name; the engine returns them by left-joining the goal list. |
| Details modal | Columns: Goal, Uniques, Total, CR, Revenue, Avg revenue (revenue columns only when applicable); search by goal name; sort; Export CSV (Section 9.6.5). |
| Comparison | Not shown in the panel (Section 9.13.3). |
| Empty (no goals) | Viewers: "No goals have been set up for this site." Editors and above: the setup prompt with "Add goal" and the top undeclared events (10.2.7). |
When the date range shows imported data, goal rows for imported custom events follow 10.2.4.
10.2.9 Goal API shapes #
POST /sites/{domain}/goals request:
{
"type": "event",
"display_name": "Pro signup",
"event_name": "Signup",
"property_conditions": [{"key": "plan", "value": "pro"}],
"revenue_currency": "USD"
}Pageview example: {"type": "pageview", "page_path": "/thank-you*"}. Scroll example: {"type": "scroll", "page_path": "/blog/**", "scroll_threshold": 75, "display_name": "Read blog posts"}.
Success (201): {"data": {"id": "...", "type": "event", "display_name": "Pro signup", "event_name": "Signup", "page_path": null, "scroll_threshold": null, "property_conditions": [{"key":"plan","value":"pro"}], "revenue_currency": "USD", "created_at": "2026-11-03T10:00:00Z", "updated_at": "2026-11-03T10:00:00Z", "used_by_funnels": 0}}. GET /goals/{id} returns the same object; used_by_funnels is the number of funnel steps referencing the goal. Fields not applicable to the type are null (and [] for property_conditions).
10.3 Custom Properties #
10.3.1 Model #
A custom property is a string key and a string value attached to an event through the tracker props option or the Events API field p (Section 7 and Section 13). Each event carries at most 30 properties, keys up to 300 characters, values up to 2,000 characters (limits enforced by Section 8; larger payloads are rejected there).
10.3.2 Collection and storage #
- Properties are stored on the event row (Section 6.3) as the parallel arrays
meta.keyandmeta.valueregardless of the allowlist, so a key can be added to the allowlist later and reveal its history. - Properties with an empty-string value are dropped at ingestion (Section 8); therefore "no value" in reports and filters (
(none), Section 9.16.5) always means the event lacked the key. - Property values are exact, case-sensitive strings; no trimming is applied after ingestion's own normalization.
- Pageviews and custom events can carry properties;
engagementevents do not and are ignored by property reports and conditions.
10.3.3 Allowlist behaviour #
The allowlist (custom_property_allowlist, Section 5) decides which stored keys are queryable and visible.
| Aspect | Rule |
|---|---|
| Visible in | Properties report, the Property filter, goal conditions, funnel step conditions, segments, and the Stats API (event:props:<key>) for dashboard and API alike. |
| Not allowlisted | The key is never returned by any report or filter; querying it returns 422 property_not_allowed. |
| Add | POST /properties with {"key": "plan"} (key rules below). Idempotent: adding an existing key returns the existing record with 200; a new key returns 201. Takes effect immediately for all history. |
| Remove | DELETE /properties/{key}. Blocked with 409 property_in_use and details: {"goal_ids": [...], "funnel_ids": [...], "segment_ids": [...]} when a goal condition, funnel step condition or segment filter uses the key. The stored data is not deleted. |
| Limit | 50 allowlisted keys per site (Section 5.16); the 51st returns 409 limit_reached with details.limit_name custom_properties. The two reserved auto-collected keys (10.3.4) do not count. |
| Key validity | 1 to 300 characters, no leading or trailing whitespace, no control characters, must not start with qs_, quietstat_ or $. Violations return 422 reserved_property_name for the reserved prefixes and 400 validation_failed otherwise. |
| Discovery | GET /properties/discovered?limit= returns up to 100 keys seen on the site's events in the last 30 days that are not allowlisted, each {"key": "plan", "events": 1210, "sample_values": ["free", "pro", "team"]} (up to 3 sample values, truncated to 100 characters each), ordered by events descending. The Settings > Properties screen lists them with an Add button each. |
List response item: {"key": "plan", "created_at": "...", "auto_collected": false}.
10.3.4 Reserved and auto-collected properties #
The tracker adds properties automatically to certain events (Section 7). They are always available without the allowlist, appear in the Properties report as selectable keys when data exists, can be used in goal conditions and funnel step conditions, and cannot be removed.
| Event name | Auto-collected key | Value |
|---|---|---|
Outbound Link: Click |
url |
Destination URL of the clicked link. |
File Download |
url |
URL of the downloaded file. |
404 |
path |
Pathname of the page that returned 404. |
Form: Submission |
path |
Pathname of the page containing the submitted form. |
The keys url and path are always available and are not removable. A site that sends its own url or path property on other events sees those values reported under the same keys. The reserved prefixes qs_, quietstat_ and $ are rejected for allowlisting; the tracker never sends them (Section 7), and Section 8 drops any incoming property whose key starts with one of those prefixes.
10.3.5 Properties report panel #
Placement: Section 9.3 order 8; shown when the site has at least one allowlisted key or a reserved key with data in range.
| Aspect | Specification |
|---|---|
| Tabs | One tab per property key that has data in the date range, ordered by events descending, plus the key picker dropdown when there are more than 6 keys (first 6 as tabs, the rest in a "More" menu). Remembers the last selected key per site (localStorage). |
| Dimension | event:props:<selected key>. |
| Default columns | Value, Visitors, Events, Percentage (percentage, computed against all visitors of the filtered query, Section 9.2.2). The absent value is excluded from rows unless the tab toggle "Show events without this property" is on (then it appears as (none)). |
| Conversion mode (goal filter) | Value, Conversions (visitors), Total conversions (events), CR (group_conversion_rate), plus Revenue and Avg revenue for a single revenue currency (Section 9.6.2). |
| Event scope | Without a goal filter the panel counts pageviews and custom events together (all non-engagement events). With a goal filter, only events matching the goal. |
| Row click | Adds event:props:<key> is <value>. |
| Details modal | All columns above; search by value (case-insensitive contains); sort; Export CSV. |
| Limits | Top 10,000 values per key. |
| Empty | No allowlisted keys: editors see "Add properties in Settings > Properties to break down events by custom data." with a link; viewers see "No custom properties yet." Allowlisted but no data in range: "No data for this property in the selected period." |
10.3.6 Property API shapes #
POST /sites/{domain}/properties with {"key": "plan"} returns {"data": {"key": "plan", "created_at": "...", "auto_collected": false}}. GET /properties returns the list envelope (limit default 50, max 200) ordered by key ascending, with the two reserved keys listed first with auto_collected: true.
10.4 Revenue #
10.4.1 Sending revenue #
A revenue value is sent on a custom event as $: {"amount": "12.50" | number, "currency": "EUR"} (Section 7 and Section 13). Section 8.14 validates it: amount is a decimal string or a number with 0 <= amount <= 999,999,999,999.999 (rounded half up to 3 fractional digits), and currency is an ISO 4217 code from the supported currency table. An event with an invalid $ is rejected as a whole with 400 $ is invalid (the tracker drops invalid revenue before sending, Section 7). The product does not model refunds or negative revenue.
Storage (Section 6.3): revenue_amount (Decimal(18,3), major units, exactly as sent) and revenue_currency, plus revenue_reporting_amount and revenue_reporting_currency, which hold the conversion into the site's reporting currency (site_settings.reporting_currency, Section 5.5.2) computed at ingestion (Section 8.14) and are NULL when no rate was available.
10.4.2 Revenue goals #
A revenue goal is an event goal with revenue_currency set (10.2.2). The currency is chosen per goal, is immutable, and defines the currency of every revenue number for that goal. Multiple revenue goals may use different currencies on the same site. Only events matching the goal definition (name and conditions) count toward that goal's revenue; revenue values on events that match no revenue goal are stored but not reported.
10.4.3 Currency conversion #
Revenue for a goal is always reported in the goal's revenue_currency. For each event matching the goal the engine takes the first applicable source:
revenue_amount, when the event'srevenue_currencyequals the goal currency (no conversion, no rate).revenue_reporting_amount, when the event'srevenue_reporting_currencyequals the goal currency.- Otherwise it converts
revenue_amounton read with the rate rules of Section 8.14: the daily rate of the event's UTC date; if none exists for that date, the most recent earlier rate within a 10-day lookback; if none exists in the lookback either, the conversion has no rate.
If the conversion has no rate (no rate within the 10-day lookback, so the event's revenue_reporting_amount is NULL), the event's revenue is excluded from revenue totals and averages (the event still counts as a conversion). When any event was excluded for this reason the query response carries meta.warnings containing revenue_rate_missing, and the UI shows a warning icon on the revenue cells: "Some revenue values could not be converted."
Per-event amounts are rounded half away from zero to the goal currency's minor unit (ISO 4217 exponent) before summing; totals are exact sums of those rounded values. Because the reporting columns are written at ingestion with the same rate table and rules as the read-time conversion, a report does not depend on which of the three sources produced a number (the reporting amount is rounded to 3 decimals at ingestion and to the goal's minor unit when reported).
10.4.4 Metrics #
| Metric | Definition (full formulas in 9.2.2) |
|---|---|
total_revenue |
Sum of converted revenue over matching events. |
average_revenue |
total_revenue / number of matching events that carry a revenue value. Events matching the goal without a revenue value are excluded from the denominator. A revenue value of 0 counts. |
Return format for APIs and for CSV (Section 4): {"value": 1234.56, "currency": "USD", "short": "$1.2K", "long": "$1,234.56"}. Rounding of average_revenue: total in minor units divided by the denominator, rounded half away from zero to the minor unit. Display format follows Section 9.2.4.
10.4.5 Where revenue appears #
| Place | Behaviour |
|---|---|
| Top stats (Section 9.4.1) | Total revenue and Average revenue tiles in Conversion mode when exactly one revenue currency applies. |
| Main chart | total_revenue and average_revenue are selectable metrics in that mode (Section 9.5.3). |
| Goals panel | Revenue and Avg revenue columns (10.2.8). |
| Sources, Channels, Referrers, AI Assistants, Campaign tabs, Pages, Locations, Devices, Properties | Revenue breakdown column in Revenue mode (Section 9.6.2). Revenue by source is the Sources and Channels tabs in Revenue mode, sorted by revenue descending in the details modal. |
| Funnels | Revenue and revenue per visitor per step (10.5.7). |
| CSV export | Revenue number and currency columns (Section 9.6.5). |
| Stats API | total_revenue and average_revenue metrics (Section 13). |
10.4.6 Mixed currencies and edge cases #
- If the active goal filter selects revenue goals in more than one currency, revenue metrics are not computed and the revenue UI is hidden (Section 9.2.3); selecting goals of one currency restores it.
- Filtering by a goal that is not a revenue goal hides the revenue UI.
- Switching an existing event goal into a revenue goal is not possible; create a second goal with the currency (the original and the new goal can coexist because their display names must differ).
- Revenue values for deleted goals disappear from reports with the goal.
10.5 Funnels #
10.5.1 Concepts #
A funnel is an ordered list of steps. For every session in scope the engine determines how far along the steps the session progressed (10.5.6); for every step it reports how many distinct visitors reached it. Steps can be built directly from pages, from events, or from existing goals; no goal needs to be created first.
10.5.2 Funnel fields #
Persisted in funnels and funnel_steps (Section 5).
| Field | Type | Rules |
|---|---|---|
id |
uuid | Server generated. |
name |
string | Required; trimmed; 1 to 120 characters; unique per site, case-insensitive (409 funnel_name_taken). |
order_mode |
enum ordered, strict |
Default ordered; semantics in 10.5.4. |
steps |
array of 2 to 8 step objects | Order of the array is the step order; position (1-based) is derived. |
created_at, updated_at |
timestamptz | Server managed. |
10.5.3 Step definition #
| Field | Type | Rules |
|---|---|---|
type |
enum page, event, goal |
Required. |
page_path |
string | Required for page; same grammar and matching as goal path patterns (10.2.3). Null otherwise. |
event_name |
string | Required for event; 1 to 120 characters; must not equal pageview or engagement (use a page step for pageviews). Null otherwise. |
goal_id |
uuid | Required for goal; must be a goal of the same site (any goal type, including scroll goals). Null otherwise. A goal step uses the goal's current definition at evaluation time. |
property_conditions |
array of {key, value} |
0 to 3 for page and event steps (rules identical to 10.2.5); must be [] for goal steps (the goal carries its own). |
is_optional |
boolean | Default false. See 10.5.4. |
label |
string or null | Optional display override, trimmed, 1 to 120 characters. Default label: the path pattern for page, the event name for event, the goal's display name for goal. |
Step validation:
- The first and the last step must have
is_optional = false(422 invalid_optional_step). Only middle steps may be optional. - Two adjacent steps may not have an identical definition (
typeplus matching fields plus conditions), because one event cannot satisfy two consecutive steps (422 duplicate_adjacent_steps). - A
goalstep referencing a deleted goal or a goal of another site returns400 validation_failedwithdetails.fieldsnamingsteps.N.goal_id. - Property keys in conditions follow
property_not_allowed(10.3.3).
10.5.4 Order modes and optional steps #
order_mode |
UI label | Semantics |
|---|---|---|
ordered |
"Allow other events between steps" (default) | Each step must be matched by an event that occurs after the event matched for the previous step, within the same session. Any number of other events may occur in between. |
strict |
"Steps must follow each other directly" | Each step must be matched by the very next event in the session after the event matched for the previous step, with no other event in between. |
There is deliberately no mode in which steps may occur in arbitrary order; "did the visitor do A and B in any order" is answered with goal filters (Section 9.16) and the Goals report. A funnel is always a sequence.
Event stream: the sequence considered for a session consists of its pageview events and custom events, ordered by timestamp ascending. Ties (identical timestamps) are ordered with pageview first, then custom events by name ascending. engagement events are excluded from the stream, with one exception: an engagement event that satisfies a scroll goal used as a goal step is included at its own timestamp (after custom events on ties).
Optional steps (only in ordered mode): an optional step may be matched or skipped. A funnel using strict mode cannot contain optional steps (422 strict_with_optional_steps), because "must be the very next event" and "may be skipped" conflict; the editor disables the optional checkbox in strict mode and clears it on switching to strict (after a confirm).
10.5.5 Time window #
A funnel is always evaluated within one session (a visit, Section 8: consecutive events of one visitor with gaps under 30 minutes). A "within N days" window is not offered: visitor identifiers rotate daily (Section 8) and the product intentionally cannot link a person across days. The UI states this under the editor's order-mode setting: "Steps are counted within a single visit." The maximum time between steps is therefore bounded by the session rules.
10.5.6 Evaluation algorithm (normative) #
Scope: all sessions whose start falls inside the date range and which match the active filters (10.5.9). Their full event streams are used even for events after the range end. Per session, with stream[0..m-1] the ordered events and n the step count, each event gets a bitmask mask where bit s (0-based) is set when the event satisfies step s's definition. A single event may satisfy several steps.
Reference implementation (the production ClickHouse query must produce identical results; tests use this function as an oracle, Section 18):
type StepResult = { reached: boolean; eventIndex: number | null };
export function evaluateSession(
stream: { mask: number }[],
isOptional: boolean[],
mode: 'ordered' | 'strict',
): StepResult[] {
const n = isOptional.length;
const res: StepResult[] = Array.from({ length: n }, () => ({ reached: false, eventIndex: null }));
const has = (i: number, s: number) => (stream[i].mask & (1 << s)) !== 0;
if (mode === 'strict') {
let bestStart = -1;
let bestLen = 0;
for (let start = 0; start < stream.length; start++) {
if (!has(start, 0)) continue;
let k = 1;
while (k < n && start + k < stream.length && has(start + k, k)) k++;
if (k > bestLen) { bestLen = k; bestStart = start; }
if (bestLen === n) break;
}
for (let s = 0; s < bestLen; s++) res[s] = { reached: true, eventIndex: bestStart + s };
return res;
}
// ordered: pass 1, required steps ("gates") in order
let cursor = -1;
for (let s = 0; s < n; s++) {
if (isOptional[s]) continue;
let found = -1;
for (let i = cursor + 1; i < stream.length; i++) if (has(i, s)) { found = i; break; }
if (found === -1) break;
res[s] = { reached: true, eventIndex: found };
cursor = found;
}
// ordered: pass 2, optional steps, bounded by the surrounding required steps
for (let s = 0; s < n; s++) {
if (!isOptional[s]) continue;
let prev = s - 1; while (isOptional[prev]) prev--; // first step is always required
let next = s + 1; while (isOptional[next]) next++; // last step is always required
if (!res[prev].reached) continue;
let lower = -1;
for (let t = 0; t < s; t++) if (res[t].reached && res[t].eventIndex! > lower) lower = res[t].eventIndex!;
const upper = res[next].reached ? res[next].eventIndex! : stream.length;
for (let i = lower + 1; i < upper; i++) {
if (has(i, s)) { res[s] = { reached: true, eventIndex: i }; break; }
}
}
return res;
}Consequences:
- In
orderedmode a session "reaches" a required step when all earlier required steps were matched in order; the matches are chosen greedily (earliest possible), which maximizes progress. - An optional step counts as reached only when an event matching it occurs after everything matched before it and before the event matched for the next required step; if the next required step is not reached, an optional step still counts when it occurs after the last matched step.
- In
strictmode the session's progress is the longest run of consecutive stream events that matches steps 1, 2, 3, ... starting at any start position; the earliest start achieving the maximum is used. - Visitor level: a visitor has "reached step
s" when any of their sessions in scope reached it. Because sessions of one visitor are bounded to one day by the daily hash, a visitor has at most a handful of sessions. - Step counts of required steps are non-increasing along the funnel; an optional step's count can be lower than the following required step's count.
10.5.7 Result metrics, drop-off and revenue #
For step i (1-based), let P(i) be the nearest earlier required step (none for step 1) and V(i) the number of distinct visitors that reached step i.
| Result field | Definition |
|---|---|
visitors |
V(i). |
from_first_rate |
V(i) / V(1) * 100, 1 decimal; null when V(1) = 0. |
from_previous_rate |
V(i) / V(P(i)) * 100, 1 decimal (step 1: null); null when the denominator is 0. |
dropoff_visitors |
For required steps with i > 1: V(P(i)) - V(i). Null for step 1 and for optional steps. |
dropoff_rate |
dropoff_visitors / V(P(i)) * 100, 1 decimal; null when not applicable. |
overall |
{visitors_first, visitors_last, conversion_rate} where conversion_rate = V(last) / V(1) * 100 (1 decimal; null if V(1) = 0). |
revenue |
Present per step only when a revenue currency can be resolved for the step (below); otherwise null. Object: {"total": Money, "per_visitor": Money} using the money shape of Section 4. |
Revenue per step:
- A step has a revenue currency when it is a
goalstep referencing a revenue goal, or aneventstep whoseevent_nameequals theevent_nameof one or more revenue goals that all share one currency (the goals' property conditions are ignored for this purpose; the step's own conditions apply). Page steps have no revenue. total= the sum, over all sessions in scope where the step was reached, of the converted revenue of the matched event (eventIndex) of that step (conversion per 10.4.3). Only the matched event contributes, not later repeat events in the same session.per_visitor=total / V(i)rounded half away from zero to the minor unit. Null whenV(i) = 0.- Revenue is never summed across steps or currencies; the funnel header shows the last step's revenue as "Revenue" only when that step has a currency.
10.5.8 Period comparison #
The request may carry comparison_date_range (resolved by the dashboard per Section 9.13.2). The engine evaluates the funnel for both ranges independently and returns comparison with the same structure. The UI shows, per step, the previous visitors and the change in from_first_rate in percentage points (rules of Section 9.4.3 for colors), and overall conversion change. Comparison honors the same filters. If the comparison range has V(1) = 0 the deltas show New/- per Section 9.4.3.
10.5.9 Scope and filters #
- Session-level filters (source, channel, referrer, UTM, entry/exit page, country, region, city, browser, OS, screen size) restrict the sessions in scope.
- Event-level filters (page, hostname, goal, property) restrict to sessions that contain at least one matching event; they do not alter which events satisfy steps.
- A segment filter resolves into its constituent filters. The
with_importedtoggle is ignored (imports have no event streams). - Date range: sessions by start time (Section 9.2.1). Funnels do not support the
realtimeperiod (the panel is hidden there).
10.5.10 Funnels panel (dashboard) #
Placement: Section 9.3 order 9.
- Header: funnel selector (dropdown of saved funnels sorted by name; last selection remembered per site in
localStorage), overall conversion ("12.4% of visitors completed all steps (1,230 of 9,900)"), and for editors an Edit and New funnel action. - Drop-off visualization: a column chart with one column per step, left to right. The full-height of the tallest column equals
V(1). Each column shows the reached portion as a solid bar and the lost portion (relative to the previous required step) as a lighter hatched bar stacked above it labelled with-{dropoff_rate}%in red (the text sign is always shown). Under each column: the step label,visitors(compact),from_first_rate, andfrom_previous_rate. Optional steps are drawn with a dashed outline and anOptionaltag, no hatched drop-off bar. If revenue exists the step shows two extra lines "Revenue" and "Per visitor". When comparison is on, a narrow gray bar to the right of each column shows the previous period's value, and the delta under it. - Table view: a "View as table" toggle lists the same numbers (Step, Visitors, % of first, % of previous, Drop-off visitors, Drop-off %, Revenue, Revenue per visitor, and previous-period columns when comparison is on) for accessibility and for CSV export (the panel's Export CSV exports this table, Section 9.6.5).
- States: loading skeleton columns; "No visitors reached the first step in this period." when
V(1) = 0; error with Retry; for sites without funnels: editors see "Create a funnel" with the New funnel button, viewers do not see the panel. - Clicking a column does nothing (no individual-level drill-down exists).
10.5.11 Funnel editor #
A modal opened from the panel or from Settings > Funnels (a list screen with name, steps summary, created date, Edit, Duplicate, Delete). Content: name field; order mode radio; ordered step list with drag handles and Move up / Move down buttons for keyboard use; Add step (disabled at 8 steps with the hint "A funnel can have at most 8 steps"); per step: type select, the type-specific input (path combobox from top pages; event-name combobox from suggestions, 10.2.7; goal select), property conditions editor (0 to 3 rows, allowlisted keys only), Optional checkbox (is_optional) for middle steps in ordered mode, optional label, Remove button (disabled below 3 steps; a funnel needs at least 2). A live preview on the right evaluates the unsaved definition over the last 30 days via POST /funnels/preview (debounced 700 ms, editor+ only). Saving validates server-side; errors map to fields. Duplicate creates "Copy of {name}" (truncated to 120 characters, suffixed with a number on conflict).
10.5.12 Limits #
| Limit | Value |
|---|---|
| Funnels per site | 50 (Section 5.16); the 51st returns 409 limit_reached with details.limit_name funnels |
| Steps per funnel | 2 to 8 |
| Conditions per step | 3 |
| Name | 120 characters |
| Sessions evaluated per funnel query | 5,000,000; a larger scope returns 422 query_too_large ("Narrow the date range or add filters.") |
| Execution time | 30 seconds (503 query_timeout) |
| Cache | 60 seconds for ranges including today; 10 minutes for past ranges (Section 9.23.2); invalidated by the site epoch |
10.5.13 API shapes #
POST /sites/{domain}/funnels:
{
"name": "Checkout",
"order_mode": "ordered",
"steps": [
{"type": "page", "page_path": "/pricing", "property_conditions": [], "is_optional": false},
{"type": "event", "event_name": "Add to cart", "property_conditions": [{"key": "plan", "value": "pro"}], "is_optional": true, "label": "Added to cart"},
{"type": "goal", "goal_id": "0190c6f0-0000-7000-8000-000000000001"}
]
}201 returns {"data": {"id": "...", "name": "Checkout", "order_mode": "ordered", "steps": [{"position": 1, "type": "page", "page_path": "/pricing", "event_name": null, "goal_id": null, "property_conditions": [], "is_optional": false, "label": null}, ...], "created_at": "...", "updated_at": "..."}}. PATCH accepts any of name, order_mode, steps (replaces all steps) and revalidates the whole funnel.
POST /funnels/{id}/results and POST /funnels/preview (the latter takes {"order_mode", "steps"} instead of an id):
{
"date_range": ["2026-10-05", "2026-11-03"],
"comparison_date_range": ["2026-09-05", "2026-10-04"],
"filters": [["is", "visit:country", ["DE"]]]
}Response 200:
{
"data": {
"funnel_id": "...",
"date_range": ["2026-10-05", "2026-11-03"],
"steps": [
{"position": 1, "label": "/pricing", "type": "page", "is_optional": false, "visitors": 9900, "from_first_rate": 100.0, "from_previous_rate": null, "dropoff_visitors": null, "dropoff_rate": null, "revenue": null},
{"position": 2, "label": "Added to cart", "type": "event", "is_optional": true, "visitors": 3100, "from_first_rate": 31.3, "from_previous_rate": 31.3, "dropoff_visitors": null, "dropoff_rate": null, "revenue": null},
{"position": 3, "label": "Purchase", "type": "goal", "is_optional": false, "visitors": 1230, "from_first_rate": 12.4, "from_previous_rate": 12.4, "dropoff_visitors": 8670, "dropoff_rate": 87.6,
"revenue": {"total": {"value": 61500.0, "currency": "USD", "short": "$61.5K", "long": "$61,500.00"}, "per_visitor": {"value": 50.0, "currency": "USD", "short": "$50", "long": "$50.00"}}}
],
"overall": {"visitors_first": 9900, "visitors_last": 1230, "conversion_rate": 12.4},
"comparison": null
},
"meta": {"cached": false, "warnings": []}
}filters and date_range use the dashboard conventions of Section 9.1.3 and 9.16; comparison_date_range is optional. Errors: 404 not_found (unknown funnel), 422 invalid_date_range, 422 invalid_filter, 422 query_too_large, 503 query_timeout.
10.6 User Journeys #
10.6.1 Definition #
A user journey is an aggregated diagram of the sequences of pages visitors view within a session. It answers "where do visitors go after page A?" (forward) and "where did visitors come from before page B?" (backward). It is built only from pageview events; custom events and engagement events are ignored. It is aggregated: no individual session or visitor is ever shown.
Constants (defined in @quietstat/shared, not configurable per site):
| Constant | Value |
|---|---|
JOURNEY_MIN_VISITS |
5 |
JOURNEY_MAX_NODES_PER_COLUMN |
10 (9 pages plus one Other node) |
JOURNEY_MIN_DEPTH, JOURNEY_DEFAULT_DEPTH, JOURNEY_MAX_DEPTH |
2, 5, 8 |
JOURNEY_SESSION_SCAN_LIMIT |
5,000,000 sessions |
10.6.2 Path construction #
- For each session (start time in range, filters per 10.6.7), take its
pageviewevents ordered bytimestamp(ties: bypathnameascending). Sessions with no pageview (events sent only through the Events API without pageviews) are excluded. - Collapse consecutive identical pathnames into one step (page reloads and repeated SPA navigations to the same path). Non-consecutive repeats are kept (A, B, A is a three-step path).
- Node identity is the stored pathname (query strings removed, hash included only for hash-routing sites). No grouping by wildcard is applied to nodes.
- The resulting array is the session's path
[p1, p2, ..., pk].
10.6.3 Inputs #
| Input | Values | Default |
|---|---|---|
direction |
forward ("Starts at") or backward ("Ends at") |
forward |
anchor |
A page pattern (same grammar as 10.2.3) or null |
null |
depth |
Integer 2 to 8 | 5 |
The "starting point / ending point" selection is the combination of direction and anchor; only one anchor exists per diagram.
- Forward, anchor
null("Any entry page"): column 1 holds each session's entry page; subsequent columns hold the following pages. - Forward, anchor pattern: for every session whose path contains a page matching the pattern, the journey starts at the first matching page of the path (the matched pathname becomes the node in column 1; a wildcard pattern may yield several nodes in column 1). Pages before it are ignored.
- Backward, anchor
null("Any exit page"): the last column (columndepth) holds each session's exit page; earlier columns hold the preceding pages. - Backward, anchor pattern: the journey ends at the last matching page of the path; pages after it are ignored.
10.6.4 Columns, nodes and links #
- Forward: column
c(1-based) holds the page at offsetc - 1from the start page. A session whose path ends before offsetc - 1contributes an(exit)node in the first column after its last page and does not continue. A session that has exactlydepthpages from the start contributes no(exit)node (it is truncated by depth). - Backward: column
depthholds the end page; columndepth - jholds the pagejsteps earlier. A session that began fewer thanjsteps earlier contributes a(entry)node in the first column to the left of its first page and does not extend further left. - A link from node X in column
cto node Y in columnc + 1has value equal to the number of sessions whose path contains that consecutive pair at those offsets. A node's value is the number of sessions passing through it. Values are visits (sessions), displayed as "visits". - The anchor column's total ("N visits") is shown above the diagram and equals the sum of its node values before suppression.
10.6.5 Aggregation thresholds (privacy) #
The diagram only ever displays values of at least JOURNEY_MIN_VISITS visits:
- In each column, nodes are ranked by value descending (ties by label ascending). The top 9 nodes with value >=
JOURNEY_MIN_VISITSare shown. All other nodes of the column (including those below the threshold and those beyond the top 9) are folded into oneOthernode whose value is their sum;Otheris shown only if its value is >=JOURNEY_MIN_VISITS, otherwise it is omitted.(exit)and(entry)nodes follow the same rules and are never folded intoOtherwhile they qualify. - A link is drawn only if its value is >=
JOURNEY_MIN_VISITS. Links into or out ofOtherare aggregated and follow the same rule;Othernodes have no outgoing links (traversal beyond anOthernode is not traced). - A column whose nodes all fall below the threshold ends the diagram: later columns are not shown.
- If the anchor column itself has no node with at least
JOURNEY_MIN_VISITSvisits the panel shows the empty state: "Not enough traffic to show journeys. Paths with fewer than 5 visits are hidden to protect visitor privacy." - The response reports
hidden: {"threshold": 5, "nodes_hidden": <count>, "visits_hidden": <sum>}so the UI can show "N paths with fewer than 5 visits are hidden." Exact hidden counts are acceptable because they are aggregates of at least one session across many nodes; counts per hidden node are never returned. - Thresholds apply identically for shared links and embeds.
10.6.6 Visualization #
- Rendered as a Sankey diagram with
d3-sankey(Section 3): columns left to right, node rectangles 12 px wide, 12 px vertical padding, link width proportional to visits, link color a 35%-opacity gradient from source to target node color,(exit)and(entry)nodes in neutral gray,Othernodes in a lighter tint. Column headers: "Step 1" through "Step N"; in backward direction the last column header is "End page". - Node labels show the page path truncated to 28 characters (middle ellipsis) with the visit count; full path in the tooltip. Link tooltip: "{source} -> {target}: {visits} visits ({share}% of {source})".
- Node click opens a small menu: "Start journeys here", "End journeys here", "Filter dashboard by this page" (adds
event:page is <path>; hidden forOther,(exit),(entry)). The two journey actions set the panel's anchor and direction. - Controls above the diagram: direction toggle (Starts at / Ends at), anchor combobox (type to search pages, "Any entry page" or "Any exit page" per direction, accepts wildcard patterns), depth select (2 to 8), "View as table".
- Table view (and the accessible equivalent): rows for each link (From step, From page, To step, To page, Visits, % of source) ordered by visits descending; Export CSV per Section 9.6.5.
- Responsive: below 768 px the diagram scrolls horizontally inside the panel with a minimum column width of 140 px; the table view is the default below 480 px.
- Keyboard: nodes are focusable in column order with Tab; arrow keys move between adjacent nodes; Enter opens the node menu.
- Colors meet the contrast requirements in Section 4 and never encode information by color alone (node type is also given by labels).
10.6.7 Filters, range and comparison #
Date range and filters apply exactly as for funnels (10.5.9): session-level filters restrict sessions; event-level filters keep sessions containing at least one matching event; segments resolve to filters. The anchor pattern is independent of the filter list (a page filter and an anchor can differ). Comparison is not supported for journeys (the panel ignores the comparison setting and shows "Comparison is not available for journeys." in a muted note when comparison is on). Imported data is never included. The panel is hidden in the Realtime view.
10.6.8 Limits and performance #
| Limit | Value |
|---|---|
| Sessions scanned per query | 5,000,000 (422 query_too_large when exceeded) |
| Path length considered | depth pages from the anchor (the query slices the compacted array; ClickHouse arrayCompact is the reference approach) |
| Execution time | 30 seconds (503 query_timeout) |
| Cache | Same TTLs and epoch invalidation as funnels (10.5.12) |
| Result size | At most 8 columns x 10 nodes, 80 nodes and 1,000 links |
10.6.9 API #
POST /sites/{domain}/journeys request:
{
"direction": "forward",
"anchor": {"page_path": "/pricing"},
"depth": 5,
"date_range": ["2026-10-05", "2026-11-03"],
"filters": [["is", "visit:channel", ["Organic Search"]]]
}anchor is null or {"page_path": "<pattern>"}. Response 200:
{
"data": {
"direction": "forward",
"depth": 5,
"anchor_visits": 4120,
"columns": [
{"index": 1, "nodes": [{"id": "c1:/pricing", "kind": "page", "label": "/pricing", "visits": 4120}]},
{"index": 2, "nodes": [
{"id": "c2:/signup", "kind": "page", "label": "/signup", "visits": 1320},
{"id": "c2:(exit)", "kind": "exit", "label": "(exit)", "visits": 1900},
{"id": "c2:(other)", "kind": "other", "label": "Other", "visits": 900}
]}
],
"links": [
{"source": "c1:/pricing", "target": "c2:/signup", "visits": 1320},
{"source": "c1:/pricing", "target": "c2:(exit)", "visits": 1900},
{"source": "c1:/pricing", "target": "c2:(other)", "visits": 900}
],
"hidden": {"threshold": 5, "nodes_hidden": 14, "visits_hidden": 36}
},
"meta": {"cached": false}
}kind is one of page, exit, entry, other. Node ids are c{column}:{label}. Validation: direction enum, depth 2 to 8, anchor.page_path per 10.2.3 (invalid pattern is 400 validation_failed), range and filters per Section 9 (422 invalid_date_range, 422 invalid_filter).
10.6.10 Panel placement and empty states #
Placement: Section 9.3 order 10, always visible. States: loading skeleton; "No pageviews in this period." when no session has a pageview; the threshold message of 10.6.5 item 4; error with Retry.
10.7 Permissions per Role #
The authoritative permission matrix is in Section 11; this table states what this section requires from it. guest_editor has the permissions of editor and guest_viewer the permissions of viewer, scoped to the one site.
| Action | owner | admin | editor | viewer | guest_editor | guest_viewer | Shared-link viewer |
|---|---|---|---|---|---|---|---|
| View Goals, Properties, Funnels and Journeys panels | yes | yes | yes | yes | yes | yes | yes (within the link's segment restriction, Section 12) |
| Create, edit, delete goals | yes | yes | yes | no | yes | no | no |
| Read goal suggestions and discovered properties | yes | yes | yes | no | yes | no | no |
| Add or remove allowlisted properties | yes | yes | yes | no | yes | no | no |
| Create, edit, duplicate, delete funnels | yes | yes | yes | no | yes | no | no |
| Run funnel preview (editor live preview) | yes | yes | yes | no | yes | no | no |
| Evaluate a saved funnel | yes | yes | yes | yes | yes | yes | yes |
| Compute journeys | yes | yes | yes | yes | yes | yes | yes |
| Export a panel as CSV | yes | yes | yes | yes | yes | yes | yes |
Rules: a role without the permission receives 403 forbidden with details.required_role: "editor"; a user without any access to the site receives 404 not_found. Goal, funnel and property mutations write audit entries (10.2.6) and increment the site's cache epoch (Section 9.23.2). Mutations, funnel previews and journey queries are rate limited by policy dashboard_query (Section 17.6).
10.8 Validation and Errors #
All errors use the envelope of Section 4.4. Field-level issues are in details.fields.
| Code | Status | When |
|---|---|---|
validation_failed |
400 | Malformed JSON, wrong types, string length violations, unsupported characters, invalid page pattern grammar, invalid currency format, missing required field for the type, field present for the wrong type, a funnel step whose goal_id is deleted or belongs to another site (details.fields names steps.N.goal_id). |
not_found |
404 | Unknown goal id, funnel id or property key, or a site the caller cannot access (10.7). |
goal_name_taken |
409 | Duplicate display name (case-insensitive). |
goal_exists |
409 | Same definition tuple already exists (10.2.2). |
limit_reached |
409 | A limit of Section 5.16: 100 goals (limit_name goals), 50 allowlisted properties (custom_properties), 50 funnels (funnels); details carries limit_name, limit and current. |
goal_in_use |
409 | Deleting a goal referenced by funnel steps; details.funnel_ids. |
immutable_field |
422 | Changing type or revenue_currency after creation. |
revenue_not_supported |
422 | revenue_currency on a non-event goal. |
unsupported_currency |
422 | Currency code not in the supported list of Section 8. |
conditions_not_supported |
422 | Property conditions on a scroll goal or on a goal-type funnel step. |
duplicate_property_condition |
422 | Two conditions with the same key. |
too_many_property_conditions |
422 | More than 3 conditions. |
property_not_allowed |
422 | Key is not allowlisted nor reserved (goal conditions, funnel step conditions, filters, segments, queries). |
reserved_property_name |
422 | Key starts with qs_, quietstat_ or $. |
property_in_use |
409 | Removing a key used by goal conditions, funnel conditions or segments; details lists ids by kind. |
reserved_event_name |
422 | Event goal or funnel event step named pageview or engagement. |
funnel_name_taken |
409 | Duplicate funnel name. |
invalid_step_count |
422 | Fewer than 2 or more than 8 steps. |
invalid_optional_step |
422 | First or last step marked optional. |
strict_with_optional_steps |
422 | order_mode = strict with an optional step. |
duplicate_adjacent_steps |
422 | Two adjacent steps with an identical definition. |
invalid_date_range, invalid_filter, query_too_large |
422 | As defined in Section 9.24. |
query_timeout |
503 | Execution exceeded the limit (Section 9.23.3). |
Validation order for create/update requests: authentication and permissions first (401; 404 not_found for a caller without access to the site; 403 forbidden for a missing role), so a caller without access learns nothing about the request schema; then shape (Zod, 400); then semantic checks in the order: reserved names, property allowlist, immutability, limits, uniqueness (422/409). The first failing check is returned. Zod schemas are shared with the browser in @quietstat/shared so the editor shows the same messages.
Message text rules: messages are human readable English sentences ending in a period and never include raw property values, visitor data or SQL.
11. Accounts, Authentication, Teams, Roles & Permissions #
This section owns identity: who a user is, how they prove it, which team they act in, and what each role may do. It owns the canonical permission matrix (Section 11.19); every other section enforces permissions by naming an action ID from that matrix and does not restate role rules. All routes in this section live under the dashboard API family /api/app/v1 defined in Section 4 (cookie session auth, X-CSRF-Token on mutating requests, the success and error envelopes of Section 4.4). Table and column definitions are owned by Section 5; Section 11.23 lists the attributes this section requires of them.
11.1 Principles and Edition Differences #
- Email plus password is the only credential type. There is no SSO, no social login, no magic link and no passkey. TOTP is the only second factor.
- Users are global; access is per team. A user row exists once. A user may be a member of many teams and a guest on sites of other teams. The active team is a UI concept; every API call names the team or site it acts on and is authorized against that resource (Section 11.19.3).
- No existence leaks on resources. If a user has no relationship to a team or site, the API returns
404 not_found. If the user has a relationship but an insufficient role, it returns403 forbidden. - Sensitive actions require step-up. Changing the password, changing the email, changing 2FA, deleting the account, starting an ownership transfer and creating an operator impersonation each require re-entry of the current password (and a TOTP code when 2FA is on).
- Raw IP addresses of dashboard users are never stored. Login history keeps a device label and a country only (Section 11.12.3).
| Aspect | Hosted (EDITION=hosted) |
Self-hosted (EDITION=selfhosted) |
|---|---|---|
Default REGISTRATION_MODE |
open |
invite_only |
| First account | Created only through POST /api/app/v1/setup with the one-time setup token (Section 11.2) |
Same |
| Email verification | Required (REQUIRE_EMAIL_VERIFICATION defaults to true) |
Required only when SMTP is configured; otherwise accounts are created verified |
| Terms acceptance at signup | Required checkbox (accept_terms) |
Shown only when TERMS_URL is set |
| Super admin | Verified emails in SUPER_ADMIN_EMAILS whose owner has TOTP enabled get the operator console (Section 15) |
The list is ignored (Section 16.4); there is no operator console and no registration bypass |
| Licensing, caps, billing rows in the matrix | Active | Hidden and denied (the action IDs billing.view and billing.manage always evaluate to deny) |
| Email transport required at boot | Yes (Section 16 refuses to start without SMTP_HOST) |
No (Section 12.9.6 describes the no-SMTP mode) |
Configuration names used here (REGISTRATION_MODE, REQUIRE_EMAIL_VERIFICATION, SUPER_ADMIN_EMAILS, TERMS_URL, PRIVACY_URL, SECRET_KEY_BASE, SETUP_TOKEN, BASE_URL, PRODUCT_NAME, PASSWORD_BREACH_CHECK, AUDIT_RETENTION_MONTHS) are defined with types, defaults and editions in the configuration table in Section 16.4.
11.2 Registration Modes #
REGISTRATION_MODE is one of three values.
| Mode | Who may create an account | Signup endpoint behavior |
|---|---|---|
open |
Anyone who submits a valid signup form | Creates the account, personal team and session (Section 11.3) |
invite_only |
Only a person holding a valid, unexpired, unaccepted invitation token (team member, site guest, ownership-transfer or site-transfer invitation) | Requires invitation_token; the email must equal the invited email; without a valid token returns 403 registration_closed |
disabled |
Nobody via the public form | Always returns 403 registration_closed; existing users keep working; invitations to non-users cannot be accepted (the invitation page shows "Registration is disabled on this instance") |
Bootstrap and bypass rules:
- Bootstrap (no first-signup bypass). While no user row exists (
setup_required = true),POST /api/app/v1/auth/signupreturns403 setup_requiredin every mode and edition, so nobody can claim a fresh instance by reaching the signup page first. The first account is created only throughPOST /api/app/v1/setupwith the one-time setup token (generated on first boot and printed once to thewebprocess log whenSETUP_TOKENis unset, Section 16.3.5). A wrong token returns403 setup_token_invalid(rate-limit policysetup_token). That account is verified without an email, owns its personal team, is signed in, and is audited asuser.registeredwithvia = setup.POST /api/app/v1/setupuses the anonymous CSRF token (Section 11.7); once any user exists it returns409 setup_already_completedand the/setuppage returns404. The registration-mode rules below apply only after setup. - Super admins (hosted only). In the hosted edition an email present in
SUPER_ADMIN_EMAILS(case-insensitive) may always sign up, even indisabledmode, where this is the only open path. In the self-hosted editionSUPER_ADMIN_EMAILSis ignored and there is no bypass.
GET /api/app/v1/bootstrap is the runtime configuration endpoint. It is public and returns what the SPA needs before and after login; the same JSON is inlined into index.html as a non-executable <script type="application/json" id="qs-bootstrap"> element so no inline script exists (Section 17.5). Responses carry Cache-Control: private, no-store when a session cookie is present and public, max-age=60 otherwise:
{
"data": {
"product_name": "Quietstat",
"edition": "hosted",
"base_url": "https://app.example.com",
"registration_mode": "open",
"features": { "licensing": true, "stripe": true, "google_import": false, "email": true },
"geoip_attribution": "IP geolocation by DB-IP",
"setup_required": false,
"email_verification_required": true,
"password_min_length": 12,
"password_max_length": 128,
"terms_url": "https://example.com/terms",
"privacy_url": null,
"dpa_url": null,
"hosting_region_label": "EU (Germany)",
"banner": { "key": "none", "severity": "info", "params": {} },
"csrf_token": null,
"user": null
}
}This section owns the full body. Fields: product_name, edition (hosted or selfhosted), base_url, registration_mode, features (licensing, stripe, google_import, email; features.email is true when SMTP_HOST is set), geoip_attribution, setup_required, email_verification_required, password_min_length, password_max_length, terms_url, privacy_url, dpa_url and hosting_region_label (null when unset), banner, csrf_token and user. The response contains every key of the public runtime-config subset of Section 4.10 unchanged, plus setup_required, banner, csrf_token, user and the password-policy fields. user is null when no valid session cookie accompanies the request, otherwise the user object of Section 11.12.1 with the caller's teams; csrf_token is then the session CSRF token of Section 11.7 and is null for anonymous callers (who obtain the anonymous token from GET /auth/csrf). banner is the licensing banner object defined by Section 15.12 (always { "key": "none", ... } in the self-hosted edition). When setup_required is true the SPA redirects every route to /setup.
The signup page hides itself and shows "Registration is closed. Ask an administrator for an invitation." when the mode is disabled, or when it is invite_only and no invitation_token query parameter is present (an invitation email link always carries it).
Changing the mode takes effect on the next process start; it is not a runtime setting. Existing accounts are never affected by a mode change.
11.3 Signup and Email Verification #
11.3.1 Signup request #
POST /api/app/v1/auth/signup (unauthenticated; anonymous CSRF token required, Section 11.7).
| Field | Type | Rules |
|---|---|---|
name |
string | Trimmed, NFC-normalized, 1–100 characters, no control characters (U+0000–U+001F, U+007F) |
email |
string | Trimmed and lowercased entirely; maximum 254 characters; Zod email syntax; domain must contain a dot; uniqueness is case-insensitive among all rows including soft-deleted ones awaiting anonymization |
password |
string | Per Section 11.5 |
accept_terms |
boolean | Must be true when terms acceptance is required (Section 11.1 table); otherwise ignored |
invitation_token |
string, optional | Required in invite_only; when present in any mode the email must match the invitation (403 invitation_email_mismatch otherwise) |
There is no disposable-email blocklist: disposable-email domains are not blocked (decision: no maintained blocklist in the product). Throwaway signups on the hosted edition are bounded by mandatory email verification before a site can be created (Section 11.3.3) and by the signup_ip policy (Section 17.6).
Processing order:
- If
setup_requiredis true, return403 setup_required. Otherwise check the registration mode and the hosted super-admin bypass (Section 11.2); failure returns403 registration_closed. - Validate fields; failures return
400 validation_failedwithdetails.fields. - Check email uniqueness; a duplicate returns
409 email_taken. This reveals account existence by decision (login and password reset do not); thesignup_ippolicy of Section 17.6 bounds enumeration. - Hash the password (Section 11.5), insert the user, and in the same transaction insert the personal team (Section 11.14.1) and the owner membership.
- If an
invitation_tokenwas supplied and valid, mark the email verified, then apply the invitation exactly as an acceptance (Section 11.15.4) in the same transaction. Asite_transferinvitation accepted at signup moves the site into the new personal team, subject to the site-limit check of Section 15; if that check fails the account is still created and the invitation stays pending. - If verification is required and the signup was not via an invitation, create a
verify_emailtoken and enqueue theverify_emailemail (Section 12.8). - Create a session (Section 11.6) and return
201.
Response body:
{
"data": {
"user": {
"id": "0190...",
"name": "Ada Lovelace",
"email": "ada@example.com",
"email_verified": false,
"two_factor_enabled": false,
"theme_preference": "system",
"is_super_admin": false,
"created_at": "2026-11-03T10:00:00Z"
},
"team": { "id": "0190...", "name": "Ada Lovelace's team", "role": "owner" },
"csrf_token": "..."
}
}The signup writes audit event user.registered (Section 11.20).
11.3.2 Verification email and token #
| Property | Value |
|---|---|
| Token | 32 random bytes, base64url (43 characters) |
| Storage | HMAC-SHA-256 (key purpose email-token-hmac, Section 17.7) in email_tokens with kind verify_email; plaintext is never stored |
| Lifetime | 24 hours from creation |
| Use | Single use; used_at is set on success |
| Link | {BASE_URL}/verify-email/<token> |
| Superseding | Creating a new verify_email token deletes all earlier unused tokens for the user |
POST /api/app/v1/auth/verify-email with { "token": "..." } (unauthenticated, no session needed; the token proves mailbox control). Success returns 200 {"data": {"email_verified": true}} and writes user.email_verified. Failures: 422 token_invalid (unknown, used or malformed) and 422 token_expired. Both render a page with a "Send a new link" button when the visitor is signed in, or a login prompt otherwise.
POST /api/app/v1/auth/verify-email/resend (authenticated): returns 202 with empty data; 409 already_verified if already verified; limited to 3 per hour per user.
11.3.3 What an unverified user can and cannot do #
| Allowed | Blocked (403 email_not_verified) |
|---|---|
| Sign in, view their teams, change password, enable 2FA, manage sessions, read dashboards of sites they already have access to, accept invitations (which verifies the email) | Create a site, create a team, create or rotate an API key, send invitations, start a license purchase or redeem a code (Section 15), start an import, create a shared link or email report subscription for an external recipient, change email |
The SPA shows a persistent banner "Verify your email to unlock all features" with a "Resend email" action.
Unverified accounts with no sites, no memberships in other teams and age over 14 days are deleted by the unverified-user-cleanup job (Section 11.23.2).
11.4 Login #
11.4.1 Credential step #
POST /api/app/v1/auth/login with { "email": "...", "password": "..." }. The email is normalized as in signup.
Processing:
- Apply the rate limits and lockout of Section 11.8. A locked email returns
429 rate_limitedwithdetails.reason = "account_locked"and aRetry-Afterheader, even when the password would have been correct. - Look up the user by email among rows with
anonymized_at IS NULL(so an account inside its 30-day deletion window can still log in and restore itself). If no row exists, verify the submitted password against a fixed dummy argon2id hash (same parameters) so the response time does not reveal existence. - Verify the password. On any failure return
401 invalid_credentials("Email or password is incorrect.") and increment the failure counters. 3a. Ifusers.suspended_atis set, return403 account_suspended(after the password was verified, so suspension is not revealed to wrong-password attempts). The suspension is set and lifted by the operator console (Section 15). - If the account is soft-deleted and still within its 30-day window, the credentials are accepted and the response sets
account_pending_deletion: true(Section 11.13.3). - If 2FA is enabled, do not create a session; return
401 two_factor_requiredwith the challenge (Section 11.4.2). - Otherwise create a session and return
200with the same body shape as signup, minusteam, plusteams(array of{id, name, role}) andsites_shared_with_me_count.
On success the failure counters for that email are cleared, users.last_login_at is updated, and audit event user.login is written with metadata { "method": "password" }.
Verified-email status does not block login (Section 11.3.3).
11.4.2 Second-factor step #
When 2FA is enabled the first step returns 401 two_factor_required (the code of Section 4.4.3) with the challenge in details:
{ "error": { "code": "two_factor_required", "message": "A verification code is required.", "details": { "challenge_token": "chl_...", "expires_in": 300 }, "request_id": "req_..." } }The challenge token is valid for 300 seconds. It is 32 random bytes, base64url, prefixed chl_, stored in Valkey under login_challenge:<sha256> for 300 seconds with fields user_id and attempts. It is not a session and grants nothing but the right to call the second step.
POST /api/app/v1/auth/login/2fa with { "challenge_token": "...", "code": "123456" } or { "challenge_token": "...", "recovery_code": "ABCD-EFGH-JKMN-PQRS" } (exactly one of code and recovery_code).
| Rule | Value |
|---|---|
| TOTP code format | Six digits; spaces and hyphens are stripped before validation |
| Recovery code format | 16 Crockford base32 characters in four groups of four joined by hyphens; case-insensitive; the characters I, L, O, U are mapped as in Crockford normalization before lookup |
| Attempts per challenge | 5; the fifth failure deletes the challenge and returns 401 challenge_expired; the user must restart at the credential step |
| Wrong code | 401 invalid_two_factor_code for a TOTP code, 401 invalid_recovery_code for a recovery code |
| Replay | A TOTP code whose time step is less than or equal to users.totp_last_used_step is rejected with 401 invalid_two_factor_code |
| Success | Session created; challenge deleted; user.login audited with { "method": "password+totp" } or { "method": "password+recovery_code" } |
| Recovery code use | The code is marked used (single use), the recovery_code_used email is sent, and the response includes recovery_codes_remaining |
If the user belongs to a team that enforces 2FA and has not enabled it, no challenge happens (there is nothing to verify). The session is created with two_factor_enrollment_required: true and the restrictions of Section 11.14.4 apply.
11.4.3 Logout #
POST /api/app/v1/auth/logout (authenticated, CSRF required) deletes the current auth_sessions row and its Valkey cache entry, clears the cookie (Max-Age=0), and returns 204. Audit event user.logout. Logging out never affects other sessions.
11.5 Password Policy and Hashing #
11.5.1 Policy #
| Rule | Decision |
|---|---|
| Minimum length | 12 characters (counted as Unicode code points after NFKC normalization) |
| Maximum length | 128 characters; longer input returns 400 validation_failed (bounds hashing cost) |
| Composition rules | None (no required digit, symbol or case mix) |
| Normalization | NFKC before hashing and before every comparison, so the same visible password on different keyboards verifies identically |
| Whitespace | Allowed anywhere, including leading and trailing; never trimmed |
| Forbidden | Passwords equal to the account email, the email local part, or the account name (case-insensitive comparison); passwords made of a single repeated character; passwords found in the common-password list |
| Common-password list | A bundled list of the 100,000 most common passwords at packages/shared/data/common-passwords.txt, stored lowercase, loaded into a Set at boot; comparison uses the lowercased normalized candidate. Returns 422 password_too_common |
| Breached-password check (decision) | Local list only by default. No outbound call is made, which keeps the product free of third-party requests. An operator may set PASSWORD_BREACH_CHECK=hibp to additionally query the Have I Been Pwned range API with k-anonymity (only the first 5 hex characters of the SHA-1 are sent); the call has a 2-second timeout, a hit returns 422 password_breached, and any timeout or error fails open (the password is accepted and a warning is logged without the candidate). Default value is local |
| Strength meter | The SPA shows a length-based meter only (no entropy library); the meter never blocks submission |
| Reuse | New password must differ from the current one (422 password_unchanged); no history of older passwords is kept |
Error precedence when several rules fail: length, forbidden relation, common list, breach check. One error is returned.
11.5.2 Hashing parameters #
Passwords are hashed with argon2id via the argon2 package (Section 3).
| Parameter | Value |
|---|---|
| Algorithm | argon2id |
| Memory cost | 65,536 KiB (64 MiB) |
| Time cost | 3 iterations |
| Parallelism | 1 |
| Salt | 16 random bytes per hash |
| Output length | 32 bytes |
| Encoding | Standard PHC string stored in users.password_hash |
| Pepper | None (decision) |
| Rehash on login | After a successful verification, if the stored hash's parameters differ from the current constants, rehash with the current parameters and update the row |
| Concurrency | A per-process semaphore allows at most 4 concurrent hash or verify operations; further callers wait up to 5 seconds and then receive 503 unavailable with Retry-After: 2 |
| Timing equalization | Unknown-email logins verify against a precomputed dummy hash with identical parameters |
The parameters live in constants ARGON2_PASSWORD_PARAMS in packages/shared. A shared-link password uses lighter parameters defined in Section 12.2.4.
11.6 Sessions and Cookies #
11.6.1 Session cookie #
| Property | Value |
|---|---|
| Name | __Host-qs_session when BASE_URL is https; qs_session when BASE_URL is http (local development only) |
| Value | 32 random bytes, base64url (43 characters). The server stores only its SHA-256 hash |
HttpOnly |
Yes |
Secure |
Yes whenever the name carries the __Host- prefix |
SameSite |
Lax |
Path |
/ |
Domain |
Not set (host-only) |
Max-Age |
Remaining absolute lifetime in seconds |
| Absolute lifetime | 30 days from creation. After that the session is invalid and the user must log in again |
| Idle timeout | 7 days since last_seen_at. Each authenticated request refreshes last_seen_at, written to Postgres at most once per 5 minutes per session |
| "Remember me" | Not offered; every session has the same lifetime |
| Concurrent session cap | 50 per user. Creating the 51st deletes the session with the oldest last_seen_at. Impersonation sessions (rows with impersonator_user_id) do not count toward the cap |
| Impersonation cookie | Operator impersonation (Section 15) uses a separate cookie, __Host-qs_imp (qs_imp over plain HTTP), with the same attributes and Max-Age=1800. When present it takes precedence over the session cookie, and the operator's own session cookie is left untouched |
11.6.2 Storage and lookup #
- Each request carrying the cookie hashes the value and looks up
session:<hash>in Valkey (TTL 300 seconds). A miss falls back toauth_sessionsin Postgres, joins the user, and repopulates the cache. - A session is valid when the row exists,
expires_at > now(),last_seen_at > now() - interval '7 days', the user hasdeleted_at IS NULL(except the deletion-restore flow, Section 11.13.3) andsuspended_at IS NULL(otherwise the request returns403 account_suspended). - Revoking a session deletes the Postgres row and the Valkey key in the same operation. Revocation is therefore immediate on every web process.
- An invalid cookie is cleared in the response and the request is treated as unauthenticated (
401 unauthenticated). - Every request on an impersonation session first re-checks that the impersonating user is still a super admin (email verified and listed in
SUPER_ADMIN_EMAILS, TOTP enabled); otherwise the server deletes the session and returns401 unauthenticated.
11.6.3 Rotation #
| Trigger | Behavior |
|---|---|
| Login | A brand-new session token is issued; any pre-login anonymous cookie is discarded |
| Password change or reset | All other sessions of the user are deleted; the current session gets a new token |
| 2FA enabled or disabled; recovery codes regenerated | Current session gets a new token; other sessions are kept |
| Age | On the first request after a session token is 24 hours old, the server issues a new token for the same session row, stores the old hash in a previous-token slot valid for 60 seconds (so concurrent in-flight requests still authenticate), and sets the new cookie |
The csrf_token is bound to the session row ID, not the cookie value, so rotation never invalidates it (Section 11.7).
11.7 CSRF Protection #
Cookie-authenticated mutating requests (POST, PUT, PATCH, DELETE) on /api/app/v1 are protected by a synchronizer token plus origin validation.
- Authenticated token.
csrf_token = base64url(HMAC-SHA256(csrf_key, session_id))wherecsrf_keyis derived fromSECRET_KEY_BASEwith HKDF (purposecsrf, Section 17.7). It is returned by login, signup andGET /api/app/v1/auth/session. The SPA keeps it in memory and sends it in headerX-CSRF-Token. - Anonymous token. Unauthenticated mutating endpoints (setup, signup, login, login/2fa, password forgot and reset, email verification, invitation decline and acceptance for new users) require an anonymous token.
GET /api/app/v1/auth/csrfsets cookie__Host-qs_anon(qs_anonover plain HTTP;HttpOnly,Secure,SameSite=Lax,Path=/,Max-Age=86400; 32 random bytes) if absent, and returns{"data": {"csrf_token": "<HMAC of the cookie value>"}}. - Origin check. If an
Originheader is present it must equal the origin ofBASE_URL; otherwise403 csrf_invalid. IfOriginis absent andSec-Fetch-Siteis present, the value must besame-originornone. If neither header is present the token alone decides. - Comparison. Constant-time. A missing or wrong token returns
403 csrf_invalid. The SPA reacts to the first such response by re-fetching the token (GET /auth/sessionor/auth/csrf) and retrying the request once. - Exempt routes. Any route that authenticates with a bearer secret instead of the cookie is exempt:
POST /api/event,/api/v1/*,/api/v2/*,/api/share/v1/*,POST /api/app/v1/unsubscribe/*(the unsubscribe token is the credential),POST /api/app/v1/csp-report(unauthenticated, changes no state), and the Stripe webhook route of Section 15.GET,HEADandOPTIONSrequests never need a token and must never change state. The one exception isGET /api/app/v1/integrations/google/callback(Section 14.5.2), which creates an import row and is protected by its single-usestatetoken bound to the session user instead of a CSRF token. - CORS.
/api/app/v1sends no CORS headers; browsers block cross-origin reads.
11.8 Rate Limits and Lockout #
Limits and windows are owned by the rate-limit table in Section 17.6; this section assigns a policy to each flow and the policy ID is the rule name used in details.reason. Limits use fixed windows in Valkey (@fastify/rate-limit, Section 3). The client IP is resolved per Section 8 (trusted-proxy rules). Exceeding a limit returns 429 rate_limited with a Retry-After header in seconds and details = { "retry_after_seconds": N, "reason": "<policy ID>" }.
| Policy ID | Applies to | Scope key |
|---|---|---|
signup_ip |
POST /auth/signup |
IP |
login_ip |
POST /auth/login |
IP |
login_email |
POST /auth/login (failures, then lock) |
SHA-256 of the normalized email (whether or not the account exists) |
login_2fa_ip |
POST /auth/login/2fa |
IP |
login_2fa_user |
POST /auth/login/2fa (failures) |
user ID |
password_forgot_email |
POST /auth/password/forgot |
SHA-256 of the normalized email |
password_forgot_ip |
POST /auth/password/forgot |
IP |
password_reset_ip |
POST /auth/password/reset |
IP |
verify_resend_user |
POST /auth/verify-email/resend |
user ID |
verify_token_ip |
POST /auth/verify-email |
IP |
email_change_user |
POST /account/email |
user ID |
reauth_user |
Failed current-password or code checks across all step-up endpoints | user ID |
invitation_create_team |
Creating invitations of all kinds | team ID |
invitation_resend |
POST /invitations/:id/resend |
invitation ID |
invitation_lookup_ip |
GET /invitations/lookup and POST /invitations/decline |
IP |
unsubscribe_ip |
POST /unsubscribe/:token |
IP |
Lockout behavior (login_email). The fifth failed attempt inside 15 minutes sets login_lock:<hash> for 15 minutes. While the lock exists every login attempt for that email returns 429 with details.reason = "account_locked", whether or not the password is right. The lock cannot be lifted by the user except by completing a password reset (Section 11.9), which clears it. Counters apply identically to unknown emails, so lockout state does not reveal existence. When a lock is set for an existing account, the login_lockout_notice email is enqueued once per lock (Section 12.8) and audit event user.login_locked is written. Failed individual attempts are not audited.
Locking by email makes targeted denial of login possible for 15 minutes at a time; this is accepted in exchange for brute-force resistance, and the legitimate user can reset the password to regain access immediately.
A successful 2FA second step clears login_2fa_user. Outbound email limits per recipient are enforced by the per-flow limits above, not by a global limiter.
All other dashboard API routes use the app_api_user and app_api_ip_unauth policies of Section 17.6.
11.9 Password Reset #
- Request.
POST /api/app/v1/auth/password/forgotwith{ "email": "..." }. The response is always202with empty data, whether or not an account exists, so the endpoint reveals nothing. Rate limits:password_forgot_emailandpassword_forgot_ip(Section 11.8). - Token. When a non-anonymized account exists (including one inside its 30-day deletion window): create an
email_tokensrow of kindpassword_reset; 32 random bytes base64url; HMAC-SHA-256 stored (key purposeemail-token-hmac, Section 17.7); valid 1 hour; single use; creating a new token deletes earlier unused reset tokens for the user. Enqueue thepassword_resetemail containing{BASE_URL}/reset-password/<token>. Audituser.password_reset_requested(only when an account exists). - Completion.
POST /api/app/v1/auth/password/resetwith{ "token": "...", "password": "..." }. The password must satisfy Section 11.5. Errors:422 token_invalid,422 token_expired,422 password_too_common,422 password_breached,422 password_unchanged,400 validation_failed. - Effects of success (one transaction): replace
password_hash; mark the token used; mark the email verified (the token proves mailbox control); delete all sessions of the user; clear thelogin_emailfailure counters and any lock; send thepassword_changednotice; audituser.password_reset_completed. - No automatic login and no 2FA bypass. The user is redirected to the login page. If 2FA is enabled it is still required at the next login. A password reset never disables 2FA.
- Accounts pending deletion (Section 11.13.3) can reset their password and then restore the account.
Authenticated password change: POST /api/app/v1/account/password with { "current_password", "new_password", "code"? } (code is the TOTP or recovery code, required when 2FA is enabled). On success: all other sessions are deleted, the current session token is rotated, the password_changed email is sent, audit user.password_changed. A wrong current password returns 422 current_password_incorrect and a wrong code returns 422 two_factor_code_incorrect; both count against reauth_user.
11.10 Email Change #
POST /api/app/v1/account/emailwith{ "new_email", "current_password", "code"? }(coderequired when 2FA is on). The new address is normalized like signup. Errors:422 email_unchangedwhen equal to the current address,409 email_takenwhen already used,422 current_password_incorrect,422 two_factor_code_incorrect,403 email_not_verifiedwhen the current email is unverified (verify first).- The server creates a
change_emailtoken storing the new address (32 random bytes prefixedce_, HMAC-SHA-256 stored with key purposeemail-token-hmac, valid 24 hours, single use; a newer request supersedes the earlier one), sendsemail_change_confirmto the new address andemail_change_requested_noticeto the current address. Returns202. The current address stays active until confirmation. - The confirmation link is
{BASE_URL}/verify-email/<token>; the verification page callsPOST /api/app/v1/account/email/confirmfor tokens beginningce_andPOST /api/app/v1/auth/verify-emailfor all others.POST /api/app/v1/account/email/confirmtakes{ "token" }(anonymous CSRF token, no session required). The server re-checks uniqueness (409 email_takenif another account took the address in the meantime), then in one transaction updates the email, sets the email verified, deletes all sessions of the user (the user logs in again with the new email), and sendsemail_changed_noticeto the old address. Audituser.email_changedwith metadataold_emailandnew_email. GET /api/app/v1/accountincludespending_email(nullable) with its expiry;DELETE /api/app/v1/account/email/pendingcancels it (204).- Pending invitations addressed to the old email are not rewritten; the user must ask for a new invitation if needed.
- Hosted: operator status follows the email: a user whose new verified address is not in
SUPER_ADMIN_EMAILSimmediately losesis_super_admin, and vice versa (subject to the TOTP requirement of Section 11.19.1).
11.11 Two-Factor Authentication (TOTP) #
11.11.1 Algorithm parameters #
| Parameter | Value |
|---|---|
| Standard | RFC 6238 TOTP via otplib (Section 3) |
| HMAC | SHA-1 (maximum authenticator-app compatibility) |
| Digits | 6 |
| Period | 30 seconds |
| Accepted window | The current step plus or minus one step (90 seconds total) |
| Secret | 20 random bytes (160 bits), base32 encoded for display |
| Issuer label | PRODUCT_NAME (Section 16.4); account label is the user email |
| Provisioning URI | otpauth://totp/<PRODUCT_NAME>:<email>?secret=<base32>&issuer=<PRODUCT_NAME>&algorithm=SHA1&digits=6&period=30 (URL-encoded) |
| Secret storage | Encrypted at rest per Section 17.7.2 (AES-256-GCM, key purpose totp-encryption) and stored in users.totp_secret_encrypted. Rotation of SECRET_KEY_BASE re-encrypts the value without re-enrolling users (Section 17.7.4) |
| Replay protection | users.totp_last_used_step stores the last accepted step; a code from a step less than or equal to it is rejected |
11.11.2 Setup #
POST /api/app/v1/account/2fa/setupwith{ "current_password" }. Errors:422 current_password_incorrect,409 two_factor_already_enabled. Returns{"data": {"secret": "<base32>", "otpauth_uri": "...", "expires_in": 600}}. The pending secret is held encrypted in Valkey for 10 minutes (keytotp_setup:<user_id>); calling setup again replaces it. The SPA renders the QR code client-side fromotpauth_uriand also displays the secret as groups of four characters for manual entry.POST /api/app/v1/account/2fa/enablewith{ "code" }. The code is checked against the pending secret (window ±1). Errors:400 two_factor_setup_expired(no pending secret),422 two_factor_code_incorrect.- On success, in one transaction: persist the encrypted secret and
totp_enabled_at; settotp_last_used_stepto the matched step; delete any old recovery codes and create 10 new ones; delete the pending Valkey key. The response contains the codes once:
{ "data": { "recovery_codes": ["ABCD-EFGH-JKMN-PQRS", "..."], "recovery_codes_remaining": 10 } }- The current session token is rotated, the
two_factor_enabledemail is sent, audituser.2fa_enabled.
11.11.3 Recovery codes #
| Property | Value |
|---|---|
| Count | 10 per generation |
| Format | 16 Crockford base32 characters (80 bits) displayed as XXXX-XXXX-XXXX-XXXX, uppercase |
| Storage | SHA-256 of the normalized code (no hyphens, uppercase) in user_recovery_codes, plus used_at |
| Use | Single use at login (Section 11.4.2) or in place of a TOTP code in disable and regenerate calls |
| Regeneration | POST /api/app/v1/account/2fa/recovery-codes with { "current_password", "code" } (a TOTP code only; recovery codes cannot regenerate recovery codes). All previous codes, used or not, are deleted and 10 new ones are returned once. Email recovery_codes_regenerated, audit user.2fa_recovery_codes_regenerated |
| Low-stock warning | When recovery_codes_remaining is 2 or fewer, the account security page shows a warning banner and the login success response carries recovery_codes_remaining |
| UI | The codes dialog offers Copy, Download as .txt (filename <product>-recovery-codes.txt, generated client-side) and Print; closing requires ticking "I have saved these codes" |
GET /api/app/v1/account/2fa returns {"data": {"enabled": true, "enabled_at": "...", "recovery_codes_remaining": 8}}.
11.11.4 Disabling #
POST /api/app/v1/account/2fa/disable with { "current_password", "code" | "recovery_code" }.
| Condition | Result |
|---|---|
| 2FA not enabled | 409 two_factor_not_enabled |
User is a member (any role) of any team with require_2fa = true |
403 two_factor_required_by_team, details.teams = [{id, name}] |
| Wrong password or code | 422 current_password_incorrect or 422 two_factor_code_incorrect; counts against reauth_user |
| Success | Delete secret and all recovery codes; rotate the session; send two_factor_disabled; audit user.2fa_disabled |
11.11.5 Lost authenticator and lost codes #
There is no self-service bypass. Recovery paths, in order:
- Use a saved recovery code at login.
- Hosted: an operator resets 2FA from the operator console (
POST /api/app/v1/admin/users/:user_id/reset-2fa, super admin only, with step-up by password and TOTP code and a mandatory reason, owned by Section 15). The reset deletes the TOTP secret, the recovery codes and all sessions of the user. The user receivestwo_factor_disabled, the action is audited asadmin.user_2fa_reset, and teams with enforcement force re-enrollment at the next login. - Self-hosted: the administrator runs the bundled command-line tool through the image entrypoint:
docker compose run --rm web cli users:reset-2fa <email>. The same tool offersusers:set-password <email>(prompts for the new password without echo) for instances without SMTP, andusers:test-email <address>to check SMTP delivery. The first two write the same audit events with actor typesystem, no actor user, and metadata{ "source": "cli" }.
11.12 Account Settings #
The account area has five panels: Profile, Security, Login management, Appearance and Delete account. Routes follow the route table of Section 4.13.3: /account/profile (profile, email change, appearance, delete account) and /account/security (password, two-factor, recovery codes, login management); the report subscriptions of the user across sites are listed at /account/notifications (Section 12.12).
11.12.1 Profile #
GET /api/app/v1/account returns the current user object (the shape in Section 11.3.1 plus last_login_at, pending_email, pending_deletion_at). PATCH /api/app/v1/account accepts name (rules of Section 11.3.1), theme_preference and last_team_id (Section 11.14.2). Email and password change through their own endpoints. An empty body returns 400 validation_failed.
11.12.2 Appearance #
| Property | Decision |
|---|---|
| Options | light, dark, system |
| Default | system |
| Storage | users.theme_preference is the source of truth and is applied after login; the SPA mirrors it into localStorage key qs_theme, which the static file /assets/theme-init.js (loaded before the app bundle; no inline script exists, Section 17.5) reads so the first paint uses the right theme without flashing |
| Application | A data-theme="light" or data-theme="dark" attribute on the <html> element selects the design-token set of Section 4. For system, the SPA listens to prefers-color-scheme changes and updates live |
| Logged-out pages and shared or embedded dashboards | Follow prefers-color-scheme, except embeds, which honor their theme parameter (Section 12.4.2) |
| Endpoint | PATCH /api/app/v1/account with { "theme_preference": "dark" }; the change is applied optimistically and does not rotate any session |
11.12.3 Login management #
GET /api/app/v1/account/sessions returns the user's sessions (not paginated; at most 50 by the cap in Section 11.6.1). Impersonation sessions (Section 15) are never listed, cannot be revoked by the user and are not touched by "revoke all others":
{
"data": [
{
"id": "0190...",
"current": true,
"device_label": "Chrome 141 on macOS 15",
"country": "DE",
"created_at": "2026-11-01T08:00:00Z",
"last_seen_at": "2026-11-03T09:55:00Z",
"expires_at": "2026-12-01T08:00:00Z"
}
],
"meta": { "next_cursor": null, "limit": 50 }
}| Rule | Decision |
|---|---|
device_label |
Built at session creation with ua-parser-js as <Browser> <major> on <OS> <major>; Unknown device when parsing fails. The raw User-Agent is not stored |
country |
ISO 3166-1 alpha-2 from the GeoIP database at login; null for private or unresolvable addresses. The IP address itself is never stored |
| Revoke one | DELETE /api/app/v1/account/sessions/:id returns 204; revoking the current session also logs out the caller; unknown or foreign ID returns 404 |
| Revoke all others | DELETE /api/app/v1/account/sessions revokes every session except the current one and returns {"data": {"revoked": 3}}. Audit user.sessions_revoked_all |
| Sort | last_seen_at descending, current session pinned first |
11.13 Account Deletion #
11.13.1 Request #
DELETE /api/app/v1/account with { "current_password", "code"?, "confirm_email", "acknowledge_license_loss"? }. confirm_email must equal the account email (422 confirm_email_mismatch); a wrong password or code returns 422 current_password_incorrect or 422 two_factor_code_incorrect.
Preconditions, evaluated in this order:
- For every team the user owns that has at least one other member or a pending invitation, the request fails with
409 owner_must_transfer,details.teams = [{id, name, member_count}]. The user must transfer ownership (Section 11.16.4) or delete the team first. - In the hosted edition, if an owned sole-member team holds an active license and
acknowledge_license_lossis nottrue, the request fails with409 license_acknowledgement_required. The UI explains that the license is bound to the team and is released as defined in Section 15 when the team is purged.
Effects at request time (one transaction):
| Effect | Detail |
|---|---|
| Account flag | users.deleted_at = now() |
| Sessions | All sessions deleted, including the current one |
| Teams owned alone | Scheduled for deletion with the 7-day undo window of Section 11.14.5 |
| Memberships in other teams | Removed immediately with the cleanup of Section 11.16.5 (not restored by a later cancellation; re-invitation is needed) |
| Site guest access | Removed immediately |
| API keys created by the user | Deleted immediately |
| Exports requested by the user | Deleted immediately (Section 11.16.5) |
| Email report and spike subscriptions of the user | Deleted immediately |
| Notification | account_deletion_scheduled email to the account address, naming the date after which the account is anonymized |
| Audit | user.deletion_requested |
The response is 202 with {"data": {"deleted_at": "...", "anonymize_after": "<deleted_at + 30 days>"}}.
11.13.2 Anonymization #
The user-anonymize job (Section 11.23.2) anonymizes users whose deleted_at is older than 30 days. The user row is kept so audit and ownership references stay intact:
| Field | After anonymization |
|---|---|
email |
deleted-<user id>@deleted.invalid |
name |
Deleted user |
password_hash, totp_secret_encrypted |
NULL |
| Recovery codes, email tokens, sessions | Deleted |
| Audit log rows where the user is the actor | Kept; old_email and new_email metadata keys removed; actor displays as "Deleted user" |
anonymized_at |
Set to now |
The original email address becomes available for new signups only after anonymization.
11.13.3 Restoring within the window #
During the 30 days the user can still log in with the old credentials. The login succeeds with account_pending_deletion: true and a restricted session: every route except GET /auth/session, POST /account/restore, POST /auth/logout returns 403 account_pending_deletion. The SPA shows a single screen: "This account is scheduled for deletion on . Restore it?" with Restore and Log out buttons.
POST /api/app/v1/account/restore clears deleted_at, restores the teams that this deletion scheduled (those still inside their 7-day window), and audits user.deletion_cancelled. Teams already purged cannot be restored; the response lists restored_teams and unrecoverable_teams.
11.14 Teams #
11.14.1 Model and creation #
A team owns sites, goals, funnels, segments, API keys, members, invitations and (hosted) one license. Every site belongs to exactly one team. A team has exactly one owner at all times: the owner is the membership with role owner (there is no separate owner column), and the one-owner constraint of Section 5 is checked at commit.
| Rule | Decision |
|---|---|
| Personal team | Created at signup, named <Name>'s team where <Name> is the user's name truncated to 53 characters. It is an ordinary team in every respect (can be renamed, can receive members, can be deleted); teams.is_personal only records origin and is shown nowhere in the UI |
| Additional teams | POST /api/app/v1/teams with { "name" }; requires a verified email; the creator becomes owner. Name: trimmed, 1–60 characters, no control characters, not required to be unique |
| Limit | A user may own at most 20 non-deleted teams (409 limit_reached with details.limit_name = "teams_owned"; the value is listed with the service limits of Section 5.16) |
| Membership limit | None (no per-member pricing exists) |
| New team state, hosted | Unlicensed until a license is redeemed or purchased (Section 15); self-hosted teams have no licensing |
| Last team | A user may delete every team they own; they then see the empty state "Create a team to add your first site" |
GET /api/app/v1/teams returns the teams the caller is a member of, sorted by name case-insensitively, each as:
{
"id": "0190...", "name": "Acme", "role": "admin", "is_personal": false,
"member_count": 4, "site_count": 12, "require_2fa": false,
"deletion_scheduled_at": null, "created_at": "2026-10-01T09:00:00Z"
}Hosted responses add a license summary object defined by Section 15.
GET /api/app/v1/teams/:team_id returns one team (404 for non-members).
11.14.2 Switching #
- The app shell of Section 4 scopes every team URL by team ID; the team switcher is a menu in the top-left of the shell listing all teams the user belongs to (alphabetical, with the role as a secondary label) and, below a divider, a "Shared with me" entry when the user holds any site guest access (Section 11.18.4).
- Choosing a team navigates to that team's sites list and records the choice with
PATCH /api/app/v1/accountfieldlast_team_id(must reference a team the user belongs to; otherwise422 validation_failed). - After login the SPA opens
last_team_id. If it is null, deleted or no longer accessible, it opens the first team alphabetically; if there is none, it opens "Shared with me" when guest access exists, else the empty state. - Switching never changes permissions of in-flight requests: authorization is always evaluated against the team or site named in the request (Section 11.19.3).
- A team scheduled for deletion appears in the switcher (owner only) with a "Scheduled for deletion" badge and opens a restore screen (Section 11.14.5); other members no longer see it.
11.14.3 Team settings #
PATCH /api/app/v1/teams/:team_id accepts name (owner or admin; action team.settings.update) and require_2fa (owner only; action team.require_2fa.set). Fields the caller may not change return 403 forbidden for the whole request. Audit events team.renamed and team.require_2fa_changed.
11.14.4 Enforcing two-factor authentication #
| Rule | Decision |
|---|---|
| Who | Only the owner can toggle require_2fa, and only while the owner has 2FA enabled (422 owner_two_factor_required otherwise) |
| On enable | All current members (not guests) without 2FA receive the team_2fa_enforced email immediately; there is no grace period |
| Member without 2FA, restriction | Every request to that team's resources (any route scoped to the team or to one of its sites) returns 403 two_factor_enrollment_required with details.team_id. Still allowed: login, logout, the account security endpoints of Section 11.11, listing teams, POST /teams/:team_id/leave, and account deletion. The SPA redirects such users to a full-page "Set up two-factor authentication to continue" screen that embeds the setup flow |
| Invitations | A new member accepting an invitation to an enforcing team is admitted, then restricted until 2FA is enabled |
| Disabling own 2FA | Blocked while the user belongs to any enforcing team (403 two_factor_required_by_team) |
| Operator reset | After an operator reset (Section 11.11.5) the user is restricted again |
| Guests | Site guests are outside the team and are not subject to enforcement |
| Ownership transfer | When require_2fa is on, the transfer target must already have 2FA (422 target_two_factor_required) |
| On disable | Takes effect immediately; no email |
| API keys and shared links | Unaffected |
11.14.5 Team deletion and undo #
DELETE /api/app/v1/teams/:team_id (owner; action team.delete) with { "current_password", "code"?, "confirm_name" }. confirm_name must equal the team name exactly (422 confirm_name_mismatch). A wrong password or code returns 422 current_password_incorrect or 422 two_factor_code_incorrect. Hosted teams holding an active license additionally require "acknowledge_license_loss": true (409 license_acknowledgement_required).
Effects: teams.deleted_at = now(), teams.purge_after = now() + 7 days.
| During the 7-day window | Behavior |
|---|---|
| Visibility | Only the owner can see the team (restore screen). Members, guests and API keys receive 404 |
| Tracking | Events for the team's sites are dropped with 202 (Section 8 treats the site as inactive) |
| Shared links, embeds, public dashboards | Return 404 |
| Scheduled emails | Email reports and spike notifications are suspended (the jobs skip sites of deleted teams) |
| Domains | Remain reserved; no other team can add the same domain |
| Notification | team_deletion_scheduled email to every member at deletion time |
| Restore | POST /api/app/v1/teams/:team_id/restore by the owner clears deleted_at; sites resume collecting events immediately; the gap is not backfilled. Audit team.deletion_cancelled |
The team-purge job (Section 11.23.2) hard-deletes teams past purge_after: sites (calling freezeSiteUsage(team_id, site_id) of Section 15.9.4 first on the hosted edition, then enqueueing ClickHouse purge per Section 6), goals, funnels, segments, annotations, shared links, API keys, memberships, invitations, guests, subscriptions and the team's audit rows, except rows whose action starts with license., purchase., code. or admin. and the team.purged row. Those rows are kept with their team_id value retained (the column has no foreign key) so refund and chargeback evidence survives, and they are removed only by the audit retention job. License handling at purge follows Section 15. Nothing else is recoverable after purge.
11.15 Invitations #
11.15.1 Kinds and shared rules #
All invitations share one table (invitations, Section 5) and one acceptance flow.
| Kind | Purpose | Roles allowed | Created by (action) |
|---|---|---|---|
team_member |
Add a person to a team | admin, editor, viewer |
team.members.invite (create); team.invitations.manage (list, resend, revoke) |
site_guest |
Give a person access to one site | guest_viewer, guest_editor |
site.guests.manage |
ownership_transfer |
Hand team ownership to an existing member | target becomes owner |
team.ownership.transfer |
site_transfer |
Offer a site to a person who will choose a destination team | not applicable | site.transfer |
| Property | Value |
|---|---|
| Token | 32 random bytes, base64url; only the HMAC-SHA-256 (key purpose email-token-hmac, Section 17.7) is stored |
| Lifetime | 7 days from creation or last resend |
| The invited address, lowercased and normalized as in Section 11.3.1 | |
| Pending limit | 100 pending invitations of all kinds per team (Section 5.16; 409 limit_reached with details.limit_name = "pending_invitations") |
| Duplicates | One pending invitation per (team, email, kind, site); a second returns 409 invitation_exists with details.invitation_id |
| Link | {BASE_URL}/invitations/<token> |
| Statuses (computed) | pending (not past expires_at), expired (past expires_at; remains listed until deleted by the invitation-expire job 30 days after expiry) |
| Verified inviter | Only users with a verified email may invite (403 email_not_verified) |
11.15.2 Creating a team invitation #
POST /api/app/v1/teams/:team_id/invitations with { "email", "role" }.
| Check | Error |
|---|---|
Caller lacks team.members.invite |
403 forbidden |
Role not in admin, editor, viewer |
422 invalid_role |
| Email equals the caller's own | 422 cannot_invite_self |
| Email already belongs to a member | 409 member_exists |
| Duplicate pending invitation | 409 invitation_exists |
| Pending limit reached | 409 limit_reached (details.limit_name = "pending_invitations") |
| Team scheduled for deletion | 404 not_found |
Admins may invite admins, editors and viewers. No invitation can create an owner. The response is 201 with the invitation object (id, kind, email, role, status, expires_at, invited_by, created_at), never the token. When email delivery is not configured (Section 12.9.6) the object also carries accept_url so the inviter can share the link manually. The team_invitation email is enqueued. Audit team.member_invited.
11.15.3 Listing, resending, revoking #
| Operation | Endpoint | Rules |
|---|---|---|
| List | GET /api/app/v1/teams/:team_id/invitations?status=pending|expired |
Cursor pagination per Section 4; returns the invitations of the kinds the caller may manage (site-scoped ones carry site_domain). Owners and admins see team_member, site_guest and site_transfer; only the owner also sees ownership_transfer. Permission by kind: team.invitations.manage for team_member, site.guests.manage for site_guest, team.ownership.transfer for ownership_transfer, site.transfer for site_transfer |
| Resend | POST /api/app/v1/invitations/:id/resend |
Generates a new token (the old link stops working), resets expires_at to now plus 7 days, resends the email; also revives expired invitations; at most 1 per minute and 5 per 24 hours per invitation (429 rate_limited); audit team.invitation_resent |
| Revoke | DELETE /api/app/v1/invitations/:id |
Hard delete; the link then returns 404 not_found; 204; audit team.invitation_revoked |
Permission to list, resend or revoke is the per-kind action named in the list row above; an inviter who has since been demoted can no longer manage the invitation, but the owner and admins always can (the owner alone for ownership_transfer).
11.15.4 Looking up and accepting #
GET /api/app/v1/invitations/lookup?token=<token>(unauthenticated) returns{"data": {"kind", "status", "team_name", "inviter_name", "role", "site_domain", "invited_email", "account_exists"}}. Unknown token returns404 not_found.account_existstells the SPA whether to show login or signup.- No account: the invitation page shows the signup form with the email prefilled and read-only and
invitation_tokenattached (Section 11.3.1). Signup accepts the invitation in the same request. This works in every registration mode except thatdisabledmode refuses (Section 11.2). - Has an account: the user logs in (the invitation page returns to itself after login) and clicks Accept.
POST /api/app/v1/invitations/acceptwith{ "token", "target_team_id"? }(target_team_idonly forsite_transfer). The authenticated account's email must equal the invited email, else403 invitation_email_mismatch; the SPA then offers "Log out and use the correct account". An unverified account becomes verified by accepting. - In-app acceptance:
GET /api/app/v1/account/invitationslists pending invitations addressed to the caller's verified email (shown as a banner and in account settings);POST /api/app/v1/account/invitations/:id/acceptand.../declineact on them without a token. - Decline:
POST /api/app/v1/invitations/declinewith{ "token" }(no login needed) or the in-app decline deletes the invitation. Auditteam.invitation_declined.
Effects of acceptance (one transaction, then the invitation row is marked accepted_at = now(), so the link returns 404 not_found from then on and the invitation-expire job removes the row later):
| Kind | Effect |
|---|---|
team_member |
Insert the membership with the invited role. If the user already holds guest access on any site of that team, those guest rows are deleted (membership supersedes). Already a member: 409 member_exists and the invitation is deleted |
site_guest |
Insert the guest row (Section 11.18) |
ownership_transfer |
Section 11.16.4 |
site_transfer |
Section 11.17.2 |
Errors: 422 token_expired (the invitation is past expires_at), 404 not_found (unknown, revoked, already accepted, or belonging to a team scheduled for deletion). After acceptance the inviter receives the invitation_accepted email (not for ownership and site transfers, which have their own completion emails). Audit team.invitation_accepted.
If the accepting user's team now enforces 2FA and they lack it, the restriction of Section 11.14.4 starts immediately.
11.16 Membership Management #
11.16.1 Member list #
GET /api/app/v1/teams/:team_id/members (any member; guests receive 404) returns {user_id, name, email, role, joined_at} for every member, sorted owner first, then by name. Owners and admins additionally see two_factor_enabled and last_login_at per member. The list never includes guests (they are listed on each site, Section 11.18).
11.16.2 Changing roles and removing members #
PATCH /api/app/v1/teams/:team_id/members/:user_id with { "role" } and DELETE /api/app/v1/teams/:team_id/members/:user_id.
| Rule | Result |
|---|---|
Caller must have team.members.change_role (owner, admin) or team.members.remove (owner, admin) |
403 forbidden otherwise |
| Target is the owner | 403 owner_protected (the owner changes only through ownership transfer) |
| Target is the caller | Role change: 409 cannot_change_own_role; removal: 409 use_leave |
New role is owner |
422 use_ownership_transfer |
New role not in admin, editor, viewer |
422 invalid_role |
| Admin changing or removing another admin | Allowed (admins manage all members except the owner) |
| Same role as current | 200 no-op, no audit row |
| Target not a member | 404 not_found |
A role change takes effect on the target's next request (authorization reads the membership each time; the Valkey session cache holds no roles). The target receives the member_role_changed email; removal sends member_removed. Audit team.member_role_changed (metadata from, to) and team.member_removed.
11.16.3 Leaving a team #
POST /api/app/v1/teams/:team_id/leave by any non-owner member (action team.leave). The owner receives 409 owner_cannot_leave and is directed to transfer ownership or delete the team. No email is sent to the leaver; audit team.member_left. A user pending 2FA restriction may leave (Section 11.14.4).
11.16.4 Ownership transfer #
- The owner starts it:
POST /api/app/v1/teams/:team_id/ownership-transferwith{ "user_id", "current_password", "code"? }.
| Check | Error |
|---|---|
| Target must be a current member other than the caller | 422 invalid_target |
| Target email verified | 422 target_email_unverified |
| Team enforces 2FA and target lacks it | 422 target_two_factor_required |
| A transfer is already pending for the team | 409 transfer_pending |
| Wrong password or code | 422 current_password_incorrect or 422 two_factor_code_incorrect |
- An
ownership_transferinvitation addressed to the target's email is created (7 days) and theownership_transfer_requestemail sent. The target also sees it in-app (Section 11.15.4 item 4). Ownership does not change yet; the owner keeps all rights. - The target accepts or declines. The owner may cancel with
DELETE /api/app/v1/teams/:team_id/ownership-transfer(204; auditteam.ownership_transfer_cancelled). Expiry deletes nothing visible; the owner may start again. - On acceptance, atomically: the target's membership role becomes
ownerand the previous owner's becomesadmin(the one-owner constraint is checked at commit); the hosted license stays attached to the team and the billing contact becomes the new owner (Section 15). Both parties receiveownership_transferred. Auditteam.ownership_transferredwith metadatafrom_user_id,to_user_id. - If the target is removed from or leaves the team before accepting, the pending transfer is deleted.
- Super admins cannot transfer ownership by impersonation (impersonation is read-only).
11.16.5 Cleanup when a user stops being a member #
Applies to removal, leaving, and account deletion (Section 11.13.1). For every site of the team:
| Item | Result |
|---|---|
| Membership | Deleted |
| The user's personal segments and personal annotations | Deleted |
| Shared segments and shared annotations authored by the user | Kept; the author shows as "Former member" (created_by_user_id set to null) |
| The user's email report and spike notification subscriptions | Deleted |
| API keys created by the user in this team | Deleted immediately, so a departing member cannot keep programmatic access; audit api_key.revoked with metadata reason = "creator_removed" |
| Exports requested by the user for sites of this team | Deleted immediately (files and rows), so a departing member cannot download them later |
| Invitations the user sent | Kept (still valid) |
| Sessions | Unaffected (they belong to the user, not the team) |
last_team_id |
Set to null if it pointed at this team |
11.17 Site Transfer Between Teams #
Action ID site.transfer (owner, admin of the source team). A site keeps its domain, settings, data and tracker snippet; only its owning team changes. ClickHouse data is keyed by the site UUID and is not touched.
11.17.1 Direct transfer #
POST /api/app/v1/sites/:domain/transfer with { "target_team_id" }.
| Check | Error |
|---|---|
Caller has site.transfer on the source team |
403 forbidden |
| Caller is owner or admin of the target team | 403 target_team_forbidden |
| Target equals source | 422 same_team |
| Target team scheduled for deletion | 404 not_found |
| Hosted: target is at its site limit, or unlicensed with one site already | 409 site_limit_reached (rules in Section 15) |
| A transfer invitation is already pending for the site | 409 transfer_pending |
On success the response is 200 with the updated site. On the hosted edition the service calls freezeSiteUsage(team_id, site_id) (Section 15.9.4) for the source team inside the transfer, before the owning team changes; the same applies to the invitation flow of Section 11.17.2 at acceptance.
11.17.2 Transfer by invitation #
POST /api/app/v1/sites/:domain/transfer with { "email" } (exactly one of target_team_id and email; both or neither returns 400 validation_failed) creates a site_transfer invitation (7 days) and sends site_transfer_request. The recipient must be a verified user whose email matches, and an owner or admin of the destination team they pick: POST /api/app/v1/invitations/accept with { "token", "target_team_id" }. The same target checks as the direct transfer apply at acceptance time, evaluated then, not at invitation time. DELETE /api/app/v1/sites/:domain/transfer cancels a pending invitation (204). Only one pending site transfer exists per site.
11.17.3 What happens to related data #
| Item | After transfer |
|---|---|
| Goals, funnels, shared segments, shared annotations, property allowlist, exclusions, shared links, public-dashboard setting, imports, settings | Unchanged (they reference the site) |
| Shared links and embeds | Keep working with the same tokens |
| Site guests | Keep their access |
| Team API keys | Keys of the source team lose access at once; keys of the target team gain access (Section 13) |
| Personal segments and annotations | Kept for users who are members of the target team or guests of the site; deleted for everyone else |
| Email report and spike subscriptions | User recipients without access in the target team (not a member, not a guest) are deleted; external-email recipients are kept |
| Cached site configuration | Invalidated in Valkey so ingestion sees the new team immediately |
| Usage counting (hosted) | Before the transfer the service calls freezeSiteUsage(team_id, site_id) (Section 15.9.4) for the source team. Events ingested after the transfer count against the new team; events already counted stay with the team that owned the site when they were ingested (no retroactive moves, Section 15) |
Both teams' owners and admins receive site_transferred. Audit site.transferred is written once for each team with metadata from_team_id, to_team_id, domain.
11.18 Site Guests #
Guests are for clients of agencies: access to one site without belonging to the team.
11.18.1 Roles and boundaries #
| Role | Capabilities (full detail in Section 11.19) |
|---|---|
guest_viewer |
View that one site's dashboard, use filters, create personal segments and annotations, export CSV, subscribe themselves to reports and spike notifications |
guest_editor |
Everything a guest viewer can do plus manage goals, funnels, properties, shared segments, shared annotations, shared links, public-dashboard toggle, email reports, spike configuration, exclusions and general site settings of that one site |
A guest never sees the team name, other sites, the member list, API keys, licensing, imports, site transfer, domain change or deletion. Guests do not count toward any limit.
11.18.2 Managing guests #
Action site.guests.manage (owner, admin). Maximum 50 guests plus pending guest invitations per site (409 limit_reached with details.limit_name = "site_guests"; the value is listed with the service limits of Section 5.16).
| Operation | Endpoint | Rules |
|---|---|---|
| List | GET /api/app/v1/sites/:domain/guests |
Returns active guests and pending invitations: {user_id | null, invitation_id | null, name | null, email, role, status: "active" | "pending" | "expired", created_at} |
| Invite | POST /api/app/v1/sites/:domain/guests with { "email", "role" } |
Role in guest_viewer, guest_editor (422 invalid_role). An email that belongs to a team member returns 409 member_exists; an existing guest returns 409 already_guest; creates a site_guest invitation and sends site_guest_invitation. Audit site.guest_invited |
| Change role | PATCH /api/app/v1/sites/:domain/guests/:user_id with { "role" } |
Audit site.guest_role_changed |
| Remove | DELETE /api/app/v1/sites/:domain/guests/:user_id |
Also deletes the guest's personal segments, personal annotations and subscriptions for that site (Section 11.16.5 rules); audit site.guest_removed |
| Leave | POST /api/app/v1/sites/:domain/guests/leave |
By the guest themselves (action site.guest.leave); same cleanup |
Pending guest invitations are revoked and resent through the invitation endpoints of Section 11.15.3.
11.18.3 Combined access #
A user holds both a team role and a guest role on one site only when a site is transferred into a team the user already belongs to (accepting a team invitation deletes the user's guest rows for that team's sites, Section 11.15.4). In that case authorization evaluates every role the user holds for the site and allows an action if at least one of those roles allows it (union of permissions per Section 11.19.3). Guest rows are not deleted automatically by a transfer.
11.18.4 Shared with me #
GET /api/app/v1/account/shared-sites returns the sites on which the caller is a guest: {domain, role} only (team names are withheld). The team switcher shows them under "Shared with me" and opens the single-site dashboard. If the caller has no team memberships the "Shared with me" view is the landing page after login.
11.19 Authorization Model and Canonical Permission Matrix #
11.19.1 Roles #
| Role | Scope | Source |
|---|---|---|
owner |
Team (exactly one per team) | team_memberships.role |
admin |
Team, all sites of the team | team_memberships.role |
editor |
Team, all sites of the team | team_memberships.role |
viewer |
Team, all sites of the team | team_memberships.role |
guest_editor |
One site | site_guests.role |
guest_viewer |
One site | site_guests.role |
super_admin |
Instance (hosted only) | Not stored. Computed per request: the user's email is verified (by consuming a verify_email token, never by the no-SMTP shortcut), the email is listed in SUPER_ADMIN_EMAILS, and TOTP is enabled (otherwise every operator route returns 403 admin_2fa_required). Access to team data is through read-only impersonation (Section 15) |
Team members (owner, admin, editor, viewer) have access to every site of the team; there is no per-site restriction for members. Site-level restriction exists only for guests.
11.19.2 The matrix #
Legend: ✓ allowed. — denied. SA column: ✓r means allowed while impersonating (read-only, audited, Section 15); — means denied even during impersonation. Rows marked H exist only in the hosted edition; in the self-hosted edition they always evaluate to deny and their UI is absent. For "personal" rows the permission covers only items the caller owns; every role that holds it can see only their own personal items. Action IDs are the keys of the shared PERMISSIONS constant; every endpoint in the specification declares exactly one action ID.
| Action ID | Action | owner | admin | editor | viewer | guest_editor | guest_viewer | super admin |
|---|---|---|---|---|---|---|---|---|
| Site data and analysis | ||||||||
site.view |
View dashboard, all reports, realtime, journeys, funnel results, revenue | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓r |
site.export_csv |
Export stats as CSV (Section 14) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | — |
site.export.download_any |
Download exports requested by other users (Section 14) | ✓ | ✓ | — | — | — | — | — |
segment.personal.manage |
Create, edit, delete own personal segments | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | — |
segment.shared.manage |
Create, edit, delete site-shared segments | ✓ | ✓ | ✓ | — | ✓ | — | — |
annotation.personal.manage |
Create, edit, delete own personal annotations | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | — |
annotation.shared.manage |
Create, edit, delete any shared annotation | ✓ | ✓ | ✓ | — | ✓ | — | — |
| Site configuration | ||||||||
goal.manage |
Create, edit, delete goals | ✓ | ✓ | ✓ | — | ✓ | — | — |
property.manage |
Manage the custom property allowlist | ✓ | ✓ | ✓ | — | ✓ | — | — |
funnel.manage |
Create, edit, delete funnels | ✓ | ✓ | ✓ | — | ✓ | — | — |
shared_link.manage |
Create, edit, revoke shared links and embed settings (Section 12.2) | ✓ | ✓ | ✓ | — | ✓ | — | — |
public_dashboard.toggle |
Turn the public dashboard on or off (Section 12.3) | ✓ | ✓ | ✓ | — | ✓ | — | — |
email_report.subscribe_self |
Subscribe or unsubscribe oneself to weekly and monthly reports | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | — |
email_report.manage |
Configure report schedules and manage other recipients, including non-members | ✓ | ✓ | ✓ | — | ✓ | — | — |
spike.subscribe_self |
Subscribe or unsubscribe oneself to traffic spike emails | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | — |
spike.manage |
Set the spike threshold and manage other recipients | ✓ | ✓ | ✓ | — | ✓ | — | — |
exclusion.manage |
Manage IP and page exclusions | ✓ | ✓ | ✓ | — | ✓ | — | — |
site.settings.update |
Change site name, timezone, tracker options, allowed hostnames, datacenter-filter toggle and other general settings | ✓ | ✓ | ✓ | — | ✓ | — | — |
site.domain.change |
Change the site's domain | ✓ | ✓ | — | — | — | — | — |
| Site lifecycle and access | ||||||||
site.create |
Add a site to the team | ✓ | ✓ | — | — | — | — | — |
site.delete |
Delete a site and its data (hosted: the service first calls freezeSiteUsage(team_id, site_id), Section 15.9.4) |
✓ | ✓ | — | — | — | — | — |
site.transfer |
Transfer a site to another team (Section 11.17) | ✓ | ✓ | — | — | — | — | — |
site.guests.manage |
Invite, change and remove site guests | ✓ | ✓ | — | — | — | — | — |
site.guest.leave |
Leave a site one was invited to as a guest | — | — | — | — | ✓ | ✓ | — |
site.import.manage |
Start, view and delete data imports (Section 14) | ✓ | ✓ | — | — | — | — | — |
| Team | ||||||||
team.view |
See the team, its sites list and its member list | ✓ | ✓ | ✓ | ✓ | — | — | ✓r |
team.members.invite |
Invite members (admin, editor, viewer) | ✓ | ✓ | — | — | — | — | — |
team.invitations.manage |
List, resend and revoke pending team-member invitations (site-guest, ownership-transfer and site-transfer invitations follow their own actions, Section 11.15.3) | ✓ | ✓ | — | — | — | — | — |
team.members.change_role |
Change a member's role (never the owner's) | ✓ | ✓ | — | — | — | — | — |
team.members.remove |
Remove a member (never the owner) | ✓ | ✓ | — | — | — | — | — |
team.leave |
Leave the team | — | ✓ | ✓ | ✓ | — | — | — |
team.settings.update |
Rename the team | ✓ | ✓ | — | — | — | — | — |
team.require_2fa.set |
Require two-factor authentication for members (Section 11.14.4) | ✓ | — | — | — | — | — | — |
team.audit_log.view |
View the team audit log (Section 11.20) | ✓ | ✓ | — | — | — | — | ✓r |
team.api_keys.manage |
Create, list and revoke team API keys (Section 13) | ✓ | ✓ | — | — | — | — | — |
billing.view H |
View the license summary, usage meter and invoices list (Section 15) | ✓ | ✓ | — | — | — | — | ✓r |
billing.manage H |
Checkout, upgrade, redeem a code, download invoice and credit-note PDFs (Section 15) | ✓ | — | — | — | — | — | — |
team.ownership.transfer |
Start or cancel an ownership transfer | ✓ | — | — | — | — | — | — |
team.delete |
Schedule team deletion and restore it | ✓ | — | — | — | — | — | — |
| Instance | ||||||||
admin.console H |
Open the operator console /admin (Section 15) |
— | — | — | — | — | — | ✓ |
admin.impersonate H |
Start a read-only impersonation session (Section 15) | — | — | — | — | — | — | ✓ |
Account-level actions (profile, password, email, 2FA, sessions, appearance, account deletion, creating a team, accepting invitations addressed to oneself) are not role-gated; they apply to the authenticated user only and use the step-up rules of Section 11.1. Operator actions of the console (users, teams, licenses, code batches, refunds, usage) are enumerated and owned by Section 15 and all require admin.console.
API keys (Section 13) act with the authority of a team administrator limited to the endpoints that Section 13 assigns to keys; keys never gain account-level actions.
11.19.3 Evaluation algorithm #
Every dashboard-API handler calls authorize(request, actionId, resource) before any work. Signature in packages/shared:
export type Role = 'owner' | 'admin' | 'editor' | 'viewer' | 'guest_editor' | 'guest_viewer';
export const PERMISSIONS = {
'site.view': ['owner', 'admin', 'editor', 'viewer', 'guest_editor', 'guest_viewer'],
'site.create': ['owner', 'admin'],
'billing.manage': ['owner'],
// ...one key per row of the matrix above, transcribed cell for cell
} as const satisfies Record<string, readonly Role[]>;
export type ActionId = keyof typeof PERMISSIONS; // a union of the literal keys, so a mistyped action ID does not compile
export type Resource =
| { kind: 'team'; teamId: string }
| { kind: 'site'; siteId: string }
| { kind: 'none' };
export function can(roles: readonly Role[], action: ActionId): boolean; // true if any role is listedOrder of checks:
- Resolve the team. A site resource resolves to its owning team. A team scheduled for deletion is visible only to its owner (Section 11.14.5); to anyone else it does not exist.
- Collect the caller's roles for the resource: the membership role in the team (if any) and the guest role on the site (if any).
- No relationship: if the caller holds no role and is not an impersonating super admin, return
404 not_found. - Two-factor enforcement: if the team enforces 2FA and the caller is a member without it, return
403 two_factor_enrollment_required(Section 11.14.4). - Email verification: actions listed as blocked in Section 11.3.3 return
403 email_not_verifiedwhen the caller is unverified. - Impersonation: a request made through an impersonation session is allowed only for actions whose super-admin cell is ✓r, and only when the method is
GET,HEADorOPTIONSor the route carries the registry flagreadOnly: true. That flag is set on the dashboard query endpointsPOST /sites/{domain}/queryand/query/batchand on the read endpoints of funnels and journeys, which use POST only to carry a request body and never change state. Any other call returns403 impersonation_read_only. - Role check: allowed if
can(roles, action); otherwise403 forbiddenwithdetails = { "action": "<id>" }. - Edition check: rows marked H are not registered in the self-hosted edition, so their routes return
404 not_found(Section 4.10) andcan()evaluates to deny for them.
The unit test suite (Section 18) must assert that the PERMISSIONS constant equals this table cell for cell, and that every route in the dashboard API has a declared action ID.
11.20 Audit Log #
11.20.1 Record and access #
Audit rows are written in the same transaction as the change they describe, to the audit_log table (Section 5). Fields:
| Field | Meaning |
|---|---|
id |
UUIDv7 (also the pagination cursor) |
occurred_at |
UTC timestamp |
actor_type |
user, super_admin, impersonation, api_key or system (CLI and jobs write system) |
actor_user_id |
Acting user; null for system, CLI, job and API-key actors. Under impersonation it is the super admin |
actor_api_key_id |
uuid, nullable; set when a team API key performed a Sites API mutation (the key prefix goes in metadata.key_prefix) |
on_behalf_of_user_id |
Under impersonation, the user being viewed; otherwise null (read-only impersonation sessions write only admin.impersonation_* events) |
team_id |
Owning team; null for account-level events. No foreign key, so team purge cannot cascade |
site_id |
Site UUID when relevant, else null |
action |
Event name from the catalogue below |
target_type, target_id |
The affected object (user, team, site, invitation, goal, and so on) and its ID |
reason |
Text, nullable, 1–500 characters; required for operator console actions (Section 15), null otherwise |
metadata |
JSON object with the keys named in the catalogue; never contains passwords, tokens, secrets, codes, IP addresses or user agents |
request_id |
The request ID of Section 4.4.2 when the event came from an HTTP request |
| Property | Decision |
|---|---|
| Visibility | Team events: owners and admins (team.audit_log.view) via GET /api/app/v1/teams/:team_id/audit-log?action=&actor_id=&site=&from=&to=&cursor=&limit= (cursor pagination, newest first, default limit 50, max 200). Account-level events (team_id null) are visible only to the user themself in the Security panel (GET /api/app/v1/account/audit-log, same parameters, last 90 days) and to super admins through the console |
| Retention | Hosted: AUDIT_RETENTION_MONTHS (default 24, Section 16.4); older rows are deleted by the audit-prune job (Section 16.6). Self-hosted: audit rows are kept indefinitely |
| Immutability | No update or delete endpoints exist; rows disappear only by retention, team purge (which keeps the licensing evidence rows named in Section 11.14.5) or anonymization scrubbing (Section 11.13.2) |
| Failures | Audit writes are part of the transaction; if the insert fails the whole action fails |
| High-volume events | Failed logins, page views and read operations are never audited |
11.20.2 Event catalogue #
| Action | Written when | Metadata keys |
|---|---|---|
user.registered |
Signup or first-run setup completes | via (open, invitation, setup, super_admin) |
user.email_verified |
Verification token consumed or invitation accepted | none |
user.login |
Successful login | method |
user.login_locked |
Lockout set | none |
user.logout |
Logout | none |
user.password_changed |
Authenticated change | none |
user.password_reset_requested |
Reset requested for an existing account | none |
user.password_reset_completed |
Reset token used | none |
user.email_change_requested |
Change requested | new_email |
user.email_changed |
Change confirmed | old_email, new_email |
user.2fa_enabled |
2FA enabled | none |
user.2fa_disabled |
2FA disabled by the user | none |
user.2fa_recovery_code_used |
Recovery code consumed at login | remaining |
user.2fa_recovery_codes_regenerated |
Codes regenerated | none |
user.session_revoked |
One session revoked | session_id |
user.sessions_revoked_all |
All other sessions revoked | count |
user.deletion_requested |
Account deletion requested | none |
user.deletion_cancelled |
Account restored | restored_teams |
user.anonymized |
Anonymization job ran | none |
team.created |
Team created | name |
team.renamed |
Name changed | from, to |
team.require_2fa_changed |
Enforcement toggled | enabled |
team.deletion_scheduled |
Team deletion requested | purge_after |
team.deletion_cancelled |
Team restored | none |
team.purged |
Purge job ran (written without team_id) |
team_name, site_count |
team.member_invited |
Team invitation created | email, role, kind |
team.invitation_resent |
Resent | email |
team.invitation_revoked |
Revoked | email, kind |
team.invitation_accepted |
Accepted | kind, role |
team.invitation_declined |
Declined | kind |
team.member_role_changed |
Role changed | from, to |
team.member_removed |
Member removed | role |
team.member_left |
Member left | role |
team.ownership_transfer_requested |
Transfer started | to_user_id |
team.ownership_transfer_cancelled |
Cancelled | to_user_id |
team.ownership_transferred |
Accepted | from_user_id, to_user_id |
site.created |
Site created | domain, timezone |
site.updated |
Settings changed | changed (list of field names; values omitted) |
site.domain_changed |
Domain changed | from, to |
site.deleted |
Site deleted | domain |
site.transfer_requested |
Invitation transfer started | email |
site.transferred |
Transfer completed (one row per team) | from_team_id, to_team_id, domain |
site.guest_invited |
Guest invited | email, role |
site.guest_role_changed |
Guest role changed | from, to |
site.guest_removed |
Guest removed or left | role, self |
goal.created, goal.updated, goal.deleted |
Goal changes | name |
funnel.created, funnel.updated, funnel.deleted |
Funnel changes | name |
segment.created, segment.updated, segment.deleted |
Site-shared segment changes only (personal segments are not audited) | name |
shared_link.created, shared_link.updated, shared_link.revoked |
Shared link changes | name, password_protected, segment_id, embed_mode |
public_dashboard.toggled |
Public dashboard toggled | enabled |
email_report.recipient_added, email_report.recipient_removed |
Report recipient changes by someone other than the recipient | frequency, recipient (user ID or external email) |
spike.updated |
Spike config changed | enabled, threshold |
exclusion.created, exclusion.deleted |
IP or page exclusion changes | kind (ip or page), pattern |
property.allowlist_changed |
Custom property allowlist changed | added, removed |
api_key.created, api_key.revoked |
Key lifecycle (Section 13) | name, reason on revocation |
export.requested, export.deleted |
CSV exports (Section 14) | export_id |
import.created, import.completed, import.failed, import.cancelled, import.deleted, import.ga4_authorized |
Imports (Section 14) | source, import_id |
license.granted, license.upgraded, license.tier_changed, license.revoked, license.reinstated, license.moved, license.purge_extended, license.purge_warning_sent, license.data_purged, purchase.created, purchase.paid, purchase.partially_refunded, purchase.refunded, purchase.auto_refunded, purchase.disputed, code.redeemed, code.revoked, code.bruteforce_suspected, code_batch.created, code_batch.exported, code_batch.revoked_unredeemed, tier.updated, license.dispute_closed_lost, license.allowance_granted, purchase.refund_requested |
Licensing events, written by Section 15 into this table | defined in Section 15 (ids and numbers only) |
team.suspended, team.unsuspended, user.suspended, user.unsuspended, user.force_logout |
Operator console actions (Section 15); reason column required |
none |
admin.impersonation_started, admin.impersonation_ended, admin.impersonation_expired |
Operator impersonation (written to the impersonated team's log; actor_type is impersonation, on_behalf_of_user_id is the viewed user). Blocked write attempts are counted by a metric and not audited individually |
none (reason column) |
admin.user_2fa_reset |
Operator or CLI reset of 2FA | source (console or cli) |
audit.exported |
Operator audit CSV export (Section 15) | row_count |
11.21 Endpoint Index #
All paths are under /api/app/v1. "Anon" requires the anonymous CSRF token; "Session" requires the cookie session and X-CSRF-Token on mutating methods; "Token" authenticates by the secret in the body or path.
| Method and path | Auth | Purpose | Section |
|---|---|---|---|
GET /bootstrap |
none | Runtime config for the SPA (also inlined into index.html) |
11.2 |
POST /setup |
Anon | First account with the one-time setup token | 11.2 |
GET /auth/csrf |
none | Anonymous CSRF token | 11.7 |
POST /auth/signup |
Anon | Create account | 11.3 |
POST /auth/verify-email |
Token | Consume verification token | 11.3.2 |
POST /auth/verify-email/resend |
Session | Resend verification | 11.3.2 |
POST /auth/login |
Anon | Password step | 11.4.1 |
POST /auth/login/2fa |
Anon | Second factor step | 11.4.2 |
POST /auth/logout |
Session | Log out | 11.4.3 |
GET /auth/session |
Session | Current user, teams, CSRF token | 11.7 |
POST /auth/password/forgot |
Anon | Request reset | 11.9 |
POST /auth/password/reset |
Anon | Complete reset | 11.9 |
GET /account, PATCH /account |
Session | Profile, appearance, last team | 11.12 |
POST /account/password |
Session | Change password | 11.9 |
POST /account/email |
Session | Request email change | 11.10 |
POST /account/email/confirm |
Anon (token in body) | Confirm email change | 11.10 |
DELETE /account/email/pending |
Session | Cancel email change | 11.10 |
GET /account/2fa |
Session | 2FA status | 11.11.3 |
POST /account/2fa/setup, /enable, /disable, /recovery-codes |
Session | 2FA lifecycle | 11.11 |
GET /account/sessions, DELETE /account/sessions, DELETE /account/sessions/:id |
Session | Login management | 11.12.3 |
GET /account/audit-log |
Session | Own account audit events | 11.20 |
GET /account/data-export |
Session | Download the user's own account data (owned by Section 17.9) | 11.13 |
DELETE /account, POST /account/restore |
Session | Delete and restore account | 11.13 |
GET /account/invitations, POST /account/invitations/:id/accept, POST /account/invitations/:id/decline |
Session | In-app invitations | 11.15.4 |
GET /account/shared-sites |
Session | Sites where the caller is a guest | 11.18.4 |
GET /teams, POST /teams |
Session | List and create teams | 11.14.1 |
GET/PATCH/DELETE /teams/:team_id, POST /teams/:team_id/restore |
Session | Team settings, deletion, undo | 11.14 |
GET /teams/:team_id/members, PATCH/DELETE /teams/:team_id/members/:user_id, POST /teams/:team_id/leave |
Session | Membership | 11.16 |
GET/POST /teams/:team_id/invitations, POST /invitations/:id/resend, DELETE /invitations/:id |
Session | Invitation management | 11.15 |
GET /invitations/lookup, POST /invitations/accept, POST /invitations/decline |
Anon for lookup and decline; Session for accept | Invitation acceptance | 11.15.4 |
POST/DELETE /teams/:team_id/ownership-transfer |
Session | Ownership transfer | 11.16.4 |
POST/DELETE /sites/:domain/transfer |
Session | Site transfer | 11.17 |
GET/POST /sites/:domain/guests, PATCH/DELETE /sites/:domain/guests/:user_id, POST /sites/:domain/guests/leave |
Session | Site guests | 11.18 |
GET /teams/:team_id/audit-log |
Session | Team audit log | 11.20 |
POST /admin/users/:user_id/reset-2fa |
Session (super admin, hosted) | Operator 2FA reset (owned by Section 15) | 11.11.5 |
Error codes used by this section, with the statuses of the catalogue in Section 4.4.3: 400 validation_failed, two_factor_setup_expired; 401 unauthenticated, invalid_credentials, two_factor_required (details.challenge_token, details.expires_in), invalid_two_factor_code, invalid_recovery_code, challenge_expired; 403 forbidden, csrf_invalid, registration_closed, setup_required, setup_token_invalid, email_not_verified, two_factor_enrollment_required (details.team_id), two_factor_required_by_team, owner_protected, target_team_forbidden, invitation_email_mismatch, account_pending_deletion, account_suspended, admin_2fa_required, impersonation_read_only; 404 not_found; 409 email_taken, member_exists, already_guest, already_verified, invitation_exists, limit_reached (details.limit_name is teams_owned, pending_invitations or site_guests), site_limit_reached, owner_cannot_leave, owner_must_transfer, license_acknowledgement_required, transfer_pending, cannot_change_own_role, use_leave, two_factor_already_enabled, two_factor_not_enabled, setup_already_completed; 422 token_invalid, token_expired, current_password_incorrect, two_factor_code_incorrect, password_too_common, password_breached, password_unchanged, email_unchanged, invalid_role, invalid_target, use_ownership_transfer, confirm_email_mismatch, confirm_name_mismatch, same_team, owner_two_factor_required, target_two_factor_required, target_email_unverified, cannot_invite_self; 429 rate_limited; 503 unavailable.
11.22 Screens #
| Screen | Route | Behavior |
|---|---|---|
| First-run setup | /setup |
Shown only while setup_required is true (otherwise 404): name, email, password, setup token; posts to POST /api/app/v1/setup (Section 11.2) |
| Signup | /register |
Fields name, email, password (show/hide toggle, length meter), terms checkbox when required; honors Section 11.2 visibility rules; errors inline from details.fields; a duplicate email shows the email_taken message inline |
| Login | /login |
Email, password; link to forgot password; locked state shows "Too many attempts. Try again in N minutes." using Retry-After |
| Second factor | /login/2fa |
Six-digit input with auto-submit on the sixth digit; link "Use a recovery code" swaps to the recovery field; five failures return to /login with a message |
| Forgot / reset | /forgot-password, /reset-password/:token |
Forgot always shows "If an account exists, we sent a link." Reset shows the password policy hints; expired token shows "Link expired. Request a new one." |
| Verify email | /verify-email/:token |
Success, expired and invalid states described in Section 11.3.2; also confirms an email change (Section 11.10) |
| Invitation | /invitations/:token |
Summary card (team or site, role, inviter), then login, signup or Accept per Section 11.15.4; expired state offers "Ask the inviter to resend" |
| Account settings | /account/profile, /account/security |
Profile page: name, email change, Appearance, Delete account; Security page: password, 2FA and recovery codes, Login management; pending invitations banner |
| Team settings | /teams/:teamId/settings, /teams/:teamId/members |
Settings: name, enforce 2FA, transfer ownership, delete, and the audit log (filters, load more). Members: table with role dropdown, remove, invite button, and the Invitations list (status, resend, revoke). The hosted billing page (/teams/:teamId/billing) and API keys page (/teams/:teamId/api-keys) are owned by Sections 15 and 13. Pages and controls not permitted for the role are hidden, not disabled |
| Site access | /sites/:domain/settings/access |
Guests table with role dropdown, invite, remove; transfer site action with the two modes of Section 11.17 |
| Restricted states | Banners and full-page states | Unverified email banner; 2FA setup required page; account pending deletion page; team deleted restore page |
Empty states: Members shows only the owner row with "Invite your first teammate"; Invitations shows "No pending invitations"; Guests shows "Give clients access to this site only"; Sessions always has at least the current one.
11.23 Data Requirements and Jobs #
11.23.1 Attributes this section requires of the Section 5 tables #
| Table | Required attributes (beyond identifiers and timestamps) |
|---|---|
users |
name, email (unique, lowercase), email_verified_at, password_hash, totp_secret_encrypted, totp_enabled_at, totp_last_used_step, theme_preference (light, dark, system; default system), last_team_id, last_login_at, suspended_at (set by the operator console, Section 15), deleted_at, anonymized_at |
user_recovery_codes |
user_id, code_hash, used_at |
auth_sessions |
user_id, token_hash, previous_token_hash, previous_valid_until, rotated_at, device_label, country, last_seen_at, expires_at, reauth_at (last operator step-up, Section 15), impersonator_user_id |
email_tokens |
user_id, kind (verify_email, password_reset, change_email), token_hash (HMAC-SHA-256, Section 17.7), new_email, expires_at, used_at |
teams |
name, is_personal, require_2fa, deleted_at, purge_after, suspended_at, suspended_reason (the owner is derived from the membership with role owner; there is no owner column) |
team_memberships |
team_id, user_id, role (owner, admin, editor, viewer), unique (team_id, user_id) |
invitations |
kind (team_member, site_guest, ownership_transfer, site_transfer), team_id, site_id, email, role, token_hash, invited_by_user_id, expires_at, last_sent_at, resend_count |
site_guests |
site_id, user_id, role (guest_viewer, guest_editor), unique (site_id, user_id) |
audit_log |
The fields of Section 11.20.1 (including reason and actor_api_key_id) |
11.23.2 Background jobs used by this section #
Names, schedules, queues and retries are canonical in the jobs table of Section 16.6; the work described here is owned by this section.
| Job | Work |
|---|---|
session-cleanup |
Delete expired or used email_tokens older than 7 days; delete sessions past expires_at or idle over 7 days, including expired impersonation sessions |
invitation-expire |
Delete invitations 30 days past expiry; delete external report and spike recipients still awaiting confirmation after 7 days (Section 12.5.1) |
unverified-user-cleanup |
Delete unverified users older than 14 days that have no memberships beyond their own sole-member personal team, deleting that team with them (Section 11.3.3) |
user-anonymize |
Anonymize users deleted over 30 days ago (Section 11.13.2) |
team-purge |
Purge teams past purge_after (Section 11.14.5) |
audit-prune |
Hosted only: delete audit rows older than AUDIT_RETENTION_MONTHS (Section 11.20.1) |
12. Sharing, Embeds, Email Reports & Notifications #
This section owns every way analytics leave the signed-in dashboard (shared links, the public dashboard, embeds), every scheduled or triggered email about analytics (reports and traffic spike notifications), the canonical inventory of all emails the product sends (Section 12.8) and the email delivery machinery (Section 12.9). Who may perform each action is decided by the action IDs of the permission matrix in Section 11.19. Out of scope and never implemented: Slack reports, Looker Studio connectors, and any marketing or newsletter email.
12.1 Surfaces and Route Map #
| Surface | Audience | Viewer route (SPA) | Data API | Management API |
|---|---|---|---|---|
| Shared link | Anyone holding the link; optional password; optional fixed segment | /share/:token |
/api/share/v1/link/:token/... |
/api/app/v1/sites/:domain/shared-links |
| Public dashboard | Anyone on the internet | /public/:domain |
/api/share/v1/public/:domain/... |
PATCH /api/app/v1/sites/:domain/public-dashboard |
| Embed | A shared link rendered chrome-less inside an iframe | /embed/:token |
Same as shared link | Part of the shared link |
| Email reports | Chosen recipients | none (email) | none | /api/app/v1/sites/:domain/email-reports |
| Spike notifications | Chosen recipients | none (email) | none | /api/app/v1/sites/:domain/spike-notification |
| Unsubscribe | Anyone with an email link | /unsubscribe/:token |
/api/app/v1/unsubscribe/:token |
none |
Viewer routes are served by the web process like the rest of the SPA. The static file /embed.js (Section 12.4.5) is served from the same origin. Viewer APIs never use cookies and never require the dashboard CSRF token (Section 11.7).
12.2 Shared Links #
12.2.1 Resource #
GET /api/app/v1/sites/:domain/shared-links returns the site's links ({"data": [...], "meta": {"next_cursor": null, "limit": 50}}; at most the shared-link limit of Section 5.16, 50, exist). Objects:
| Field | Type | Notes |
|---|---|---|
id |
uuid | |
name |
string | 1–60 characters after trimming; no control characters; default Shared link when omitted on create |
token |
string | 32 characters, see Section 12.2.2 |
url |
string | {BASE_URL}/share/<token> |
password_protected |
boolean | True when a password hash exists; the password and hash are never returned |
segment_id |
uuid or null | Fixed segment limiting the link (Section 12.2.5) |
segment_name |
string or null | Convenience for the management UI only; never exposed on the viewer API |
embed_mode |
disabled | any | allowlist |
Default disabled (Section 12.4.3) |
embed_origins |
string[] | Allowed framing origins; empty unless embed_mode = allowlist |
include_imports |
boolean | Default true. When true, queries made through the link include imported data (Section 14) and the viewer may switch it off; when false the server forces imported data off for every request made through the link |
created_by |
uuid or null | Null once the creator has left the team |
created_at, last_accessed_at |
timestamps | last_accessed_at is updated at most once per hour per link when the viewer API serves a request; null if never opened |
| Operation | Endpoint | Action ID | Notes |
|---|---|---|---|
| Create | POST /api/app/v1/sites/:domain/shared-links |
shared_link.manage |
Body: name?, password?, segment_id?, embed_mode?, embed_origins?, include_imports?. 201 with the object. Beyond the limit of Section 5.16: 409 limit_reached with details.limit_name = "shared_links". Audit shared_link.created |
| Read one | GET /api/app/v1/sites/:domain/shared-links/:id |
shared_link.manage |
|
| Update | PATCH /api/app/v1/sites/:domain/shared-links/:id |
shared_link.manage |
Fields name, password (string sets or changes; null removes), embed_mode, embed_origins, include_imports. Sending segment_id returns 422 segment_immutable. Audit shared_link.updated |
| Revoke | DELETE /api/app/v1/sites/:domain/shared-links/:id |
shared_link.manage |
Hard delete, 204. Audit shared_link.revoked |
Names need not be unique. The unverified-email restriction of Section 11.3.3 applies only to creating links for external audiences: an unverified user cannot create or update shared links.
12.2.2 Token format #
| Property | Value |
|---|---|
| Entropy | 24 random bytes from the operating system CSPRNG (192 bits) |
| Encoding | base64url without padding, exactly 32 characters; valid tokens match ^[A-Za-z0-9_-]{32}$; anything else returns 404 without a database lookup |
| Storage | Plaintext in a uniquely indexed column. Decision: the token is a capability identifier that the management UI must be able to show again, and the link password (when set) is the second line of defence; it is not a credential reused anywhere else |
| Rotation | Not supported. To change a link, revoke it and create a new one |
| Lookup cache | The token-to-link record is cached in Valkey for 30 seconds under share_link:<token>; revoke and update delete the key, so revocation is immediate on every web process |
12.2.3 Viewer API surface #
All viewer endpoints are rooted at /api/share/v1/link/:token.
| Endpoint | Purpose |
|---|---|
GET /meta |
Returns {"data": {"password_required": bool, "limited": bool, "site": {...} | null}}. For a password-protected link without a valid unlock token, site is null and nothing else is revealed. With access, site holds domain, name, timezone, the default period, whether goals, funnels and revenue exist, and the geolocation attribution text (GEOIP_ATTRIBUTION, Section 16.4) |
POST /unlock |
Body { "password" }; Section 12.2.4 |
GET and POST read-only report endpoints |
A mirror of the read-only dashboard endpoints of Section 9: for every dashboard endpoint /api/app/v1/sites/:domain/<path> in the mirrored set below, the viewer endpoint /api/share/v1/link/:token/<path> accepts identical requests and returns identical responses |
Mirrored set (anything not listed returns 404 not_found, never 403): the stats query endpoint and its batch variant (/query and /query/batch), the realtime endpoint, filter-suggestion lookups, funnel results (read), user journey data (read), the goals list (names and types only) and the revenue-currency lookup. Annotations are not part of the mirror: shared annotations are hidden from link viewers and public-dashboard visitors (Section 9.15.2), and personal annotations are never visible. The mirror applies the link's fixed segment to every one of these endpoints (Section 12.2.5), forbids CSV export and every mutation, and strips fields that identify members. Viewer requests reject the segment pseudo-dimension and any saved-segment reference with 400 validation_failed, so a viewer cannot widen the view by guessing a segment ID. Imported data follows the link's include_imports option: when true (the default) the viewer sees imported data and may switch it off with the imported-data toggle; when false the server forces it off for every request. Each query inside a /query/batch request counts as one request toward the viewer rate limit (Section 12.2.9).
Error behavior: an unknown, malformed or revoked token returns 404 not_found with the standard error envelope. A protected link without a valid unlock token returns 401 share_password_required on every endpoint except /meta and /unlock.
12.2.4 Password protection #
| Property | Decision |
|---|---|
| Length | 8 to 128 characters, NFKC-normalized, whitespace preserved; no composition rules and no common-password check (this password gates read-only data and is distinct from account passwords) |
| Hash | argon2id with memory 19,456 KiB, time cost 2, parallelism 1, 16-byte salt, 32-byte output; parameters in ARGON2_SHARE_PARAMS; stored in shared_links.password_hash. The lighter cost keeps the public unlock endpoint from becoming a denial-of-service lever; it shares the hashing semaphore of Section 11.5.2 |
| Unlock | POST /api/share/v1/link/:token/unlock with { "password" }. Success returns {"data": {"unlock_token": "...", "expires_at": "..."}}. Failure returns 401 share_password_invalid. A link with no password returns 200 with an empty-string unlock token so clients can use one code path |
| Unlock token | v1.<link_id>.<expires_unix>.<signature> where the signature is HMAC-SHA256 (key derived from SECRET_KEY_BASE with HKDF, purpose share-unlock, Section 17.7) over the link ID, the expiry and the first 16 hex characters of the SHA-256 of the stored password hash. Lifetime 12 hours. Changing or removing the password therefore invalidates all issued tokens; revoking the link invalidates them too |
| Transport | Header X-Share-Unlock: <token> on every viewer request. No cookies. The SPA keeps the token in sessionStorage under qs_share_unlock_<link_id> and falls back to memory when storage is unavailable (an embed in a browser that blocks storage then asks again after a reload) |
| Timing | A verification always runs (against a dummy hash for unknown tokens) so response time does not distinguish unknown links from wrong passwords |
| Rate limit | Policies share_password (5 failures per IP and link per 15 minutes) and share_password_link (30 failures per link per hour from all IPs) of Section 17.6; exceeding either returns 429 rate_limited with Retry-After |
12.2.5 Segment-limited links #
| Rule | Decision |
|---|---|
| Eligible segments | Only site-shared segments of the same site (the kind managed by segment.shared.manage). A personal segment returns 422 segment_not_shared; a segment of another site returns 422 validation_failed |
| Immutability | segment_id is set at creation only; changing the audience means creating a new link |
| Enforcement | The server resolves the segment's current filters on every request and appends them to the query filters with AND semantics before the query engine runs (Section 13). Viewer-supplied filters are added on top and can only narrow further. There is no request parameter that removes the limit. The locked filters apply to every mirrored endpoint, including filter suggestions, realtime, funnels and journeys, and the imported-data merge counts them as filter dimensions (Section 13) |
| Scope of the limit | Every number on the page: time series, reports, realtime counts, funnels, journeys, goal conversions and revenue are all computed inside the segment |
| Disclosure | The segment's name and definition are never sent to viewers. The page shows a neutral badge "Limited view" |
| Editing the segment | Changes to the segment's filters apply to all links using it immediately |
| Deleting the segment | Blocked while any shared link uses it: 409 segment_in_use with details.shared_links = [{id, name}] (rule applied by the segment delete endpoint of Section 9). The user must revoke or replace those links first |
| Combination | A link may have both a password and a segment |
12.2.6 What viewers can and cannot do #
| Capability | Link viewer |
|---|---|
| Change date range, comparison and interval | Yes (state lives in the URL and in memory only) |
| Apply and remove filters | Yes, on every dimension the dashboard offers |
| Open all reports, goals conversions, funnels, journeys, revenue reports, realtime | Yes, subject to the link's segment |
| See annotations | No (shared annotations are hidden from link viewers; personal annotations are never visible) |
| Show imported data | Yes while the link's include_imports option is on (default); the viewer may switch it off (Section 12.2.1) |
| Create or save segments, annotations, goals, funnels | No |
| Export CSV | No |
| See site settings, team name, members, API keys, other sites, email addresses | No |
| Switch to another site or open the site list | No |
| Sign in from the page | The page shows a small "Sign in" link to the product login only if the visitor has no session; it grants nothing |
12.2.7 Revoking #
Deleting a link makes the next request to its token return 404 (Valkey entry deleted in the same operation, Section 12.2.2). Unlock tokens stop working immediately because they are bound to the link ID. Open pages show "This link is no longer available." after their next data request fails. Deleting a site or purging a team deletes its links. Links do not expire automatically.
12.2.8 Indexing and headers #
Every response from /share/*, /embed/*, /public/* and /api/share/* carries:
| Header | Value |
|---|---|
X-Robots-Tag |
noindex, nofollow, noarchive |
Referrer-Policy |
no-referrer (so the token is not leaked in Referer to links clicked from the page) |
Cache-Control |
no-store for API responses; no-cache for HTML |
The SPA shell for these routes also contains <meta name="robots" content="noindex,nofollow,noarchive">, and /robots.txt disallows /share/, /embed/ and /public/. Other security headers follow Section 17; the framing rules for these routes are in Section 12.4.3.
12.2.9 Viewer rate limits #
Viewer API requests are limited by the share_view policy of Section 17.6 (120 requests per minute per IP and link; the signed-in dashboard counterpart is dashboard_query) (scoped per IP and link, and per IP and site for public dashboards), enforced in Valkey across all viewer API endpoints. Exceeding it returns 429 rate_limited with Retry-After. /meta counts toward the limit, and each query inside a mirrored batch request counts separately. Unlock failures follow Section 12.2.4.
12.3 Public Dashboard #
| Property | Decision |
|---|---|
| Toggle | PATCH /api/app/v1/sites/:domain/public-dashboard with { "enabled": true }; action public_dashboard.toggle; returns {"data": {"enabled": true, "url": "{BASE_URL}/public/<domain>"}}; the domain is URL-encoded. The flag is stored as site_settings.public_dashboard (Section 5) and is readable on the site object. Audit public_dashboard.toggled |
| Default | Off for every site, including newly created ones |
| Access | Anyone, no password, no segment limit; same read-only capabilities as a shared-link viewer (Section 12.2.6) |
| Not available | Password protection, segment limiting, embedding and imported data (use a shared link for those) |
| API | /api/share/v1/public/:domain/meta and the mirrored read-only endpoints of Section 12.2.3 with the same allow-list and 404 behavior. A site whose public dashboard is off, whose team is scheduled for deletion, or that does not exist returns 404 not_found |
| Domain change | The public URL changes with the domain; the old URL returns 404 |
| Indexing | Always noindex (Section 12.2.8); there is no setting to allow indexing |
| Enable confirmation | The UI confirms: "Anyone with the link can see all stats for this site, including pages, sources and locations." |
| Rate limit | Policy share_view, Section 12.2.9 |
12.4 Embeds #
12.4.1 Model #
An embed is a presentation mode of a shared link, not a separate object. A link can be framed only when its embed_mode is not disabled. Password protection and segment limiting work unchanged inside the frame. The public dashboard cannot be embedded.
12.4.2 Embed URL and parameters #
{BASE_URL}/embed/<token> renders the dashboard without app chrome. Optional query parameters (invalid values are ignored silently and the default is used):
| Parameter | Values | Default |
|---|---|---|
theme |
light, dark, system |
light |
background |
transparent, or a 3- or 6-digit hexadecimal color without # (such as ffffff or fff; both forms are valid) |
The theme's page surface color |
period |
Any period key defined in Section 9's time range list | The site's default period |
The embed ignores the signed-in user's theme preference and cookies entirely.
12.4.3 Allowed frame ancestors #
embed_mode |
Content-Security-Policy sent on /embed/:token |
Meaning |
|---|---|---|
disabled (default) |
frame-ancestors 'none' |
Cannot be framed; opened directly the page says "Embedding is not enabled for this link." |
any |
frame-ancestors * |
Any site may embed it; the UI warns "Any website can embed this link." |
allowlist |
frame-ancestors <origin> <origin> ... |
Only listed origins |
Rules for embed_origins:
| Rule | Decision |
|---|---|
| Count | 1 to 10 when embed_mode = allowlist; empty otherwise (the server clears the list when the mode changes away from allowlist) |
| Format | Lowercase https://host with optional :port; host may begin with a single *. wildcard label; the origins http://localhost, http://127.0.0.1 and http://[::1] (optional port) are also allowed for development; no path, query, fragment or trailing slash; at most 255 characters each |
| Errors | 422 invalid_embed_origin naming the offending value in details.fields; 422 embed_origins_required when mode is allowlist with an empty list |
The header is computed from the database record on each request (not cached by the HTTP layer). X-Frame-Options is not sent on /embed/* because it cannot express allowlists. Every other route, including /share/:token, is sent with frame-ancestors 'none' and X-Frame-Options: DENY (Section 17). The /embed/:token response for an unknown token is a 404 page with frame-ancestors 'none'.
12.4.4 Layout inside the frame #
The embed view removes the header, site switcher, navigation and settings. It keeps the date range picker, comparison toggle, filters and all reports. The only footer content is the geolocation attribution line (GEOIP_ATTRIBUTION, Section 16.4) required by the database license (Section 3). Links that leave the dashboard open in a new tab. The background parameter is applied to the document body. The password form, when required, renders inside the frame.
12.4.5 Snippet and automatic resizing #
The snippet generated by the Embed tab of the share dialog:
<iframe src="{BASE_URL}/embed/<token>?theme=light" data-quietstat-embed loading="lazy"
style="width:1px;min-width:100%;height:1600px;border:0" scrolling="no"
title="Analytics dashboard"></iframe>
<script async src="{BASE_URL}/embed.js"></script>Resize protocol:
- Inside the frame, a
ResizeObserverwatches the document element and, debounced by 100 ms and only when the height changes by 2 px or more, postswindow.parent.postMessage({ "source": "quietstat", "type": "resize", "height": <integer px> }, "*"). The target origin is*because the message carries only a number. /embed.js(served withAccess-Control-Allow-Origin: *,Cache-Control: public, max-age=86400, under 1 KB gzipped, MIT license like the tracker) listens formessageevents. It accepts a message only ifevent.data.source === "quietstat",type === "resize",heightis a finite number, andevent.sourceequals thecontentWindowof aniframe[data-quietstat-embed]whosesrcorigin equalsevent.origin. It clamps the height to 400–10,000 px and setsiframe.style.height.- Without the helper script the iframe keeps its fixed 1600 px height and the dashboard scrolls inside it normally.
12.4.6 Snippet UI #
The share dialog has two tabs. Link: name, password toggle and field, segment select (only shared segments), copy button. Embed: embed mode radio group (Off, Anyone, Specific sites with a tag input for origins), theme select, background (transparent toggle or hex input), a live preview iframe, and a read-only snippet textarea with Copy. Changing options updates the snippet immediately; saving writes embed_mode and embed_origins.
12.5 Scheduled Email Reports #
12.5.1 Schedules and subscriptions #
Each site has two independent schedules, weekly and monthly. A schedule is active when it has at least one active recipient. There is no separate on/off switch. Subscriptions are rows of email_report_subscriptions (Section 5), one per (site, frequency, recipient).
| Field | Notes |
|---|---|
frequency |
weekly or monthly |
| Recipient | Exactly one of user_id (a team member of the site's team, or a guest of the site) or email (an external address, normalized as in Section 11.3.1) |
status |
pending_confirmation, active, unsubscribed, bounced |
confirmed_at |
Set when an external recipient confirms (Section 12.5.1 below); null for recipients who are users |
unsubscribe_token |
32 random bytes, base64url, 43 characters (Section 12.10) |
last_sent_period |
Period key of the last report sent: YYYY-Www (ISO week) for weekly, YYYY-MM for monthly |
consecutive_failures |
Permanent delivery failures in a row (Section 12.9.4) |
added_by_user_id |
User who created the row (the recipient themself for self-subscriptions); null after that user leaves the team |
Limits: the number of report subscriptions per site is capped by Section 5.16 (409 limit_reached with details.limit_name = "report_subscriptions"); adding external recipients is limited by the email_report_external_add (per site) and external_recipient_add (per team) policies of Section 17.6 (429 rate_limited).
External recipients must confirm. A recipient added by email address who is not a user with access to the site starts in status pending_confirmation. The email_report_subscribed_notice email asks them to confirm, and no report is sent until they press the Confirm button on the confirmation page, which calls POST /api/app/v1/unsubscribe/:token/confirm (Section 12.10) and sets the status to active and confirmed_at. Rows still unconfirmed after 7 days are deleted by the invitation-expire job. In the hosted edition, adding an external recipient also requires an active license on the team (403 license_required). Users with access to the site (members and guests) need no confirmation. This keeps the product from mailing arbitrary addresses on the operator's sending reputation.
12.5.2 Schedule and period definitions #
| Property | Weekly | Monthly |
|---|---|---|
| Sent | Every Monday at 08:00 in the site's timezone | On the 1st of each month at 08:00 in the site's timezone |
| Period covered | The previous complete ISO week, Monday 00:00:00 through Sunday 23:59:59.999, site timezone | The previous complete calendar month in the site timezone |
| Comparison period | The week before that | The month before that |
| Period label in emails | Nov 2 – Nov 8, 2026 style (English month abbreviations, ISO week start Monday) |
October 2026 |
| Period key | ISO year and week (2026-W45) |
2026-10 |
The send time is fixed and not configurable (decision). If the local time 08:00 does not exist on that date because of a daylight-saving gap, the report is sent at the first valid minute after it. If the time occurs twice, the earlier occurrence is used.
The email-reports-dispatch job (schedule in Section 16.6) runs frequently enough to honor the send time within its tick. For each site with active subscriptions it computes whether the schedule's send time has passed in the site's timezone and the period key differs from the subscription's last_sent_period. Due sites get one email-report-send job per (site, frequency, period key), which queries the statistics once and fans out one email-send job per active recipient. When the email-report-send job is enqueued, last_sent_period of the due active subscriptions is set in the same transaction, which makes the schedule idempotent.
| Edge case | Decision |
|---|---|
| Worker down at the scheduled time | Reports are caught up on the next tick for up to 72 hours after the scheduled time; after that the period is skipped and a warning is logged |
| Site created inside the period | A report is still sent for that period if the site has data in it |
| No traffic at all | If the site recorded zero visitors in both the period and the comparison period, no email is sent and last_sent_period is still advanced |
| Recipient added after the send time but within the catch-up window | The recipient waits for the next period (new rows start with last_sent_period set to the current due period) |
| Team scheduled for deletion, or site deleted | Skipped |
| Bots and excluded traffic | Not counted, as everywhere (Section 8) |
| Statistics query fails | The email-report-send job is retried by pg-boss (Section 16.6). last_sent_period was already set when the job was enqueued, and each email-send job carries the singleton key <template>:<recipient>:<period key>, so retries never send a report twice. A build job that fails permanently is logged at error level and that period is skipped |
12.5.3 Report content #
The email shows (all metric definitions from Section 9; all numbers in the site's timezone and with comparison deltas against the comparison period). Dimension values such as page paths, referrer names and goal names originate from visitor-controlled input, so both the HTML and the text part render them as escaped plain text and never as links:
- Header: site domain and the period label.
- Four headline metrics: visitors, pageviews, bounce rate, visit duration. Each shows the value, and the change versus the comparison period as a signed percentage with an arrow; when the comparison value is zero the change reads "New".
- Top 5 sources by visitors, top 5 pages by visitors, top 5 countries by visitors, each with visitor counts.
- Goals: the top 5 goals by unique conversions, with conversions and conversion rate. Omitted when the site has no goals or none converted.
- Revenue: total revenue per currency, up to three currencies, only when revenue was recorded in the period.
- A button "Open dashboard" linking to the site's dashboard with the report period applied (query parameters per Section 9). It is shown only to recipients who are users with access; external recipients do not receive the button.
- Footer: why the recipient receives the email, the site timezone, a link to manage report settings (users only) and the unsubscribe link of Section 12.10.
Subjects: weekly Weekly report for {site}: {period_label}; monthly Monthly report for {site}: {period_label}.
12.5.4 Managing recipients #
| Operation | Endpoint | Action ID | Rules |
|---|---|---|---|
| Read configuration | GET /api/app/v1/sites/:domain/email-reports |
email_report.manage |
Returns {"data": {"timezone", "send_time_local": "08:00", "weekly": {"recipients": [...]}, "monthly": {"recipients": [...]}}}. Recipient objects: id, type (user or external), user_id, name, email, status, added_at |
| Add recipient | POST /api/app/v1/sites/:domain/email-reports/recipients |
email_report.manage |
Body { "frequency", "user_id" } or { "frequency", "email" }. A user_id must be a member of the team or a guest of the site (422 invalid_recipient). An email that belongs to a user with access to the site is stored as that user. An external address creates an external row in status pending_confirmation (Section 12.5.1) and enqueues the email_report_subscribed_notice email, which asks the recipient to confirm; in the hosted edition this requires an active license (403 license_required). Duplicate: 409 recipient_exists. Previously unsubscribed external address: 409 recipient_unsubscribed (only the recipient can undo it through the unsubscribe page). Audit email_report.recipient_added |
| Remove recipient | DELETE /api/app/v1/sites/:domain/email-reports/recipients/:id |
email_report.manage |
204. Audit email_report.recipient_removed |
| Reactivate bounced | POST /api/app/v1/sites/:domain/email-reports/recipients/:id/reactivate |
email_report.manage |
Sets status active, resets the failure counter; 409 not_bounced otherwise |
| Self status | GET /api/app/v1/sites/:domain/email-reports/me |
email_report.subscribe_self |
{"data": {"weekly": true, "monthly": false}} |
| Self subscribe | PUT /api/app/v1/sites/:domain/email-reports/me |
email_report.subscribe_self |
Body { "weekly": bool, "monthly": bool }; creates, reactivates or deletes the caller's rows; a row in status unsubscribed is set back to active when the caller re-enables it |
| Send a test | POST /api/app/v1/sites/:domain/email-reports/test |
email_report.manage |
Body { "frequency" }; sends the report for the last complete period to the caller only, with subject prefix [Test] ; limited by the email_report_test policy of Section 17.6 |
Users who lose access to the site (removed from the team, guest removed, site transferred without access, account deleted) have their rows deleted (Sections 11.16.5, 11.17.3, 11.13.1). External rows are unaffected by access changes.
12.6 Traffic Spike Notifications #
12.6.1 Configuration #
One configuration per site, stored in spike_notifications (Section 5).
| Field | Rules |
|---|---|
enabled |
Boolean, default false |
threshold |
Integer 1 to 1,000,000; required when enabling; the UI pre-fills 50. Compared with the current visitors count (Section 12.6.2) |
| Cooldown | Fixed 60 minutes (not configurable) |
| Recipients | Up to 10 rows (Section 5.16) in spike_notification_recipients, each { "id", "user_id" | "email", "status": "pending_confirmation" | "active" | "unsubscribed" | "bounced", "unsubscribe_token", "consecutive_failures" }; users must have access to the site, external addresses are allowed and follow the confirmation rules of Section 12.5.1 |
last_triggered_at, armed, recent_triggers |
Internal state: last send time, whether a new trigger is allowed, and the timestamps of triggers within the past 24 hours |
| Operation | Endpoint | Action ID | Notes |
|---|---|---|---|
| Read | GET /api/app/v1/sites/:domain/spike-notification |
spike.manage |
{"data": {"enabled", "threshold", "cooldown_minutes": 60, "recipients": [...], "last_triggered_at"}} |
| Update | PUT /api/app/v1/sites/:domain/spike-notification |
spike.manage |
Body { "enabled", "threshold" }. Enabling without a threshold returns 400 validation_failed. Changing the threshold sets armed = true. Audit spike.updated |
| Add recipient | POST /api/app/v1/sites/:domain/spike-notification/recipients |
spike.manage |
Body { "user_id" } or { "email" }; same validation, duplicate, unsubscribed and confirmation rules as Section 12.5.4; external recipients start in pending_confirmation and receive the email_report_subscribed_notice email with kind traffic spike; more than 10 recipients returns 409 limit_reached with details.limit_name = "spike_recipients" |
| Remove recipient | DELETE /api/app/v1/sites/:domain/spike-notification/recipients/:id |
spike.manage |
204 |
| Reactivate bounced | POST /api/app/v1/sites/:domain/spike-notification/recipients/:id/reactivate |
spike.manage |
As in Section 12.5.4 |
| Self status and subscribe | GET and PUT /api/app/v1/sites/:domain/spike-notification/me |
spike.subscribe_self |
Body { "subscribed": bool }; adds or removes the caller as a recipient |
A site with enabled = true and no active recipient sends nothing; the settings page shows a warning "Add at least one recipient."
12.6.2 Detection #
The spike-check job runs every minute (Section 16.6). For each site with enabled = true and at least one active recipient (and an active team):
- Read the site's current visitors: the number of distinct visitors with an event in the last 5 minutes, the same definition and Valkey counters as the real-time view in Section 9.
- If the count is below the threshold, set
armed = trueand stop. - If the count is at or above the threshold, send only when all hold:
armedis true; at least 60 minutes have passed sincelast_triggered_at; fewer than 6 triggers exist inrecent_triggerswithin the last 24 hours. - On trigger: gather the top 3 sources and top 3 pages of the last 5 minutes from ClickHouse, enqueue one
email-sendjob per active recipient with thespike_notificationtemplate, setlast_triggered_at, setarmed = falseand append torecent_triggers.
Consequences: a sustained spike produces one email, then at most one more per hour after the count has dropped below the threshold and risen again. Enabling the feature while traffic is already above the threshold sends the first email within a minute.
12.6.3 Content #
Subject Traffic spike on {site}: {count} visitors right now. Body: the current visitors and the configured threshold, the top 3 sources and top 3 pages of the last 5 minutes, a button "Open real-time view" (users only; external recipients receive no button) and the unsubscribe link (Section 12.10).
12.7 Cap Warning Emails #
The usage-cap emails belong to the licensing system. Section 15 owns the triggers (usage reaching 80 percent and 100 percent of the monthly cap), their evaluation, their deduplication per month and level, and their recipients (the team owner and all team admins). This section owns the template names, subjects and content, which are listed in the inventory in Section 12.8 as usage_cap_80 and usage_cap_100. They are delivered through the same queue as every other email (Section 12.9) and have no unsubscribe link, because they are operational notices that the account holders must receive.
12.8 Email Template Inventory (Canonical) #
Rules for the inventory:
- This table is complete. The product sends no email whose template ID is not listed here. A section that needs a new email must add a row here.
{product}isPRODUCT_NAME(Section 16.4).{site}is the site domain. Names and domains inserted into subjects are truncated to 60 characters with an ellipsis. Subjects contain no line breaks.- Every template has an HTML part and a plain-text part (Section 12.9.1). Every action link is absolute and begins with
BASE_URL. - "Account" means the user's current email address.
| Template ID | Trigger | Owning section | Recipients | Subject | Content summary |
|---|---|---|---|---|---|
verify_email |
Signup when verification is required; resend | 11.3 | Account | Verify your email for {product} |
Greeting by name, button "Verify email", link, 24-hour expiry, ignore-if-not-you line |
password_reset |
Forgot-password request for an existing account | 11.9 | Account | Reset your {product} password |
Button "Choose a new password", link, 1-hour expiry, ignore-if-not-you line, note that 2FA remains required |
password_changed |
Password changed or reset | 11.9 | Account | Your {product} password was changed |
Time in UTC, approximate country, statement that other sessions were signed out, link to reset the password if it was not the user |
email_change_confirm |
Email change requested | 11.10 | The new address | Confirm your new email address for {product} |
Button "Confirm email", link, 24-hour expiry, the old address shown masked |
email_change_requested_notice |
Email change requested | 11.10 | Current address | A change of your {product} email address was requested |
New address masked (a***@domain), what happens next, link to reset the password if not the user |
email_changed_notice |
Email change confirmed | 11.10 | The old address | Your {product} email address was changed |
New address masked, time, instructions for contacting the administrator if not the user |
two_factor_enabled |
2FA enabled | 11.11 | Account | Two-factor authentication is on for your {product} account |
Confirmation, time, reminder to keep recovery codes safe |
two_factor_disabled |
2FA disabled by the user, operator or CLI | 11.11 | Account | Two-factor authentication was turned off for your {product} account |
Time, who performed it (you, an administrator), link to re-enable and to reset the password |
recovery_code_used |
Login with a recovery code | 11.4 | Account | A recovery code was used on your {product} account |
Time, remaining count, link to regenerate codes |
recovery_codes_regenerated |
Recovery codes regenerated | 11.11 | Account | Your {product} recovery codes were regenerated |
Time, statement that old codes no longer work |
login_lockout_notice |
Login lock set after 5 failed attempts | 11.8 | Account | Sign-in attempts to your {product} account were blocked |
15-minute lock, link to reset the password |
account_deletion_scheduled |
Account deletion requested | 11.13 | Account | Your {product} account is scheduled for deletion |
Anonymization date (30 days), how to cancel by signing in, list of teams scheduled for deletion |
team_invitation |
Team invitation created or resent | 11.15 | Invited address | {inviter_name} invited you to join {team_name} on {product} |
Inviter, team, role, button "Accept invitation", 7-day expiry, note that an account will be created if none exists |
site_guest_invitation |
Guest invitation created or resent | 11.18 | Invited address | {inviter_name} invited you to view {site} on {product} |
Inviter, site domain, guest role, button, 7-day expiry |
invitation_accepted |
Team or guest invitation accepted | 11.15 | Inviter | {invitee_name} accepted your invitation to {target_name} |
Who accepted, target team or site, role, link to members or guests |
member_role_changed |
Role changed | 11.16 | The member | Your role in {team_name} changed to {role} |
Old and new role, who changed it, summary of what the new role can do |
member_removed |
Member removed (not on leaving) | 11.16 | The removed user | You were removed from {team_name} on {product} |
Team name, time, statement that access ended |
team_2fa_enforced |
Owner turns on 2FA enforcement | 11.14.4 | Every member without 2FA | {team_name} now requires two-factor authentication |
Explanation, button to set up 2FA, statement that access is paused until done |
team_deletion_scheduled |
Team deletion requested | 11.14.5 | All members; the owner's copy includes the restore button | {team_name} is scheduled for deletion on {date} |
Purge date (7 days), what happens, restore link for the owner |
ownership_transfer_request |
Ownership transfer started | 11.16.4 | The target member | {owner_name} wants to make you the owner of {team_name} |
Explanation of owner duties, button "Review request", 7-day expiry |
ownership_transferred |
Transfer accepted | 11.16.4 | Previous and new owner | Ownership of {team_name} was transferred |
Who is the owner now, previous owner's new role (admin), time |
site_transfer_request |
Site transfer by invitation | 11.17.2 | Invited address | {inviter_name} wants to transfer {site} to you on {product} |
Site, from team, button to choose a destination team, 7-day expiry |
site_transferred |
Site transfer completed | 11.17 | Owners and admins of both teams | {site} moved from {from_team} to {to_team} |
Site, both teams, time, who performed it |
email_report_weekly |
Weekly schedule (Section 12.5) | 12.5 | Active weekly recipients | Weekly report for {site}: {period_label} |
Section 12.5.3 content |
email_report_monthly |
Monthly schedule | 12.5 | Active monthly recipients | Monthly report for {site}: {period_label} |
Section 12.5.3 content |
email_report_subscribed_notice |
An external address is added as a report or spike recipient | 12.5 | The external address | {adder_name} added you to {kind} notifications for {site} |
Who added them, what they will receive and when, a Confirm button (nothing is sent until the recipient confirms; unconfirmed requests expire after 7 days), unsubscribe link (kind is weekly report, monthly report or traffic spike) |
spike_notification |
Spike detection (Section 12.6) | 12.6 | Active spike recipients | Traffic spike on {site}: {count} visitors right now |
Section 12.6.3 content |
smtp_test |
Hosted operator console or the self-hosted command-line tool sends a test message | 12.9.7 | The requesting super admin (hosted) or the address given to the tool (self-hosted) | {product} email test |
Confirmation that delivery works, timestamp, host name |
usage_cap_80 |
Usage reaches 80 percent of the monthly cap | 15 | Team owner and admins | You have used 80% of this month's events on {team_name} |
Usage numbers, cap, reset date (first of next month, UTC), upgrade button for the owner |
usage_cap_100 |
Usage reaches 100 percent | 15 | Team owner and admins | You have reached your monthly event cap on {team_name} |
Cap reached, 10 percent grace explained, stop point, reset date, upgrade button |
license_activated |
First license grant by purchase or first code redeemed | 15 | Purchaser or redeemer | Your {product} license is active |
Tier, site and event limits, link to the usage page |
license_revoked |
License refunded or revoked | 15 | Team owner | Your {product} license was revoked |
Effect (ingestion stops; data is kept for 30 days, or on hold while a dispute is open), reason class, link to activate a new license |
unlicensed_purge_warning |
7 days and 1 day before an unlicensed team's data is purged | 15 | Team owner and admins | Data for {team_name} will be deleted on {date} |
Purge date, how to relicense, export link |
import_completed |
Import finished | 14 | User who started it | Your import for {site} finished |
Rows imported, date range covered, link to the site |
import_failed |
Import failed | 14 | User who started it | Your import for {site} failed |
Failure reason summary, link to the import page |
export_ready |
Asynchronous export finished | 14 | User who requested it | Your export for {site} is ready |
Download link available for 7 days, file size, row count |
export_failed |
Asynchronous export failed | 14 | User who requested it | Your export for {site} failed |
Failure summary, link to start the export again from the site's data page |
license_upgraded |
Upgrade paid, or second or third code redeemed | 15 | Purchaser or redeemer | Your {product} license was upgraded to {tier} |
New tier, new site and event limits, link to the usage page |
refund_processed |
Full refund applied | 15 | Purchaser | Your {product} refund was processed |
Refunded amount, effect on the license and data, link to the billing page |
purchase_auto_refunded |
Duplicate or invalid-state payment refunded automatically | 15 | Purchaser | Your duplicate {product} payment was refunded |
Reason, amount, expected timing of the refund |
unlicensed_purge_completed |
After an unlicensed team's data was purged | 15 | Team owner | Data for {team_name} was deleted |
What was deleted and when, how to start again |
impersonation_notice |
An operator starts a read-only impersonation (at most one per operator per user per 24 hours) | 15 | The impersonated user | {product} support viewed your account |
When, the stated reason, statement that the session was read-only |
operator_alert |
Dispute, automatic refund, brute-force suspicion or out-of-band refund anomaly | 15 | All super admins | [{product} operator] {alert_title} |
Alert title, affected team or purchase IDs, link to the console |
Operational notes: payment receipts are sent by the payment provider; the product sends its own refund and license notices listed above (refund_processed, purchase_auto_refunded and the license_* templates). There is no welcome email, no product-news email and no re-engagement email.
12.9 Email Delivery #
12.9.1 Rendering #
Templates are TypeScript functions in apps/server/src/email/templates/, one module per template ID, exposed through one function:
export function renderEmail<T extends TemplateId>(
id: T,
locals: TemplateLocals[T],
): { subject: string; html: string; text: string };| Rule | Decision |
|---|---|
| Locals | Each template has a Zod schema in packages/shared; the schema is validated when the job is enqueued and again at render time; a validation failure fails the job permanently and logs an error |
| Layout | One shared layout: 600 px wide table layout, system font stack, inline CSS only, light appearance only (no dark-mode variant), text wordmark using PRODUCT_NAME, a hidden preheader line, a bulletproof anchor-based button, footer with the reason for the email and the optional operator address MAILER_FOOTER_ADDRESS |
| Images and tracking | No images, no tracking pixels, no click-tracking redirects, no external fonts or styles; every link points to BASE_URL |
| Escaping | Every inserted value (names, domains, team names) is HTML-escaped in the HTML part; the text part is generated from the same locals, not by stripping HTML |
| Subject | CR and LF removed; maximum 150 characters; inserted names truncated to 60 characters |
| Plain text | Required for every email; links on their own lines |
| Language | English only (Section 4 covers localization policy) |
12.9.2 Transport #
| Property | Decision |
|---|---|
| Library | Nodemailer over SMTP (Section 3) |
| Configuration | SMTP_HOST, SMTP_PORT, SMTP_SECURE (starttls, tls or none), SMTP_USER, SMTP_PASSWORD, MAILER_FROM (default PRODUCT_NAME <no-reply@<host of BASE_URL>>), MAILER_REPLY_TO (optional, no default), MAILER_FOOTER_ADDRESS (optional); all in Section 16.4 |
| TLS | SMTP_SECURE=tls uses implicit TLS; starttls requires a successful STARTTLS upgrade before any message is sent; none (no encryption) is accepted only in the self-hosted edition |
| Pool | Pooled transport, 5 connections, 100 messages per connection |
| Timeouts | Connection 10 s, greeting 10 s, socket inactivity 30 s |
| Throughput | Limited globally by EMAIL_MAX_PER_MINUTE (Section 16.4) |
| Envelope sender | The address in MAILER_FROM |
12.9.3 Queue and retries #
All email goes through the pg-boss queue email-send (worker process, Section 16.6). The payload is { template, to, locals, idempotency_key }. The idempotency_key is used as the pg-boss singleton key, so enqueueing the same logical email twice (a retried transaction, a double click) sends once; keys are built as <template>:<recipient>:<event ID or period key>.
| Property | Decision |
|---|---|
| Concurrency | 5 jobs in parallel per worker process |
| Retry limit, backoff and attempt timeout | 8 retries with exponential backoff starting at 30 seconds (up to 20 percent random jitter) and a 120-second SMTP timeout, matching the email-send row of Section 16.6 |
| Retained | Completed jobs 7 days, failed jobs 14 days |
| Retryable failures | Connection errors, timeouts, SMTP 4xx responses, and SMTP 530, 534 and 535 (authentication failures are an operator configuration error; they log at error level and retry) |
| Permanent failures | SMTP 5xx rejections of the recipient or message (550, 551, 552, 553, 554 and enhanced codes 5.1.x, 5.2.x, 5.7.x): no retry, recorded as bounced for subscription recipients (Section 12.9.4) |
| User-visible effect of final failure | None for transactional email (users can request it again); the failure is logged with template ID, recipient domain only, job ID and SMTP code |
Logs never contain recipient local parts, subjects with names, tokens in links or message bodies (Section 17), except in the no-SMTP mode of Section 12.9.6.
12.9.4 Bounce handling (decision) #
The product speaks plain SMTP and does not operate an inbound mailbox, so only synchronous rejections are handled.
- A permanent SMTP rejection (Section 12.9.3) of a report or spike email increments
consecutive_failureson that subscription or recipient. Three consecutive permanent failures set statusbouncedand stop further sends. A successful SMTP acceptance resets the counter to zero. - Bounced recipients appear in the management UI with a "Delivery failed" badge. A user with
email_report.manageorspike.managecan reactivate them (Section 12.5.4), which resets the counter. - Transactional and security emails keep no bounce state.
- Asynchronous delivery-status notifications sent back to the envelope sender after SMTP acceptance, and spam complaints, are not processed. Operators who want that use an SMTP provider that handles suppression itself. The
List-Unsubscribeheader on reports and spike emails is the complaint-reduction mechanism.
12.9.5 Headers #
| Header | Value |
|---|---|
From |
MAILER_FROM |
Reply-To |
MAILER_REPLY_TO when set; absent otherwise |
Auto-Submitted |
auto-generated on every message |
X-Auto-Response-Suppress |
All |
Precedence |
bulk on report, spike and subscription-notice emails only |
List-Unsubscribe |
<{BASE_URL}/api/app/v1/unsubscribe/<token>> on report and spike emails only |
List-Unsubscribe-Post |
List-Unsubscribe=One-Click on the same emails |
Message-ID |
Generated with the domain of MAILER_FROM |
12.9.6 Mode without SMTP (self-hosted only) #
When SMTP_HOST is unset in the self-hosted edition, email is disabled (GET /bootstrap reports features.email: false). The hosted edition refuses to start without SMTP_HOST (Section 16).
| Area | Behavior |
|---|---|
| Sending | The email-send worker logs an info event named email_not_sent with the template ID and recipient domain; for the templates verify_email, password_reset, email_change_confirm, team_invitation, site_guest_invitation, ownership_transfer_request and site_transfer_request it also logs the action_url. This is the only place an action link is logged and it is intentional so an operator can retrieve it; the redaction rules of Section 17 exempt exactly this field |
| Invitations | The create response includes accept_url (Section 11.15.2) so the inviter can pass the link on |
| Verification | REQUIRE_EMAIL_VERIFICATION defaults to false |
| Password reset | The forgot-password page says "Email is not set up on this server. Ask your administrator to reset your password." and the administrator uses the command-line tool of Section 11.11.5 |
| Reports and spike notifications | The settings pages show a notice "Email is not configured on this server" and the jobs skip these sites |
12.9.7 Test message #
POST /api/app/v1/admin/test-email (hosted edition only; super admin with action admin.console; session and CSRF required) sends the smtp_test template synchronously, bypassing the queue, to the caller's address. It returns 200 {"data": {"delivered": true}} on SMTP acceptance, or 503 unavailable with details = { "reason": "smtp_failed", "smtp_code": 535 } (the code only; never the server response text). It is limited by the system_test_email policy of Section 17.6. The self-hosted edition has no operator console; there the administrator runs docker compose run --rm web cli users:test-email <address> (Section 11.11.5), which sends the same template to the given address and prints the SMTP result code.
12.10 Unsubscribe Handling #
Only emails with optional content carry an unsubscribe link: weekly and monthly reports, spike notifications and the subscription notice sent to external recipients. Transactional, security, invitation, licensing and import emails cannot be unsubscribed from; they are sent only because of an action or state that concerns the recipient. There is no global suppression list; an unsubscribe applies to one site and one kind of email.
| Endpoint | Behavior |
|---|---|
GET /api/app/v1/unsubscribe/:token |
Unauthenticated. Returns {"data": {"kind": "report_weekly" | "report_monthly" | "spike", "site_domain", "email_masked", "status"}}. Unknown token: 404 not_found. Never changes state |
POST /api/app/v1/unsubscribe/:token |
Unauthenticated, exempt from CSRF (Section 11.7). Body { "scope": "this" | "site_all" } (default this) as JSON, or the form body List-Unsubscribe=One-Click sent by mail clients. this sets that one subscription to unsubscribed; site_all does so for every report and spike subscription of the same recipient on the same site. Idempotent. Returns {"data": {"status": "unsubscribed"}}. Audit is not written (the actor is unauthenticated) |
POST /api/app/v1/unsubscribe/:token/resubscribe |
Sets status active again unless the status is bounced (409 not_resubscribable); lets a recipient undo a mistake |
POST /api/app/v1/unsubscribe/:token/confirm |
Unauthenticated, exempt from CSRF. Moves a recipient from pending_confirmation to active and sets confirmed_at; any other status returns 409 conflict with details.reason = "not_pending_confirmation" |
| Rule | Decision |
|---|---|
| Token | Per recipient and per subscription; 32 random bytes, base64url (43 characters); stored on the subscription row (reports) or on the spike_notification_recipients row (spike); never expires; deleted with the row |
| Confirmation page | /unsubscribe/:token calls GET and shows the site, the masked address and a button "Unsubscribe" (for a recipient in pending_confirmation it also shows the primary button "Confirm subscription", which calls /confirm); a second button "Unsubscribe from everything for this site" sends scope = site_all; after success it shows "You have been unsubscribed" with an "Undo" button. State changes only on explicit button press so mail scanners that follow links change nothing |
| User recipients | May turn reports back on in the site's settings (PUT .../me); the unsubscribed row becomes active |
| External recipients | Editors and above cannot re-add an unsubscribed external address (409 recipient_unsubscribed); only the recipient can undo through the page |
| Rate limit | Policy unsubscribe_ip (Section 17.6) |
12.11 Endpoint Index #
| Method and path | Auth | Action ID | Section |
|---|---|---|---|
GET/POST /api/app/v1/sites/:domain/shared-links, GET/PATCH/DELETE .../shared-links/:id |
Session | shared_link.manage |
12.2.1 |
PATCH /api/app/v1/sites/:domain/public-dashboard |
Session | public_dashboard.toggle |
12.3 |
GET /api/share/v1/link/:token/meta, POST .../unlock, mirrored read endpoints |
Share token (and unlock header) | none | 12.2.3 |
GET /api/share/v1/public/:domain/meta, mirrored read endpoints |
none | none | 12.3 |
GET /api/app/v1/sites/:domain/email-reports, POST .../recipients, DELETE .../recipients/:id, POST .../recipients/:id/reactivate, POST .../test |
Session | email_report.manage |
12.5.4 |
GET/PUT /api/app/v1/sites/:domain/email-reports/me |
Session | email_report.subscribe_self |
12.5.4 |
GET/PUT /api/app/v1/sites/:domain/spike-notification, POST .../recipients, DELETE .../recipients/:id, POST .../recipients/:id/reactivate |
Session | spike.manage |
12.6.1 |
GET/PUT /api/app/v1/sites/:domain/spike-notification/me |
Session | spike.subscribe_self |
12.6.1 |
GET/POST /api/app/v1/unsubscribe/:token, POST .../resubscribe, POST .../confirm |
Token | none | 12.10 |
POST /api/app/v1/admin/test-email |
Session (super admin, hosted only) | admin.console |
12.9.7 |
Error codes used here (statuses per Section 4.4.3): 400 validation_failed; 401 share_password_required, share_password_invalid; 403 license_required (hosted external recipients), forbidden; 404 not_found; 409 limit_reached (details.limit_name is shared_links, report_subscriptions or spike_recipients), segment_in_use, recipient_exists, recipient_unsubscribed, not_bounced, not_resubscribable, conflict (details.reason = "not_pending_confirmation"); 422 segment_immutable, segment_not_shared, invalid_embed_origin, embed_origins_required, invalid_recipient; 429 rate_limited; 503 unavailable.
12.12 Settings Screens #
| Screen | Location | Behavior |
|---|---|---|
| Share dialog | Dashboard header "Share" button | Tabs Link and Embed (Section 12.4.6); lists existing links with name, protection badges, last opened, copy and revoke buttons; empty state "No shared links yet" |
| Public dashboard | Site settings, shared-links page (/sites/:domain/settings/shared-links), Public dashboard panel |
Toggle with confirmation (Section 12.3); shows the public URL with copy button when on |
| Email reports | Site settings, notifications page (/sites/:domain/settings/notifications), Email reports panel |
Two sections (Weekly, Monthly), each with the recipient table (name, email, status including an "Awaiting confirmation" badge for external addresses, remove), an add-recipient input accepting a member name or an email address, a self-subscribe switch visible to every role that has email_report.subscribe_self, and a "Send test" button; the next send time is displayed in the site timezone |
| Spike notifications | Site settings, notifications page (/sites/:domain/settings/notifications), Spike panel |
Enable switch, threshold number input (1–1,000,000), recipient table with the same controls, "last triggered" text; the cooldown is shown as fixed "60 minutes" |
| Unsubscribe | /unsubscribe/:token |
Section 12.10 |
Viewers and guest viewers see only their own self-subscribe switches for reports and spike notifications on these panels; all other controls are hidden.
12.13 Data Requirements and Jobs #
12.13.1 Attributes this section requires of the Section 5 tables #
| Table | Required attributes |
|---|---|
shared_links |
site_id, name (1–60 characters), token (unique, 32 characters), password_hash (nullable), segment_id (nullable, foreign key to segments), embed_mode (disabled, any, allowlist; default disabled), embed_origins (text array, default empty), include_imports (boolean, default true), created_by_user_id (nullable), last_accessed_at |
site_settings |
public_dashboard boolean, default false (Section 5) |
email_report_subscriptions |
site_id, frequency (weekly, monthly), user_id (nullable), email (nullable; exactly one of the two is set), status (pending_confirmation, active, unsubscribed, bounced), confirmed_at (nullable), unsubscribe_token (unique), last_sent_period, consecutive_failures, added_by_user_id (nullable); unique (site_id, frequency, user_id) and unique (site_id, frequency, email) |
spike_notifications |
site_id (unique), enabled, threshold, armed, last_triggered_at, recent_triggers (timestamp array) |
spike_notification_recipients |
id, site_id, user_id (nullable), email (nullable; exactly one of the two is set), status (pending_confirmation, active, unsubscribed, bounced), confirmed_at (nullable), unsubscribe_token (unique, so a token resolves in one indexed lookup), consecutive_failures, added_by_user_id (nullable); unique (site_id, user_id) and unique (site_id, email) |
12.13.2 Background jobs used by this section #
Names, schedules, queues and retries are canonical in the jobs table of Section 16.6; the work described here is owned by this section.
| Job | Trigger | Work |
|---|---|---|
email-reports-dispatch |
Schedule | Finds due (site, frequency, period) combinations and enqueues email-report-send (Section 12.5.2) |
email-report-send |
Queue, one per site, frequency and period | Queries statistics once, renders per-recipient messages and enqueues email-send jobs |
spike-check |
Every minute | Detection of Section 12.6.2 |
email-send |
Queue | Delivery with the retry policy of Section 12.9.3; used by every template in Section 12.8 |
13. Public APIs: Stats API, Events API, Sites API, API Keys & Rate Limits #
This section owns every machine-facing HTTP interface of the product and the query engine that powers all statistics. The query engine (13.4 and 13.5) is canonical: the dashboard (Section 9), shared links and embeds (Section 12), email reports (Section 12), funnels and journeys (Section 10) and CSV export (Section 14) all obtain their numbers by calling it. No other section defines its own SQL for site statistics.
13.1 Surface Overview #
| Family | Route(s) | Auth | Scope / permission | Rate-limit policy | Owner of behavior |
|---|---|---|---|---|---|
| Stats API | POST /api/v2/query |
Bearer API key | stats:read |
policies stats_api_key, stats_api_team, stats_concurrency (Section 17.6) |
13.4, 13.5 |
| Sites API | /api/v1/sites..., /api/v1/teams |
Bearer API key | sites:provision (read-only subset also accepts stats:read, see 13.3.2) |
policy sites_api_key (Section 17.6) |
13.8 |
| Events API | POST /api/event |
None (public, CORS *) |
none | policy event_ingest (Section 17.6) |
13.7 for usage; Section 8 for processing |
| Dashboard query API | POST /api/app/v1/sites/{domain}/query, POST /api/app/v1/sites/{domain}/query/batch |
Cookie session + X-CSRF-Token |
action site.view (any member or guest of the site, Section 11) |
policy dashboard_query (Section 17.6) |
13.6 |
| Consolidated query API | POST /api/app/v1/consolidated/query |
Cookie session + X-CSRF-Token |
the caller's own sites only (rules in Section 9.20) | policy dashboard_query (Section 17.6) |
13.5.1 (engine), Section 9.20 (UI) |
| API key management | /api/app/v1/teams/{team_id}/api-keys... |
Cookie session + X-CSRF-Token |
action team.api_keys.manage (owner, admin) |
dashboard-API limits (Section 17.6) | 13.2 |
| OpenAPI document | GET /api/openapi.json |
None | none | policy openapi (Section 17.6) |
13.9 |
The numeric values of every rate-limit policy named in this section live in the single rate-limit table of Section 17.6; this section defines only which policy applies to which route and the response headers (13.3.3).
Decisions that apply to the whole section:
- The Stats API is a single endpoint. There are no separate aggregate, breakdown or timeseries endpoints and no
/api/v1/stats/*routes; those three shapes are all expressed by one query (no dimensions = aggregate, one non-time dimension = breakdown, onetime:*dimension = timeseries). - Events API write access needs no key.
events:writeis not a scope and must not be created. - All public JSON uses snake_case, ISO 8601 timestamps in UTC with a
Zsuffix (except where a response is explicitly expressed in the site timezone, 13.4.2), the success envelope and error envelope of Section 4.4, and theX-Request-Idresponse header (areq_...value equal toerror.request_idon failures). - Request bodies must be
Content-Type: application/json(with or without; charset=utf-8). Any other type returns415with error codeunsupported_media_type. The Events API is the sole exception (it also acceptstext/plain). - Public Stats and Sites request bodies are strict: unknown top-level or nested fields return
400 validation_failedwith the offending path indetails.fields. Responses are additive over time (13.10), so clients must ignore unknown response fields. - Maximum request body size for the Stats and Sites APIs: 256 KB (
413, error codepayload_too_large; the body-limit table is in Section 17.4). - All endpoints are served over the same origin as the dashboard. CORS:
/api/v2/*and/api/v1/*send no CORS headers, because API keys are secrets that belong in server-side code and must never be embedded in browser code (Section 17.5.3); the in-product playground (13.6.4) uses the cookie-authenticated dashboard endpoint instead. OnlyGET /api/openapi.jsonsendsAccess-Control-Allow-Origin: *(methodsGET, OPTIONS, no credentials,Access-Control-Max-Age: 86400) so documentation tools can load it. The Events API has its own CORS rules (13.7.1).
13.2 API Keys #
API keys are owned by a team, not a user. A key can read statistics for, and provision, only sites that belong to the key's team. When the creating user is removed from the team, leaves it, or deletes their account, the keys that user created in that team are deleted immediately (Section 11.16.5), because a departing member knows the plaintext. The audit entry for such a deletion is api_key.revoked with reason = "creator_removed" (Section 11.20).
13.2.1 Key format and storage #
| Aspect | Decision |
|---|---|
| Format | qs_live_ followed by 43 characters of base64url (alphabet A-Z a-z 0-9 - _, no padding), generated from 32 bytes of crypto.randomBytes. Total length 51. Regex: ^qs_live_[A-Za-z0-9_-]{43}$. The prefix qs_live_ is a fixed constant and is NOT affected by renaming the product via PRODUCT_NAME. |
| Display prefix | The first 12 characters of the key (qs_live_ plus 4 characters) are stored in api_keys.key_prefix and shown in the UI and in logs as qs_live_AbCd…. |
| Stored secret | SHA-256 of the full key string, stored as a 32-byte bytea in api_keys.key_hash (unique index; hex only in the Valkey cache key). The plaintext key is never stored, never logged (pino redaction in Section 17 covers the Authorization header and any field named key), and is shown to the user exactly once, in the creation response. A fast hash is correct here because the key has 256 bits of entropy; a slow password hash would only add latency. |
| Scopes | stats:read, sites:provision (stored as text[]). At least one is required. There is no events:write scope. |
| Per-team limit | The API-keys-per-team limit of Section 5.16 (50). Exceeding it returns 409 limit_reached with details.limit_name = "api_keys". |
| Name | 1 to 60 characters after trimming, unique per team case-insensitively (409, code api_key_name_taken). |
| Expiry | Optional. expires_in_days accepts null (never), 30, 90, 365. Stored as api_keys.expires_at. An expired key behaves exactly like a deleted key. |
| Last used | api_keys.last_used_at is updated at most once per 60 seconds per key: on a successful authentication the server runs SET qs:apikey:lu:{key_id} 1 NX EX 60 in Valkey and, only if that call set the key, writes last_used_at = now() to Postgres without blocking the request. The UI renders "Never" when null. |
| Revocation | "Revoke" is a hard delete of the row (Section 4 delete policy) plus immediate deletion of the Valkey cache entry qs:apikey:{key_hash}. A revoked key returns 401 invalid_api_key on the very next request on every web instance. The audit log (Section 11.20) keeps api_key.created and api_key.revoked events with the key id, name and prefix. |
| Auth cache | After a successful Postgres lookup the key record (key_id, team_id, scopes, expires_at) is cached in Valkey at qs:apikey:{key_hash} for 60 seconds. Negative results are not cached. |
Columns required from the api_keys table (authoritative DDL in Section 5): id, team_id, name, key_prefix, key_hash, scopes, created_by_user_id (nullable), created_at, last_used_at, expires_at. The scope strings stored in scopes are exactly stats:read and sites:provision. There is no per-key rate-limit column: every key of a policy gets the same fixed limit (13.3.3), so a team administrator cannot raise a key's quota.
13.2.2 Management endpoints (dashboard API) #
Permissions: action team.api_keys.manage (owner and admin of the team only, matrix in Section 11). Editors, viewers and guests receive 403 forbidden; the settings page for API keys is hidden for them.
| Method and path | Purpose | Success |
|---|---|---|
GET /api/app/v1/teams/{team_id}/api-keys |
List keys, newest first, cursor pagination (default 50, max 200) | 200 list envelope |
POST /api/app/v1/teams/{team_id}/api-keys |
Create a key | 201 single envelope including key |
PATCH /api/app/v1/teams/{team_id}/api-keys/{key_id} |
Rename only (body {"name": "..."}) |
200 single envelope |
DELETE /api/app/v1/teams/{team_id}/api-keys/{key_id} |
Revoke | 204 empty body |
Create request:
{
"name": "Reporting dashboard",
"scopes": ["stats:read"],
"expires_in_days": 90
}Create response (the only time key appears):
{
"data": {
"id": "0192f7c0-5a1e-7c3b-9d44-0a5f2c1b7e10",
"name": "Reporting dashboard",
"key": "qs_live_Zk3b1uY8s0Qm4pLw7vN2rT9xA5cJ6dHeFgIoKiXyUzM",
"key_prefix": "qs_live_Zk3b",
"scopes": ["stats:read"],
"created_by": { "id": "0192f7be-1111-7000-8000-000000000001", "name": "Ada Lovelace" },
"created_at": "2026-11-03T09:12:44Z",
"last_used_at": null,
"expires_at": "2027-02-01T09:12:44Z"
}
}List items have the same shape without key. Validation errors: unknown scope value, empty scopes, duplicate scope values, name outside 1 to 60 characters, expires_in_days not in {null, 30, 90, 365} all return 400 validation_failed with field paths.
UI (Settings, API keys page, per team): table with columns Name, Key (prefix plus …), Scopes (chips), Created by, Created, Last used, Expires, and a Revoke action (confirmation dialog: "Revoke ? Applications using this key will stop working immediately."). The creation modal has Name, Scopes (checkboxes with one-line explanations), Expiry (select: Never, 30 days, 90 days, 1 year). After creation the modal shows the key once in a read-only field with a Copy button and the text "Copy this key now. You will not be able to see it again." The modal cannot be dismissed by clicking the backdrop. Empty state: "No API keys yet. Create a key to query your statistics or provision sites from your own code." A Playground tab is described in 13.6.4.
Team state effects: while a team is in its 7-day scheduled-deletion window (Section 11.14.5), all of its keys receive 404 not_found on every route, as the team itself is invisible to everyone but its owner; undoing the deletion restores them. When a hosted team is unlicensed (Section 15), stats:read keeps working (dashboards stay readable); site creation through the Sites API follows the license rules of Section 15.
13.3 Authentication, Scopes and Rate Limits #
13.3.1 Authentication #
- Header:
Authorization: Bearer qs_live_.... The key is never accepted in a query string, request body or cookie. - Missing or malformed header:
401, codeinvalid_api_key, headerWWW-Authenticate: Bearer realm="quietstat-api". - Well-formed but unknown, revoked or expired key:
401, codeinvalid_api_key, message "Invalid or expired API key."; the message is identical in all cases so a caller cannot probe key existence. A valid key whose team is in its deletion window receives404 not_found(13.2.2). - Key valid but scope missing:
403, codeapi_key_scope_insufficient,details: {"required_scope": "stats:read"}. - Authentication-failure throttle: policy
api_auth_failof Section 17.6 (failed authentications per client address per minute) returns429 rate_limitedfor that address for the remainder of the window. The counter lives in Valkey under an HMAC of the address (never the raw address).
13.3.2 Scope matrix #
| Endpoint | stats:read |
sites:provision |
|---|---|---|
POST /api/v2/query |
required | not accepted alone |
GET /api/v1/sites, GET /api/v1/sites/{domain}, GET /api/v1/teams |
accepted | accepted |
Every other /api/v1/* route (create, update, delete, goals, shared links, custom properties) |
not accepted | required |
A key may carry both scopes. A site is addressed by its domain (the public site identifier of Section 4). A site that does not exist, or exists in another team, returns 404 not_found with details.resource = "site"; both cases are indistinguishable by design.
13.3.3 Rate limits #
| Policy (Section 17.6) | Applies to | Window | Counted by |
|---|---|---|---|
stats_api_key |
POST /api/v2/query |
fixed clock hour, UTC | key id |
stats_api_team |
POST /api/v2/query, aggregate over all keys of a team |
fixed clock hour, UTC | team id |
sites_api_key |
every /api/v1/* route |
fixed clock hour, UTC | key id |
stats_concurrency |
in-flight Stats queries | instantaneous | key id and team id |
The numeric limits are set once, in the rate-limit table of Section 17.6, and are fixed policy constants (API_RATE_LIMITS in packages/shared): identical on hosted and self-hosted editions, not environment-configurable and not adjustable per key.
- A request is counted when it is authenticated, whether it later succeeds or fails validation. A batch is not available on the public API, so one call is one unit. Requests rejected with
401or403do not consume quota. A429response does not consume quota. - Counters are Valkey keys
qs:apikey:rl:{policy}:{key_id or team_id}:{unix_hour}with a 3,700-second TTL and are incremented atomically withINCR. - Every response from a Stats or Sites route carries the headers below. They describe the per-key policy of the route (
stats_api_keyorsites_api_key); when the team aggregate policy is the one that rejects,X-RateLimit-Remainingis0andX-RateLimit-Resetis that policy's window end.
| Header | Value |
|---|---|
X-RateLimit-Limit |
the limit of the applicable policy |
X-RateLimit-Remaining |
requests left in the current window, never negative |
X-RateLimit-Reset |
Unix time (seconds) when the window ends (next full UTC hour) |
- When exceeded:
429, coderate_limited, headerRetry-After: <seconds until reset>,X-RateLimit-Remaining: 0,details: {"retry_after_seconds": 1832, "policy": "stats_api_key", "limit": 600, "reset_at": "2026-11-03T10:00:00Z"}(the example shows the per-key Stats policy at its value of Section 17.6). - Concurrency overflow (
stats_concurrency) returns429withdetails: {"retry_after_seconds": 1, "reason": "concurrency", "policy": "stats_concurrency", "limit": 5}andRetry-After: 1. - The Events API has its own policy
event_ingest(Section 17.6) and returns429with a plain-text body; it has no key and noX-RateLimit-*headers. - Valkey unavailable: Stats and Sites requests fail closed with
503 unavailableanddetails: {"dependency": "valkey"}(Valkey is a required dependency of the auth cache and the limiter; the matching exception to the fail-open rule for ingestion is stated in Section 17.6).
13.4 Stats API: POST /api/v2/query #
One endpoint answers every statistical question. The request describes which site, which period, which numbers (metrics), how to group them (dimensions) and which visitors to include (filters).
Auth: Bearer key with stats:read (13.3). Rate-limit policies stats_api_key, stats_api_team and stats_concurrency (Section 17.6). Request and response Content-Type: application/json.
13.4.1 Request schema #
type StatsQuery = {
site_id: string; // site domain such as "example.com" (public API only)
metrics: Metric[]; // 1..15, unique
date_range: DateRangePreset | [string, string];
relative_date?: string; // "YYYY-MM-DD", anchors preset ranges; default = today in site timezone
dimensions?: Dimension[]; // 0..4, unique, at most one time:*
filters?: Filter[]; // implicit AND of the listed filters
order_by?: [string, "asc" | "desc"][]; // 0..5 entries
include?: {
imports?: boolean; // default false
time_labels?: boolean; // default false
total_rows?: boolean; // default false
comparisons?: {
mode: "previous_period" | "year_over_year" | "custom";
date_range?: [string, string]; // required iff mode = "custom"
match_day_of_week?: boolean; // default true; only meaningful for "year_over_year"
} | null;
};
pagination?: { limit?: number; offset?: number }; // limit 1..10000 default 10000; offset >= 0 default 0
};| Field | Required | Default | Rules |
|---|---|---|---|
site_id |
yes | none | Site domain, matched case-insensitively against the sites of the key's team (13.3.2). Forbidden on the dashboard endpoints (13.6). |
metrics |
yes | none | 1 to 15 entries, each from 13.4.3, no duplicates (400 invalid_query, path metrics.<i>, message "Duplicate metric."). Result metrics arrays follow this order exactly. |
date_range |
yes | none | A preset string (13.4.2) or a two-element array of ISO dates or ISO date-times. |
relative_date |
no | today in the site timezone | Only valid with preset date_range values other than all and 24h; combining with an array form returns 400 invalid_query. Lets a client ask for "the month containing 2026-02-14" or "the day 2026-02-14". Must not be later than today plus 1 day in the site timezone. |
dimensions |
no | [] |
0 to 4 entries from 13.4.4, no duplicates, at most one time:* entry. |
filters |
no | [] |
Grammar in 13.4.5. |
order_by |
no | see 13.4.6 | Each field must be one of the requested metrics or dimensions. |
include |
no | all false / null | 13.4.6. |
pagination |
no | {limit: 10000, offset: 0} |
Applies to result rows after ordering. |
Minimal valid request:
{ "site_id": "example.com", "metrics": ["visitors"], "date_range": "7d" }The Zod schema is statsQuerySchema in @quietstat/shared (Section 3). It validates shape and enumerations. Semantic validation that needs site data or cross-field logic (date ordering, metric and dimension compatibility, goal existence) is performed by packages/query (13.5) and reported as in 13.4.9.
13.4.2 Date ranges and timezone handling #
Every site has an IANA timezone (Section 4). All range resolution, day/week/month bucketing and "today" logic uses the site timezone current at query time. Changing a site's timezone re-buckets all history on the next query; no stored data changes.
Preset date_range values (the current day is always included, and "today" and "now" mean the site timezone):
| Preset | Resolved range (inclusive of the whole last day unless stated) |
|---|---|
day |
The single day relative_date (default today), 00:00:00 to 23:59:59.999 |
24h |
Rolling 24 hours ending at the start of the next full hour after now (so exactly 24 hour buckets, the last one being the current hour). Use only with time:hour or no time dimension; relative_date not allowed. |
7d, 28d, 30d, 91d |
The N calendar days ending on relative_date (default today), so 7d is today and the six preceding days |
month |
The calendar month containing relative_date, first to last day |
6mo, 12mo |
The N calendar months ending with the month containing relative_date, starting on the first day of the oldest month |
year |
Jan 1 to Dec 31 of the year containing relative_date |
all |
From the site's first day of data to today. The first day of data is the earliest of the first native event and the first imported row (when include.imports is true); with no data at all the range is today only. |
Custom ranges:
["2026-01-01", "2026-01-31"]: two dates (YYYY-MM-DD), both inclusive, interpreted in the site timezone as 00:00:00 on the first day to 23:59:59.999 on the last day.["2026-01-01T00:00:00+01:00", "2026-01-02T00:00:00+01:00"]: two date-times with an explicit offset orZ. The interval is half-open: start inclusive, end exclusive. Date-times are converted to instants, then re-expressed in the site timezone for bucketing.- Both elements must use the same form (
400 invalid_queryotherwise). The end must be after the start for date-times and not before the start for dates (422 invalid_date_range). Earliest accepted date:2000-01-01. Maximum span of a custom range: 10 years (3,660 days) (422 invalid_date_range,details.reason = "range_too_long"). A range may extend into the future; it is not clamped inquery.date_rangebut future time buckets are never generated or returned. - Granularity restrictions:
time:minuterequires a custom date-time range of at most 6 hours (no preset is allowed withtime:minute);time:hourrequires a span of at most 366 days; the number of time buckets produced by any single query must not exceed 10,000 (422 too_many_time_buckets).
Time dimension values (labels):
| Dimension | Label format | Notes |
|---|---|---|
time:minute |
ISO 8601 with offset, 2026-03-29T01:05:00+01:00 |
Local minute start |
time:hour |
ISO 8601 with offset, 2026-03-29T03:00:00+02:00 |
Buckets start at the top of each local hour, including in timezones with a 30- or 45-minute offset. The repeated hour on a daylight-saving fall-back produces two buckets with distinct offsets in their labels, and the skipped hour never appears. |
time:day |
2026-03-29 |
Local calendar day; days with 23 or 25 hours bucket correctly |
time:week |
2026-03-23 |
ISO weeks (Monday start). The label is the Monday of the week, except the first bucket whose label is the range start date when the range starts mid-week; likewise the last bucket is limited by the range end. |
time:month |
2026-03-01 |
Label is the first day of the month, except the first bucket which is labeled with the range start date when the range starts mid-month. |
Boundary rule: a pageview, event or session belongs to the bucket and range of its timestamp (events) or its start time (sessions) converted to the site timezone. A session that starts on one day and ends on the next is counted in the start day for visits and session metrics, while its pageviews are counted on the days they occurred.
Resolved range: the response query.date_range always contains the resolved range as two ISO dates (for date-based presets) or two offset date-times (for 24h and date-time forms), and meta.site_timezone echoes the timezone used.
13.4.3 Metrics #
Definitions of the business meaning are canonical in Section 9; this table fixes the engine contract: result type, null behavior and data source. Value types: int, number (JSON number, rounded as stated), money (object, 13.4.8).
| Metric | Type | Rounding / null rule | Source |
|---|---|---|---|
visitors |
int | Count of distinct user_id (exact, 13.4.11) |
events |
visits |
int | Count of sessions | sessions, or events when an event-level dimension is present (13.4.7) |
pageviews |
int | Count of pageview events |
events |
events |
int | Count of all non-engagement events, pageviews included |
events |
views_per_visit |
number | 2 decimals; null when visits = 0 | sessions |
bounce_rate |
number | Percent 0 to 100, 1 decimal; null when visits = 0 | sessions |
visit_duration |
int | Average seconds per visit, rounded to integer; null when visits = 0 | sessions |
scroll_depth |
int | Average of each page view's maximum scroll depth, percent 0 to 100, integer; null when there are no engagement events | events (engagement) |
time_on_page |
int | Average engagement seconds per page view, integer; null when none | events (engagement) |
conversion_rate |
number | Percent 0 to 100, 1 decimal; null when the denominator is 0 | events |
group_conversion_rate |
number | Percent 0 to 100, 1 decimal; null when the denominator is 0 | events |
total_revenue |
money or null | Sum in the goal reporting currency; null when no revenue events or mixed currencies | events |
average_revenue |
money or null | Total divided by revenue-bearing conversions; null as above | events |
percentage |
number | Percent of total visitors, 1 decimal | events |
exit_rate |
number | Percent 0 to 100, 1 decimal; null when the denominator is 0 | events plus sessions |
Count metrics return 0 (never null) for an empty result group; ratio and average metrics return null when undefined. A metric name outside this list returns 400 invalid_query with the allowed values in the message.
Internal-only component metrics (bounces, total_visit_duration, total_pageviews_in_visits, total_time_on_page, total_time_on_page_visits) are used by CSV export (Section 14) and the import merge (13.4.10); total_time_on_page is the summed engagement seconds and total_time_on_page_visits the number of page views that had engagement, the numerator and denominator of time_on_page. They are accepted by the internal executeStatsQuery function with allowInternalMetrics: true and are rejected by every HTTP endpoint with the same 400 as any unknown metric.
13.4.4 Dimensions #
A dimension groups results. Each result row's dimensions array holds one string per requested dimension, in request order.
Event-level dimensions (event:*):
| Dimension | Value | Notes |
|---|---|---|
event:name |
Event name (pageview, custom names; never engagement) |
|
event:page |
Pathname (no query string; hash included only for hash-routing sites per Section 8) | |
event:hostname |
Hostname of the page | |
event:goal |
Display name of a goal of the site (Section 10) | One row per goal the event satisfies; an event matching two goals is counted in both rows. Events matching no goal are excluded. Resolution in 13.5.3. |
event:props:<name> |
Value of the custom property <name> |
<name> is 1 to 300 characters, no control characters. Events without the property group under (none). A name that is not on the site's property allowlist (Section 10.3.3) and is not one of the reserved auto-collected keys (Section 10.3.4) returns 422 property_not_allowed with details.property; removing a property from the allowlist therefore makes it unqueryable while the stored values remain in the event store. |
Visit-level dimensions (visit:*), taken from the session's first event (denormalized per Section 6; the physical column of every dimension is in the mapping of 13.5.3):
| Dimension | Value |
|---|---|
visit:source |
Referrer source name; absent referrer groups as Direct / None |
visit:channel |
One of the channels of Section 8 (Direct, Organic Search, Paid Search, Organic Social, Paid Social, Email, Affiliates, Display, Organic Video, Paid Video, Cross-network, Organic Shopping, Paid Shopping, Audio, SMS, Mobile Push Notifications, Referral, AI Assistants) |
visit:referrer |
Referrer as stored (host plus path, no scheme or query string, Section 6), (none) when absent |
visit:utm_medium, visit:utm_source, visit:utm_campaign, visit:utm_content, visit:utm_term |
UTM values, (none) when absent |
visit:device |
Device class from the user agent: Desktop, Mobile, Tablet (stored lowercase in device, displayed capitalized), (none) when unknown (smart TVs, consoles, wearables and embedded devices are stored as unknown). Classification in Section 8. |
visit:screen_size |
Class derived from the reported screen width: Mobile, Tablet, Laptop, Desktop (stored lowercase in screen_size), (none) when no width was sent. This is the dimension of the dashboard "Devices > Screen size" report. Not available for imported data. |
visit:browser, visit:browser_version |
Browser name; version as major.minor; (none) when unknown |
visit:os, visit:os_version |
Operating system name; version as major.minor; (none) when unknown |
visit:country |
ISO 3166-1 alpha-2 code (column country_code), (none) when unknown |
visit:region |
ISO 3166-2 code such as US-CA (column subdivision1_code), (none) when unknown |
visit:city |
GeoNames ID as a string (column city_geoname_id; 0 is unknown), (none) when unknown |
visit:country_name, visit:region_name, visit:city_name |
English display names resolved after the query from the bundled location dataset (Section 8). Output only: usable as dimensions, rejected in filters (400, message "visit:country_name cannot be used in filters; filter by visit:country instead."). If an ID has no name, the ID is returned. |
visit:entry_page, visit:exit_page |
Pathname of the session's first and last pageview |
visit:entry_page_hostname, visit:exit_page_hostname |
Hostnames of those pages |
Time dimensions: time:minute, time:hour, time:day, time:week, time:month (labels in 13.4.2). Empty buckets are not returned in results; request include.time_labels to get the full label list for zero-filling.
Dimension and goal values are compared exactly as stored; the engine never lower-cases or trims them.
13.4.5 Filters #
A filter is a JSON array tree.
type Filter =
| [Operator, FilterDimension, (string | number)[], FilterModifiers?]
| ["and" | "or", Filter[]]
| ["not", Filter];
type Operator = "is" | "is_not" | "contains" | "contains_not" | "matches" | "matches_not";
type FilterModifiers = { case_sensitive?: boolean }; // default trueFilterDimension is any dimension of 13.4.4 except time:* and the *_name dimensions, plus the pseudo-dimension segment (below). Top-level filters is a list implicitly combined with AND.
| Operator | Meaning for a single clause | Multiple clauses |
|---|---|---|
is |
Value equals the clause | OR across clauses |
is_not |
Value does not equal the clause | AND across clauses (value differs from every clause) |
contains |
Value contains the clause as a substring | OR |
contains_not |
Value does not contain the clause | AND |
matches |
Value matches the clause as a regular expression (RE2 syntax, unanchored) | OR |
matches_not |
Value does not match the clause | AND |
Rules:
- The clause list has 1 to 100 entries. Each clause is converted to a string; strings are 1 to 2,000 characters (the empty string is invalid; use
(none)to match absent values). Numbers are accepted forvisit:cityand converted with their decimal representation. (none)is a reserved clause meaning "absent" for props, UTM fields, referrer, device, browser, OS and location dimensions.["is", "event:props:plan", ["(none)"]]matches events without the property.containsandmatchesoperators treat(none)as a literal.case_sensitivedefaults totruefor every operator. Whenfalse, comparison lower-cases both sides (lower(x)in SQL). The dashboard sendsfalseforcontainsfilters entered in its UI.matchespatterns are at most 250 characters and a query may contain at most 5matches/matches_notclauses in total. A pattern that does not compile returns422 invalid_regex(the engine maps ClickHouse regexp-compilation errors to this code).event:goalfilters take goal display names.iswith several names matches events or visitors satisfying any listed goal. A name that does not exist returns422 unknown_goalwith the name indetails.goal.event:goalaccepts onlyisandis_not.event:pageandvisit:entry_page/visit:exit_pageclause values are pathnames and must start with/forisandis_not(otherwise400 invalid_query).visit:countryclauses must be two uppercase letters;visit:regionclauses match^[A-Z]{2}-[A-Z0-9]{1,3}$;visit:cityclauses are digits only;visit:deviceclauses are one ofDesktop,Mobile,Tablet,(none)andvisit:screen_sizeclauses one ofMobile,Tablet,Laptop,Desktop,(none). These checks apply toisandis_notonly; the enum-backedvisit:deviceandvisit:screen_sizeaccept no other operators (400 invalid_query).segmentaccepts onlyisandis_notwith segment ids (UUIDs) of the site. The engine replaces the node with the segment's stored filter tree (Section 9) before validation, which counts toward the limits below. A segment id that does not belong to the queried site returns404 not_found(details.resource = "segment"). Through the public API only segments visible to every member of the site (site-wide segments) resolve; personal segments also return404 not_found. A segment containing asegmentfilter is rejected when the segment is saved, so expansion is one level deep. Shared-link and embed requests never accept thesegmentpseudo-dimension (13.6.3).- Limits: nesting depth at most 5
and/or/notlevels; at most 50 leaf filters in total after segment expansion;and/orlists have at least 1 and at most 50 children. - Event-level versus visit-level semantics: a filter on an
event:*dimension restricts the events counted. For session-based metrics it restricts to sessions that contain at least one matching event (13.5.3). A filter on avisit:*dimension restricts to sessions (and therefore their events) with that attribute. - A filter that is satisfiable by no row (such as
iswith a value that never occurs) returns empty results, not an error.
Examples:
["is", "visit:country", ["US", "CA"]]
["contains", "event:page", ["/blog/"], { "case_sensitive": false }]
["or", [["is", "visit:channel", ["Organic Search"]], ["is", "visit:source", ["Newsletter"]]]]
["not", ["is", "event:props:plan", ["free"]]]
["matches", "event:page", ["^/docs/(v1|v2)/"]]13.4.6 Ordering, pagination and include options #
order_by:
- Each entry is
[field, direction]wherefieldis a metric or dimension that is present in the request anddirectionisascordesc. A field not in the request returns400 invalid_querywith pathorder_by.<i>.0. A field listed twice returns400 invalid_query. - Default when
order_byis omitted: if atime:*dimension is present, ascending by that dimension; otherwise descending by the first metric. Ties are always broken by the dimensions in request order, ascending, so pagination is deterministic. - Ordering by a dimension sorts strings in byte order, except
time:*which sorts chronologically. Metrics with null values sort last in both directions. - Ordering by
visit:*_namedimensions sorts by the resolved name (applied in the application after name resolution; this forces the engine to fetch all rows before slicing when it is used).
pagination: limit is an integer 1 to 10,000 (default 10,000); offset is an integer 0 to 1,000,000 (default 0). A page beyond the data returns results: [].
include:
| Option | Default | Effect |
|---|---|---|
imports |
false |
Merge imported data where compatible (13.4.10). meta.imports_included reports what happened. |
time_labels |
false |
Adds meta.time_labels with every bucket label from range start to the last bucket not in the future. Requires exactly one time:* dimension, otherwise 422 time_labels_requires_time_dimension. |
total_rows |
false |
Adds meta.total_rows: number of result rows before pagination. Costs one extra query. |
comparisons |
null |
Adds a comparison object to every row (below). |
Comparisons:
previous_period: for day-count presets and custom ranges, the immediately preceding range of the same number of days (or the same length in hours for24hand date-time ranges), ending when the current range begins. For calendar presets (month,6mo,12mo,year) the preceding calendar unit(s): previous month, the previous 6 or 12 months, previous year. Fordaythe previous day.year_over_year: the same dates one year earlier (29 February maps to 28 February). Withmatch_day_of_weektrue (default) the comparison start is moved to the nearest date with the same weekday as the current start, searching up to 3 days either way and preferring the earlier date on a tie; the range length is kept.custom:date_rangeis required, uses the absolute forms of 13.4.2 (presets are rejected with400 invalid_query), and is not required to have the same length.date_range: "all"with any comparison mode returns422 comparison_not_supported.- The comparison range is queried with the same metrics, dimensions and filters. Rows are matched by dimension values; for a
time:*dimension rows are matched by bucket position (the i-th bucket of the current range against the i-th bucket of the comparison range), andcomparison.dimensionscarries the comparison bucket's label. - Only the dimension tuples present in the returned page of current rows are fetched for the comparison (they are passed as an
INlist), so comparisons never exceed the page size. - Each row gets
"comparison": {"dimensions": [...], "metrics": [...], "change": [...]}.metricshas the comparison values (0 for count metrics and null for ratio and revenue metrics when the tuple had no data).changeis per metric:round((current - previous) / previous * 100)as an integer percent;0when both are 0;nullwhen previous is 0 and current is not, when either value is null, or formoneymetrics computed onvalue. meta.comparison_date_rangereturns the resolved comparison range in the same form asquery.date_range.- Imports apply to the comparison query on the same terms as the main query.
13.4.7 Metric and dimension compatibility #
Dimensions fall into families: none, time (time:*), visit (visit:*), page (event:page, event:hostname) and event (event:name, event:goal, event:props:*). The rules below are enforced by packages/query before any SQL runs; a violation returns 422 unsupported_combination with details: {"metric": "...", "dimension": "..."} naming the first offending pair.
| Metric | none / time / visit | page (event:page, event:hostname) |
event (event:name, event:goal, event:props:*) |
|---|---|---|---|
visitors, pageviews, events |
allowed | allowed | allowed |
visits |
allowed (sessions) | allowed (events-based) | allowed (events-based) |
views_per_visit, bounce_rate, visit_duration |
allowed | allowed with entry-page semantics (below) | not allowed |
scroll_depth, time_on_page |
allowed | allowed | not allowed |
conversion_rate, group_conversion_rate |
allowed when a goal is in scope | allowed when a goal is in scope | allowed only with event:goal among the dimensions (and no event:name or event:props:*), or with a goal filter and no event:* dimension besides event:goal; otherwise not allowed |
total_revenue, average_revenue |
allowed | allowed | allowed |
percentage |
allowed with at least one non-time dimension and no time:* dimension |
allowed | allowed |
exit_rate |
not allowed | allowed when the only non-time dimension is event:page |
not allowed |
Further rules:
- A goal is in scope when the query has a positive
event:goalfilter (is, possibly insideand/or) or theevent:goaldimension.conversion_rateorgroup_conversion_ratewithout a goal in scope returns422 goal_required. - Events-based
visits. When anyevent:*dimension is present,visitsis the number of distinct sessions having at least one counted event in the group (so a visit that viewed the page twice counts once). - Entry-page semantics. When
event:page(orevent:hostname) is a dimension,bounce_rate,visit_durationandviews_per_visitdescribe the visits that entered on that page (visit:entry_page/visit:entry_page_hostnamereplace the dimension in the sessions sub-query). This matches the "Pages" report convention of Section 9. - Revenue.
total_revenueandaverage_revenuesum events carrying a revenue amount. When the rows of a group contain more than one reporting currency the value isnulland the responsemeta.warningsincludes{"code": "revenue_mixed_currencies", ...}. Grouping byevent:goalmakes each row single-currency because a goal has one currency (Section 10). visit:entry_page,visit:exit_pageand their hostname forms may be combined only with the metricsvisitors,visits,pageviews,events,views_per_visit,bounce_rate,visit_durationandpercentage; any other metric returns422 unsupported_combination.- At most one
time:*dimension. Atime:*dimension may be combined with any other dimensions. percentageis the group'svisitorsdivided by the totalvisitorsof the whole filtered query scope (all rows, ignoring pagination), times 100, 1 decimal.
13.4.8 Response shape #
{
"results": [
{ "dimensions": ["2026-10-30"], "metrics": [1204, 3380, 41.3] },
{ "dimensions": ["2026-10-31"], "metrics": [987, 2511, 44.0] }
],
"meta": {
"site_timezone": "Europe/Berlin",
"imports_included": false,
"imports_skip_reason": null,
"imports_exist": true,
"time_labels": ["2026-10-30", "2026-10-31", "2026-11-01"],
"total_rows": 3,
"comparison_date_range": null,
"warnings": [],
"query_ms": 38
},
"query": {
"site_id": "example.com",
"metrics": ["visitors", "pageviews", "bounce_rate"],
"date_range": ["2026-10-30", "2026-11-01"],
"dimensions": ["time:day"],
"filters": [],
"order_by": [["time:day", "asc"]],
"include": { "imports": false, "time_labels": true, "total_rows": true, "comparisons": null },
"pagination": { "limit": 10000, "offset": 0 }
}
}| Field | Rules |
|---|---|
results[].dimensions |
Array of strings, one per requested dimension, in request order. Empty array when no dimensions were requested (a single result row is then always returned, even for an empty period). |
results[].metrics |
Array in the order of request.metrics. Counts are integers; ratios are numbers or null; revenue metrics are money objects or null. |
results[].comparison |
Present only with include.comparisons. Shape in 13.4.6. |
meta.site_timezone |
IANA name. |
meta.imports_included |
true only when imported rows were actually merged. |
meta.imports_skip_reason |
null or one of the codes in 13.4.10 when include.imports was true but nothing was merged. Always null when include.imports is false. |
meta.imports_exist |
true when the site has at least one completed import (so a UI can offer a toggle). |
meta.time_labels |
Present only when requested. |
meta.total_rows |
Present only when requested. |
meta.comparison_date_range |
null or the resolved comparison range. |
meta.warnings |
Array of { "code": string, "message": string }. Codes: revenue_mixed_currencies, imports_partial_goals, results_truncated_by_server (never occurs for the 10,000 row limit; reserved for the dashboard batch endpoint when a response would exceed 8 MB, in which case rows are cut and this warning is added). |
meta.query_ms |
Wall-clock milliseconds spent in the engine, including ClickHouse time. |
query |
The normalized request after defaults are applied and presets resolved: date_range is the resolved pair, order_by the effective ordering, include and pagination fully populated. relative_date is omitted. |
Revenue (money) objects:
{ "value": 1234.56, "currency": "USD", "short": "$1.2K", "long": "$1,234.56" }value is a decimal number rounded to the currency's minor unit; short and long are formatted with Intl.NumberFormat using locale en-US (compact notation for short, currency style for long). Section 4 owns the money formatting rules.
13.4.9 Validation errors #
All errors use the envelope of Section 4.4. Schema-level failures are 400 validation_failed with details.fields; unknown metric, dimension, operator and preset values are 400 invalid_query with the same details.fields shape, as in this example:
{
"error": {
"code": "invalid_query",
"message": "The query is invalid.",
"details": {
"fields": [
{ "path": "metrics.1", "message": "Unknown metric \"visitor\". Allowed values: visitors, visits, pageviews, events, views_per_visit, bounce_rate, visit_duration, scroll_depth, time_on_page, conversion_rate, group_conversion_rate, total_revenue, average_revenue, percentage, exit_rate." },
{ "path": "filters.0.0", "message": "Unknown operator \"equals\"." }
]
},
"request_id": "req_01JBX8Z2K4"
}
}Every issue found by schema validation is reported in one response (not just the first), up to 50 issues. Codes specific to this API (invalid_date_range, too_many_time_buckets, time_dimension_too_fine, time_labels_requires_time_dimension, unsupported_combination, goal_required, unknown_goal, invalid_regex, comparison_not_supported, query_too_large) are part of the closed catalogue of Section 4.4.3.
Semantic and runtime errors:
| Status | code |
Trigger | details |
|---|---|---|---|
| 400 | malformed_json |
Body is not parseable JSON | null |
| 400 | validation_failed |
Shape, type, length, count, nesting or unknown-field violation | fields |
| 400 | invalid_query |
Well-formed query that is semantically unusable: unknown metric, dimension, operator or preset value, duplicate metric or dimension, order_by field not in the request, *_name dimension used in a filter, malformed country/region/city/page/device clause, relative_date with an array range or mixed range forms, preset used as a custom comparison range |
fields |
| 401 | invalid_api_key |
13.3.1 | null |
| 403 | api_key_scope_insufficient |
Missing scope | required_scope |
| 404 | not_found |
Site (or segment) missing or not in the key's team; team in its deletion window | resource |
| 413 | payload_too_large |
Body over 256 KB | max_bytes |
| 415 | unsupported_media_type |
Not JSON | null |
| 422 | invalid_date_range |
End before start, range over 10 years, date before 2000-01-01, future relative_date beyond limit |
reason: end_before_start, range_too_long, before_minimum, relative_date_in_future |
| 422 | too_many_time_buckets |
More than 10,000 buckets | buckets, max |
| 422 | time_dimension_too_fine |
time:minute over 6 hours, time:hour over 366 days |
dimension, max_span |
| 422 | time_labels_requires_time_dimension |
include.time_labels without exactly one time dimension |
null |
| 422 | unsupported_combination |
13.4.7 | metric, dimension |
| 422 | property_not_allowed |
event:props:<name> names a property that is not allowlisted for the site |
property |
| 422 | goal_required |
Conversion-rate metric without a goal in scope | metric |
| 422 | unknown_goal |
Goal name not defined on the site | goal |
| 422 | invalid_regex |
matches pattern fails to compile |
pattern |
| 422 | comparison_not_supported |
Comparison with all |
date_range |
| 422 | query_too_large |
ClickHouse memory, group-by, result-size or rows-to-read limit hit (13.4.11) | limit |
| 429 | rate_limited |
13.3.3 | retry_after_seconds, policy |
| 503 | query_timeout |
ClickHouse exceeded the 30-second execution limit | timeout_seconds; message advises a shorter range or fewer dimensions |
| 503 | unavailable |
ClickHouse, Postgres or Valkey unreachable, or the engine queue timed out (13.4.11) | dependency |
13.4.10 Imported data merge #
When include.imports is true, the engine tries to merge rows from the imported_* tables of Section 6 (populated by Section 14) into the native result. Merging is all-or-nothing per query: if any rule below fails, no imported data is used, meta.imports_included is false and meta.imports_skip_reason carries the first failing reason. Results are never partially merged.
Rules:
- Site has imports. At least one import with status
completed(Section 14). Otherwiseimports_skip_reason = "no_imports"(andmeta.imports_existisfalse). - Date overlap. Imported rows are only used for dates strictly earlier than the first native day of the site (the site-timezone date of its earliest event in the
eventstable, cached in Valkey undersite:{site_uuid}:first_native_dayand cleared when site data is purged). Imported rows on or after that day are stored but ignored at query time. If the query range ends before the first imported date or starts after the last usable imported date, nothing is merged (imports_skip_reason = "no_imported_data_in_range"). When the site has no native events at all, every imported date is usable. - Granularity.
time:minuteandtime:hourcannot use imports ("unsupported_granularity"); imported data is daily.24hpresets are treated as hourly. - Dimensions and filters must map to one imported table. The union of the query's non-time dimensions and its filter dimensions must be a subset of exactly one row of this table (
time:day,time:weekandtime:monthare available for every table because each imported table has adatecolumn):
| Imported table | Dimensions it can answer |
|---|---|
imported_visitors |
none (site totals) |
imported_sources |
visit:source, visit:channel, visit:referrer, visit:utm_medium, visit:utm_source, visit:utm_campaign, visit:utm_content, visit:utm_term |
imported_pages |
event:page, event:hostname |
imported_entry_pages |
visit:entry_page |
imported_exit_pages |
visit:exit_page |
imported_locations |
visit:country, visit:region, visit:city, and their _name forms |
imported_devices |
visit:device (visit:screen_size has no imported table, so any query using it skips imports) |
imported_browsers |
visit:browser, visit:browser_version |
imported_operating_systems |
visit:os, visit:os_version |
imported_custom_events |
event:name, event:goal, event:page |
A query with no dimensions and no filters uses imported_visitors. A query with no dimensions but a filter on, say, visit:country uses imported_locations. Otherwise "incompatible_dimensions". Filters that a shared link or embed enforces (lockedFilters, 13.5.1) count as filter dimensions here, so a segment-limited link can never receive unfiltered imported totals.
Column mapping used by the merge (physical imported column to the engine component or dimension, Section 6.5):
| Imported table | Dimension columns | Components |
|---|---|---|
imported_visitors |
none | visitors, pageviews, bounces, visits, visit_duration read as total_visit_duration |
imported_sources |
source as visit:source, referrer, channel, utm_* |
visitors, visits, pageviews, bounces, visit_duration read as total_visit_duration |
imported_pages |
hostname as event:hostname, page as event:page |
visitors, pageviews, total_time_on_page, total_time_on_page_visits (together the numerator and denominator of time_on_page) |
imported_entry_pages |
entry_page |
visitors, entrances read as visits, pageviews, bounces, visit_duration read as total_visit_duration |
imported_exit_pages |
exit_page |
visitors, exits read as visits, pageviews |
imported_locations |
country_code as visit:country, subdivision1_code as visit:region, city_geoname_id as visit:city |
as imported_sources |
imported_devices |
device |
as imported_sources |
imported_browsers |
browser, browser_version |
as imported_sources |
imported_operating_systems |
os, os_version |
as imported_sources |
imported_custom_events |
name as event:name and event:goal, path as event:page |
visitors, events |
| 5. Metrics. Every requested metric must be supported by the chosen table: |
| Imported table | Supported metrics |
|---|---|
imported_visitors, imported_sources, imported_entry_pages, imported_locations, imported_devices, imported_browsers, imported_operating_systems |
visitors, visits, pageviews, bounce_rate, visit_duration, views_per_visit, percentage |
imported_pages |
visitors, pageviews, time_on_page, percentage |
imported_exit_pages |
visitors, visits (exits), pageviews, percentage |
imported_custom_events |
visitors, events, percentage |
Any other metric (scroll_depth, the revenue metrics, conversion_rate, group_conversion_rate, exit_rate) yields "unsupported_metric". include.comparisons is allowed; each of the two queries is evaluated independently under these rules and the comparison range inherits the main query's decision (if the main query skipped imports, so does the comparison).
6. Goals. For event:goal dimension or filters only goals of type event without property narrowing can be matched, by comparing the goal's event name with imported name. Other goals contribute only native data and the response gets the warning imports_partial_goals. If no goal in scope is mergeable, "unsupported_goal".
7. Segment and filter shapes. not, or, contains, matches and negative operators are evaluated against the imported table's columns using the same SQL builder as native data, so they are supported when the dimension maps to the table.
Merge algorithm: the engine runs the native query and the imported query with the same dimensions, range and filters (the imported query additionally restricted to import_id IN {completed_import_ids} and date < first_native_day), both requesting component values (visitors, visits, pageviews, events, bounces, total_visit_duration, total_pageviews_in_visits, total_time_on_page, total_time_on_page_visits; the imported side supplies the subset its table has, per the mapping above). Rows are merged by dimension tuple (for time:*, by bucket label); additive components are summed. Ratio metrics are then derived from the merged components (bounce_rate = bounces / visits * 100, visit_duration = total_visit_duration / visits, views_per_visit = pageviews / visits, time_on_page = total_time_on_page / total_time_on_page_visits) and percentage is recomputed over the merged total. visitors is summed across the two sources and across imported days; imported visitors cannot be de-duplicated against native visitors, a documented limitation also shown in the dashboard help text for the imports toggle (Section 14). Sorting, pagination and total_rows are applied after merging.
meta.imports_skip_reason codes: no_imports, no_imported_data_in_range, unsupported_granularity, incompatible_dimensions, unsupported_metric, unsupported_goal.
13.4.11 Sampling, accuracy, limits and timeouts #
- No sampling. Every query reads all matching rows. There is no sampling parameter, no sampled-data flag, and distinct counts are exact:
uniqExact(user_id)anduniqExact(session_id), neveruniq,uniqCombinedoruniqHLL12. Exactness is affordable because hosted tiers cap volume (Section 15) and self-hosted installations accept the cost. - Read-only database user. Every statement of the engine, for the dashboard, shared links, email reports, exports and the public Stats API alike, runs under the user
CLICKHOUSE_RO_USER(hosted: required; self-hosted: falls back to theCLICKHOUSE_URLuser, Section 16.4) with thereadonly = 2setting applied in both cases, so an injection defect in the query builder cannot alter or drop data. - ClickHouse settings applied to every query from the engine (constants in
packages/query/src/limits.ts, also the settings profile of the read-only user):readonly = 2,max_execution_time = 30(seconds),max_memory_usage = 4000000000(4 GB),max_rows_to_read = 2000000000,max_rows_to_group_by = 5000000withgroup_by_overflow_mode = 'throw',max_result_rows = 1000000,result_overflow_mode = 'throw',max_ast_elements = 100000,max_query_size = 524288. Export queries (Section 14) run withmax_execution_time = 600. - Mapping of ClickHouse failures: timeout (
TIMEOUT_EXCEEDED) becomes503 query_timeout; memory, rows-to-read, group-by or result overflow become422 query_too_large; regexp compilation failure becomes422 invalid_regex; connection failures become503 unavailable. Any other ClickHouse error becomes500 internal_error(the ClickHouse message is logged, never returned). - Concurrency: at most 32 ClickHouse queries in flight per web process, and at most 8 in flight per site across all callers of that process; additional queries wait in a FIFO queue for up to 5 seconds and then fail with
503 unavailable. The public API additionally enforces policystats_concurrency(13.3.3). - Result cache: the engine itself never caches. Dashboard and share results are cached by the server cache of Section 9.23.2 (60 seconds for ranges that include today, 5 seconds for realtime; all other TTLs are defined there), whose key includes the site configuration epoch and the caller scope (share-link id and enforced segment), so a segment-limited viewer can never receive an unrestricted cached result. The realtime queries of Section 9 and the public API are subject to that section's rules (the public API is never cached).
- Query log: each query writes one pino
infoline withrequest_id,site_uuid,metrics,dimensions,filter_count,range_days,imports_included,rows,ms. Filter values and dimension values are never logged.
13.5 Query Engine (packages/query) #
packages/query is a pure TypeScript library with no HTTP code. It turns a validated StatsQuery into one to four parameterized ClickHouse statements, runs them, merges the results and returns the response of 13.4.8. It is the only code allowed to read the events, sessions and imported_* tables for statistics.
13.5.1 Pipeline and public signatures #
// packages/query/src/index.ts
export interface QueryContext {
site: { uuid: string; domain: string; timezone: string; createdAt: Date };
goals: GoalDefinition[]; // all goals of the site (Section 10), loaded by the caller
segments?: (id: string) => Promise<Filter | null>; // resolves segment ids visible to the caller
lockedFilters?: Filter[]; // always ANDed (shared links limited to a segment, Section 12); also treated as filter dimensions by the imports decision (13.4.10)
completedImportIds: string[]; // completed imports of the site (Section 14)
firstNativeDay: string | null; // "YYYY-MM-DD" in site timezone
now: Date;
source: "public_api" | "dashboard" | "share" | "export" | "email_report";
allowInternalMetrics?: boolean;
}
export function parseStatsQuery(input: unknown): StatsQuery; // Zod, throws ValidationError
export function resolveDateRange(q: StatsQuery, tz: string, now: Date): ResolvedRange;
export function buildPlan(q: StatsQuery, ctx: QueryContext): QueryPlan; // semantic checks, 13.4.7
export function compile(plan: QueryPlan): CompiledStatement[]; // { sql, params, role }
export function executeStatsQuery(q: StatsQuery, ctx: QueryContext): Promise<StatsResponse>;
export function executeConsolidatedQuery(q: ConsolidatedQuery, ctx: ConsolidatedContext): Promise<StatsResponse>;executeConsolidatedQuery serves the all-sites overview of Section 9.20 (POST /api/app/v1/consolidated/query) and is the only other entry point to the engine; it follows the same pipeline with these differences:
export interface ConsolidatedContext {
sites: { uuid: string; domain: string; timezone: string; completedImportIds: string[]; firstNativeDay: string | null }[]; // at most 100, already filtered to the sites the caller may view (Section 9.20)
now: Date;
source: "dashboard";
}The consolidated query accepts at most 100 sites, only the metrics visitors, visits, pageviews, views_per_visit, bounce_rate, visit_duration, no filters, and the dimensions none, one of time:day, time:week, time:month (any other interval returns 422 invalid_interval), or group_by_site (rows labelled by domain). Each site's range and buckets are resolved in that site's own timezone: each site is compiled as its own statements using that site's IANA timezone, grouped by site_id; the application sums rows by local calendar date and applies the visit-weighted ratios of Section 9.20.3. Imports are merged per site under the rules of 13.4.10. The response shape is that of 13.4.8.
Stages:
- Parse with
statsQuerySchema. - Resolve the date range with
Intl.DateTimeFormatand the IANA timezone database of the Node runtime (no extra date library). Result:{ fromUtc, toUtc, fromLocalDate, toLocalDate, buckets? }. Local-midnight to UTC conversion handles daylight-saving gaps by moving a nonexistent local midnight forward to the first valid instant. - Plan. Expand segments, apply
lockedFilters, validate semantics (13.4.7, goals, regexes), classify dimensions and filters by family, decide the CTE layout (13.5.3) and the imports decision (13.4.10). - Compile to parameterized SQL. Every user-supplied value, including property names, is bound as a ClickHouse query parameter (
{p3:String}). Column and function names come from a fixed whitelist incolumns.ts; no user string is ever concatenated into SQL text. Regex andLIKEvalues: forcontains, the clause is escaped (\,%,_) and bound as aLIKEpattern%value%withilikefor case-insensitive matching; formatches,match(column, {p:String})(with the(?i)prefix added for case-insensitive matching). - Execute statements concurrently where independent, with the settings in 13.4.11 and a ClickHouse
query_idof{request_id}-{n}for tracing. - Merge and finalize: imports, name resolution for
*_namedimensions,percentage, comparisons, ordering and pagination when they could not be pushed to SQL,time_labels, money formatting.
13.5.2 Logical columns #
The sketches in this section use the physical column names of the Section 6 DDL exactly. The whitelist of columns the compiler may reference lives in packages/query/src/columns.ts.
| Table | Columns used by the engine |
|---|---|
events |
site_id (UUID), timestamp (DateTime64(3) UTC), name, session_id, user_id, hostname, pathname, referrer, referrer_source, channel, utm_medium, utm_source, utm_campaign, utm_content, utm_term, browser, browser_version, os, os_version, device (Enum8 desktop, mobile, tablet, or empty), screen_size (Enum8 mobile, tablet, laptop, desktop, or empty), country_code, subdivision1_code, city_geoname_id (UInt32), meta.key and meta.value (parallel Array(String) columns holding the custom properties), revenue_amount, revenue_currency, revenue_reporting_amount (Nullable Decimal(18,3)), revenue_reporting_currency, scroll_depth (UInt8), engagement_time (UInt32 ms), interactive (Bool) |
sessions |
site_id, session_id, user_id, started_at (DateTime UTC), ended_at, duration (UInt32 seconds), pageviews, events, interactive_events, is_bounce (UInt8), entry_page, entry_hostname, exit_page, exit_hostname, the same visit attributes as events (referrer through city_geoname_id), sign (Int8) and version (UInt32), engine VersionedCollapsingMergeTree |
imported_* |
site_id, import_id, date (Date) and the columns of Section 6.5 (mapping to engine components in 13.4.10) |
Rules for the physical model that the engine relies on: events carry the visit attributes of their session (denormalized at ingestion, Section 8); engagement events share the session's session_id and carry scroll_depth and engagement_time; sessions are collapsed with sign, so every sessions aggregate uses sum(sign * x) and every group is wrapped in HAVING sum(sign) > 0. Custom properties are read with has(meta.key, {p:String}) and meta.value[indexOf(meta.key, {p:String})].
Every statement starts with the same two predicates, written once as {site_id:UUID} and timestamp >= {from:DateTime('UTC')} AND timestamp < {to:DateTime('UTC')} (for sessions started_at replaces timestamp). They are omitted from the sketches below only where noted.
13.5.3 Planning rules #
Base predicates. Events statements for visitors, pageviews, events, visits (events-based), revenue and conversion metrics add name != 'engagement'. Statements for scroll_depth and time_on_page use name = 'engagement' plus the page filter. pageviews is countIf(name = 'pageview').
CTE layout. The planner splits the requested metrics into an events group (events table) and a sessions group (sessions table) per 13.4.3 and 13.4.7. Each group becomes one statement grouped by the requested dimensions; if both groups exist the two statements run in parallel and are joined in the application on the dimension tuple (a full outer join, with absent sides filled as 0 or null). A third statement (totals) computes denominators for percentage, conversion_rate and group_conversion_rate; a fourth computes exit_rate sessions. Joining in the application avoids ClickHouse join memory limits; each statement is bounded by the 10,000-row pagination limit where possible.
Dimension expressions:
| Dimension | Events expression | Sessions expression |
|---|---|---|
time:day |
toDate(timestamp, {tz:String}) |
toDate(started_at, {tz:String}) |
time:week |
toStartOfWeek(timestamp, 1, {tz:String}) clamped to range start in the application |
same with started_at |
time:month |
toStartOfMonth(timestamp, {tz:String}) clamped to range start in the application |
same |
time:hour |
toStartOfHour(timestamp, {tz:String}) (labels formatted with offset in the application) |
same |
time:minute |
toStartOfMinute(timestamp, {tz:String}) |
same |
visit:country, visit:region, visit:city, visit:channel, visit:referrer, visit:utm_*, visit:browser, visit:browser_version, visit:os, visit:os_version |
the column named in the mapping below; an empty string (or 0 for city_geoname_id) is returned as (none) via if(x = '', '(none)', x) |
same column |
visit:device, visit:screen_size |
toString(device) / toString(screen_size), relabelled in the application (desktop to Desktop, and so on; the empty enum value to (none)) |
same column |
visit:source |
if(referrer_source = '', 'Direct / None', referrer_source) |
same |
visit:entry_page, visit:exit_page, visit:entry_page_hostname, visit:exit_page_hostname |
not available (these attributes exist only on sessions): a query that groups by one of them is answered entirely from the sessions table (see below) | entry_page, exit_page, entry_hostname, exit_hostname |
event:page |
pathname |
entry_page (entry semantics) |
event:hostname |
hostname |
entry_hostname |
event:name |
name |
not available |
event:props:<name> |
if(has(meta.key, {pN:String}), meta.value[indexOf(meta.key, {pN:String})], '(none)') |
not available |
event:goal |
arrayJoin(arrayFilter(g -> g != '', [if(<goal 1 predicate>, {g1:String}, ''), if(<goal 2 predicate>, {g2:String}, '')])) |
not available |
When a query groups by visit:entry_page, visit:exit_page or one of their hostname forms, every metric comes from one sessions statement: visits = sum(sign), visitors = uniqExactIf(user_id, sign = 1), pageviews = sum(sign * pageviews), events = sum(sign * events), and the session ratios as usual.
Dimension to physical column mapping (events and sessions use the same names): visit:source to referrer_source; visit:referrer to referrer; visit:channel to channel; visit:utm_medium, visit:utm_source, visit:utm_campaign, visit:utm_content, visit:utm_term to the column of the same name; visit:device to device; visit:screen_size to screen_size; visit:browser, visit:browser_version to browser, browser_version; visit:os, visit:os_version to os, os_version; visit:country to country_code; visit:region to subdivision1_code; visit:city to city_geoname_id; visit:entry_page and visit:exit_page to entry_page and exit_page; visit:entry_page_hostname and visit:exit_page_hostname to entry_hostname and exit_hostname. The visit:*_name dimensions have no column; they are resolved after the query (13.4.4).
Goal predicates (from the goal definitions of Section 10, compiled by compileGoal(goal)):
| Goal type | Predicate on an events row |
|---|---|
event |
name = {event_name} |
pageview |
name = 'pageview' AND match(pathname, {regex}), where regex is the anchored regular expression compiled from the goal's path pattern by compileGoalPattern in @quietstat/shared (pattern syntax in Section 10) |
scroll |
name = 'engagement' AND scroll_depth >= {threshold} AND match(pathname, {regex}) |
event or pageview, with property conditions |
the above AND has(meta.key, {k1:String}) AND meta.value[indexOf(meta.key, {k1:String})] = {v1:String} AND ... (up to 3 pairs; conditions are not allowed on scroll goals) |
For the event:goal dimension the array contains every site goal, or only the goals named by a positive event:goal filter when one exists, to keep the expression small. When the active goal set contains a scroll goal the base predicate name != 'engagement' is not applied, because the goal predicates already fix the event name. For scroll goals the converting unit is a visitor (visitors counts distinct visitors who reached the threshold) and events counts engagement events at or above the threshold.
Filter compilation. A filter tree compiles to one boolean expression per table:
- Event-level leaf in the events statement: column predicate. In the sessions statement:
session_id IN (SELECT session_id FROM events WHERE <site and range predicates> AND name != 'engagement' AND <event-level subtree>). When anor/and/notsubtree mixes event-level and visit-level leaves, the whole subtree is evaluated on events and wrapped in the samesession_id IN (...)sub-select for the sessions statement. - Visit-level leaf in the events statement: column predicate on the denormalized column, except
visit:entry_page,visit:exit_pageand their hostname forms, which compile tosession_id IN (SELECT session_id FROM sessions WHERE <site and range predicates> AND <leaf> GROUP BY session_id HAVING sum(sign) > 0). In the sessions statement: column predicate. event:goal is [...]in the events statement:(<goal 1 predicate> OR <goal 2 predicate>). In the sessions statement: thesession_id IN (...)form above.containsandison(none)compile tocol = ''(city_geoname_id = 0, andNOT has(meta.key, {p:String})for properties) accordingly;is_not (none)to the negation.visit:deviceandvisit:screen_sizeclauses are lower-cased and compared with the enum column (device = 'mobile').- The
case_sensitive: falsemodifier wraps both sides inlower(...). - Locked filters (
ctx.lockedFilters) are ANDed at the root after segment expansion.
Session sub-select cost guard. When a session_id IN (...) sub-select would be used and the range is longer than 366 days, the planner adds the sub-select's range predicate widened by 1 day on both sides to bound the scan while still catching sessions that straddle the boundary.
13.5.4 Worked examples #
Example 1: daily timeseries with a filter. Visitors, pageviews and bounce rate per day for the last 7 days, US visitors only (site timezone Europe/Berlin, "now" is 2026-11-03 14:20 local).
{
"site_id": "example.com",
"metrics": ["visitors", "pageviews", "bounce_rate"],
"date_range": "7d",
"dimensions": ["time:day"],
"filters": [["is", "visit:country", ["US"]]],
"include": { "time_labels": true }
}Resolved range: 2026-10-28 to 2026-11-03 local, i.e. from = 2026-10-27 23:00:00 UTC, to = 2026-11-03 23:00:00 UTC (exclusive). Two statements:
-- role: events
SELECT
toDate(timestamp, {tz:String}) AS d0,
uniqExact(user_id) AS visitors,
countIf(name = 'pageview') AS pageviews
FROM events
WHERE site_id = {site_id:UUID}
AND timestamp >= {from:DateTime('UTC')} AND timestamp < {to:DateTime('UTC')}
AND name != 'engagement'
AND country_code = {p0:String}
GROUP BY d0
ORDER BY d0 ASC
LIMIT {limit:UInt32} OFFSET {offset:UInt32};
-- role: sessions
SELECT
toDate(started_at, {tz:String}) AS d0,
if(sum(sign) = 0, NULL, round(sum(sign * is_bounce) / sum(sign) * 100, 1)) AS bounce_rate
FROM sessions
WHERE site_id = {site_id:UUID}
AND started_at >= {from:DateTime('UTC')} AND started_at < {to:DateTime('UTC')}
AND country_code = {p0:String}
GROUP BY d0
HAVING sum(sign) > 0
ORDER BY d0 ASC;The application joins the two result sets on d0, formats labels (2026-10-28, ...), and meta.time_labels lists all seven days. Days with no traffic are absent from results.
Example 2: conversion by source (goal filter, group conversion rate). Top sources among visitors who completed the goal "Signup", with the conversion rate inside each source.
{
"site_id": "example.com",
"metrics": ["visitors", "group_conversion_rate"],
"date_range": "30d",
"dimensions": ["visit:source"],
"filters": [["is", "event:goal", ["Signup"]]],
"order_by": [["visitors", "desc"]],
"pagination": { "limit": 10 }
}Signup is an event goal with event_name = 'Signup'. Statements:
-- role: events (converters)
SELECT
if(referrer_source = '', 'Direct / None', referrer_source) AS d0,
uniqExact(user_id) AS visitors
FROM events
WHERE site_id = {site_id:UUID}
AND timestamp >= {from:DateTime('UTC')} AND timestamp < {to:DateTime('UTC')}
AND name != 'engagement'
AND (name = {g0_event:String}) -- goal predicate
GROUP BY d0
ORDER BY visitors DESC, d0 ASC
LIMIT 10 OFFSET 0;
-- role: totals (denominator per group, goal filter removed)
SELECT
if(referrer_source = '', 'Direct / None', referrer_source) AS d0,
uniqExact(user_id) AS visitors_all
FROM events
WHERE site_id = {site_id:UUID}
AND timestamp >= {from:DateTime('UTC')} AND timestamp < {to:DateTime('UTC')}
AND name != 'engagement'
AND if(referrer_source = '', 'Direct / None', referrer_source) IN {tuples:Array(String)} -- only the 10 returned groups
GROUP BY d0;group_conversion_rate = round(visitors / visitors_all * 100, 1) per row, computed in the application. Plain conversion_rate in the same query would divide by uniqExact(user_id) over the whole scope (a third scalar query without a GROUP BY), because it has no non-time dimensions in its denominator.
Example 3: revenue by source. Total and average revenue for the goal "Purchase" by source, with comparison to the previous period.
{
"site_id": "shop.example",
"metrics": ["visitors", "total_revenue", "average_revenue"],
"date_range": "month",
"dimensions": ["visit:source"],
"filters": [["is", "event:goal", ["Purchase"]]],
"order_by": [["total_revenue", "desc"]],
"include": { "comparisons": { "mode": "previous_period" } }
}-- role: events (current period; the comparison runs the same statement with the previous range
-- and an added predicate: AND if(referrer_source = '', 'Direct / None', referrer_source) IN {tuples:Array(String)})
SELECT
if(referrer_source = '', 'Direct / None', referrer_source) AS d0,
uniqExact(user_id) AS visitors,
if(uniqExactIf(revenue_reporting_currency, revenue_reporting_amount IS NOT NULL) > 1, NULL,
sumIf(revenue_reporting_amount, revenue_reporting_amount IS NOT NULL)) AS total_revenue_raw,
if(uniqExactIf(revenue_reporting_currency, revenue_reporting_amount IS NOT NULL) > 1, NULL,
avgIf(revenue_reporting_amount, revenue_reporting_amount IS NOT NULL)) AS average_revenue_raw,
anyIf(revenue_reporting_currency, revenue_reporting_amount IS NOT NULL) AS currency
FROM events
WHERE site_id = {site_id:UUID}
AND timestamp >= {from:DateTime('UTC')} AND timestamp < {to:DateTime('UTC')}
AND name != 'engagement'
AND (name = {g0_event:String})
GROUP BY d0
ORDER BY total_revenue_raw DESC NULLS LAST, d0 ASC
LIMIT 10000 OFFSET 0;The application converts total_revenue_raw and currency into the money object (13.4.8) and computes change (13.4.6). A row mixing currencies yields null and the warning revenue_mixed_currencies.
Example 4: property breakdown with entry-page session metrics. Pages report: visitors, pageviews, bounce rate and time on page for the top pages, ordered by pageviews, including imported data.
{
"site_id": "example.com",
"metrics": ["visitors", "pageviews", "bounce_rate", "time_on_page"],
"date_range": ["2025-01-01", "2025-12-31"],
"dimensions": ["event:page"],
"order_by": [["pageviews", "desc"]],
"include": { "imports": true, "total_rows": true },
"pagination": { "limit": 5 }
}bounce_rate is a sessions metric under entry-page semantics and time_on_page is supported by imported_pages but bounce_rate is not (13.4.10, rule 5), so imports are skipped: meta.imports_included = false, imports_skip_reason = "unsupported_metric". The native statements:
-- role: events
SELECT pathname AS d0, uniqExact(user_id) AS visitors, countIf(name = 'pageview') AS pageviews
FROM events
WHERE site_id = {site_id:UUID} AND timestamp >= {from:DateTime('UTC')} AND timestamp < {to:DateTime('UTC')}
AND name != 'engagement'
GROUP BY d0
ORDER BY pageviews DESC, d0 ASC
LIMIT 5 OFFSET 0;
-- role: engagement (time_on_page for the same 5 pages, filtered by the page tuple list)
SELECT pathname AS d0,
if(count() = 0, NULL, round(sum(page_time) / count() / 1000)) AS time_on_page
FROM (
SELECT pathname, session_id, sum(engagement_time) AS page_time
FROM events
WHERE site_id = {site_id:UUID} AND timestamp >= {from:DateTime('UTC')} AND timestamp < {to:DateTime('UTC')}
AND name = 'engagement' AND pathname IN {tuples:Array(String)}
GROUP BY pathname, session_id
)
GROUP BY d0;
-- role: sessions (entry semantics)
SELECT entry_page AS d0,
if(sum(sign) = 0, NULL, round(sum(sign * is_bounce) / sum(sign) * 100, 1)) AS bounce_rate
FROM sessions
WHERE site_id = {site_id:UUID} AND started_at >= {from:DateTime('UTC')} AND started_at < {to:DateTime('UTC')}
AND entry_page IN {tuples:Array(String)}
GROUP BY d0
HAVING sum(sign) > 0;
-- role: total_rows
SELECT count() FROM (
SELECT pathname FROM events
WHERE site_id = {site_id:UUID} AND timestamp >= {from:DateTime('UTC')} AND timestamp < {to:DateTime('UTC')}
AND name != 'engagement'
GROUP BY pathname
);The IN {tuples} lists are the page values returned by the primary events statement, which is why the secondary statements stay cheap. The primary statement is whichever statement owns the order_by metric (here pageviews, events); when ordering by a sessions metric the sessions statement is primary and the events statement receives the tuple list.
Example 5: imported data merge (site totals over a long range). Visitors and bounce rate per month for 2019 to 2026, where the site has a completed import covering 2019-01-01 to 2024-06-30 and its first native day is 2024-07-01.
{
"site_id": "example.com",
"metrics": ["visitors", "visits", "bounce_rate"],
"date_range": ["2019-01-01", "2026-11-03"],
"dimensions": ["time:month"],
"include": { "imports": true, "time_labels": true }
}Imports are compatible (no dimension besides time, metrics supported by imported_visitors). The engine runs the native statements for the full range and the imported statement below, then merges by month label; the first native month is 2024-07-01, the imported statement is bounded by date < '2024-07-01':
-- role: imported_visitors (components)
SELECT toStartOfMonth(date) AS d0,
sum(visitors) AS visitors, sum(visits) AS visits, sum(bounces) AS bounces
FROM imported_visitors
WHERE site_id = {site_id:UUID}
AND import_id IN {import_ids:Array(UUID)}
AND date >= {from_date:Date} AND date <= {to_date:Date}
AND date < {first_native_day:Date}
GROUP BY d0
ORDER BY d0;The native statements also return bounces (sum(sign * is_bounce)) as an internal component. For each month label the application adds components and derives bounce_rate = bounces / visits * 100. The first label is 2019-01-01 (range start); months entirely before the first native day come only from the import; there is no month with both sources because the native cut-off is a day boundary, and a cut-off inside a month (here the cut-off is on the 1st) would sum the imported part and the native part of that month.
13.6 Dashboard Internal Query Endpoints #
The SPA never talks to /api/v2/query (no key is available in the browser). It uses two cookie-authenticated endpoints that call the same executeStatsQuery function with source: "dashboard".
13.6.1 Single query #
POST /api/app/v1/sites/{domain}/query
{domain}is the site domain, URL-encoded (Section 4). The site is resolved from the domain together with the caller's access, never from the body.- Auth: session cookie plus
X-CSRF-Token(Section 4). Permission: actionsite.view(Section 11): any member of the site's team (owner, admin, editor, viewer) or a site guest (guest_viewer,guest_editor). No API key is involved and the public rate-limit policies are not touched. The route is flagged read-only in the route registry, so it stays available to a read-only impersonation session of an operator (Section 15). - Body: exactly the
StatsQueryof 13.4.1 withoutsite_id(asite_idfield returns400 invalid_query). Additional behavior versus the public endpoint:segmentfilters may reference the caller's personal segments as well as site-wide segments; a referenced segment must belong to this site, otherwise404 not_found.- The server result cache of Section 9.23.2 applies (13.4.11).
- Realtime widgets (Section 9) call the engine with a
date_rangeof two date-times covering the last 30 minutes andtime:minute; they passinclude.imports = false.
- Response: identical to 13.4.8, wrapped in nothing (the same
results/meta/querydocument; the success envelope exception of Section 4.4 applies). - Rate limit: policy
dashboard_queryof Section 17.6 (per user),429 rate_limitedwithRetry-Afterwhen exceeded. - Errors: the same codes as 13.4.9 (with
404 not_foundfor a site the user cannot access, so existence is not revealed).
13.6.2 Batch #
POST /api/app/v1/sites/{domain}/query/batch
{ "queries": [ { "metrics": ["visitors"], "date_range": "30d" }, { "metrics": ["visitors"], "date_range": "30d", "dimensions": ["visit:source"] } ] }- 1 to 20 queries. Executed with a concurrency of 6. Order is preserved. Each query in a batch counts as one request toward policy
dashboard_query(orshare_viewfor the shared-link mirror, 13.6.3), so a batch cannot be used to multiply query cost beyond the limit. - Response
200:{ "data": [ { "status": 200, "body": { "results": [...], "meta": {...}, "query": {...} } }, { "status": 422, "body": { "error": {...} } } ] }. The overall status is200as long as the batch itself is valid; each item carries its own status and body (a body is either a 13.4.8 document or an error envelope). An invalid batch (not 1 to 20 queries) returns400 validation_failed. - Total response size is capped at 8 MB; when exceeded, the largest
resultsarrays are truncated from the end andmeta.warningsreceivesresults_truncated_by_server. - Permissions, read-only flag and cache as in 13.6.1.
13.6.3 Shared links and embeds #
Shared-link and embed requests (Section 12) call executeStatsQuery with source: "share" and lockedFilters set to the link's fixed segment. Their routes under /api/share/v1/... are defined in Section 12; they accept the same body as 13.6.1 except that a request-supplied segment filter is always rejected with 403 forbidden (so a viewer cannot expand a guessed segment id; the link's own fixed segment, if any, is applied by the server as a locked filter) and the server forces include.imports = false for every request made through a link whose include_imports flag is false (Section 12.2.1, Section 14.7). A request filter is always ANDed with the locked filters and can never override or remove them, and the locked filters also count as filter dimensions in the imports decision (13.4.10).
13.6.4 API playground #
Settings, API keys page, tab Playground: a JSON editor (monospace, validation on keystroke using statsQuerySchema) prefilled with { "metrics": ["visitors", "pageviews"], "date_range": "7d", "dimensions": ["time:day"] }, a site selector, a Run button that posts to the single-query endpoint of 13.6.1, a formatted response viewer showing results as a table, and a "Copy as curl" button that renders the equivalent public request with the placeholder $QUIETSTAT_API_KEY and the site's domain as site_id. The playground never displays or asks for a real key.
13.7 Events API #
POST /api/event is the same endpoint the tracking script uses (Section 7); this subsection documents it for servers, mobile apps and other non-browser callers. Processing (validation order, enrichment, bot filter, caps) is owned by Section 8.
13.7.1 Request #
| Header | Required | Rule |
|---|---|---|
Content-Type |
yes | application/json or text/plain (the body is JSON in both cases) |
User-Agent |
yes for counted events | The end user's user agent string. Requests without a user agent, or with a bot user agent, are accepted and silently dropped (202); headless-client markers (such as the default agents of HTTP libraries) also cause a silent drop, except for events whose u uses the app: scheme (mobile and desktop apps), which skip only the headless-marker check (Section 8.8). Always send a real device or browser user agent, never your HTTP library's default. |
X-Quietstat-Client-IP and X-Quietstat-Proxy-Key |
yes for correct visitor counting | The end user's IP address and the site's proxy key (shown in the site's tracking settings, Section 7.15.4). The instance trusts X-Quietstat-Client-IP only for a site whose proxy key matches under a constant-time comparison (Section 8.7.2); an invalid key means the header is ignored and the request is rate-limited as a normal client. Without both headers the address of the calling server is used, which collapses all of that server's visitors into one visitor. The proxy key is a secret: keep it on your server and never ship it to browsers or mobile apps. IPs are used to compute the daily hash and are never stored. |
Body fields:
| Field | Type | Required | Rule |
|---|---|---|---|
n |
string | yes | Event name: pageview or a custom name, 1 to 120 characters. engagement is reserved for the tracking script; a server may send it only together with sd or e. |
u |
string | yes | Absolute URL of the page or screen, at most 2,000 characters, scheme http, https or app. Include UTM parameters in the query string to attribute campaigns. For mobile apps use a stable fake URL such as app://localhost/screens/home. The URL host must be allowed for the site: by default the site domain and its subdomains, otherwise the site's allowed_hostnames list (13.8.1); events with any other host are dropped with 202 (Section 8.6). Add your app's host (conventionally localhost) to allowed_hostnames when sending app: URLs. |
d |
string | yes | Site domain as registered (13.8): lowercase, without scheme, port or a leading www. (a leading www. and any port are stripped before lookup, Section 8.6). A comma-separated list of up to 10 domains records the event on each (multi-site rollup). Unknown domains are dropped silently. |
r |
string or null | no | Referrer URL; an invalid value is ignored |
w |
integer or null | no | Screen width in CSS pixels (1 to 100,000), used for the screen-size class |
h |
integer | no | 1 if the URL fragment is a route (hash routing); otherwise the fragment is dropped |
p |
object | no | Custom properties: at most 30 keys, key at most 300 characters, value a string of at most 2,000 characters. Numbers and booleans are converted to strings; null and empty-string values drop the key; keys starting with qs_, quietstat_ or $ are dropped. Objects and arrays are rejected. |
$ |
object | no | Revenue: { "amount": 12.5, "currency": "EUR" } (the amount may also be a decimal string such as "12.50", at most 3 decimals); currency is a 3-letter ISO 4217 code. Ignored for pageview and engagement. |
i |
boolean | no | Interactive, default true; set false for events that must not affect bounce rate |
sd |
integer 0 to 100 | no | Scroll depth (engagement events only) |
e |
integer | no | Engagement time in milliseconds, 0 to 86,400,000 (values above 3,600,000 are clamped); engagement events only |
The authoritative validation rules and failure reason texts are in Section 8.4.
Fixed behavior for server-side callers:
- There is no timestamp field: events are timestamped when received. Historical data is loaded with CSV import (Section 14), never through this endpoint.
- Sessions form automatically: events with the same site and the same hash of IP and user agent less than 30 minutes apart share a session (Section 8). A server sending a different IP/UA pair per event creates a visitor per pair.
- Maximum body size 32 KB (
413). The rate limit is policyevent_ingestof Section 17.6, counted per client address (429plain text; a request with a valid proxy key is counted under the proxy tier,event_ingest_proxy); batch calls on the server side are not supported, one request carries one event. - Credentials are not sent. Never place an API key on this endpoint.
Responses: 202 with an empty body for every well-formed request including silently dropped ones (bot, exclusion, unknown domain, URL host not allowed for the site, usage cap, localhost, unlicensed team); 400 with a short plain-text reason for malformed JSON or a violated limit above (reason texts defined in Section 8); 413; 429. CORS: preflight OPTIONS returns 204 with Access-Control-Allow-Origin: *, Access-Control-Allow-Methods: POST, OPTIONS, Access-Control-Allow-Headers: Content-Type, Access-Control-Max-Age: 86400.
13.7.2 Examples #
curl: pageview with a campaign and a custom event with revenue.
curl -i https://analytics.example.com/api/event \
-H 'Content-Type: application/json' \
-H 'User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 18_1 like Mac OS X) AppleWebKit/605.1.15' \
-H 'X-Quietstat-Client-IP: 203.0.113.42' \
-H "X-Quietstat-Proxy-Key: $QUIETSTAT_PROXY_KEY" \
-d '{"n":"pageview","u":"https://example.com/pricing?utm_source=newsletter&utm_medium=email","d":"example.com","r":"https://example.com/"}'
curl -i https://analytics.example.com/api/event \
-H 'Content-Type: application/json' \
-H 'User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 18_1 like Mac OS X) AppleWebKit/605.1.15' \
-H 'X-Quietstat-Client-IP: 203.0.113.42' \
-H "X-Quietstat-Proxy-Key: $QUIETSTAT_PROXY_KEY" \
-d '{"n":"Purchase","u":"https://example.com/checkout/done","d":"example.com","p":{"plan":"pro","coupon":"WELCOME"},"$":{"amount":"49.00","currency":"EUR"}}'Node (server-side, forwarding the visitor's identity from an incoming request; Node.js global fetch):
export async function track(
req: { headers: Record<string, string | undefined>; socket: { remoteAddress?: string } },
name: string,
url: string,
extra: Record<string, unknown> = {},
): Promise<void> {
const ip = req.headers["x-forwarded-for"]?.split(",")[0].trim() ?? req.socket.remoteAddress ?? "";
const res = await fetch("https://analytics.example.com/api/event", {
method: "POST",
headers: {
"Content-Type": "application/json",
"User-Agent": req.headers["user-agent"] ?? "",
"X-Quietstat-Client-IP": ip,
"X-Quietstat-Proxy-Key": process.env.QUIETSTAT_PROXY_KEY ?? "", // secret: server-side only
},
body: JSON.stringify({ n: name, u: url, d: "example.com", ...extra }),
signal: AbortSignal.timeout(3000),
});
if (res.status !== 202) console.warn("analytics event rejected", res.status, await res.text());
}Python:
import os
import requests
def track(user_agent: str, ip: str, name: str, url: str, **extra) -> None:
resp = requests.post(
"https://analytics.example.com/api/event",
json={"n": name, "u": url, "d": "example.com", **extra},
headers={
"User-Agent": user_agent,
"X-Quietstat-Client-IP": ip,
"X-Quietstat-Proxy-Key": os.environ["QUIETSTAT_PROXY_KEY"], # secret: server-side only
},
timeout=3,
)
if resp.status_code != 202:
print("analytics event rejected", resp.status_code, resp.text)
track("Mozilla/5.0 (Windows NT 10.0; Win64; x64)", "198.51.100.7", "Signup",
"https://example.com/welcome", p={"source": "api"})Operational guidance for callers: never block a user request on the analytics call (use a short timeout and ignore failures); do not retry on 202; retry once on network errors only.
13.8 Sites API #
Provisioning API for agencies and platforms. Base path /api/v1. Auth: Bearer key with sites:provision (read-only routes also accept stats:read, 13.3.2). Rate-limit policy sites_api_key (Section 17.6). All routes are scoped to the key's team. Mutations write an audit_log row (catalogue in Section 11.20.2) with actor_type = api_key and actor_api_key_id set to the key's id (the key's display prefix is recorded in the row metadata).
Routes:
| Method and path | Purpose | Success |
|---|---|---|
GET /api/v1/teams |
The team of the key | 200 list (exactly one item) |
GET /api/v1/sites |
List sites, cursor-paginated, ordered by domain ascending; optional ?domain_contains= (1 to 100 characters) |
200 list |
POST /api/v1/sites |
Create a site | 201 |
GET /api/v1/sites/{domain} |
Get a site | 200 |
PATCH /api/v1/sites/{domain} |
Update a site (partial) | 200 |
DELETE /api/v1/sites/{domain} |
Delete a site and its data | 202 |
GET /api/v1/sites/{domain}/goals |
List goals | 200 list |
POST /api/v1/sites/{domain}/goals |
Create a goal | 201 |
GET /api/v1/sites/{domain}/goals/{goal_id} |
Get a goal | 200 |
PATCH /api/v1/sites/{domain}/goals/{goal_id} |
Update a goal | 200 |
DELETE /api/v1/sites/{domain}/goals/{goal_id} |
Delete a goal | 204 |
GET /api/v1/sites/{domain}/shared-links |
List shared links | 200 list |
POST /api/v1/sites/{domain}/shared-links |
Create a shared link | 201 |
DELETE /api/v1/sites/{domain}/shared-links/{link_id} |
Delete a shared link | 204 |
GET /api/v1/sites/{domain}/custom-props |
List allowed custom properties | 200 list |
POST /api/v1/sites/{domain}/custom-props |
Allow a custom property (idempotent) | 201 new, 200 when already allowed |
DELETE /api/v1/sites/{domain}/custom-props/{key} |
Remove an allowed property (URL-encoded name) | 204 |
{domain} is the site's domain, URL-encoded. List responses use {"data": [...], "meta": {"next_cursor": "...|null", "limit": 50}} with ?cursor=&limit= (default 50, max 200). Single resources use {"data": {...}}.
13.8.1 Objects #
Team:
{ "data": [ { "id": "0192f7b0-0000-7000-8000-000000000010", "name": "Ada's team", "site_count": 7, "site_limit": 25, "created_at": "2026-01-12T10:00:00Z" } ], "meta": { "next_cursor": null, "limit": 50 } }site_limit is the sites cap of the active license tier on the hosted edition (Section 15) and null on self-hosted.
Site:
{
"domain": "example.com",
"timezone": "Europe/Berlin",
"team_id": "0192f7b0-0000-7000-8000-000000000010",
"script_id": "7k3m9q2x5d8a",
"tracker_config": {
"track_outbound_links": true,
"track_file_downloads": true,
"file_download_extensions": ["pdf", "xlsx", "docx", "csv", "zip"],
"track_form_submissions": false,
"track_404_pages": false,
"track_hash_routing": false
},
"datacenter_filter_enabled": true,
"allowed_hostnames": [],
"created_at": "2026-11-03T09:12:44Z",
"updated_at": "2026-11-03T09:12:44Z"
}Create/update rules:
| Field | Create | Update | Rule |
|---|---|---|---|
domain |
required | optional | Hostname per RFC 1123, stored lowercase without scheme, port, path or wildcard and without a leading www. (a submitted www.example.com is stored as example.com); 1 to 253 characters; internationalized names are converted to punycode; unique across the whole installation on the normalized value. Changing the domain takes effect immediately: events sent with the old domain are dropped, so tracking snippets that name the domain must be updated first (the per-site script URL, which uses script_id, keeps working). |
timezone |
optional, default UTC |
optional | A valid IANA name from the runtime's list |
tracker_config |
optional | optional (partial merge) | The keys are exactly the tracking columns of the site settings (Section 5): booleans track_outbound_links, track_file_downloads, track_form_submissions, track_404_pages, track_hash_routing, and file_download_extensions (1 to 50 entries, each 1 to 10 lower-case alphanumerics; null resets to the default list of Section 7.5.6). Reads return the effective list. Defaults on create: track_outbound_links true, track_file_downloads true, the others false. Scroll depth and engagement are always collected by the tracker and have no setting. |
datacenter_filter_enabled |
optional, default true |
optional | Per-site toggle of Section 8; the user-agent bot filter is always on and not configurable |
allowed_hostnames |
optional, default [] |
optional (replaces the list) | Hostname allowlist of Section 8.6: at most 50 entries, each a lowercase hostname without port, scheme or www.. An empty list means the site domain and all its subdomains; events whose URL host is not allowed are dropped. |
script_id |
read-only | read-only | The 12-character identifier used by the per-site tracker script URL (Section 7.2.1); generated at creation and immutable |
Errors: 400 validation_failed; 409 domain_taken (message "This domain is already registered."); 409 site_limit_reached (hosted only, including an unlicensed team that already has its one site, Section 15; details: {"limit": 5, "current": 5}); 404 not_found for an unknown site. DELETE removes the Postgres rows immediately and enqueues the purge of ClickHouse data; the response is 202 with {"data": {"domain": "example.com", "deleted": true, "purge": "queued"}}. The previous domain is reserved for the same team for 30 days (Section 5.5.5): that team may add it again, while other teams receive 409 domain_taken until the reservation expires. Old data cannot reappear because it is keyed by the deleted site's UUID.
Goal:
{
"id": "0192f7c2-8d0b-7a11-b2a4-5d9c3f7a1e22",
"type": "event",
"display_name": "Signup",
"event_name": "Signup",
"page_path": null,
"scroll_threshold": null,
"property_conditions": [{ "key": "plan", "value": "pro" }],
"revenue_currency": "USD",
"created_at": "2026-11-03T09:20:00Z"
}| Field | Rule |
|---|---|
type |
event, pageview or scroll; fixed after creation |
display_name |
1 to 120 characters, unique per site case-insensitively; defaults per Section 10.2.2 when omitted |
event_name |
Required for event; 1 to 120 characters, not pageview or engagement; null otherwise |
page_path |
Required for pageview and scroll; starts with / or **; at most 2,000 characters; wildcard syntax per Section 10.2.3 |
scroll_threshold |
Required for scroll; integer 1 to 100 |
property_conditions |
Optional array of at most 3 {key, value} pairs with distinct keys; keys must be allowed properties (custom-props below) or reserved auto-collected keys, else 422 property_not_allowed; not allowed on scroll goals (422 conditions_not_supported) |
revenue_currency |
Optional ISO 4217 code, only for event goals; null disables revenue. Immutable after creation. |
Errors: 409 goal_exists when another goal has the same definition; 409 goal_name_taken for a duplicate display name; 409 limit_reached with details.limit_name = "goals" when the per-site goal limit of Section 5.16 is reached; PATCH may change display_name, event_name, page_path, scroll_threshold and property_conditions but not type or revenue_currency (422 immutable_field), exactly as in Section 10.2.6; deleting a goal used by a funnel returns 409 goal_in_use.
Shared link:
{ "id": "0192f7c3-0000-7000-8000-00000000aa01", "name": "Client report", "url": "https://analytics.example.com/share/Qm3xT9vLw2KpZr8YcB5nHd1AfJe7GsUo", "password_protected": true, "segment_id": null, "embed_mode": "disabled", "embed_origins": [], "include_imports": true, "created_at": "2026-11-03T09:25:00Z" }Create body: name (1 to 60 characters, default "Shared link"), password (optional, 8 to 128 characters, stored as a hash and never returned), segment_id (optional UUID of a site-wide segment; when set the link only exposes that segment, Section 12.2.5), and optionally embed_mode, embed_origins and include_imports with the meaning and defaults of Section 12.2.1. The url format and token (32 characters) are defined in Section 12.2.2. A site may have at most the number of shared links in Section 5.16 (409 limit_reached with details.limit_name = "shared_links").
Custom property: POST body {"key": "plan"}; response {"data": {"key": "plan", "created_at": "...", "auto_collected": false}}. Behavior, key rules and errors are those of Section 10.3.3: names are 1 to 300 characters without control characters and must not start with qs_, quietstat_ or $ (422 reserved_property_name); adding an existing key is idempotent (200); at most the registered-property limit of Section 5.16 per site (409 limit_reached with details.limit_name = "custom_properties"). Deleting a property is blocked while a goal condition, funnel step condition or segment uses it (409 property_in_use); otherwise it makes the property unqueryable and the stored values remain in the event store.
Example: provision a site and a goal.
curl -s https://analytics.example.com/api/v1/sites \
-H "Authorization: Bearer $QUIETSTAT_API_KEY" -H 'Content-Type: application/json' \
-d '{"domain":"client.example","timezone":"America/New_York"}'
curl -s https://analytics.example.com/api/v1/sites/client.example/goals \
-H "Authorization: Bearer $QUIETSTAT_API_KEY" -H 'Content-Type: application/json' \
-d '{"type":"event","event_name":"Signup","revenue_currency":"USD"}'13.9 OpenAPI 3.1 Generation #
- The Stats API, Sites API and Events API are described by one OpenAPI 3.1 document (
GET /api/openapi.jsonitself is also listed), generated from the Zod schemas in@quietstat/sharedwith Zod's JSON Schema converter (target JSON Schema 2020-12, which OpenAPI 3.1 adopts). Route definitions carryoperationId,summary,tags, request and response schemas and error responses from one registry (apps/server/src/openapi/registry.ts) so a route cannot exist without documentation. - Generation command:
pnpm openapi:buildwritesapps/server/openapi/openapi.json(committed).pnpm openapi:checkregenerates in memory and fails if the file differs; it runs in CI (Section 18). - Required content:
info.title= product name fromPRODUCT_NAME,info.version= the API document revision asYYYY.MM.DDof the last change,servers= the instance's own base URL,components.securitySchemes.bearerAuth(http,bearer), globalsecurityfor/api/v2/queryand/api/v1/*, empty security for/api/eventand/api/openapi.json. Every operation lists400,401,403,404,422,429and503responses where applicable using shared components for the error envelope (Error, with the closed code list of Section 4.4.3 as an enum) and rate-limit headers (X-RateLimit-*,Retry-After). - Operation ids:
statsQuery,eventsCreate,teamsList,sitesList,sitesCreate,sitesGet,sitesUpdate,sitesDelete,goalsList,goalsCreate,goalsGet,goalsUpdate,goalsDelete,sharedLinksList,sharedLinksCreate,sharedLinksDelete,customPropsList,customPropsCreate,customPropsDelete. GET /api/openapi.jsonserves the committed document withCache-Control: public, max-age=300,Content-Type: application/jsonandAccess-Control-Allow-Origin: *(13.1); it is subject to policyopenapiof Section 17.6. Dashboard-internal routes (/api/app/v1/*,/api/share/v1/*) are not part of the public document.- Contract tests (Section 18): every example request and response in this section is replayed against a test instance and validated against the document; a response that does not validate fails the build.
- The filter grammar (recursive tuple types) is expressed in OpenAPI with
$refrecursion andprefixItems;date_rangeis aoneOfof a preset enum and a two-item array.
13.10 Versioning and Deprecation Policy #
- Major versions in the path. The Stats API is
v2(/api/v2/query); the Sites API isv1(/api/v1/...); the Events API is unversioned and stable. A breaking change creates a new path version for that API only. - Non-breaking (no version bump): adding optional request fields, new metrics, dimensions, operators or presets, new response fields, new enum values in responses, new error codes, new endpoints, tightening nothing that was previously accepted. Clients must ignore unknown response fields and unknown
meta.warningscodes. - Breaking: removing or renaming a field, metric or dimension; changing a type, meaning or default; adding a required request field; tightening a limit below the documented value; changing the ordering or rounding rules.
- Support window. After a new major version is generally available the previous major keeps working for at least 12 months. Announcement happens in the release notes of the version that introduces the successor.
- Signals while deprecated. Every response of a deprecated route carries
Deprecation: @<unix timestamp of deprecation>(RFC 9745 form),Sunset: <HTTP date>(RFC 8594) andLink: <successor documentation URL>; rel="successor-version". The API keys settings page shows a banner when any key of the team called a deprecated route in the last 30 days. - After sunset. The route returns
404 not_found(the closed error catalogue of Section 4.4.3 has no 410) with aLink: <successor documentation URL>; rel="successor-version"header; the Postgres and ClickHouse data model is not affected. - Experimental routes (none at release) would live under
/api/experimental/with no stability promise. - Metric and dimension semantics in this section are part of the contract: changes to how a metric is computed follow the breaking-change rules even if the name stays.
14. Data Import & Export #
Three features move aggregate statistics in and out of a site: CSV export (14.3), CSV import in the format of the previous generation of privacy-first analytics tools (14.4) and Google Analytics 4 import (14.5). Imported data is stored in the imported_* ClickHouse tables of Section 6 and shown in the dashboard through the merge rules of 13.4.10 (14.7). Raw per-event export and raw per-event import are not offered; the Events API (13.7) covers live ingestion and self-hosters can query ClickHouse directly. Google Search Console, Looker Studio and other connectors are out of scope.
14.1 Permissions, Jobs and State #
| Action (action ID, Section 11) | owner | admin | editor | viewer | guest_editor | guest_viewer |
|---|---|---|---|---|---|---|
Request a CSV export, download own exports (site.export_csv) |
yes | yes | yes | yes | yes | yes |
Download exports requested by others (site.export.download_any) |
yes | yes | no | no | no | no |
See the imports list; start, cancel, retry or delete imports, CSV and GA4 (site.import.manage) |
yes | yes | no | no | no | no |
Use the "Imported data" toggle in the dashboard (site.view) |
yes | yes | yes | yes | yes | yes |
The authoritative matrix is in Section 11; the action IDs above are the ones this section uses. Unauthorized calls return 403 forbidden. Imports are owner and admin only: editors, viewers and guests never see the imports list or its controls. A shared-link viewer (Section 12) can neither export nor see imports; whether imported data appears for a shared link follows the link's include_imports flag (14.7).
Background jobs (pg-boss queues of apps/worker; names, schedules, retries and timeouts are owned by the jobs table of Section 16.6): export-run, import-run, ga4-import-run (with continuations), import-purge, export-expire (hourly; marks exports past expires_at expired, deletes their files and hard-deletes expired rows 30 days later) and import-sweep (hourly; removes stale configuring imports, revokes and deletes GA4 credentials 24 hours after a retryable failure, fails imports stalled for over 6 hours, and removes orphan upload files). Worker concurrency: 2 jobs per queue per worker process.
Files (uploads in progress, generated export archives) live in the instance file store: a local directory under the data directory (DATA_DIR, Section 16.2.2), accessed through a FileStore interface in apps/server (put, get, delete, list) with one implementation, local disk. Multi-instance hosted topologies must give web and worker the same mounted volume (Section 16). Directory layout: exports/{site_uuid}/{export_id}.zip and imports/{import_id}/{file_id}.csv, where {file_id} is a server-generated random identifier (UUID) assigned per uploaded file; the table each file belongs to is taken from the validated file-name pattern of 14.4.1 and held in the import's job payload, never in the path. Client-supplied file names are never used in a path.
New libraries needed beyond the base list of Section 3: a streaming ZIP reader/writer, a streaming RFC 4180 CSV parser and the Fastify multipart plugin. Google APIs are called with plain fetch (no Google SDK).
Audit log entries (catalogue in Section 11.20.2): export.requested, export.deleted, import.created, import.completed, import.failed, import.cancelled, import.deleted, import.ga4_authorized.
Emails (template inventory in Section 12; sent to the requesting user): export_ready, export_failed, import_completed, import_failed. Each contains the site domain, a one-line result and an authenticated link to the relevant settings page; none contains data.
14.2 Shared File Format (export = import) #
Export writes and import reads the same ten files. File names follow imported_<table>_<YYYYMMDD>_<YYYYMMDD>.csv where the two dates are the first and last day covered (inclusive, site timezone for exports).
CSV dialect: UTF-8, optional BOM accepted on import and never written on export; field separator ,; row terminator \n on export and either \n or \r\n accepted on import; the first row is a header with lower-case column names; fields containing ,, ", CR or LF are quoted with " and embedded quotes doubled; empty field = empty string (string columns) or 0 (numeric columns that are optional); no trailing spreadsheet formulas (formula-injection rule below).
File (<table>) |
Columns in order | Required on import |
|---|---|---|
visitors |
date, visitors, pageviews, bounces, visits, visit_duration |
all |
sources |
date, source, referrer, utm_source, utm_medium, utm_campaign, utm_content, utm_term, channel, pageviews, visitors, visits, visit_duration, bounces |
date, source, visitors, visits, pageviews |
pages |
date, hostname, page, visitors, pageviews, exits, time_on_page |
date, page, visitors, pageviews |
entry_pages |
date, entry_page, visitors, entrances, visit_duration, bounces, pageviews |
date, entry_page, visitors, entrances |
exit_pages |
date, exit_page, visitors, exits, pageviews |
date, exit_page, visitors, exits |
locations |
date, country, region, city, visitors, visits, pageviews, bounces, visit_duration |
date, country, visitors, visits |
devices |
date, device, visitors, visits, pageviews, bounces, visit_duration |
date, device, visitors, visits |
browsers |
date, browser, browser_version, visitors, visits, pageviews, bounces, visit_duration |
date, browser, visitors, visits |
operating_systems |
date, operating_system, operating_system_version, visitors, visits, pageviews, bounces, visit_duration |
date, operating_system, visitors, visits |
custom_events |
date, name, link_url, path, visitors, events |
date, name, visitors, events |
Column semantics (identical for export and import):
| Column | Type and rules |
|---|---|
date |
YYYY-MM-DD, a calendar day in the site timezone |
visitors, visits, pageviews, bounces, events, entrances, exits |
Integers 0 to 4,294,967,295 per row. visitors is the unique visitors of that day for that dimension value; visits the sessions that started that day; entrances equals visits for entry pages and exits equals visits for exit pages. In the pages file exits is the number of visits that ended on that page that day. |
visit_duration |
Integer: total seconds of all visits in the row (not an average), 0 to 4,294,967,295 |
time_on_page |
Integer: total engagement seconds on the page that day. Stored as total_time_on_page; the matching denominator total_time_on_page_visits is set to the row's pageviews (the engine derives the average as total divided by that denominator, 13.4.10). |
source, referrer, utm_* |
Strings at most 255 characters (referrer 2,000). source empty means direct. |
channel |
One of the channel names of Section 8 or empty. On import, empty is replaced by classification of source and utm_medium with the Section 8 classifier. |
hostname |
At most 253 characters |
page, entry_page, exit_page, path, link_url |
Pathname starting with / (at most 2,000 characters); link_url may be an absolute URL or empty |
country |
Two uppercase letters or empty |
region |
ISO 3166-2 code matching ^[A-Z]{2}-[A-Z0-9]{1,3}$ or empty |
city |
GeoNames ID (integer) or empty (0 also means unknown) |
device |
desktop, mobile, tablet (lowercase, the device classes of Section 6) or empty. Smart TVs, consoles, wearables and embedded devices are written as empty. |
browser, browser_version, operating_system, operating_system_version, name |
Strings at most 255 characters; versions major.minor at most 32 characters; name is the event name, at most 120 characters, and must not be pageview or engagement |
Export-specific: files are sorted by date ascending, then visitors descending, then dimension columns ascending. Formula-injection guard: a text cell whose first character is =, +, -, @, tab or CR is written with a leading apostrophe; import removes one leading apostrophe when the next character is one of those. Revenue, scroll depth, funnels and goal configuration are not part of the format.
Stored column names. The CSV format keeps the column names of the previous tool generation; the ClickHouse tables of Section 6.5 use these names: country is stored as country_code, region as subdivision1_code, city as city_geoname_id, operating_system and operating_system_version as os and os_version; every other column keeps its name. The time_on_page rule above applies to pages; the scroll-depth columns of imported_pages are written as 0.
Import-specific leniency (reads exports of the previous tool generation): unknown columns are ignored (a visits column in a pages file is ignored and not stored); missing optional columns are treated as empty or 0; columns may appear in any order; file name matching is case-insensitive; device is matched case-insensitively and the value Laptop is read as desktop.
14.3 CSV Export #
14.3.1 Flow #
- Request. From the dashboard top bar menu "Export as CSV…" (modal prefilled with the current date range and filters) or Site settings, Data tab, Export card (date range picker with default "All time", no filters).
POST /api/app/v1/sites/{domain}/exports(actionsite.export_csv;{domain}is the site domain, URL-encoded):
{
"date_range": ["2026-01-01", "2026-10-31"],
"filters": [["is", "visit:country", ["DE"]]],
"include_imports": true
} date_range uses the absolute date form of 13.4.2 (both dates inclusive; "all" is also accepted and resolved to the site's first day of data through today); filters follow 13.4.5 (default empty); include_imports defaults to true. Validation: invalid range 422 invalid_date_range; filters beyond the limits of 13.4.5 400 validation_failed. Response 202: {"data": <export object>} with status: "queued".
2. Concurrency rules. One active export (queued or running) per site (409 export_in_progress); the request rate is limited by policy export_start of Section 17.6 (429 rate_limited); at most 5 completed exports retained per site, the oldest being deleted when a sixth completes.
3. Generation (export-run job). Streams ClickHouse results for each file into a ZIP written to the file store. Each file is produced by the query engine of 13.5 with source: "export" and the internal component metrics (bounces, total_visit_duration), grouped by time:day plus the file's dimensions, with max_execution_time = 600 and no pagination (cursor-style streaming). Date bucketing uses the site timezone. Time limit for the whole job: 30 minutes. Maximum archive size: 2 GB uncompressed (the job fails with error_code = "export_too_large").
4. Imports in exports. When include_imports is true and no filters are present, imported rows for dates before the first native day (13.4.10, rule 2) are merged into the matching files; otherwise native data only. The manifest records which happened.
5. Completion. Status becomes completed, expires_at = completed_at + 7 days, the requester gets the export_ready email with a link to the Data settings page of the site (login required; the link carries no secret). Failure: status failed, error_code and a user-readable error_message, the export_failed email, partial file removed.
6. Download. GET /api/app/v1/sites/{domain}/exports/{export_id}/download streams the ZIP with Content-Type: application/zip and Content-Disposition: attachment; filename="<domain> export <from> to <to>.zip". The permission is re-checked at download time: the caller must currently hold site.export_csv on the site and be either the requester or hold site.export.download_any (owners and admins). A requester who has since been removed from the team or demoted to a role without export access therefore gets 404 not_found, as do expired or deleted exports. Downloads are logged only as a counter, not per request.
7. Expiry and deletion. DELETE .../exports/{export_id} (requester, or a holder of site.export.download_any) deletes the row and file immediately (204). The hourly export-expire job deletes files and sets status = expired for exports past expires_at; expired rows are hard-deleted 30 days later. Exports requested by a user in a team are also deleted when that user is removed from the team (Section 11.16.5).
Status polling: GET .../exports/{export_id} and GET .../exports (list, newest first, cursor pagination; a caller sees their own exports, and holders of site.export.download_any see all). The UI polls every 2 seconds for 60 seconds, then shows "This is taking a while. We will email you at when it is ready."
14.3.2 Export object and columns required from the exports table #
{
"id": "0192f7d0-1111-7000-8000-00000000e001",
"status": "completed",
"date_from": "2026-01-01",
"date_to": "2026-10-31",
"filters": [["is", "visit:country", ["DE"]]],
"include_imports": true,
"imports_included": false,
"file_size_bytes": 482113,
"row_counts": { "visitors": 304, "sources": 5120, "pages": 20311 },
"error_code": null,
"error_message": null,
"requested_by": { "id": "0192f7be-1111-7000-8000-000000000001", "name": "Ada Lovelace" },
"created_at": "2026-11-03T09:00:00Z",
"completed_at": "2026-11-03T09:00:12Z",
"expires_at": "2026-11-10T09:00:12Z"
}Statuses: queued, running, completed, failed, expired. Values of error_code (a job result shown in the UI, not an HTTP error code): export_too_large, export_timeout, internal_error. The exports table of Section 5 needs these columns: id, site_id, requested_by_user_id, status, date_from, date_to, filters (jsonb), include_imports, imports_included, file_path, file_size_bytes, row_counts (jsonb), error_code, error_message, created_at, started_at, completed_at, expires_at.
14.3.3 Archive contents #
Archive name <domain> export <date_from> to <date_to>.zip, containing the ten files of 14.2 named imported_<table>_<YYYYMMDD>_<YYYYMMDD>.csv and manifest.json:
{ "format": "csv-v1", "site": "example.com", "timezone": "Europe/Berlin", "date_from": "2026-01-01", "date_to": "2026-10-31", "filters": [], "imports_included": true, "generated_at": "2026-11-03T09:00:12Z", "files": { "visitors": 304, "sources": 5120 } }Every file is always present, header-only when it has no rows. Derivation from native data (per day and dimension value):
| File | Source query (13.5) |
|---|---|
visitors |
metrics visitors, pageviews, bounces, visits, total_visit_duration; dimension time:day |
sources |
the same metrics; dimensions time:day, visit:source, visit:referrer, visit:utm_source, visit:utm_medium, visit:utm_campaign, visit:utm_content, visit:utm_term, visit:channel |
pages |
events statement: visitors, pageviews, total_time_on_page; dimensions time:day, event:hostname, event:page. exits comes from a second sessions statement (visits per time:day, visit:exit_page_hostname, visit:exit_page) merged in the application on the (day, hostname, page) tuple |
entry_pages |
visitors, visits (entrances), total_visit_duration, bounces, total_pageviews_in_visits (written as pageviews); dimensions time:day, visit:entry_page |
exit_pages |
visitors, visits (exits), pageviews; dimensions time:day, visit:exit_page |
locations |
session metrics; dimensions time:day, visit:country, visit:region, visit:city (written as country, region, city) |
devices, browsers, operating_systems |
session metrics; dimensions time:day plus visit:device (written as the lowercase stored value), or visit:browser and visit:browser_version, or visit:os and visit:os_version (written as operating_system and operating_system_version) |
custom_events |
visitors, events; dimensions time:day, event:name, event:page, plus link_url read from the event property url (outbound link and file download events carry it, Section 7); pageview and engagement excluded |
Rows where visits and visitors are both 0 are omitted. Rows with an empty dimension value are written with an empty string (never (none)).
14.3.4 Export UI #
Site settings, Data tab, Export card: date range picker, "Include imported data" checkbox (shown only when the site has completed imports), a "Request export" button (disabled while an export is active), and a table of exports: Requested (date), Range, Requested by, Status chip (Queued, Running with spinner, Ready, Failed, Expired), Size, Expires, actions Download and Delete. Empty state: "No exports yet. Export your aggregated statistics as CSV files you can open in a spreadsheet or import into another site." Failed rows show the error_message. Texts are in the English UI catalogue of Section 4.
14.4 CSV Import #
14.4.1 Accepted input #
Either one or more .csv files named as in 14.2, or a single .zip containing them (an archive produced by 14.3 is accepted unchanged; manifest.json and unknown entries are ignored with a warning). Rules enforced before the job starts (synchronously, in the upload request):
| Rule | Limit / behavior | Error |
|---|---|---|
| File name | Must match ^imported_(visitors|sources|pages|entry_pages|exit_pages|locations|devices|browsers|operating_systems|custom_events)_(\d{8})_(\d{8})\.csv$, case-insensitive |
422 import_invalid_file_name with the file in details.file |
| Same range | Every file shares identical date range in its name | 422 import_files_range_mismatch |
| Required file | imported_visitors_...csv is required |
422 import_visitors_file_missing |
| One file per table | A table appears once | 422 import_duplicate_table |
| Header | Required columns of 14.2 present; at least one data row | 422 import_invalid_header with missing columns |
| Size | 200 MB per file, 500 MB total upload, 5,000,000 data rows per file | 413 payload_too_large or 422 import_too_many_rows (row limit is detected by the worker) |
| ZIP safety | At most 50 entries, no nested archives, no path separators in entry names, no absolute names, .. segments, symbolic links, duplicate names or encrypted entries. Limits are enforced on the bytes actually inflated while streaming: the upload is aborted at 200 MB for any entry, at 2 GB in total, and when inflated bytes exceed 100 times the compressed bytes of any entry. Sizes declared in ZIP headers are ignored. |
422 import_archive_rejected |
| Date range | date_from <= date_to, date_from >= 2010-01-01, span at most 10 years |
422 invalid_date_range |
| Overlap | The range must not intersect the range of any existing import of the site whose status is queued, validating, importing, waiting_quota, completed (any kind) |
409 import_overlap with the conflicting import id |
| Count | At most 10 non-deleted imports per site | 409 limit_reached with details.limit_name = "imports" |
| Concurrency | One import (any kind) in an active status per site | 409 import_in_progress |
| License (hosted) | The team must hold an active license (Section 15) | 403 license_required |
| Rate | Policy import_start of Section 17.6 |
429 rate_limited |
| Free space | The upload is refused when free space under DATA_DIR would fall below 5 GB |
507 insufficient_storage |
Upload: POST /api/app/v1/sites/{domain}/imports/csv (action site.import.manage; {domain} is the site domain, URL-encoded) as multipart/form-data with one part named files per file (streamed to the file store, never buffered in memory). Response 202 with the import object (14.6). The route's body limit is 500 MB (524,288,000 bytes) and all other routes keep their usual limit (the body-limit table is in Section 17.4).
Ordering of checks, so that a rejected upload costs nothing: permission, license, import_in_progress, the import count, the import_start rate limit and the free-space check run before the request body is read; the file-name pattern, the shared range, the date-range rules and the overlap rule run as soon as each part header arrives, before any byte of that part is written to disk. A part whose name fails the pattern is rejected without being stored, and every accepted file is stored under a server-generated random name (imports/{import_id}/{file_id}.csv, 14.1), never under the client's file name.
14.4.2 Processing (import-run job) #
Two passes, all-or-nothing:
- Validate (status
validating; progress 0 to 50 %, by bytes read). Streams every file. Row rules:dateis a valid calendar date inside the import range, not later than yesterday in the site timezone, not earlier than 2010-01-01.- Numeric columns parse as non-negative integers within range; strings obey the length rules;
country,region,city,device,channel,pageforms obey 14.2 (adevicethat is notdesktop,mobile,tablet,Laptopor empty is an error); no control characters (code points below 32 except tab) in strings. - Rows on or after the first native day (13.4.10, rule 2) are not errors: they are counted in
warnings(rows_ignored_native_overlap) and skipped in pass 2. - Duplicate keys (same
dateand same dimension columns) within a file are merged by summing the numeric columns and counted inwarnings(duplicate_rows_merged). - Soft consistency warnings, never errors:
bounces > visitsandvisitors > visits(withvisitsdefined) are clamped tovisitsand counted (values_clamped). - Up to 100 errors are collected, each
{file, line, column, message}; validation stops after the 100th (therows_with_errorscount reports how many were seen at that point). If any error exists the import fails witherror_code = "import_validation_failed", nothing is written to ClickHouse and the errors are stored inimport.error_details.
- Insert (status
importing; progress 50 to 100 %). Before inserting, deletes any rows already tagged with thisimport_id(makes retries idempotent). Inserts batches of 50,000 rows per table withimport_id,site_idand normalized values written to the stored column names of 14.2 (emptysourcestays empty, versions reduced tomajor.minor,pages.time_on_pagestored astotal_time_on_pagewithtotal_time_on_page_visitsset topageviews). After each batch the worker checks for cancellation and updates progress (at most every 2 seconds). When all tables are done the import becomescompletedand the first-native-day cache is refreshed; the uploaded files are deleted.
Job settings (the import-run row of Section 16.6): retry limit 2 with 60-second back-off for infrastructure errors (never for validation failures), job timeout 4 hours. Imported rows are invisible to queries until completed because the engine filters on completed_import_ids (13.5.1). Imported data is never billable and does not count towards usage caps (Section 15).
14.5 Google Analytics 4 Import #
Imports aggregate history from a GA4 property through the Google Analytics Data API and Admin API. Universal Analytics (the previous generation) is not importable because Google no longer serves its data.
14.5.1 Operator configuration #
The feature requires an OAuth client of type "Web application" created by the instance operator in their own Google Cloud project, with the Analytics Data API and Analytics Admin API enabled and the redirect URI {BASE_URL}/api/app/v1/integrations/google/callback. Credentials are supplied by the environment variables GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET (listed in Section 16.4). When either is missing the "Google Analytics 4" option in the import modal is disabled with the text "Google import is not configured on this instance. The instance operator must add Google OAuth credentials." For the hosted edition the operator's OAuth consent screen must be verified by Google for the scope below before launch (item of the launch checklist, Section 20.10.1). The requested scope is only https://www.googleapis.com/auth/analytics.readonly.
14.5.2 User flow and endpoints #
All routes are dashboard-API routes under /api/app/v1/, action site.import.manage (owner and admin only, 14.1); {domain} in the paths below is the site domain, URL-encoded.
| Step | Route | Behavior |
|---|---|---|
| 1. Authorize | POST sites/{domain}/imports/ga4/authorize |
Checks permissions, the license (hosted), the import_in_progress and limit_reached (imports) rules of 14.4.1 and the import_start rate limit, then returns {"data": {"authorization_url": "..."}}. The URL targets Google's OAuth endpoint with response_type=code, access_type=offline, prompt=consent, include_granted_scopes=false, the scope above, PKCE (code_challenge_method=S256) and a state token (32 random bytes, stored in Valkey for 10 minutes bound to user id, site id and the PKCE verifier). The SPA navigates the browser to it. |
| 2. Callback | GET integrations/google/callback?code=&state= |
This GET changes state and is therefore protected by the single-use state token instead of a CSRF header (exception listed in Section 11.7). It validates state (single use), that the state record's user id equals the session user and that the user still holds site.import.manage on the recorded site (otherwise 302 to the Data settings page with ?import_error=state_mismatch, and the Google code is discarded); exchanges the code (with the PKCE verifier) for tokens; requires a refresh token; creates an imports row with kind = "ga4", status = "configuring" and encrypted credentials (14.5.3); responds 302 to the site's Data settings page with ?import={id} so the SPA opens the configuration step. User denial or an error from Google redirects to the same page with ?import_error=access_denied (or google_error) and a toast; no row is created. |
| 3. List properties | GET sites/{domain}/imports/{import_id}/ga4/properties |
Calls the Admin API account-summaries listing with a fresh access token, pages through all results (at most 1,000 properties) and returns {"data": [{"property_id": "properties/123456789", "display_name": "Marketing site", "account_name": "Acme Inc"}]} sorted by account then property name. Empty list: the UI shows "No Google Analytics 4 properties found for this Google account." with a "Use another account" link that restarts step 1. |
| 4. Start | POST sites/{domain}/imports/{import_id}/start |
Body {"property_id": "properties/123456789", "date_from": "2020-01-01", "date_to": "2026-10-31"}. Validates the property is in the listed set and that a minimal runReport against it succeeds (422 unprocessable with details.reason = "ga4_permission_denied" if not); validates the date range (date_from >= 2020-01-01, date_from <= date_to, span at most 10 years, date_to clamped to the latest importable day, 422 invalid_date_range if date_from is after it; overlap rule of 14.4.1). Stores the property's reporting timezone and currency read from the Admin API, sets status = "queued" and enqueues the ga4-import-run job (Section 16.6). Response 202 with the import object. |
| Abandonment | import-sweep |
A configuring import older than 1 hour is deleted and its tokens revoked at Google. A new authorize for the same site replaces any existing configuring import of the same user. |
The latest importable day of a site is the day before its first native day (13.4.10, rule 2) or, when it has no native data, yesterday in the site timezone. The configuration step shows it and pre-fills date_to with it. Dates are taken from GA4 as-is in the property's reporting timezone with no conversion; the configuration step displays the property timezone and a note when it differs from the site timezone ("Day boundaries may differ slightly").
14.5.3 Token storage and deletion #
- Refresh and access tokens are encrypted per Section 17.7.2 (AES-256-GCM, key purpose
oauth-token-encryption, additional authenticated dataimports.oauth_credentials:<import id>) and stored as text in the versioned format of that section inimports.oauth_credentials. Plain tokens exist only in worker or request memory, are never logged (pino redaction, Section 17.8) and are never returned by any endpoint. - Access tokens are refreshed on demand (they live about one hour); the refreshed access token is held in memory only.
- When an import reaches
completed,cancelledor a non-retryablefailed, the worker first calls Google's token revocation endpoint (failure is ignored and logged without the token), then setsoauth_credentials = NULL. Retryable failures (14.5.6) keep the credentials for at most 24 hours, after whichimport-sweeprevokes and deletes them. Deleting an import also deletes its credentials.
14.5.4 Reports fetched and mapping #
The worker runs one report per target table per date window. A window is a block of days processed as a unit (14.5.5). Every request sets returnPropertyQuota: true, the window as its date range, the page size to 100,000 rows and an offset for pagination. GA4 metrics are used as follows: visitors = activeUsers; visits = sessions; pageviews = screenPageViews; bounces = sessions - engagedSessions; visit_duration = userEngagementDuration (seconds, rounded, total). The "five core metrics" are activeUsers, sessions, screenPageViews, engagedSessions and userEngagementDuration.
| Report | GA4 dimensions | GA4 metrics | Target table and mapping |
|---|---|---|---|
| Site totals | date |
activeUsers, sessions, screenPageViews, engagedSessions, userEngagementDuration |
imported_visitors row per day |
| Sources | date, sessionSource, sessionMedium, sessionCampaignName, sessionManualAdContent, sessionManualTerm |
same five | imported_sources: source normalized with the Section 8 source-name normalizer ((direct), (not set) and (none) become empty); utm_medium kept only when it is not organic, referral, (none) or (not set); utm_campaign, utm_content, utm_term set to empty when (not set), (direct), (organic), (referral); utm_source equals source when utm_medium is kept, else empty; referrer empty (GA4 exposes no session-level referrer URL); channel classified by Section 8 from source and medium, not taken from Google's channel groups |
| Pages | date, hostName, pagePath |
screenPageViews, activeUsers, userEngagementDuration |
imported_pages: page = pagePath (no query string), pageviews = screenPageViews, total_time_on_page = userEngagementDuration, total_time_on_page_visits = screenPageViews; exits and the scroll-depth columns are 0 |
| Entry pages | date, landingPage |
five core metrics | imported_entry_pages: entry_page = landingPage stripped of any query string; entrances = sessions, pageviews, bounces and visit_duration as in the core mapping |
| Locations | date, countryId, regionId, city |
five core metrics | imported_locations: country_code = countryId; subdivision1_code = regionId only if it matches the ISO 3166-2 form, else empty; city_geoname_id resolved by matching (country, region, normalized name) against the bundled city dataset of Section 8.10.3 and set to 0 when not exactly one match |
| Devices | date, deviceCategory |
five core metrics | imported_devices: desktop, mobile and tablet kept as the lowercase stored values, anything else (smart TV and other categories) empty |
| Browsers | date, browser, browserVersion |
five core metrics | imported_browsers: versions reduced to major.minor. If Google rejects browserVersion for the property (INVALID_ARGUMENT) the report is re-run with browser only and version empty |
| Operating systems | date, operatingSystem, operatingSystemVersion |
five core metrics | imported_operating_systems: os through the Section 8 OS-name normalizer, os_version as major.minor |
| Custom events | date, eventName, pagePath, linkUrl |
activeUsers, eventCount |
imported_custom_events: visitors = activeUsers, events = eventCount, path = pagePath, link_url = linkUrl. Event name mapping: click to Outbound Link: Click, file_download to File Download, form_submit to Form: Submission; every other name is kept. Excluded names: page_view, session_start, first_visit, user_engagement, scroll, form_start. |
Exit pages are not imported: GA4 provides no exit-page dimension, so imported_exit_pages stays empty for GA4 imports and the Exit pages report skips imports (the engine treats an imported table that has no rows for the completed imports in the range as unavailable and reports no_imported_data_in_range). Revenue, scroll depth and goals are likewise not imported.
Rows that map to the same key after normalization (same date and dimensions) are summed before insertion. A row whose date is on or after the first native day is dropped at insertion time.
14.5.5 Windows, pagination and resumability #
- Order of work: for each report in the order of the table above, for each window from
date_fromtodate_to. Window size starts at 31 days. If a window'srowCount(from the first response) exceeds 2,000,000, the window is halved (down to 1 day) before any data is fetched; a single day above 2,000,000 rows fails the import withga4_window_too_large. - Within a window the worker fetches all pages sequentially (offset 0, 100,000, ...), maps and aggregates in memory, then inserts the window's rows in batches of 50,000 with the import's
import_id. - Checkpoint (
imports.checkpoint, jsonb, updated after each window):{ "report": "sources", "window_from": "2024-01-01", "window_to": "2024-01-31", "window_state": "in_flight" | "done", "completed": { "visitors": "2026-10-31", "sources": "2024-01-31" } }. A window is markedin_flightbefore its first insert anddoneafter its last. On resume the worker deletes any rows with thisimport_idin that table for the in-flight window's dates (DELETE FROMlightweight delete), then restarts that window. Windows alreadydoneare never re-fetched. - Progress:
floor(100 * done_windows / total_windows), wheretotal_windowsis computed up front per report from the window plan and recalculated after any halving. Updated at most every 2 seconds. - A job attempt runs for at most 4 hours; on timeout or after entering a quota wait the worker saves the checkpoint and enqueues a continuation job.
14.5.6 Quotas, errors and failure handling #
All calls are sequential (concurrency 1 per property). After every response the worker reads propertyQuota and acts:
| Condition | Action |
|---|---|
Hourly tokens remaining below 2 times the cost of the last request, or HTTP 429 RESOURCE_EXHAUSTED mentioning hourly or concurrent quota |
Status waiting_quota, resume_at = now + 15 minutes; the job saves its checkpoint and re-enqueues itself with startAfter. Up to 48 consecutive hourly waits (12 hours); then fail ga4_quota_exhausted (retryable) |
Daily tokens remaining below 2 times last cost, or RESOURCE_EXHAUSTED mentioning daily quota |
Status waiting_quota, resume_at = next midnight in America/Los_Angeles plus 5 minutes (Google's quota reset); up to 2 daily waits, then fail ga4_quota_exhausted (retryable) |
| HTTP 5xx or network error or timeout (120 seconds per request) | Retry with exponential back-off 2, 4, 8, 16, 32, 60 seconds (6 attempts), then fail ga4_transient_failure (retryable) |
| HTTP 401 | Refresh the access token once and retry; if the refresh returns invalid_grant, fail ga4_authorization_revoked (not retryable; message "Google access was revoked. Connect again to restart the import.") |
HTTP 403 PERMISSION_DENIED |
Fail ga4_permission_denied (not retryable) |
HTTP 400 INVALID_ARGUMENT for any reason other than browserVersion |
Fail ga4_invalid_request with Google's message stored in error_message (not retryable) |
Response metadata reports sampling or dataLossFromOtherRow |
Import continues; warnings ga4_sampled_data and ga4_other_row_truncation are added once per report to import.warnings and shown in the UI |
On any failed status the rows inserted so far remain stored but invisible (the import is not completed). A retryable failure shows a Retry button for 24 hours (POST sites/{domain}/imports/{import_id}/retry resumes from the checkpoint, status back to queued, credentials kept); afterwards, and for non-retryable failures, import-sweep enqueues import-purge for the partial rows and deletes the credentials, leaving the import row visible with can_retry = false until the user deletes it.
The values of error_code listed in this section (import_validation_failed, import_stalled, export_too_large, ga4_* and similar) are job results stored on the import or export record and shown in the UI. They are not HTTP error codes: the HTTP responses of the routes in this section use only the closed catalogue of Section 4.4.3.
14.6 Import Management #
Import object (list and detail share the shape):
{
"id": "0192f7e0-2222-7000-8000-00000000f001",
"kind": "ga4",
"status": "importing",
"status_detail": null,
"resume_at": null,
"source_label": "Google Analytics 4: Marketing site (properties/123456789)",
"date_from": "2020-01-01",
"date_to": "2026-10-31",
"progress_pct": 62,
"rows_imported": { "imported_visitors": 2496, "imported_sources": 118204 },
"warnings": [ { "code": "ga4_sampled_data", "message": "Google returned sampled data for part of this range." } ],
"error_code": null,
"error_message": null,
"error_details": null,
"can_retry": false,
"created_by": { "id": "0192f7be-1111-7000-8000-000000000001", "name": "Ada Lovelace" },
"created_at": "2026-11-03T09:30:00Z",
"started_at": "2026-11-03T09:30:05Z",
"finished_at": null
}Statuses: configuring (GA4 only, waiting for property and dates), queued, validating (CSV only), importing, waiting_quota (GA4 only), completed, failed, cancelled, deleting. Active statuses: queued, validating, importing, waiting_quota. error_details holds the validation error list of 14.4.2 (CSV) as [{ "file", "line", "column", "message" }]. source_label is CSV upload (<n> files) for CSV imports. warnings entries: rows_ignored_native_overlap (with count), duplicate_rows_merged, values_clamped, ga4_sampled_data, ga4_other_row_truncation.
Columns required from the imports table (Section 5): id, site_id, kind (csv, ga4), status, status_detail, resume_at, source_label, date_from, date_to, progress_pct, rows_imported (jsonb), warnings (jsonb), error_code, error_message, error_details (jsonb), checkpoint (jsonb), oauth_credentials (text, nullable, format of Section 17.7.2), ga4_property_id, ga4_property_timezone, can_retry, cancel_requested_at, created_by_user_id, created_at, started_at, finished_at. ClickHouse imported tables carry the import's UUID in import_id (Section 6).
Routes (under /api/app/v1/sites/{domain}/, {domain} URL-encoded; every route requires site.import.manage, so editors, viewers and guests receive 403 forbidden):
| Method and path | Purpose |
|---|---|
GET imports |
List (newest first, cursor pagination) |
GET imports/{import_id} |
Detail; the UI polls every 3 seconds while the status is active |
POST imports/csv |
Start a CSV import (14.4.1) |
POST imports/ga4/authorize, GET imports/{import_id}/ga4/properties, POST imports/{import_id}/start, POST imports/{import_id}/retry |
GA4 flow (14.5.2) |
POST imports/{import_id}/cancel |
Cancel |
DELETE imports/{import_id} |
Delete the import and its data |
Cancel. Allowed in configuring, queued, validating, importing, waiting_quota; otherwise 409 import_not_cancellable with details: null (the current status is in the message). Sets cancel_requested_at and returns the import object. The worker checks the flag before each batch and window (at most 10 seconds of delay); on seeing it, enqueues import-purge for the rows already inserted, removes uploaded files, clears credentials and sets status = "cancelled", finished_at = now. A cancelled import keeps its row (visible in the list) until deleted and does not block a new import over the same dates.
Delete imported data. Allowed in completed, failed, cancelled (and configuring); active imports return 409 import_in_progress (cancel first). Sets status = "deleting" and enqueues import-purge, which removes the import's rows from every imported_* table (DELETE FROM ... WHERE import_id = ..., waiting for the mutation to finish), then hard-deletes the Postgres row, clears the first-native-day and imports_exist caches, and writes import.deleted. Statistics stop including the data as soon as the status leaves completed. import-purge handles one import only: it removes that import's rows and files and, only when the purge was started by a delete request, the imports row itself (a cancellation keeps the row). Site deletion does not use it: the site-purge job with scope all (Section 16.6) removes the site's imported rows, uploaded files, export files and credentials together with its native data.
14.7 Imported Data in the Dashboard #
- Toggle. When the response of any dashboard query has
meta.imports_exist = true, the top bar shows an "Imported data" switch (label includes the count of imports in a tooltip). Default on in the dashboard. The state lives in the URL query parameterwith_imported(trueorfalse) so it is shareable; the SPA keeps the last choice per site inlocalStorageas the default for URLs without the parameter. Shared links and embeds are controlled by the link'sinclude_importsflag (a checkbox "Include imported data" in the shared-link form of Section 12.2.1, default on): when it is on, the viewer sees the switch and may turn imported data off; when it is off, the server forces native data only and the switch is not shown. - Applying. The SPA sends
include.imports = truewhenever the switch is on. Each report's response then reportsmeta.imports_includedandmeta.imports_skip_reason. - Indicators. A report whose response has
imports_included = trueshows a small "Includes imported data" icon in its header (tooltip: source labels and covered date span). In the main time-series chart, points for dates covered by imported data render with a dashed line segment and the tooltip footnote "Imported". When imports are on but a report cannot use them, its header shows a muted notice with the text mapped from the reason code:
imports_skip_reason |
Text |
|---|---|
no_imported_data_in_range |
"No imported data for this period." |
unsupported_granularity |
"Imported data is available per day, not per hour." |
incompatible_dimensions |
"Imported data cannot be combined with these filters." |
unsupported_metric |
"Imported data does not include this metric." |
unsupported_goal |
"Imported data does not include these goals." |
The skip reason no_imports never reaches the UI because the switch is hidden in that case.
- Help text beneath the switch (popover): "Imported visitors cannot be de-duplicated against visitors tracked here, so totals over long periods can be slightly higher than unique people. Imported data covers dates before tracking started on this site."
- Not imported into the dashboard: funnels, journeys, realtime, goals of types
pageviewandscroll, revenue and scroll-depth reports always use native data only. - Imports page. Site settings, Data tab, Imports card (rendered only for owner and admin; the Data tab itself is open to every role because of the Export card, 14.3.4): table with Source, Date range, Status chip, a progress bar for active imports (with "Waiting for Google quota, resumes " for
waiting_quota), Rows imported, Created by and date, and actions: View details (modal with warnings, errors and counts per table), Cancel, Retry (whencan_retry), Delete (confirmation: "Delete imported data from ? This cannot be undone."). "Import data" opens a modal with two choices: Upload CSV files (drop zone accepting.csvand.zip, list of recognized files with row counts after validation of headers, Start import button) and Connect Google Analytics 4 (disabled with an explanation when unconfigured, 14.5.1). Empty state: "No imports yet. Bring your history from Google Analytics 4 or from CSV files so your charts start before today." After the first completed import a toast shows "Import finished. Turn on 'Imported data' in the dashboard to see it."
14.8 Limits and Edge Cases Summary #
| Item | Value |
|---|---|
| Exports | 1 active per site; request rate per policy export_start (Section 17.6); 5 retained per site; retained 7 days after completion; archive at most 2 GB; job at most 30 minutes |
| CSV import | 200 MB per file; 500 MB per upload; 5,000,000 rows per file; 50 archive entries; 2 GB expanded; range at most 10 years; dates 2010-01-01 or later |
| GA4 import | Range at most 10 years, starting 2020-01-01 or later; 100,000 rows per API page; 2,000,000 rows per window; up to 12 hours of hourly quota waiting and 2 daily quota waits; credentials kept at most 24 hours after a retryable failure |
| Per site | 10 imports (any status except deleting; 409 limit_reached with details.limit_name = "imports"); 1 active import; imports must not overlap in date range |
| Native overlap | Imported rows from the first native day onward are stored (CSV: skipped during insert; GA4: dropped at insert) and never shown |
| Deleting a site or team | Purges imported rows (site-purge with scope all), uploaded files, export files and OAuth credentials; running imports are cancelled first |
| Removing a member | Exports that member requested in the team are deleted (Section 11.16.5) |
| Changing the site timezone | Imported rows keep their calendar dates; native data re-buckets; the first native day is recomputed |
| Domain change | No effect on imported data (keyed by site UUID) |
| Server restart mid-job | pg-boss re-delivers the job; CSV import restarts its insert pass from a clean state, GA4 resumes from its checkpoint |
| Worker down | Statuses remain queued or importing; the UI shows the last progress; a job stuck in importing for over 6 hours without progress update is failed by import-sweep with error_code = "import_stalled" |
| Self-hosted | All features work; GA4 requires the operator's Google OAuth credentials (14.5.1) |
| Hosted usage caps | Imports and exports never count as billable events (Section 15) |
15. Lifetime Deal Licensing, Payments, Usage Caps & Operator Console #
This section defines everything that exists only in the hosted edition: the three lifetime-deal tiers, how a team obtains a license (Stripe Checkout or marketplace codes), how usage is metered and capped, refunds, invoices, the billing UI, and the operator console. The tables referenced here live in Postgres; this section owns the column semantics of license_tiers, licenses, purchases, license_code_batches, license_codes, usage_monthly and usage_site_monthly; Section 15.19 gives their authoritative definitions (columns, types, nullability, enums and checks), and Section 5 references these definitions and does not repeat them. Throughout, "the entitlement" means the license-derived limits of a team (site limit and monthly event cap).
15.1 Scope, Editions and Principles #
15.1.1 Edition gating #
| Aspect | EDITION=hosted |
EDITION=selfhosted |
|---|---|---|
| Licensing, tiers, codes, purchases | Active | Not registered. Billing and admin routes return 404 not_found; the SPA hides every billing, plan and usage element. |
| Site limit | Enforced per Section 15.10 | None (unlimited) |
| Event cap and 110% hard stop | Enforced per Section 15.9 | None (unlimited) |
Usage counters in Valkey, usage_monthly and usage_site_monthly writes |
Written | Not written. The ingestion hook short-circuits to accept. |
Stripe webhook route POST /api/webhooks/stripe |
Registered | Not registered (404) |
Operator console /admin, /api/app/v1/admin/* |
Registered | Not registered (404). SUPER_ADMIN_EMAILS is ignored. |
| Jobs owned by this section | Scheduled | Not scheduled (the worker skips registration; Section 16.6 lists the Edition column per job) |
Migrations and seed rows (license_tiers) |
Run | Run (identical schema), but the rows are never read |
The SPA learns the edition from the edition field ("hosted" or "selfhosted") of the app bootstrap response (GET /api/app/v1/bootstrap, Section 11.2). All edition checks on the server use one helper, isHosted(), that reads the validated EDITION value (Section 16.4).
15.1.2 Principles (binding decisions) #
- All features in every tier. Tiers differ only in
max_sitesandmonthly_event_cap. No feature is gated by tier or by license state except data collection, site creation beyond the limit, data import and adding external email recipients (Sections 15.11, 12.5.1). - Lifetime means "for the life of the hosted product". There are no recurring fees, subscriptions, trials or renewals. Nothing in this product expires a license by date.
- One active license per team. Enforced by a partial unique index (Section 15.3.3). A license belongs to exactly one team.
- Licenses are never transferred between teams by customers. A license is bound to its team. Transferring team ownership (Section 11) carries the license with the team. Support-driven moves are possible only through the operator console action "Move license" (Section 15.17.5). Redeemed codes and paid purchases are bound to the license they created.
- Licenses are never deleted. They change status. Purchases, codes, invoices and audit rows are retained for accounting.
- Currency is USD only. All tier prices, charges, invoices and credit notes are in USD. Stripe Adaptive Pricing is disabled. Revenue goals in customer analytics (Section 10) are unrelated and support many currencies.
- Webhooks are the only source of truth for payment state. The browser redirect to the success URL never grants a license.
- Fail open on infrastructure uncertainty, fail closed on explicit state. If entitlement state cannot be read (Valkey and Postgres both unreachable), ingestion accepts the event. If state is read and says unlicensed or blocked, ingestion drops it.
- A paid checkout always ends with an active license or an automatic refund, never neither. Every path of Section 15.6.4 either grants the license or refunds the payment, and
stripe-reconcilerepeats the function until one of the two has happened.
15.1.3 Permissions (matrix rows owned by Section 11) #
| Permission key | Owner | Admin | Editor | Viewer | Guests |
|---|---|---|---|---|---|
billing.view (license summary, usage meter, invoices list) |
yes | yes | no | no | no |
billing.manage (checkout, upgrade, redeem code, download invoice and credit-note PDFs) |
yes | no | no | no | no |
Editors, viewers and guests receive only the generic banner text from Section 15.12.2 (no numbers, no call to action). Super admins reach billing data only through the operator console (Section 15.17), never through team routes.
15.2 Tiers #
15.2.1 Seed values #
Seeded into license_tiers by the migration that creates it; editable by the operator in the console without a code change (Section 15.17.8).
code |
name |
rank |
price_minor |
currency |
max_sites |
monthly_event_cap |
sort_order |
is_active |
|---|---|---|---|---|---|---|---|---|
solo |
Solo | 1 | 5900 | USD | 5 | 100000 | 1 | true |
pro |
Pro | 2 | 14900 | USD | 25 | 500000 | 2 | true |
agency |
Agency | 3 | 34900 | USD | 100 | 2000000 | 3 | true |
Marketing names in the UI: "Tier 1 · Solo", "Tier 2 · Pro", "Tier 3 · Agency". The rank equals the number of stacked marketplace codes that produce the tier (Section 15.8.1).
15.2.2 license_tiers columns used by this section #
| Column | Type | Rule |
|---|---|---|
id |
uuid | PK |
code |
text | Unique, ^[a-z][a-z0-9_]{1,31}$. Immutable after creation. The three seeded codes are permanent and cannot be deleted. |
name |
text | 1–40 characters |
rank |
smallint | 1, 2 or 3. Unique. Immutable. The product supports exactly three tiers. |
price_minor |
integer | 100 ≤ value ≤ 10,000,000 (USD cents). Stripe minimum is 50 cents; the product floor is $1.00. |
currency |
char(3) | Always USD |
max_sites |
integer | 1 ≤ value ≤ 100,000 |
monthly_event_cap |
bigint | 1,000 ≤ value ≤ 100,000,000,000 |
sort_order |
smallint | Display order |
is_active |
boolean | Inactive tiers cannot be purchased or upgraded to; existing licenses are unaffected. At least one tier must remain active; at least the highest-ranked active tier must exist. |
created_at, updated_at |
timestamptz |
Constraints enforced in the console service (and by a DB check where possible): price_minor strictly increases with rank among active tiers; max_sites and monthly_event_cap are non-decreasing with rank. Violations return 422 tier_constraint_violation with details.fields.
15.2.3 Effect of editing a tier #
A license snapshots its entitlement at grant time (max_sites, monthly_event_cap, credit_minor). Editing a tier therefore affects:
- New purchases and new code stacks: immediately (the snapshot is read from the tier at grant time).
- Existing licenses: not at all by default. The edit form contains a checkbox "Also apply new site limit and event cap to existing licenses of this tier". It is enabled only when both new values are greater than or equal to the old values (raising only). When checked, a single transaction updates
licenses.max_sitesandlicenses.monthly_event_capfor allactivelicenses with thattier_id, writes one audit entrytier.updatedwithapply_to_existing: trueand the affected count, and publisheslicense:changedfor each affected team (Section 15.9.6). Price changes never touch existing licenses; upgrade credit uses the license's owncredit_minor(Section 15.7.2).
15.3 License Model and State Machine #
15.3.1 licenses columns used by this section #
| Column | Type | Rule |
|---|---|---|
id |
uuid NOT NULL | PK |
team_id |
uuid null | FK to teams, ON DELETE SET NULL; null only on revoked rows (the team-purge job revokes the license in the transaction that deletes the team; Section 15.16) |
team_name_snapshot |
text NOT NULL | Team name at grant; kept for the console after team deletion |
owner_email_snapshot |
text null | Owner email at grant; cleared by the user anonymization job when that user is anonymized |
tier_id |
uuid NOT NULL | FK to license_tiers, ON DELETE RESTRICT |
source |
text NOT NULL | stripe, codes or operator |
status |
text NOT NULL | active or revoked; default active |
max_sites |
integer NOT NULL | Snapshot of the entitlement, ≥ 1 |
monthly_event_cap |
bigint NOT NULL | Snapshot of the entitlement, ≥ 1 |
credit_minor |
integer NOT NULL | Amount credited toward upgrades (Section 15.7.2), default 0. 0 for operator licenses. |
granted_at |
timestamptz NOT NULL | First time this row became active |
revoked_at |
timestamptz null | Set when status becomes revoked; cleared on reinstatement |
revoked_reason |
text null | refunded, dispute, code_revoked, operator_revoked, team_deleted |
purge_after |
timestamptz null | revoked_at + 30 days when revoked; operator may extend (Section 15.14.3). NULL on a dispute hold and for team_deleted (no purge clock) |
purged_at |
timestamptz null | Set by the unlicensed-purge job |
created_at, updated_at |
timestamptz NOT NULL |
The complete definition with constraints is Section 15.19.2.
Codes attached to a license are license_codes.license_id rows (Section 15.8). Purchases attached to a license are purchases.license_id rows (Section 15.5).
15.3.2 States #
"None" is not a stored state: a team with no active license row is unlicensed.
| Team state | Definition | Ingestion | Dashboard | Site creation | Imports |
|---|---|---|---|---|---|
| Unlicensed, never licensed | No license row has ever existed for the team | Dropped (still 202), counted as dropped_unlicensed |
Readable (empty) | Up to 1 site | Blocked |
active |
A license row with status='active' |
Accepted per cap rules (Section 15.9) | Full | Up to max_sites |
Allowed |
Unlicensed after revocation (revoked, no active row) |
Latest license status='revoked' |
Dropped | Readable until purged_at; exports allowed |
Up to 1 site total (existing sites remain; no new site if count ≥ 1) | Blocked |
15.3.3 Transitions #
| From | Event | To | Side effects |
|---|---|---|---|
| none | Stripe checkout.session.completed (kind new, paid) |
active |
Create license; source='stripe'; email license_activated; invoice issued |
| none | Code redeemed (first code) | active |
Create license at solo (1 unit); source='codes' |
| none | Operator grants license | active |
source='operator'; audit license.granted |
active |
Stripe upgrade purchase paid | active (tier raised) |
Entitlement raised; cap-email flags re-evaluated (Section 15.9.5); email license_upgraded |
active |
Another code redeemed (stack 2 or 3) | active (tier raised) |
Same as above |
active |
Full refund of the new purchase (charge.refunded) |
revoked (refunded) |
purge_after = now + 30 d; email license_revoked |
active |
Full refund of the latest upgrade purchase |
active (tier reverted to from_tier) |
Entitlement reverts; email refund_processed |
active |
Out-of-band Stripe refund of an earlier (non-latest) upgrade purchase |
active (tier set to that of the latest remaining non-refunded purchase) |
Entitlement follows that purchase; operator_alert email (Section 15.13.3) |
active |
charge.dispute.created |
revoked (dispute) |
purge_after = NULL (hold, Section 15.14.3); operator alert email; purchase marked disputed; ingestion dropped |
revoked (dispute) |
Operator closes the dispute as lost | revoked |
purge_after = now + 30 d; audit license.dispute_closed_lost |
active |
Operator revokes code(s) leaving 0 units | revoked (code_revoked) |
As above |
active |
Operator revokes code(s) leaving 1 or 2 units | active (tier lowered to stack count) |
Audit license.tier_changed; site over-limit handling (Section 15.10.3) |
active |
Operator revokes license | revoked (operator_revoked) |
As above |
active |
Team hard-deleted by the team-purge job | revoked (team_deleted), team_id set null |
No purge clock needed (data removed with the team) |
revoked |
Operator reinstates | active |
Requires that the team has no other active license; clears revoked_at, revoked_reason, purge_after (also ends a dispute hold when the dispute was won); audit license.reinstated |
revoked |
Team buys/redeems again | stays revoked; a new license row is created active |
The older revoked row's purge_after is ignored while the team has an active license |
No other transitions exist. active → active transitions never create a new license row; the same row is updated.
Database invariants (Section 5 creates exactly these; the complete definitions are in Section 15.19):
CREATE UNIQUE INDEX licenses_one_active_per_team
ON licenses (team_id) WHERE status = 'active' AND team_id IS NOT NULL;
ALTER TABLE licenses ADD CONSTRAINT licenses_status_chk CHECK (status IN ('active','revoked'));
ALTER TABLE licenses ADD CONSTRAINT licenses_source_chk CHECK (source IN ('stripe','codes','operator'));
ALTER TABLE licenses ADD CONSTRAINT licenses_revoked_reason_chk
CHECK (revoked_reason IS NULL OR revoked_reason IN ('refunded','dispute','code_revoked','operator_revoked','team_deleted'));
ALTER TABLE licenses ADD CONSTRAINT licenses_state_chk CHECK (
(status = 'active' AND revoked_at IS NULL AND revoked_reason IS NULL AND purge_after IS NULL AND purged_at IS NULL)
OR (status = 'revoked' AND revoked_at IS NOT NULL AND revoked_reason IS NOT NULL
AND (purge_after IS NOT NULL OR revoked_reason IN ('team_deleted','dispute'))));15.3.4 Entitlement lookup contract #
All callers (ingestion, site creation, import, banners) use one module in apps/server/src/modules/licensing/entitlement.ts; its ingest decision (accept, drop or warn) is the one the ingestion pipeline applies in Section 8.9:
export type LicenseState = 'unlicensed' | 'active';
export interface Entitlement {
teamId: string;
state: LicenseState;
licenseId: string | null;
tierCode: 'solo' | 'pro' | 'agency' | null;
maxSites: number; // 1 when unlicensed; Infinity-equivalent (Number.MAX_SAFE_INTEGER) when selfhosted
monthlyEventCap: number; // 0 when unlicensed; Number.MAX_SAFE_INTEGER when selfhosted
suspended: boolean; // team suspended by operator (Section 15.17.4)
purgeAfter: string | null; // ISO timestamp, when revoked and not yet purged
}
export function getEntitlement(teamId: string): Promise<Entitlement>;
export type IngestDecision = 'accept' | 'drop_unlicensed' | 'drop_cap' | 'drop_suspended';
export function decideIngest(teamId: string, nowMs: number): Promise<IngestDecision>;Self-hosted: getEntitlement returns state: 'active', tierCode: null, unlimited maxSites and monthlyEventCap, and decideIngest returns 'accept' without touching Valkey. The same two signatures are what the milestone stubs of Section 19 implement.
Resolution order in getEntitlement: (1) in-process cache, 5 s TTL; (2) Valkey hash license:state:{team_id} (no TTL; deleted on any license/tier/suspension change); (3) Postgres (licenses where team_id and status='active', plus teams.suspended_at), then the result is written to Valkey. Invalidation: every writer calls publishLicenseChanged(teamId), which deletes the Valkey key and publishes the team id on the Valkey channel license:changed; every web process subscribes and evicts its in-process cache entry. Missed messages are healed by the 5 s TTL.
15.4 Stripe Configuration #
| Item | Decision |
|---|---|
| Mode | Checkout Sessions, mode: 'payment', redirect flow (no embedded elements, no publishable key needed) |
| Payment methods | payment_method_types: ['card'] (wallets such as Apple Pay and Google Pay arrive through card). Delayed-notification methods are disabled, so every completed session is already paid. |
| Prices | Inline price_data per session (currency: 'usd', unit_amount, product_data.name). No Stripe Price or Product objects are pre-created, so editing a tier needs no Stripe sync. |
| Product name | "{PRODUCT_NAME} Lifetime — {tier name}"; for upgrades "{PRODUCT_NAME} Lifetime upgrade — {from name} to {to name}" |
| Tax | If STRIPE_TAX_ENABLED=true: automatic_tax: {enabled: true}, price_data.tax_behavior: 'exclusive', billing_address_collection: 'required', tax_id_collection: {enabled: true}. If false: none of these are sent and no tax line appears. |
| Customer | customer_creation: 'always'; customer_email = purchaser email. A returning purchaser (any prior paid purchase by the same user) reuses customer = purchases.stripe_customer_id instead of customer_email. |
| Receipts | payment_intent_data.receipt_email = purchaser email. Stripe sends its own receipt (Dashboard setting "Email customers about successful payments" must be on; stated in the launch checklist, Section 20). |
| Promotion codes | allow_promotion_codes: false |
| Session lifetime | expires_at = now + 30 minutes |
| Locale | locale: 'auto' |
| Idempotency keys | Session creation passes Idempotency-Key = "co_" + purchases.id. Console refunds pass "rf_" + purchases.id + "_" + refund_request_id, where refund_request_id is a UUID generated by the console and stored in the audit row purchase.refund_requested before Stripe is called (a console retry reuses it). The automatic duplicate refund of Section 15.6.4 passes "rf_" + purchases.id + "_auto". |
| Metadata on session and payment intent | purchase_id, team_id, kind (new or upgrade), tier_code, from_license_id (upgrade only). Metadata is advisory; the pending purchases row is authoritative. |
client_reference_id |
purchases.id |
| API errors | Any Stripe API failure during session creation returns 503 billing_unavailable; the pending purchase row is marked expired. |
| Library | The official Node SDK, per Section 3. A single StripeClient wrapper in apps/server/src/modules/billing/stripe.ts is the only importer of the SDK. |
15.5 Purchases #
15.5.1 purchases columns used by this section #
| Column | Type | Rule |
|---|---|---|
id |
uuid | PK |
team_id |
uuid null | FK to teams, ON DELETE SET NULL |
license_id |
uuid null | FK to licenses; null while pending and for auto-refunded duplicates |
user_id |
uuid null | Purchaser; FK ON DELETE SET NULL |
kind |
text | new or upgrade |
provider |
text | stripe (the only value) |
tier_id |
uuid | Tier purchased (target tier for upgrades) |
from_tier_id |
uuid null | Upgrade only: tier before the upgrade |
status |
text | pending, paid, partially_refunded, refunded, disputed, expired |
amount_minor |
integer | Subtotal excluding tax |
tax_minor |
integer | Tax charged (0 when tax disabled) |
total_minor |
integer | amount_minor + tax_minor |
currency |
char(3) | USD |
refunded_minor |
integer | Cumulative refunded total (tax included), default 0 |
stripe_checkout_session_id |
text unique null | |
stripe_payment_intent_id |
text null | |
stripe_charge_id |
text unique null | |
stripe_refund_id |
text null | Stripe refund that completed the full refund; the credit note is keyed to it |
stripe_customer_id |
text null | |
stripe_dispute_id |
text null | |
payment_method_summary |
text null | For example Visa •••• 4242; brand and last 4 only |
billing_name |
text null | From Stripe customer_details.name |
billing_email |
text null | |
billing_address |
jsonb null | {line1,line2,city,state,postal_code,country} from Stripe |
billing_tax_ids |
jsonb null | [{type,value}] from Stripe |
paid_at |
timestamptz null | |
refunded_at |
timestamptz null | Set when status becomes refunded |
invoice_year, invoice_seq, invoice_number |
smallint null, integer null, text unique null | Section 15.15.1; set together |
credit_note_year, credit_note_seq, credit_note_number |
smallint null, integer null, text unique null | Section 15.15.3; set together |
seller_snapshot |
jsonb null | Seller details frozen at invoice issue time |
expires_at |
timestamptz null | Checkout session expiry |
created_at, updated_at |
timestamptz |
Unique constraints: purchases_invoice_seq_uq on (invoice_year, invoice_seq) and purchases_credit_note_seq_uq on (credit_note_year, credit_note_seq). The complete definition with constraints is Section 15.19.3.
15.5.2 Purchase status machine #
| From | Event | To |
|---|---|---|
pending |
checkout.session.completed with payment_status='paid' |
paid |
pending |
Session expired, creation failure, replaced by a newer session, or older than 24 h at the purchase-pending-expire job |
expired |
expired |
fulfillCheckoutSession finds the Stripe session complete with payment_status='paid' (a payment that arrived after the row was expired) |
paid (row re-opened), or refunded through the automatic refund path; it is never left expired |
paid |
charge.refunded with amount_refunded < total_minor |
partially_refunded |
paid or partially_refunded |
charge.refunded with amount_refunded >= total_minor |
refunded |
paid or partially_refunded |
charge.dispute.created |
disputed |
expired rows are never reused for a new checkout; a retry creates a new row. A late payment on an expired row is re-opened only by fulfillCheckoutSession (Section 15.6.4).
15.6 Stripe Checkout Flow (New Purchase) #
15.6.1 Plans endpoint #
GET /api/app/v1/teams/{team_id}/billing/plans (permission billing.view). Hosted only.
{ "data": {
"license": { "state": "unlicensed|active", "tier_code": "pro|null", "source": "stripe|codes|operator|null" },
"tiers": [
{ "code": "solo", "name": "Solo", "rank": 1, "price_minor": 5900, "currency": "USD",
"max_sites": 5, "monthly_event_cap": 100000,
"action": "buy|upgrade|current|unavailable",
"due_minor": 5900, "unavailable_reason": null }
]
} }Rules per tier row: unlicensed → action: "buy", due_minor = price_minor. Active Stripe license: tier below current → unavailable with reason lower_tier; same tier → current; higher tier → upgrade, due_minor = price_minor − credit_minor. Active codes or operator license → higher tiers unavailable with reason license_source_codes or license_source_operator; lower and same as above. Inactive tiers are omitted unless they are the team's current tier.
15.6.2 Create checkout session #
POST /api/app/v1/teams/{team_id}/billing/checkout, permission billing.manage, CSRF required.
Request: {"tier": "solo|pro|agency"}. Response 201: {"data": {"purchase_id": "<uuid>", "checkout_url": "https://checkout.stripe.com/..."}}.
Server steps (the Stripe calls happen outside the Postgres transaction):
- Validate
tieragainst active tiers (422 validation_failedotherwise). - For every existing
pendingpurchase of this team, retrieve its Checkout Session. If the session iscompleteand paid, runfulfillCheckoutSession(Section 15.6.4) instead of expiring it, so a payment already in flight is never discarded. Otherwise callstripe.checkout.sessions.expire(ignoring an already-expired session) and set the rowexpired. - Begin a transaction and lock the team row (
SELECT ... FOR UPDATE). If the team has anactivelicense (including one granted by step 2) →409 license_already_active(use the upgrade endpoint). If the team is suspended →403 forbiddenwithdetails.reason = "team_suspended". - Insert a
pendingpurchase (kind='new',amount_minor = tier.price_minor,expires_at = now + 30 min) and commit. - Create the Stripe session with
success_urlandcancel_urlbelow and update the row withstripe_checkout_session_id. A Stripe failure marks the rowexpiredand returns503 billing_unavailable.
Rate limit: policy checkout_create (Section 17.6).
URLs:
success_url:{BASE_URL}/teams/{team_id}/billing?checkout=success&purchase={purchase_id}cancel_url:{BASE_URL}/teams/{team_id}/billing?checkout=cancelled&purchase={purchase_id}
Neither contains a Stripe session id, so the browser cannot be used to forge fulfillment.
15.6.3 Returning from Stripe #
On ?checkout=success the billing page shows a full-width "Confirming your payment…" panel and polls GET /api/app/v1/teams/{team_id}/billing/checkout-status?purchase_id=<uuid> every 2 s for at most 60 s (permission billing.view; the purchase must belong to {team_id}, otherwise 404 not_found). Response: {"data": {"status": "pending|paid|expired", "license_active": true|false}}.
The status endpoint, when the purchase is still pending and older than 10 s, retrieves the Checkout Session from the Stripe API; if it is complete and payment_status='paid', it runs the same idempotent fulfillment function the webhook uses (Section 15.6.4). So a delayed webhook never blocks the customer. The Stripe session is fetched at most once per 10 s per purchase (Valkey key co:poll:{purchase_id}). After 60 s without paid, the panel changes to: "Your payment is still being confirmed. You will receive an email once your license is active. Contact {SUPPORT_EMAIL} if this takes longer than an hour." and polling stops.
On ?checkout=cancelled the page shows a neutral notice "Checkout cancelled. You have not been charged." and the plan chooser; the pending row stays pending until replaced or expired.
15.6.4 Fulfillment function #
fulfillCheckoutSession(session: Stripe.Checkout.Session): Promise<FulfillResult> is idempotent and is called by the webhook, the status endpoint and the daily stripe-reconcile job.
- Resolve the purchase by
stripe_checkout_session_id = session.id. If none exists, logwarnand returnignored(HTTP 200 for webhooks). - Begin transaction;
SELECT ... FOR UPDATEthe purchase. Ifstatusispaid,partially_refunded,refundedordisputed→ returnalready_processed. Ifstatusispendingorexpired, continue: anexpiredrow is re-opened only when step 3 finds the session paid, so a payment arriving afterpurchase-pending-expireor session replacement is never discarded. - If
session.payment_status !== 'paid'→ returnnot_paid(no state change). - Retrieve the PaymentIntent with
expand: ['latest_charge']to obtain the charge id, card brand/last4, amounts. Storeamount_minor = session.amount_subtotal,tax_minor = session.total_details.amount_tax,total_minor = session.amount_total, customer, billing details and tax ids fromsession.customer_details. - Validate business state:
kind='new': the team must exist, not be soft-deleted, not be suspended, and have no active license. If it already has an active license (a race between two tabs, or a re-opened late payment) or is deleted → automatic duplicate refund: create a full Stripe refund (reason: 'duplicate', idempotency keyrf_<purchase_id>_auto), set the purchasestatus='refunded',refunded_minor = total_minor,refunded_at = now, write auditpurchase.auto_refunded, send emailpurchase_auto_refundedto the purchaser and anoperator_alertemail to super admins; no license is created; no invoice number is allocated. If the Stripe refund call fails, the transaction rolls back and the function fails (see the guarantee below).kind='upgrade': the target license (metadata.from_license_id) must still beactive,source='stripe', and its current tier must equalfrom_tier_id. Otherwise the same automatic refund path.
- Grant: for
new, insert the license (source='stripe', snapshotmax_sites,monthly_event_capfrom the tier,credit_minor = tier.price_minor,granted_at = now); forupgrade, update the existing license (tier_id, snapshots,credit_minor = target.price_minor). Setpurchase.status='paid',paid_at,license_id. - Allocate the invoice number and freeze
seller_snapshot(Section 15.15.1). - Commit. After commit:
publishLicenseChanged(team_id), enqueue the emaillicense_activatedorlicense_upgradedto the purchaser, write auditlicense.granted/license.upgraded, and (upgrade only) re-evaluate cap-email flags (Section 15.9.5).
Guarantee. For every Checkout Session with payment_status = 'paid', this function ends in exactly one of two states: an active license (purchase paid) or a refunded purchase (refunded, the automatic duplicate refund). If any step fails (database error, Stripe refund API failure), the transaction rolls back and nothing is partially applied; the caller then repeats the whole function: the webhook answers 500 so Stripe retries, the status endpoint retries on the next poll, and stripe-reconcile retries daily. A paid checkout is therefore never left with neither a license nor a refund.
15.6.5 Webhook endpoint #
POST /api/webhooks/stripe, unauthenticated by session; authenticated by signature. Registered only when EDITION=hosted.
| Rule | Decision |
|---|---|
| Body | Raw bytes (no JSON parsing before verification); limit 1 MiB (413 beyond) |
| Verification | Stripe-Signature header verified against STRIPE_WEBHOOK_SECRET with a tolerance of 300 s using the SDK's signature verification. Missing or invalid → 400 with {"error":{"code":"invalid_signature",...}}. |
| Handled event types | checkout.session.completed, charge.refunded, charge.dispute.created. Every other type, including checkout.session.async_payment_succeeded, checkout.session.async_payment_failed and checkout.session.expired, returns 200 {"received": true} and changes nothing. A checkout.session.completed whose payment_status is not paid grants nothing. |
| Idempotency | (a) SET NX of stripe:evt:{event.id} in Valkey with a 7-day TTL is a fast-path skip only, written after successful processing; (b) correctness comes from the database: purchases.stripe_checkout_session_id unique, row locks, status checks, and absolute (not incremental) refund amounts. Replaying any event any number of times, in any order, yields the same state. |
| Success | 200 {"received": true} after the database transaction commits |
| Business-level no-ops (unknown purchase, unpaid session, already processed) | 200 (so Stripe stops retrying) |
| Transient failure (database error, Stripe API timeout) | 500; Stripe retries with its own backoff |
| Ordering | charge.refunded or charge.dispute.created finds no purchase with that charge id: look up the purchase named by the PaymentIntent's metadata purchase_id. If it is an existing pending purchase, return 500 so Stripe retries later (up to 3 days) and the checkout is processed first; otherwise return 200 with result ignored |
| Observability | Counter quietstat_stripe_webhook_total{type,result}; log line with event_id, type, result and no payload |
15.6.6 Refund and dispute handlers #
charge.refunded: locate the purchase by stripe_charge_id (fall back to stripe_payment_intent_id; ordering per Section 15.6.5). Set refunded_minor = charge.amount_refunded (absolute). Then, in order:
- If the purchase is already
refunded: updaterefunded_minoronly and return. - If
license_idis null orinvoice_numberis null (the automatic duplicate refund of Section 15.6.4, or a purchase that was never fulfilled): set the status (refundedwhen fully refunded) and stop. No credit note, no license effect, no email. - If
refunded_minor < total_minor:status='partially_refunded'; the license is unchanged; auditpurchase.partially_refunded; no email. - Otherwise:
status='refunded',refunded_at = now; record the completing Stripe refund id instripe_refund_idand allocate a credit note number only whencredit_note_number IS NULL, so a replayed refund event never creates a second credit note (Section 15.15.3); apply the license effect per Section 15.13.3; emailrefund_processed.
charge.dispute.created: locate the purchase by charge id. If the purchase is already disputed, or has no license (an automatically refunded duplicate), return without effect. Otherwise set status='disputed' and stripe_dispute_id; revoke the license with reason dispute and purge_after = NULL (all purchases of the license are affected; the license is revoked as a whole, and its data is held, not purged, because disputes take 60 to 75 days to resolve); audit license.revoked; send an operator_alert email to all super admins with the dispute id and Stripe Dashboard link; send license_revoked to the owner (text: "Your payment was disputed with your card issuer, so the license is suspended. Contact {SUPPORT_EMAIL} to resolve this."). When the dispute is won, an operator reinstates the license (Section 15.17.5); when it is lost, the operator uses "Close dispute as lost", which starts the 30-day purge clock. The product deliberately does not handle charge.dispute.closed.
15.7 Upgrade Flow (Stripe) #
15.7.1 Rules #
- Only a team with an
activelicense whosesource='stripe'can pay to upgrade. Teams withcodeslicenses upgrade by redeeming more codes (Section 15.8.4). Teams withoperatorlicenses are changed only by an operator. Other source →409 license_source_conflict. - Target must have a higher
rankthan the current tier: a lower tier returns422 downgrade_not_allowed, the same tier409 license_already_active, and an inactive tier422 invalid_upgrade_target. - Downgrades by customers are impossible (no endpoint exists).
- Upgrade skips tiers: Solo → Agency is allowed.
15.7.2 Price difference #
due_minor = target.price_minor − license.credit_minor, where license.credit_minor is the list price of the license's current tier at the time it was granted or last upgraded (credit_minor is set to target.price_minor on every upgrade). If due_minor < 100 (for example because an operator lowered the target price below credit) → 422 invalid_upgrade_target with message "No payment is required; contact support." and the operator uses the console to change the tier.
15.7.3 Endpoint #
POST /api/app/v1/teams/{team_id}/billing/upgrade, permission billing.manage, CSRF required. Request {"tier": "pro|agency"}. Response 201 as in Section 15.6.2 with a purchase of kind='upgrade', amount_minor = due_minor, from_tier_id = current tier, tier_id = target. The Stripe line item name follows Section 15.4. The same pending-session replacement, success/cancel URLs, polling and fulfillment rules apply. A second concurrent upgrade is prevented by the pending-replacement rule; if two upgrades are paid anyway, the second fails the Section 15.6.4 step 5 check (current tier no longer equals from_tier_id) and is automatically refunded.
15.7.4 Effects of an upgrade #
Entitlement (max_sites, monthly_event_cap) rises immediately for the current UTC month, so a team in blocked state resumes ingestion within 5 s of fulfillment. The month's counters are not reset. Events dropped earlier are not recovered.
15.8 Marketplace Codes #
15.8.1 Concepts #
A marketplace code is a single-use voucher worth one unit. A team's unit count determines its tier: 1 unit = Solo, 2 units = Pro, 3 units = Agency. The maximum is 3 units per team. A batch is a group of codes generated together for a marketplace listing; the batch's tier_id is a reporting label recording the marketplace listing tier the batch is sold under and does not change the value of a code (every code is exactly one unit).
15.8.2 Tables used #
license_code_batches:
| Column | Type | Rule |
|---|---|---|
id |
uuid | PK |
name |
text | 1–80 characters, unique among batches (case-insensitive) |
marketplace |
text | 1–60 characters (free text, for example the marketplace's name) |
tier_id |
uuid | Reporting label (see 15.8.1) |
quantity |
integer | 1 ≤ value ≤ 10,000 |
notes |
text null | ≤ 1,000 characters |
created_by |
uuid null | Super admin user id |
created_at |
timestamptz |
license_codes:
| Column | Type | Rule |
|---|---|---|
id |
uuid | PK |
batch_id |
uuid | FK to license_code_batches, ON DELETE RESTRICT |
code |
text unique | Normalized display form QS-XXXX-XXXX-XXXX (see 15.8.3) |
status |
text | unredeemed, redeemed, revoked |
license_id |
uuid null | Set on redemption |
redeemed_by |
uuid null | User id (ON DELETE SET NULL) |
redeemed_team_id |
uuid null | (ON DELETE SET NULL) |
created_by |
uuid null | Super admin who generated the batch (ON DELETE SET NULL) |
redeemed_at |
timestamptz null | |
revoked_at |
timestamptz null | |
revoked_reason |
text null | ≤ 200 characters |
created_at |
timestamptz |
Codes are stored in plaintext because the operator must be able to re-export a batch. Database access is inside the trust boundary; a code is a revocable bearer voucher.
15.8.3 Code format and generation #
- Format
QS-XXXX-XXXX-XXXX. The prefix is the constantCODE_PREFIX(defaultQS, independent ofPRODUCT_NAME; 2–4 uppercase letters, set inpackages/shared). - The 12 payload characters come from the Crockford base32 alphabet
0123456789ABCDEFGHJKMNPQRSTVWXYZ(excludes I, L, O, U), each drawn uniformly from the system CSPRNG (crypto.randomInt(32)). That is 60 bits of entropy. - Normalization on input (redeem API and UI): trim; uppercase; remove spaces and hyphens; map
I→1,L→1,O→0; require exactlylen(prefix)+12characters starting with the prefix and all payload characters in the alphabet; otherwise422 code_invalid. The normalized value is re-hyphenated toQS-XXXX-XXXX-XXXXbefore lookup. - Generation inserts in chunks of 1,000 rows inside one transaction; a unique-violation on
coderegenerates only that code (up to 5 attempts per code, then the request fails with500 internal_error, which in practice cannot occur at 60 bits).
15.8.4 Redemption rules #
POST /api/app/v1/teams/{team_id}/license/redeem, permission billing.manage, CSRF required. Request: {"code": "QS-7K2M-9PQX-4TRD"}. Response 200: {"data": {"license": <license summary per 15.12.1>, "units": 2, "units_max": 3}}.
Evaluation order (this order prevents the endpoint from revealing whether a code exists):
- Brute-force check (Section 15.8.6) →
429 rate_limited. - Team preconditions, before any code lookup (fast failure; repeated under the lock in step 4):
- Active license with
source='stripe'or'operator'→409 license_source_conflict("This team already has a purchased license; redeem codes on a team without one."). - Active license with
source='codes'and 3 units →409 code_stack_limit_reached. - Team suspended →
403 forbiddenwithdetails.reason = "team_suspended".
- Active license with
- Normalize the code (
422 code_invalidon format failure; counts as a failed attempt). - One transaction, locks always taken in this order:
- (a)
SELECT ... FROM teams WHERE id = $1 FOR UPDATE, so concurrent redemptions on one team serialize; - (b) re-read the team's active license and its redeemed-unit count inside the transaction and repeat the step 2 checks (
409 code_stack_limit_reachedat 3 units,409 license_source_conflict), so two parallel redemptions on a team with 2 units can never produce 4 units and two parallel first redemptions can never create two licenses; - (c)
SELECT ... FROM license_codes WHERE code = $1 FOR UPDATE. If the row does not exist, orstatusis notunredeemed→422 code_invalid("This code is not valid or has already been used.") (counts as a failed attempt; identical response for missing, used and revoked).
- (a)
- If the team has no active license: insert a license (
source='codes', tiersolo, snapshot from the tier,credit_minor = solo.price_minor). Otherwise raise the existing license to the tier whoserank = new unit count, update snapshots andcredit_minor. - Mark the code
redeemedwithlicense_id,redeemed_by,redeemed_team_id,redeemed_at. Commit. - After commit:
publishLicenseChanged, auditcode.redeemed, emaillicense_activated(first code) orlicense_upgraded(2nd or 3rd), re-evaluate cap-email flags (Section 15.9.5). Failed attempts are not audited individually (they are rate counted; Section 15.8.6).
Units are counted as COUNT(*) FROM license_codes WHERE license_id = $1 AND status = 'redeemed'.
Redeemed codes are bound to the license. They cannot be moved, re-sold or released by customers. A code redeemed on a team that is later hard-deleted stays redeemed.
15.8.5 Code revocation (operator) #
- Revoking an
unredeemedcode setsstatus='revoked',revoked_at,revoked_reason. Redemption then fails withcode_invalid. - Revoking a
redeemedcode setsstatus='revoked', recomputes the units of its license and applies the transition in Section 15.3.3 (0 units → licenserevokedwithcode_revoked; 1 or 2 units → tier lowered to that rank and entitlement snapshots reset to that tier's current values). Unit counting uses onlyredeemedcodes. - Batch-level actions: "Revoke all unredeemed codes in this batch" (marketplace cancellation). Redeemed codes of that batch are not touched by this action.
- Marketplace refunds are handled by the marketplace; the operator revokes the specific code the marketplace reports. Each revoke requires a reason (1–200 characters) and a step-up re-authentication (Section 15.17.2).
15.8.6 Brute-force protection #
| Control | Value |
|---|---|
| Failed attempts per user and per client IP (counter key is an HMAC of the IP, never the raw IP) | Limited by the sliding-window policy code_redeem_fail of Section 17.6; above it 429 rate_limited with Retry-After equal to the seconds until the oldest counted failure leaves the window |
| Successful redemptions | Not counted as failures; limited by the policy code_redeem_success of Section 17.6 |
| Lockout escalation | 3 consecutive hourly lockouts for the same user within 24 h → audit code.bruteforce_suspected and an operator_alert email |
| Response uniformity | Missing, used and revoked codes return the same status, code and message; response time is padded to a minimum of 300 ms |
| Entropy | 60 bits makes online guessing infeasible under these limits |
The limits are the policies code_redeem_fail and code_redeem_success of Section 17.6 (sliding window); their counters live in Valkey under the rate-limit key family of that section.
15.8.7 Batch generation and CSV export (operator) #
POST /api/app/v1/admin/code-batches with {name, marketplace, tier_code, quantity, notes} → 201 {"data": {batch, "download_url": "/api/app/v1/admin/code-batches/{id}/export.csv"}}. Validation per 15.8.2. GET /api/app/v1/admin/code-batches/{id}/export.csv returns text/csv; charset=utf-8 with Content-Disposition: attachment; filename="codes-{batch-slug}-{YYYYMMDD}.csv".
CSV: RFC 4180, LF line endings, header row always present:
code,batch_name,marketplace,tier_label,created_at
QS-7K2M-9PQX-4TRD,Spring 2027 launch,Example Marketplace,solo,2027-03-01T10:15:00Ztier_label is the batch's tier code. Every export is audited (code_batch.exported, with row count). Exports of a batch are permitted any number of times by super admins. The formula guard of Section 14.2 applies to every cell (batch and marketplace names are operator-entered text). The export contains all codes of the batch regardless of status (a status column is not included, so the file is safe to hand to the marketplace).
15.9 Usage Metering and Cap Enforcement #
15.9.1 What is billable #
A billable event is every event accepted by ingestion (Section 8) whose name is pageview or any custom event name. engagement events are not billable and are not counted. Events dropped for any reason (bot, exclusion, unknown domain, hostname_not_allowed, spam referrer, cap, unlicensed, suspended, invalid) are not billable. Imported data (CSV, GA4) is not billable. A cap applies per team across all of its sites. The cap month is the UTC calendar month; "month key" is YYYYMM computed from the server's UTC clock at acceptance time.
15.9.2 Thresholds #
For a team with cap C (licenses.monthly_event_cap) and month count N:
| Name | Formula | Meaning |
|---|---|---|
warn_at |
ceil(0.8 * C) |
80% reached |
cap_at |
C |
100% reached |
hard_at |
floor(1.1 * C) + extra_allowance |
110% hard stop plus any allowance an operator granted for this month (Section 15.17.4; normally 0) |
| State | Condition | Ingestion |
|---|---|---|
ok |
N < warn_at |
Accept |
warning |
warn_at <= N < cap_at |
Accept |
grace |
cap_at <= N < hard_at |
Accept |
blocked |
N >= hard_at |
Drop (202), count as dropped_cap |
The check is accept if N < hard_at. Percentage shown in the UI is floor(100 * N / C), displayed as 999+ above 999.
15.9.3 Valkey counters (real-time path) #
For each team and month:
usage:v1:{team_id}:{YYYYMM}:base— integer: the sum of the team'susage_site_monthly.billable_eventsfor the month at the last reconciliation (Section 15.9.4), covering events beforeusage:v1:{team_id}:{YYYYMM}:cutoff.usage:v1:{team_id}:{YYYYMM}:cutoff— integer epoch seconds, a multiple of 300.usage:v1:{team_id}:{YYYYMM}:b:{bucket}— integer counters, one per 5-minute bucket (bucket = floor(epoch_seconds / 300) * 300) incremented byINCRBYfor each accepted billable event.usage:v1:{team_id}:{YYYYMM}:dcand:du— integers counting events dropped for cap and for unlicensed state.usage:v1:{team_id}:{YYYYMM}:extra— integer copy ofusage_monthly.extra_allowance, written by the console action and by reconciliation.- TTL on every key: 70 days from last write.
Current count N = base + sum(b:{bucket} for bucket >= cutoff). When base and cutoff are absent, base = 0 and cutoff = month start.
Write path: each web process accumulates increments in memory per team and flushes them with a single pipelined INCRBY every 1 s (and on graceful shutdown). The read path for enforcement caches N per team in process for 5 s and adds the process's own unflushed increments. This bounds overshoot to roughly the events accepted by all processes in a few seconds, which is covered by the 10% grace.
decideIngest returns, in order: drop_suspended if suspended; drop_unlicensed if the entitlement state is unlicensed; drop_cap if N >= hard_at; otherwise accept. Dropped events increment :du or :dc (batched like billable increments). The ingestion hook (owned by Section 8) calls decideIngest after bot and exclusion checks and before enrichment cost, and increments the billable counter only after the event is accepted into the buffer. If Valkey is unreachable the hook uses the last cached N for up to 60 s and then fails open (accept).
15.9.4 Reconciliation from ClickHouse #
Job usage-reconcile (every 10 minutes, Section 16.6). For the current UTC month, for every team that has an active license or any nonzero counter this month:
cutoff = floor((now − 600 s) / 300) * 300(at least 10 minutes old, so all buffered events have been flushed to ClickHouse).- Count per site, from the ledger
usage_site_monthly(Section 15.19.7), which has one row per(team_id, site_id, month). For every ledger row that is not frozen, recount from ClickHouse with one query per team:SELECT site_id, name = 'pageview' AS is_pv, count() FROM events WHERE name != 'engagement' AND timestamp < cutoff AND ((site_id = {s1} AND timestamp >= {counted_from1}) OR ...) GROUP BY site_id, is_pv(at most 100 site terms per team, using theeventstable of Section 6), and writebillable_events,pageviewsandcustom_eventsof the row. A site that has no row yet gets one withcounted_from = max(month start, the time the site joined the team). - Frozen rows. Site deletion and site transfer call
freezeSiteUsage(teamId, siteId)before the site's ClickHouse rows are purged or the site changes team: it counts that site's events for the current month up to now, stores the result in the row and setsfrozen_at. Frozen rows are never recomputed and never lowered, so a team cannot reduce its month by deleting a busy site or by transferring it to another team, and the receiving team starts a fresh row withcounted_from = transfer time(earlier events are not moved retroactively). A site that returns to a team in the same month reopens its frozen row (frozen_at = NULL,counted_fromunchanged), which can only over-count. base = SUM(billable_events)over all ledger rows of the team for the month, including frozen rows. Setbaseandcutoffin one ValkeyMULTIso readers never see a mixed pair.- Upsert
usage_monthlyfor (team_id, month):billable_events = base + sum(b >= cutoff),pageviewsandcustom_eventsfrom the same ledger sums,events_dropped_capandevents_dropped_unlicensedfrom:dc/:du,event_cap= current cap,reconciled_at = now. Delete bucket keys older thancutoff.
Reconciliation can lower an unfrozen row (events lost in a crashed buffer were never stored), never lowers a frozen row, and never raises beyond what ClickHouse plus recent buckets show. If a team's query fails, that team is skipped and the job logs warn; the job fails (and retries) only if more than 25% of teams fail.
Month close: job usage-month-close runs at 00:10 UTC on the 1st. For the previous month it performs one final full count with cutoff = month end, writes the final usage_monthly row with closed_at = now (the ledger rows already hold the final per-site counts), and deletes that month's Valkey keys. Teams whose license was revoked mid-month still get a row.
usage_monthly columns (authoritative DDL in Section 15.19.6):
| Column | Type | Rule |
|---|---|---|
id |
uuid | PK |
team_id |
uuid | FK ON DELETE CASCADE |
month |
date | First day of the UTC month |
billable_events |
bigint | default 0 |
pageviews |
bigint | default 0 |
custom_events |
bigint | default 0 |
events_dropped_cap |
bigint | default 0 |
events_dropped_unlicensed |
bigint | default 0 |
event_cap |
bigint | Cap in force at last update |
warned_80_at |
timestamptz null | Time the 80% email was claimed |
warned_100_at |
timestamptz null | Time the 100% email was claimed |
hard_stop_at |
timestamptz null | First time the state reached blocked this month |
extra_allowance |
bigint | Events added to this month's hard_at by an operator (default 0, ≥ 0; Section 15.17.4) |
closed_at |
timestamptz null | |
reconciled_at |
timestamptz null | |
created_at, updated_at |
timestamptz |
Unique index (team_id, month).
The per-site ledger usage_site_monthly and the full DDL of both tables are in Section 15.19.6 and 15.19.7.
15.9.5 Cap emails #
Job usage-cap-notify (every 5 minutes). For each licensed team with a cap, using the same count as enforcement:
- If
N >= warn_atandwarned_80_at IS NULL: claim and send the 80% email. - If
N >= cap_atandwarned_100_at IS NULL: claim and send the 100% email. (When a single run finds both unsent thresholds crossed, only the 100% email is sent and both flags are set.) - Set
hard_stop_atthe first timeN >= hard_at.
Claiming is atomic so concurrent workers cannot double-send:
UPDATE usage_monthly SET warned_80_at = now()
WHERE team_id = $1 AND month = $2 AND warned_80_at IS NULL
RETURNING id; -- send only when a row is returnedIf the claimed send later fails permanently (all email retries exhausted), the flag stays set; the banner still shows the state. Recipients are the team owner and all admins with verified email addresses. Emails are usage_cap_80 and usage_cap_100 (template inventory in Section 12). Content: team name, percentage, count, cap, reset date (resets_at), upgrade link {BASE_URL}/teams/{team_id}/billing; the 100% email also states that events are accepted until 110% (hard_at, stated as a number) and then dropped until the reset or an upgrade. No email is sent at the 110% hard stop; the banner covers it.
Once per month is absolute except one case: when the entitlement is raised by an upgrade or an additional code, flags are re-evaluated against the new cap: warned_80_at is cleared if N < ceil(0.8 * newCap), warned_100_at is cleared if N < newCap. A team can therefore receive at most one email per threshold per cap value per month.
15.9.6 Reset and cache invalidation #
Reset is implicit: counters are keyed by month, usage_monthly gets a new row on the first reconcile or notify run of the new month, and the in-process cache keys include the month. At 00:00:00 UTC on the 1st every team's state is ok (count 0) and events resume. Entitlement changes (grant, upgrade, revoke, reinstate, tier edits, suspension) invalidate caches through publishLicenseChanged described in Section 15.3.4.
15.9.7 Usage endpoint #
GET /api/app/v1/teams/{team_id}/usage?month=YYYY-MM (permission billing.view; default current month; months before the earliest usage_monthly row return zeros):
{ "data": {
"month": "2027-03", "billable_events": 84012, "pageviews": 70110, "custom_events": 13902,
"event_cap": 100000, "warn_at": 80000, "hard_at": 110000, "percent": 84,
"state": "warning", "resets_at": "2027-04-01T00:00:00Z",
"dropped_cap": 0, "dropped_unlicensed": 0,
"sites": [ { "domain": "example.com", "events": 51000 } ],
"history": [ { "month": "2027-02", "billable_events": 61000, "event_cap": 100000 } ]
} }sites lists the top 10 by events for the current month (ClickHouse, cached 60 s in Valkey) and is empty for past months. history has up to 12 previous months from usage_monthly. For an unlicensed team event_cap is 0, state is unlicensed, and percent is null.
15.10 Site Limit Enforcement #
15.10.1 Rule #
Any operation that adds a site to a team (create site in the UI, POST /api/v1/sites in the Sites API, accepting a site transfer into the team) takes a row lock on the team (SELECT ... FOR UPDATE), counts the team's existing sites, and fails with 409 site_limit_reached when count >= limit. The limit is licenses.max_sites for an active license and 1 for an unlicensed team. All sites count, including sites that currently receive no events. Sites in a team scheduled for deletion are not special-cased (the team is read-only anyway).
Error body: {"error": {"code": "site_limit_reached", "message": "Your plan allows 5 sites. Upgrade to add more.", "details": {"limit": 5, "current": 5, "tier": "solo", "upgrade_url": "/teams/<id>/billing"}, "request_id": "req_..."}}. For an unlicensed team the message is "Activate a license to add more sites." and tier is null.
15.10.2 UI behavior #
The "Add site" button stays enabled; submitting past the limit shows an inline error with an Upgrade button (owner) or the text "Ask the team owner to upgrade." (other roles). The sites list header shows 3 of 5 sites for roles with billing.view and nothing for other roles.
15.10.3 Over-limit state after revocation #
When an operator lowers a license (code revocation, operator tier change) so that site count > max_sites, no sites are deleted or disabled. The team is "over site limit": new sites are refused, existing sites keep collecting events, and the banner over_site_limit shows (Section 15.12.2). The state clears when the count drops to the limit or below.
15.11 Unlicensed Behavior #
An unlicensed hosted team (never licensed, or its license is revoked):
| Capability | Behavior |
|---|---|
| Sign-up, email verification, login, 2FA | Fully available; sign-up creates the personal team per Section 11 with no license |
| Sites | At most 1 site; the snippet and setup guides are shown normally |
| Ingestion | Every event for the team's sites is dropped; POST /api/event still returns 202 |
| Dashboards, reports, filters, segments, API reads, CSV exports | Readable until the purge (Section 15.14); empty for never-licensed teams |
| Data import (CSV, GA4), starting a new import | Blocked: 403 license_required. Section 14.4.1 checks it from the request headers and query parameters before the upload body is read |
| API keys, goals, funnels, shared links, email reports | Allowed to configure (settings only); email reports for empty periods are skipped by Section 12 |
| External email-report recipients | Adding one requires an active license: 403 license_required (Section 12.5.4) |
| Stats API reads | Allowed |
| Banner | unlicensed (never licensed) or license_revoked (revoked), Section 15.12.2 |
| Billing page | Plan chooser plus the redeem-code card |
There is no free trial and no free tier. An unlicensed team that has never held a license cannot accumulate data, so no purge applies to it.
15.12 License Summary, Banners and Billing UI #
15.12.1 License summary endpoint #
GET /api/app/v1/teams/{team_id}/license (permission billing.view). Hosted only.
{ "data": {
"state": "active",
"license_id": "<uuid>|null",
"tier": { "code": "pro", "name": "Pro" },
"source": "stripe",
"max_sites": 25, "sites_used": 7,
"monthly_event_cap": 500000,
"granted_at": "2027-01-04T12:00:00Z",
"refund_eligible_until": "2027-03-05T12:00:00Z",
"units": null,
"units_max": null,
"revoked_at": null, "revoked_reason": null, "purge_after": null,
"banner": { "key": "none", "severity": "info", "params": {} }
} }Field rules: tier, source, max_sites, monthly_event_cap, granted_at are null when unlicensed. refund_eligible_until is paid_at + REFUND_WINDOW_DAYS days (default 60, Section 16.4) of the earliest refundable purchase, or null when none (also null for codes and operator licenses). units and units_max (3) are populated only for codes licenses. For revoked state, revoked_* and purge_after come from the latest revoked license. The same banner object is included in the app bootstrap response for roles that may see banners, with reduced params for roles without billing.view.
15.12.2 Banner states #
At most one banner is shown; the highest priority wins. Banners appear at the top of every authenticated page inside a team context, below the app header, and on the team's site dashboards.
| Priority | key |
Condition | Severity | Text (billing roles) | CTA (owner only) | Text (other roles) | Dismissible |
|---|---|---|---|---|---|---|---|
| 1 | team_suspended |
Team suspended by operator | error | "This team is suspended and is not collecting data. Contact {SUPPORT_EMAIL}." | none | "This team is suspended. Contact the team owner." | No |
| 2 | blocked |
State blocked (N >= hard_at) |
error | "You have reached 110% of your monthly limit. Events are being dropped until {resets_at date} or until you upgrade." | "Upgrade" | "Data collection is paused for this team. Contact the team owner." | No |
| 4 | license_revoked |
Revoked license, not purged | error | "Your license is no longer active. Data collection is off. Your data will be deleted on {purge_after date} unless you activate a license." When purge_after is null because of a payment dispute: "Your license is on hold because of a payment dispute. Data collection is off and your data is kept. Contact {SUPPORT_EMAIL}." |
"Activate a license" | "Data collection is off for this team. Contact the team owner." | No |
| 5 | unlicensed |
Never licensed | warning | "Activate a license to start collecting data." | "Activate a license" | "Data collection is off for this team. Contact the team owner." | No |
| 6 | grace |
State grace |
warning | "You have passed 100% of your monthly limit ({n} of {cap} events). Events are still collected up to {hard_at}; after that they are dropped until {resets_at date}." | "Upgrade" | none | No |
| 7 | warning |
State warning |
info | "You have used {percent}% of your monthly events." | "View usage" | none | Yes, for the browser session |
| 8 | over_site_limit |
Site count > max_sites |
warning | "Your team has {current} sites but your plan allows {limit}. You cannot add sites." | "Upgrade" or none for codes source: "Redeem a code" |
none | No |
| 3 | purge_soon |
Revoked, purge_after within 7 days |
error | Replaces license_revoked text: "Your data will be deleted in {days} days." |
"Activate a license" | as license_revoked |
No |
purge_soon has higher priority than license_revoked when its condition holds. Banners for roles without billing.view never include counts or amounts. The SPA renders severity as: info = neutral, warning = amber, error = red, each with icon and role="status" (error uses role="alert"), per the design system in Section 4.
15.12.3 Billing page /teams/:teamId/billing #
Visible to billing.view roles; actions require billing.manage. Sections from top to bottom:
- Status panel (when
checkout=success): the confirmation panel of Section 15.6.3. - Plan card. Unlicensed: heading "Choose your lifetime plan" and three tier cards (name, price, "One-time payment", sites, events per month, "All features included", Buy button). Active: tier name, source label ("Purchased", "Marketplace codes", "Granted by support"), granted date,
sites_used / max_sites, refund note "Refund available until {date}. Contact {SUPPORT_EMAIL}." when eligible (a mailto link with subject "Refund request — team {team_id}"). - Usage meter (licensed only): horizontal bar from 0 to
hard_at, tick marks at 80%, 100% and 110% of the cap, fill color ok = neutral, warning = amber, grace/blocked = red, label "{n} of {cap} events this month ({percent}%)", text "Resets on {resets_at date} (UTC)", a table of the top 10 sites, a 12-month history bar chart, and dropped counters shown only when nonzero ("{x} events dropped this month"). - Upgrade card (Stripe source): the higher tiers with "Pay {due} to upgrade" buttons. For codes source: "{units} of 3 codes redeemed" and the redeem form. For operator source: "Contact support to change your plan."
- Redeem a code card (unlicensed or codes source): single input with placeholder
QS-XXXX-XXXX-XXXX, auto-uppercase, accepts pasted lower-case and spaces, Submit button "Redeem". Errors:code_invalid→ "This code is not valid or has already been used.";code_stack_limit_reached→ "This team already has the maximum of 3 codes.";license_source_conflict→ "This team already has a purchased license.";rate_limited→ "Too many attempts. Try again in {n} minutes." - Invoices link to
/teams/:teamId/billing/invoices.
Loading state: skeletons. Error state: inline "Could not load billing information" with Retry. Buttons that start Stripe flows disable themselves after click and show a spinner; if the API fails, show "Payments are temporarily unavailable. Try again shortly." (billing_unavailable).
15.12.4 Invoices page /teams/:teamId/billing/invoices #
Table (cursor pagination per Section 4, default 50): Invoice number, Date (paid_at, YYYY-MM-DD in UTC), Description (for example "Lifetime license — Pro"), Amount (total_minor, USD), Status chip (Paid, Partially refunded, Refunded, Disputed), PDF button; refunded rows add a second link "Credit note". Empty state: "No invoices yet." Endpoints:
GET /api/app/v1/teams/{team_id}/billing/invoices(permissionbilling.view) returns purchases withstatusin (paid,partially_refunded,refunded,disputed) and a non-nullinvoice_number, newest first.GET /api/app/v1/teams/{team_id}/billing/invoices/{purchase_id}/pdfand.../credit-note/pdf(permissionbilling.manage) returnapplication/pdfwithContent-Disposition: attachment; filename="{invoice_number}.pdf"(credit note:{credit_note_number}.pdf),Cache-Control: private, no-store. A purchase from another team returns404 not_found; a missing credit note returns404.
15.12.5 Activation gate and redeem deep link #
/activate: after first login of a hosted user whose every team is unlicensed and who has never dismissed the gate, the SPA shows a full page "Activate {PRODUCT_NAME}" with the plan cards and redeem form for the personal team (a team picker appears when the user owns several unlicensed teams). The link "Skip for now" goes to the sites list and records the dismissal in the browser session only. The gate is shown at most once per login session./redeem?code=QS-...: requires login (redirect to login with anextpath, per the redirect rules in Section 17.4), pre-fills the code, and shows the team picker listing teams where the user is owner; submitting calls the redeem API. Marketplaces link to this URL. On load the SPA movescodeinto memory and removes it from the address bar withhistory.replaceState; the page is served withReferrer-Policy: no-referrer, and server logs never contain query strings (Section 17.8).
15.13 Refund Policy #
15.13.1 Rules #
| Rule | Decision |
|---|---|
| Window | REFUND_WINDOW_DAYS days (default 60, Section 16.4) from the purchase's paid_at. The window is a console rule only: a charge.refunded event for any purchase is applied as in Section 15.6.6 whether or not the window has passed |
| Who initiates | Operator only, in the console (Section 15.17.7), or directly in the Stripe Dashboard. Customers request by email via the mailto link on the billing page. |
| Amount | Full refund of the purchase (tax included). The console does not issue partial refunds; partial refunds made in Stripe are recorded but do not change the license. |
| After the window | Refused (422 refund_window_closed) unless the operator ticks "Override window" and gives a reason; the override is audited |
| Marketplace codes | Refunded by the marketplace; the operator revokes the code (Section 15.8.5). No Stripe refund exists. |
| Upgrades | Each upgrade purchase has its own REFUND_WINDOW_DAYS window |
| Order | Upgrades are refunded in reverse order: the console refuses to refund a purchase while a later non-refunded upgrade purchase exists on the same license (422 refund_upgrade_first) |
| Method | Stripe Refunds API, payment_intent = the purchase's intent, reason: 'requested_by_customer', metadata purchase_id, idempotency key rf_<purchase_id>_<refund_request_id> (Section 15.4) |
15.13.2 Console flow #
The console generates a refund_request_id (UUID), writes the audit row purchase.refund_requested containing it, then calls Stripe with the idempotency key of Section 15.4, and shows "Refund requested. The license changes when Stripe confirms." A retry from the console reuses the same refund_request_id. The license effect is applied by the charge.refunded handler (Section 15.6.6), the single code path. The console's purchase detail page refreshes every 5 s for 60 s while the refund is pending.
15.13.3 License effects of a full refund #
| Refunded purchase | Effect |
|---|---|
kind='new' and no other non-refunded purchase on the license |
License revoked (refunded), revoked_at = now, purge_after = now + 30 d; email refund_processed (its content states that the license was revoked) |
kind='upgrade' and it is the latest non-refunded purchase on the license |
License reverts to from_tier (snapshots reset to that tier's current max_sites, monthly_event_cap; credit_minor reset to from_tier.price_minor); site over-limit handling per Section 15.10.3; email refund_processed |
kind='upgrade' that is not the latest (refunded out-of-band in Stripe) |
The license is set to the tier_id of the latest remaining non-refunded purchase of the license (snapshots and credit_minor as above); the system emails all super admins operator_alert naming the purchases |
kind='new' refunded out-of-band in Stripe while a paid upgrade exists |
License revoked as above; the system emails all super admins operator_alert listing the remaining paid upgrade purchase ids for manual refund |
The refunded purchase keeps its invoice; a credit note is issued (Section 15.15.3). A team with a revoked license can purchase again immediately: a new license row and a new purchase are created; no data restore occurs for data already purged.
15.14 Unlicensed Data Purge #
15.14.1 Rule #
A revoked license sets purge_after = revoked_at + 30 days, except a dispute revocation, which sets purge_after = NULL (a hold, Section 15.14.3). If at purge_after the team still has no active license and purged_at IS NULL, the job unlicensed-purge (daily 03:00 UTC) deletes all analytics data of every site of the team. The job skips every license whose purge_after IS NULL.
15.14.2 What is purged #
Deleted: the native rows of the team's site ids in the ClickHouse tables events and sessions (through site-purge with scope native) and the imported rows (by enqueueing import-purge for every import of the team's sites, Section 16.6); Valkey keys of those sites; exports files of the team. Kept: the sites themselves and their settings, goals, funnels, segments, shared links, API keys, members, annotations, the license, purchases, invoices, usage rows and audit rows. Afterward licenses.purged_at = now, audit license.data_purged, and the owner receives unlicensed_purge_completed.
15.14.3 Warnings, holds and edge cases #
- Warning emails
unlicensed_purge_warningare sent to the owner and admins by the jobunlicensed-purge-warn(daily 09:00 UTC) whenpurge_afteris 7 days away (purge_after - nowbetween 6 and 7 days) and 1 day away (between 0 and 1 day), withdays_left= 7 or 1. Each is sent at most once per license (tracked in audit aslicense.purge_warning_sentwithdays_left). - Operator hold: the console action "Extend purge date" accepts a new date up to 90 days after
revoked_at; further extension requires repeated use of the action; every use is audited with a reason. - Dispute hold: a license revoked for
disputehaspurge_after = NULL, so ingestion is dropped but no data is deleted while the dispute is open. The operator ends the hold with one of two console actions (Section 15.17.5): reinstate the license (dispute won), or "Close dispute as lost", which setspurge_after = now + 30 dand starts the normal warning schedule. - If the team obtains a new active license before
purge_after, the older revoked row is ignored by the job and its data is retained. - If the job runs while a purchase is
pendingfor the team, it skips the team and logsinfo; it retries the next day. - The job processes at most 200 teams per run and enqueues the per-site deletion as separate retryable work. Failure for one team does not stop others.
- A purge cannot be undone and backups age out per Section 17.10.
15.15 Invoices and Credit Notes #
15.15.1 Numbering #
Invoice number format {INVOICE_PREFIX}-{YYYY}-{NNNNNN} for example QS-2026-000123. INVOICE_PREFIX is an env var (Section 16.4, default QS, 1–6 uppercase letters or digits). YYYY is the UTC year of issue; NNNNNN is a zero-padded sequence starting at 000001 for each year, gap-free. Allocation happens in the fulfillment transaction:
SELECT pg_advisory_xact_lock(7019001); -- serializes allocation
SELECT COALESCE(MAX(invoice_seq), 0) + 1 AS next
FROM purchases WHERE invoice_year = $year;An invoice number is allocated only for a purchase that becomes paid and has a license (never for auto-refunded duplicates or expired rows). Numbers are never reused or edited. seller_snapshot captures the SELLER_* values at issue time so reissued PDFs are identical.
15.15.2 PDF content #
Generated server-side on demand from stored fields by apps/server/src/modules/billing/invoice-pdf.ts using the pure-JavaScript PDF library listed in Section 3.1 (pdf-lib). Output is A4, portrait, English, deterministic, with embedded standard fonts, under 100 KB.
| Block | Content |
|---|---|
| Header | Seller name, address lines, country, seller tax ID (label "VAT / Tax ID") when set, seller email; title "Invoice"; invoice number; issue date; payment date |
| Bill to | billing_name (fallback: purchaser name), billing_email, address lines, buyer tax IDs |
| Line item | One row: description, quantity 1, unit price, amount. New: "{PRODUCT_NAME} Lifetime License — {Tier} ({max_sites} sites, {cap} events per month)". Upgrade: "{PRODUCT_NAME} Lifetime License upgrade — {From} to {To} (price difference)" |
| Totals | Subtotal; tax line with rate when tax_minor > 0 (rate computed as tax_minor / amount_minor rounded to 2 decimals, labeled "Tax"); Total; "Amount paid"; "Amount due: $0.00" |
| Payment | "Paid by {payment_method_summary} on {paid_at}" and the Stripe payment intent id |
| Footer | "One-time payment. No recurring charges." plus INVOICE_FOOTER_NOTE when set (≤ 300 characters) |
| Refunded purchases | Invoice PDF shows a diagonal-free text line "Refunded in full on {date} — see credit note {credit_note_number}" below totals |
Money is formatted $59.00 (en-US, USD). Dates are ISO YYYY-MM-DD in UTC.
15.15.3 Credit notes #
When a purchase becomes refunded and credit_note_number IS NULL, a credit note number {INVOICE_PREFIX}-CN-{YYYY}-{NNNNNN} is allocated with the same gap-free method, using the columns credit_note_year and credit_note_seq under advisory lock 7019002 (MAX(credit_note_seq) + 1 for the year), and stored in credit_note_number. The credit note PDF mirrors the invoice with the title "Credit note", references the original invoice number, and shows negative amounts. Partial refunds do not produce credit notes, and a purchase without an invoice (the automatic duplicate refund) produces none. A repeated charge.refunded never allocates a second credit note.
15.15.4 Seller details #
From env vars (Section 16.4): SELLER_NAME, SELLER_ADDRESS_LINE1, SELLER_ADDRESS_LINE2, SELLER_CITY, SELLER_REGION, SELLER_POSTAL_CODE, SELLER_COUNTRY (ISO 3166-1 alpha-2), SELLER_TAX_ID, SELLER_EMAIL, INVOICE_PREFIX, INVOICE_FOOTER_NOTE. In the hosted edition the application refuses to start when SELLER_NAME, SELLER_ADDRESS_LINE1, SELLER_CITY, SELLER_POSTAL_CODE, SELLER_COUNTRY or SELLER_EMAIL is missing while STRIPE_SECRET_KEY is set.
15.16 Team Deletion With a License #
- The delete-team confirmation (Section 11) shows, for teams with an active license: "This team has a {tier} lifetime license. Deleting the team permanently forfeits it. No refund is issued automatically." The owner must type the team name to confirm (Section 11).
- During the 7-day undo window the license stays
activeand ingestion is dropped per Section 11.14.5; undo restores the team entirely and collection resumes. - When the team-purge job hard-deletes the team (Section 16.6): the license is set to
revokedwith reasonteam_deleted,team_idbecomes null through the FK,team_name_snapshotis kept; codes stayredeemed(not reusable); purchases remain withteam_idnull and their invoices remain downloadable by super admins only; usage rows (usage_monthlyandusage_site_monthly) cascade-delete with the team; audit rows with actionslicense.*,purchase.*,code.*,admin.*andteam.purgedkeep theirteam_id(no foreign key) untilAUDIT_RETENTION_MONTHS. - An operator may still refund such a purchase inside its
REFUND_WINDOW_DAYSwindow (Section 15.13); the refund has no further license effect. - Webhooks arriving for a deleted team follow Section 15.6.4 step 5 (automatic refund for
new; forcharge.refundedthe effect is applied to the already-revoked license, a no-op).
15.17 Operator Console #
15.17.1 Access #
Routes live in the SPA under /admin and call /api/app/v1/admin/* (dashboard API family: cookie session plus CSRF, envelopes per Section 4). Only hosted. A user is a super admin when their email (case-insensitive) is in SUPER_ADMIN_EMAILS (comma-separated), that email was verified by consuming a verification token (never by the no-SMTP shortcut), and the user has TOTP enabled. SUPER_ADMIN_EMAILS is read at process start (Section 16.4.1) and the check runs on every request against that list, with no stored flag, so removing an address takes effect when the web processes restart. Non-super-admins get 404 not_found for every /api/app/v1/admin/* route and the SPA hides the Admin link. Admin pages never display visitor analytics data (no pageviews, sources or paths of customer sites); they show account, license, payment and aggregate usage data only.
Requirements: super admins must have TOTP enabled (Section 11). Without it, every admin route returns 403 admin_2fa_required and the SPA redirects to the 2FA setup page. Admin routes are limited by policy admin_api (Section 17.6).
15.17.2 Step-up re-authentication #
Destructive and sensitive actions require a fresh step-up within the last 15 minutes: revoke or reinstate a license, close a dispute as lost, move a license, change a tier, grant a license, grant a monthly allowance, refund, revoke a code or batch, suspend or unsuspend a user or team, force logout, reset 2FA, impersonate, download user data, export a code batch, export the audit log, edit tiers. Without it the API returns 403 reauth_required; the SPA shows a dialog asking for the current password and a TOTP code (or recovery code), which calls POST /api/app/v1/admin/reauth with {"password": "...", "code": "..."}. Wrong values count against policy admin_reauth (Section 17.6); success stores reauth_at on the auth session, TTL 15 minutes. Every such action also requires a reason string (1–500 characters) in the request body, stored in the audit entry.
15.17.3 Users (/admin/users, /admin/users/:id) #
List: search by email substring, name substring or exact user id; filters: verified (yes/no), suspended (yes/no), 2fa (yes/no); sort by created_at desc (default), last_login_at; columns: name, email, verified, 2FA, teams count, created, last login, status. Detail: profile fields, teams with roles, active session count, recent audit entries about the user (50), pending email tokens (type and expiry only).
Actions (all audited with reason): Resend verification email (no step-up); Force logout (deletes all auth_sessions); Suspend (sets users.suspended_at; login is refused with 403 account_suspended after the password check, sessions deleted) and Unsuspend; Reset 2FA (step-up and reason; deletes the user's TOTP secret and recovery codes and all their sessions, sends the two_factor_disabled email and writes audit admin.user_2fa_reset; the dialog states "Verify the person's identity out of band before resetting."); Impersonate (Section 15.17.11); Download user data (JSON of Section 17.9.3, step-up required). The console cannot read passwords or TOTP secrets, cannot change emails and cannot delete users (deletion is self-service per Section 11); resetting 2FA is its only credential action. Suspending a super admin is refused with 403 forbidden.
15.17.4 Teams (/admin/teams, /admin/teams/:id) #
List: search by team name, owner email or id; filter by license state (none, active, revoked), tier, suspended, deletion_scheduled; columns: name, owner email, members, sites, tier, license state, usage % this month, created. Detail: members and roles, sites (domain, created; no analytics), license card, 12-month usage chart, purchases, audit entries for the team.
Actions: Grant license (team without active license; choose tier, optional note; step-up required; creates source='operator', credit_minor=0); Suspend team (sets teams.suspended_at and suspended_reason; ingestion dropped as drop_suspended, banner team_suspended; no purge clock; dashboards remain readable) and Unsuspend; Grant monthly allowance (step-up and reason; adds N events, 1 to 10,000,000,000, to this month's hard_at through usage_monthly.extra_allowance, for example after another party flooded the team's domain with fake events; audit license.allowance_granted); Recompute usage now (runs the reconcile for the team immediately). Suspension is the abuse tool of Section 17.14.
15.17.5 Licenses (/admin/licenses, /admin/licenses/:id) #
List: filter by status, tier, source, refund_eligible (yes/no), search by team name or owner email; columns: team, tier, source, status, granted, revoked reason, purge date. Detail: all fields of Section 15.3.1, purchases, codes, timeline from audit.
Actions (step-up and reason required):
| Action | Rules |
|---|---|
| Revoke | Active only; reason operator_revoked; sets purge_after = now + 30 d |
| Reinstate | Revoked only; the team must have no active license and must not be purged (purged_at IS NULL; otherwise 422 license_purged since data is gone, in which case grant a new license instead); reinstating a dispute-held license ends the hold |
| Change tier | source='operator' licenses freely; codes and stripe licenses are refused with 422 license_source_conflict (use code revoke or refunds) |
| Move license to another team | Active only; the target team must have no active license, not be deleted and not be suspended. In one transaction the license row's team_id, team_name_snapshot and owner_email_snapshot are updated to the target, and the source team receives a copy-less state: a new revoked license row is inserted for the source team with reason operator_revoked, purge_after = now + 30 d, so the source team follows the normal unlicensed rules. Redeemed codes and purchases follow the license to the target team. Site-limit over-count on the target is handled by Section 15.10.3. |
| Extend purge date | Revoked only; per Section 15.14.3 |
| Close dispute as lost | Revoked with reason dispute only; sets purge_after = now + 30 d; audit license.dispute_closed_lost |
15.17.6 Code batches (/admin/code-batches, /admin/code-batches/:id) #
List: name, marketplace, tier label, quantity, redeemed count, revoked count, created by, created at. Create form fields per Section 15.8.2 with inline validation. Detail: counters (unredeemed/redeemed/revoked), a paginated codes table (code, status, redeemed team and user email, redeemed at), search by code, buttons Export CSV (Section 15.8.7), Revoke selected, Revoke all unredeemed. Per-code revoke and batch revoke follow Section 15.8.5.
15.17.7 Purchases and refunds (/admin/purchases, /admin/purchases/:id) #
List: filter by status, kind, date range, search by invoice number, buyer email, Stripe ids; columns: invoice number, date, team, buyer, tier, kind, total, refunded, status. Detail: all stored fields, links to the Stripe Dashboard (payment, customer, dispute), invoice and credit note PDFs, refund eligibility ("Eligible until {date}" / "Window expired"). Actions: Refund in full (Section 15.13; requires step-up, reason, optional "Override window"), Download invoice. Totals strip above the list: gross, refunded and net for the filtered range in USD.
15.17.8 Tiers (/admin/tiers) #
Table of the three tiers with inline edit of name, price_minor (entered as dollars with 2 decimals), max_sites, monthly_event_cap, is_active, sort_order, plus the "apply to existing" checkbox of Section 15.2.3. Validation per Section 15.2.2. Save requires step-up and a reason; audit tier.updated records before and after values. Price changes show a warning: "Price changes affect new purchases only."
15.17.9 Usage (/admin/usage) #
Month selector (current and past months). Table of teams from usage_monthly joined with live counts for the current month: team, tier, billable events, cap, percent, state, dropped counts, with sorting by percent (default desc) and filters state (warning, grace, blocked, unlicensed with dropped events). Totals header: total billable events across all teams, number of teams per state. The page reads Postgres and Valkey only; it runs no ClickHouse queries except the on-demand "Recompute usage now" per team.
15.17.10 Audit (/admin/audit-log) #
Reads audit_log (Section 5; action names in Section 15.18.2 and Section 11.20): filters by actor, action, target type and id, team, date range; columns: time (UTC), actor (email), on-behalf-of user when set, action, target, reason, metadata (collapsed JSON). CSV export of the filtered view (max 100,000 rows; requires step-up; the formula guard of Section 14.2 applies; audited as audit.exported). Audit rows never contain IP addresses or user agents. Rows are append-only: no update or delete route exists; the retention job audit-prune removes rows older than AUDIT_RETENTION_MONTHS (Section 16.4).
15.17.11 Impersonation #
POST /api/app/v1/admin/users/{id}/impersonate body {"reason": "<10–500 chars>"}. Requires step-up. Rules:
| Rule | Decision |
|---|---|
| Targets | Any non-super-admin, non-suspended user; super admins cannot be impersonated (403 forbidden) |
| Session | A new auth_sessions row for the target user with impersonator_user_id set and a fixed 30-minute lifetime (no sliding renewal). It is carried by its own cookie __Host-qs_imp (same attributes as the session cookie, Max-Age=1800), which takes precedence over the normal session cookie when present, so the operator's own cookie and session are untouched. Rows with impersonator_user_id are excluded from the target's session list, from "revoke all others" and from the 50-session cap: the target can neither see nor revoke them, and they never evict a real session |
| Operator check | Every request on an impersonation session re-checks that impersonator_user_id is still a super admin with TOTP enabled; otherwise the server deletes the session and answers 401 unauthenticated |
| Read-only | Allowed: GET, HEAD, OPTIONS, POST /api/app/v1/admin/impersonation/stop, and routes flagged readOnly: true in the route registry (POST /api/app/v1/sites/{domain}/query, POST /api/app/v1/sites/{domain}/query/batch and the read endpoints of funnels and journeys, so dashboards load). Only actions marked read-only for impersonation in the permission matrix (Section 11.19) are authorized. Every other request returns 403 impersonation_read_only |
| Hidden data | The API never returns secrets even normally (password hashes, TOTP secrets, API key secrets, share-link passwords); additionally, impersonation sessions cannot call the data-export endpoints, create exports, or view API keys list values (the list shows names and prefixes only as for the user) |
| UI | A persistent top bar on every page: "Viewing as {name} ({email}) — read-only. Exit" with a countdown. Exit calls the stop endpoint, which deletes the impersonation session and returns to /admin/users/:id |
| Audit | admin.impersonation_started (target, reason), admin.impersonation_ended or admin.impersonation_expired; each rejected write is counted (metric quietstat_impersonation_blocked_total) and not individually audited |
| Notification | The impersonated user receives the email impersonation_notice (at most one per operator per user per 24 hours) stating that support viewed their account, when, and for what stated reason |
| Rate | Policy impersonation_start (Section 17.6) |
15.17.12 Overview (/admin) #
Cards: total users, total teams, active licenses by tier (3 numbers), gross revenue / refunds / net for the last 30 days and all time (USD), events accepted this month (sum of usage_monthly.billable_events plus live), teams in warning/grace/blocked, Stripe webhook failures in the last 24 h (from the Valkey counter behind quietstat_stripe_webhook_total{result="error"}), pending purchases older than 1 h, and failed jobs in the last 24 h (pg-boss failed count). Each card links to its list page. All counts are computed on request with a 60-second Valkey cache.
15.17.13 Admin API summary #
All under /api/app/v1/admin, super admin only, envelopes per Section 4. Lists use cursor pagination.
| Method and path | Purpose |
|---|---|
POST /reauth |
Step-up; body { "password", "code" } |
GET /overview |
Overview cards |
GET /users, GET /users/{id} |
List, detail |
POST /users/{id}/resend-verification, /force-logout, /suspend, /unsuspend, /reset-2fa, /impersonate |
Actions |
GET /users/{id}/data-export |
JSON per Section 17.9.3 |
POST /impersonation/stop |
End impersonation |
GET /teams, GET /teams/{id} |
List, detail |
POST /teams/{id}/grant-license, /suspend, /unsuspend, /grant-allowance, /recompute-usage |
Actions |
GET /licenses, GET /licenses/{id} |
List, detail |
POST /licenses/{id}/revoke, /reinstate, /change-tier, /move, /extend-purge, /close-dispute |
Actions |
GET /code-batches, POST /code-batches, GET /code-batches/{id}, GET /code-batches/{id}/codes, GET /code-batches/{id}/export.csv |
Batches |
POST /code-batches/{id}/revoke-unredeemed, POST /codes/{code_id}/revoke |
Revocation |
GET /purchases, GET /purchases/{id}, POST /purchases/{id}/refund, GET /purchases/{id}/invoice.pdf |
Purchases |
GET /tiers, PATCH /tiers/{id} |
Tiers |
GET /usage?month=YYYY-MM |
Usage table |
GET /audit, GET /audit/export.csv |
Audit (export requires step-up) |
POST /test-email |
Send the SMTP test message to the operator's own address (policy system_test_email, Section 17.6; behaviour in Section 12.9.7) |
Every mutating action returns 200 {"data": <updated resource>} and writes an audit row in the same transaction.
15.18 Reference Tables #
15.18.1 Error codes owned by this section #
| HTTP | error.code |
When |
|---|---|---|
| 400 | invalid_signature |
Webhook signature failure |
| 403 | license_required |
Import or other data-creating action by an unlicensed team; adding an external email-report recipient without an active license |
| 403 | admin_2fa_required |
Super admin without TOTP |
| 403 | reauth_required |
Step-up missing or older than 15 minutes |
| 403 | impersonation_read_only |
Write during impersonation |
| 403 | account_suspended |
Login of a suspended user |
| 403 | setup_required |
Signup attempted while no user exists (Section 16.3.5) |
| 403 | setup_token_invalid |
Wrong setup token (Section 16.3.5) |
| 409 | site_limit_reached |
Section 15.10 |
| 409 | license_already_active |
Checkout for a team with an active license, or an upgrade to the same tier |
| 409 | license_source_conflict |
Purchase or code flow incompatible with the license source |
| 409 | code_stack_limit_reached |
Redeeming a 4th code |
| 409 | setup_already_completed |
POST /setup when a user already exists |
| 422 | code_invalid |
Malformed, unknown, used or revoked code (identical response) |
| 422 | downgrade_not_allowed |
Upgrade to a lower tier |
| 422 | invalid_upgrade_target |
Inactive tier, or due amount below $1.00 |
| 422 | refund_window_closed |
Section 15.13 |
| 422 | refund_upgrade_first |
Section 15.13 |
| 422 | license_purged |
Reinstating a purged license |
| 422 | tier_constraint_violation |
Tier edit that breaks the ordering rules of Section 15.2.2 |
| 503 | billing_unavailable |
Stripe API failure |
Every code in this table is a row of the Section 4.4.3 catalogue. Cases without a dedicated code use forbidden, conflict or validation_failed with a details.reason.
15.18.2 Audit actions written by this section #
license.granted, license.upgraded, license.tier_changed, license.revoked, license.reinstated, license.moved, license.purge_extended, license.purge_warning_sent, license.data_purged, license.dispute_closed_lost, license.allowance_granted, purchase.created, purchase.paid, purchase.refund_requested, purchase.partially_refunded, purchase.refunded, purchase.auto_refunded, purchase.disputed, code.redeemed, code.revoked, code.bruteforce_suspected, code_batch.created, code_batch.exported, code_batch.revoked_unredeemed, tier.updated, team.suspended, team.unsuspended, user.suspended, user.unsuspended, user.force_logout, admin.user_2fa_reset, admin.impersonation_started, admin.impersonation_ended, admin.impersonation_expired, audit.exported. Each row stores actor_type (user, super_admin, impersonation, api_key, system), actor_user_id (null for system/webhook), on_behalf_of_user_id (the impersonated user when actor_type is impersonation), action, target_type, target_id, team_id, reason (1–500 characters, required for console actions), and metadata (JSON, ids and numbers only, no email addresses beyond the console-visible actor, never IP or user agent).
15.18.3 Emails triggered by this section (templates listed in Section 12) #
| Template | Trigger | Recipients |
|---|---|---|
license_activated |
First license grant by purchase or first code | Purchaser / redeemer |
license_upgraded |
Upgrade paid, or 2nd/3rd code redeemed | Purchaser / redeemer |
license_revoked |
License revoked (refund, dispute, code, operator) | Owner |
refund_processed |
Full refund applied | Purchaser |
purchase_auto_refunded |
Duplicate or invalid-state payment refunded | Purchaser |
usage_cap_80, usage_cap_100 |
Section 15.9.5 | Owner and admins |
unlicensed_purge_warning |
7 days and 1 day before purge | Owner and admins |
unlicensed_purge_completed |
After purge | Owner |
impersonation_notice |
Section 15.17.11 | Impersonated user |
operator_alert |
Dispute, auto-refund, brute-force suspicion, out-of-band refund anomalies | All super admins |
15.18.4 Jobs owned by this section (schedules in Section 16.6) #
usage-reconcile, usage-cap-notify, usage-month-close, unlicensed-purge, unlicensed-purge-warn, purchase-pending-expire (marks pending purchases older than 24 h expired, hourly; a later paid session re-opens the row, Section 15.6.4), stripe-reconcile (daily safety net: lists Checkout Sessions created in the last 72 hours with status=complete and runs fulfillCheckoutSession for any whose purchase is still pending or is expired).
15.18.5 Edge-case decisions (summary) #
| Case | Decision |
|---|---|
| Duplicate webhook delivery | Idempotent by design (Section 15.6.5); returns 200 |
| Webhook before purchase row commit | Cannot happen: the pending row is committed before the Stripe session is created |
| Two tabs buy at once | Step 2 of Section 15.6.2 settles the earlier pending session first (fulfilling it when it is already paid); if both end up paid anyway, the second is refunded automatically (Section 15.6.4 step 5) |
| Customer pays after the 30-minute session expiry | Not possible; Stripe refuses expired sessions |
| Team deleted with license | Section 15.16 |
| License transfer between teams by customers | Not allowed; operator-only move (Section 15.17.5); team ownership transfer carries the license |
| Stripe license and a marketplace code on the same team | Refused (license_source_conflict) |
Upgrade while in blocked state |
Ingestion resumes within 5 s; flags re-evaluated |
| Currency | USD only everywhere in this section |
| Cap lowered by revocation below current count | State is recomputed against the new cap on the next read; events may immediately be grace or blocked; no email is sent for the lowering itself |
| Clock skew between web processes | Month key comes from each process's UTC clock; processes must run NTP; at most one second of skew is tolerated |
| User changes email or is anonymized | Purchases keep billing_email/billing_name for tax records (retention in Section 17.10); owner_email_snapshot is cleared on anonymization |
Super admin removed from SUPER_ADMIN_EMAILS |
Loses access when the web processes restart (the list is read at start). Every request on an impersonation session re-checks the operator, so their active impersonation sessions are deleted at their next request, and session-cleanup removes the rest |
| Payment completes after the pending row was expired | The row is re-opened by fulfillCheckoutSession (Section 15.6.4 step 2) and ends with a license or an automatic refund; never neither |
| Dispute opened | Ingestion dropped, data held, no purge clock until the operator reinstates the license or closes the dispute as lost (Section 15.14.3) |
| Operator deletes or transfers a busy site mid-month | The site's usage is frozen in the ledger and still counts toward the team's month (Section 15.9.4) |
| Self-hosted | All of Section 15 is off (Section 15.1.1) |
15.19 Authoritative Table Definitions #
This section is the final column list of the licensing tables. The Section 5 DDL repeats it exactly (names, types, nullability, enums, checks, indexes). Referenced tables teams, users and sites are defined in Section 5.
15.19.1 license_tiers #
CREATE TABLE license_tiers (
id uuid PRIMARY KEY DEFAULT uuidv7(),
code text NOT NULL UNIQUE CHECK (code ~ '^[a-z][a-z0-9_]{1,31}$'),
name text NOT NULL CHECK (char_length(name) BETWEEN 1 AND 40),
rank smallint NOT NULL UNIQUE CHECK (rank BETWEEN 1 AND 3),
price_minor integer NOT NULL CHECK (price_minor BETWEEN 100 AND 10000000),
currency char(3) NOT NULL DEFAULT 'USD' CHECK (currency = 'USD'),
max_sites integer NOT NULL CHECK (max_sites BETWEEN 1 AND 100000),
monthly_event_cap bigint NOT NULL CHECK (monthly_event_cap BETWEEN 1000 AND 100000000000),
sort_order smallint NOT NULL DEFAULT 0,
is_active boolean NOT NULL DEFAULT true,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now()
);
INSERT INTO license_tiers (code, name, rank, price_minor, currency, max_sites, monthly_event_cap, sort_order) VALUES
('solo', 'Solo', 1, 5900, 'USD', 5, 100000, 1),
('pro', 'Pro', 2, 14900, 'USD', 25, 500000, 2),
('agency', 'Agency', 3, 34900, 'USD', 100, 2000000, 3);15.19.2 licenses #
CREATE TABLE licenses (
id uuid PRIMARY KEY DEFAULT uuidv7(),
team_id uuid REFERENCES teams(id) ON DELETE SET NULL,
team_name_snapshot text NOT NULL,
owner_email_snapshot text,
tier_id uuid NOT NULL REFERENCES license_tiers(id) ON DELETE RESTRICT,
source text NOT NULL CHECK (source IN ('stripe','codes','operator')),
status text NOT NULL DEFAULT 'active' CHECK (status IN ('active','revoked')),
max_sites integer NOT NULL CHECK (max_sites >= 1),
monthly_event_cap bigint NOT NULL CHECK (monthly_event_cap >= 1),
credit_minor integer NOT NULL DEFAULT 0 CHECK (credit_minor >= 0),
granted_at timestamptz NOT NULL DEFAULT now(),
revoked_at timestamptz,
revoked_reason text CHECK (revoked_reason IN ('refunded','dispute','code_revoked','operator_revoked','team_deleted')),
purge_after timestamptz,
purged_at timestamptz,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT licenses_state_chk CHECK (
(status = 'active' AND revoked_at IS NULL AND revoked_reason IS NULL AND purge_after IS NULL AND purged_at IS NULL)
OR (status = 'revoked' AND revoked_at IS NOT NULL AND revoked_reason IS NOT NULL
AND (purge_after IS NOT NULL OR revoked_reason IN ('team_deleted','dispute')))),
CONSTRAINT licenses_team_chk CHECK (team_id IS NOT NULL OR status = 'revoked')
);
CREATE UNIQUE INDEX licenses_one_active_per_team ON licenses (team_id) WHERE status = 'active' AND team_id IS NOT NULL;
CREATE INDEX licenses_team_idx ON licenses (team_id);
CREATE INDEX licenses_purge_due_idx ON licenses (purge_after) WHERE status = 'revoked' AND purged_at IS NULL AND purge_after IS NOT NULL;15.19.3 purchases #
CREATE TABLE purchases (
id uuid PRIMARY KEY DEFAULT uuidv7(),
team_id uuid REFERENCES teams(id) ON DELETE SET NULL,
license_id uuid REFERENCES licenses(id) ON DELETE RESTRICT,
user_id uuid REFERENCES users(id) ON DELETE SET NULL,
kind text NOT NULL CHECK (kind IN ('new','upgrade')),
provider text NOT NULL DEFAULT 'stripe' CHECK (provider = 'stripe'),
tier_id uuid NOT NULL REFERENCES license_tiers(id) ON DELETE RESTRICT,
from_tier_id uuid REFERENCES license_tiers(id) ON DELETE RESTRICT,
status text NOT NULL DEFAULT 'pending'
CHECK (status IN ('pending','paid','partially_refunded','refunded','disputed','expired')),
amount_minor integer NOT NULL CHECK (amount_minor >= 0),
tax_minor integer NOT NULL DEFAULT 0 CHECK (tax_minor >= 0),
total_minor integer NOT NULL CHECK (total_minor >= 0),
currency char(3) NOT NULL DEFAULT 'USD' CHECK (currency = 'USD'),
refunded_minor integer NOT NULL DEFAULT 0,
stripe_checkout_session_id text UNIQUE,
stripe_payment_intent_id text,
stripe_charge_id text UNIQUE,
stripe_refund_id text,
stripe_customer_id text,
stripe_dispute_id text,
payment_method_summary text,
billing_name text,
billing_email text,
billing_address jsonb,
billing_tax_ids jsonb,
paid_at timestamptz,
refunded_at timestamptz,
invoice_year smallint,
invoice_seq integer,
invoice_number text UNIQUE,
credit_note_year smallint,
credit_note_seq integer,
credit_note_number text UNIQUE,
seller_snapshot jsonb,
expires_at timestamptz,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT purchases_upgrade_chk CHECK ((kind = 'upgrade') = (from_tier_id IS NOT NULL)),
CONSTRAINT purchases_total_chk CHECK (total_minor = amount_minor + tax_minor),
CONSTRAINT purchases_refund_chk CHECK (refunded_minor BETWEEN 0 AND total_minor),
CONSTRAINT purchases_invoice_chk CHECK ((invoice_number IS NULL) = (invoice_seq IS NULL) AND (invoice_year IS NULL) = (invoice_seq IS NULL)),
CONSTRAINT purchases_credit_note_chk CHECK ((credit_note_number IS NULL) = (credit_note_seq IS NULL) AND (credit_note_year IS NULL) = (credit_note_seq IS NULL)),
CONSTRAINT purchases_invoice_seq_uq UNIQUE (invoice_year, invoice_seq),
CONSTRAINT purchases_credit_note_seq_uq UNIQUE (credit_note_year, credit_note_seq)
);
CREATE INDEX purchases_team_idx ON purchases (team_id, created_at DESC);
CREATE INDEX purchases_license_idx ON purchases (license_id);
CREATE INDEX purchases_pending_idx ON purchases (created_at) WHERE status = 'pending';15.19.4 license_code_batches #
CREATE TABLE license_code_batches (
id uuid PRIMARY KEY DEFAULT uuidv7(),
name text NOT NULL CHECK (char_length(name) BETWEEN 1 AND 80),
marketplace text NOT NULL CHECK (char_length(marketplace) BETWEEN 1 AND 60),
tier_id uuid NOT NULL REFERENCES license_tiers(id) ON DELETE RESTRICT,
quantity integer NOT NULL CHECK (quantity BETWEEN 1 AND 10000),
notes text CHECK (notes IS NULL OR char_length(notes) <= 1000),
created_by uuid REFERENCES users(id) ON DELETE SET NULL,
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE UNIQUE INDEX license_code_batches_name_key ON license_code_batches (lower(name));15.19.5 license_codes #
CREATE TABLE license_codes (
id uuid PRIMARY KEY DEFAULT uuidv7(),
batch_id uuid NOT NULL REFERENCES license_code_batches(id) ON DELETE RESTRICT,
code text NOT NULL UNIQUE CHECK (code ~ '^[A-Z]{2,4}(-[0-9A-HJKMNP-TV-Z]{4}){3}$'),
status text NOT NULL DEFAULT 'unredeemed' CHECK (status IN ('unredeemed','redeemed','revoked')),
license_id uuid REFERENCES licenses(id) ON DELETE RESTRICT,
redeemed_by uuid REFERENCES users(id) ON DELETE SET NULL,
redeemed_team_id uuid REFERENCES teams(id) ON DELETE SET NULL,
created_by uuid REFERENCES users(id) ON DELETE SET NULL,
redeemed_at timestamptz,
revoked_at timestamptz,
revoked_reason text CHECK (revoked_reason IS NULL OR char_length(revoked_reason) BETWEEN 1 AND 200),
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT license_codes_state_chk CHECK (
(status = 'unredeemed' AND license_id IS NULL AND redeemed_at IS NULL AND revoked_at IS NULL)
OR (status = 'redeemed' AND license_id IS NOT NULL AND redeemed_at IS NOT NULL AND revoked_at IS NULL)
OR (status = 'revoked' AND revoked_at IS NOT NULL))
);
CREATE INDEX license_codes_batch_idx ON license_codes (batch_id, status);
CREATE INDEX license_codes_license_idx ON license_codes (license_id) WHERE license_id IS NOT NULL;15.19.6 usage_monthly #
CREATE TABLE usage_monthly (
id uuid PRIMARY KEY DEFAULT uuidv7(),
team_id uuid NOT NULL REFERENCES teams(id) ON DELETE CASCADE,
month date NOT NULL CHECK (month = date_trunc('month', month)::date),
billable_events bigint NOT NULL DEFAULT 0,
pageviews bigint NOT NULL DEFAULT 0,
custom_events bigint NOT NULL DEFAULT 0,
events_dropped_cap bigint NOT NULL DEFAULT 0,
events_dropped_unlicensed bigint NOT NULL DEFAULT 0,
event_cap bigint NOT NULL DEFAULT 0,
extra_allowance bigint NOT NULL DEFAULT 0 CHECK (extra_allowance >= 0),
warned_80_at timestamptz,
warned_100_at timestamptz,
hard_stop_at timestamptz,
closed_at timestamptz,
reconciled_at timestamptz,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT usage_monthly_team_month_uq UNIQUE (team_id, month)
);15.19.7 usage_site_monthly (per-site ledger) #
CREATE TABLE usage_site_monthly (
id uuid PRIMARY KEY DEFAULT uuidv7(),
team_id uuid NOT NULL REFERENCES teams(id) ON DELETE CASCADE,
site_id uuid NOT NULL, -- no foreign key: the row outlives a deleted or transferred site
month date NOT NULL CHECK (month = date_trunc('month', month)::date),
billable_events bigint NOT NULL DEFAULT 0,
pageviews bigint NOT NULL DEFAULT 0,
custom_events bigint NOT NULL DEFAULT 0,
counted_from timestamptz NOT NULL, -- start of counting for this team (month start, or the time the site joined the team)
frozen_at timestamptz, -- set when the site was deleted or transferred out; the row is then never recomputed or lowered
reconciled_at timestamptz,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT usage_site_monthly_uq UNIQUE (team_id, site_id, month)
);
CREATE INDEX usage_site_monthly_team_month_idx ON usage_site_monthly (team_id, month);16. Editions, Deployment, Configuration & Operations #
16.1 Editions #
One codebase, one Docker image, one schema. EDITION selects behavior at runtime (default selfhosted).
| Area | selfhosted (free, Docker Compose) |
hosted (operator-run, multi-tenant, EU) |
|---|---|---|
| License | AGPL-3.0-only source; no license keys | Same source; lifetime-deal licensing per Section 15 |
| Tenancy | One installation, any number of teams (personal team per user, Section 11) | Multi-tenant |
| Limits | Unlimited sites, unlimited events | Per tier: sites and monthly event cap (Section 15.2) |
| Billing, Stripe, invoices, usage meter, banners | Absent | Present |
Operator console /admin, SUPER_ADMIN_EMAILS |
Absent. SUPER_ADMIN_EMAILS is ignored; the SMTP test message is sent with the CLI users:test-email (Section 16.2.1) |
Present |
| Registration default | REGISTRATION_MODE=invite_only; the first account is created through the setup page with the one-time setup token (Section 16.3.5) |
open |
| Email verification | REQUIRE_EMAIL_VERIFICATION: default true when SMTP_HOST is set, otherwise false (accounts are then created verified and can never become super admins) |
Always required (false is refused at startup); SMTP mandatory |
| SMTP | Optional. Without it, email reports, spike alerts and email invitations are disabled; invitation links are shown in the UI for manual sharing | Required |
| GeoIP | Bundled DB-IP Lite MMDB, auto-updated by geoip-update when GEOIP_AUTO_UPDATE is on (default); or operator-supplied MMDB |
Same; auto-update on |
| TLS | Caddy in the Compose file (automatic certificates) or the operator's own proxy | Load balancer terminates TLS |
| Data retention | DATA_RETENTION_MONTHS (default 0 = keep forever) |
Same variable; default 0 |
| Metrics endpoint | Off by default (METRICS_ENABLED=false) |
On |
| Outbound connections | Only those in Section 17.9.6; no telemetry, no update checks, no license calls | Same plus Stripe, SMTP relay |
| Support | Community | Operator (SUPPORT_EMAIL) |
Server-side checks use isHosted() (Section 15.1.1). Frontend code receives the edition from the bootstrap response. There are no build-time editions.
16.2 Docker Image #
16.2.1 Image contract #
| Item | Decision |
|---|---|
| Name and tags | quietstat/quietstat, tags latest, X.Y.Z, X.Y, X. Also pushed to the GitHub Container Registry under the same name. latest is the newest stable release; there is no edge tag in the release line. |
| Platforms | linux/amd64 and linux/arm64 (multi-arch manifest) |
| Build | Multi-stage Dockerfile at the repository root: deps (pnpm install with frozen lockfile), build (TypeScript compile, Vite SPA build, tracker bundle), assets (download of the DB-IP Lite MMDB for the current month at build time, checksum-verified, plus the datacenter CIDR and spam-referrer list snapshots), runtime (production dependencies only). Base images per Section 3, pinned by digest. |
| Runtime user | Non-root quietstat (uid/gid 10001) |
| Working dir / contents | /app containing compiled server and worker, the SPA build, tracker files, migrations (Postgres SQL, ClickHouse SQL), assets/geoip/dbip-city-lite.mmdb, assets/lists/ |
| Writable paths | /data (volume; DATA_DIR) and /tmp (tmpfs). The root filesystem is read-only compatible (read_only: true works). |
| Port | 8000 (PORT), bound to 0.0.0.0 (HOST) |
| Entrypoint | docker-entrypoint.sh takes one command: web, worker, migrate or cli; any other value prints usage and exits 64. migrate accepts --reencrypt-secrets (Section 17.7.4) and --check (exit 0 when the schema is current, 1 when migrations are pending, without applying anything). cli <subcommand> [args] runs node dist/cli.js for operator tasks: users:reset-2fa <email>, users:set-password <email> and users:test-email <address>; a self-hoster runs for example docker compose run --rm web cli users:reset-2fa <email>. |
| Healthcheck | HEALTHCHECK CMD node dist/healthcheck.js calling GET http://127.0.0.1:${PORT}/healthz for web; for worker the check reads a heartbeat file in /tmp updated every 10 s and fails when older than 60 s |
| Signals | SIGTERM: web stops accepting connections, drains in-flight requests for up to 10 s, flushes the ingestion buffer to ClickHouse (bounded to 10 s), closes pools, exits 0. worker stops fetching jobs and waits up to 60 s for running jobs. Compose uses stop_grace_period: 70s for the worker and 30s for web. |
| Labels | OCI standard labels: org.opencontainers.image.source, .version, .revision, .licenses=AGPL-3.0-only |
| Supply chain | Image vulnerability scan, SBOM and signed provenance attestation per Section 17.11 |
16.2.2 Data directory layout (DATA_DIR, default /data) #
| Path | Purpose |
|---|---|
/data/geoip/ |
Downloaded newer GeoIP MMDB files and current symlink; created by the geoip-update job |
/data/lists/ |
Refreshed datacenter CIDR and spam-referrer lists written by lists-refresh; web and worker processes reload a list when its file modification time changes |
/data/exports/ |
Generated CSV export files before download (7-day expiry from completion, exports.expires_at, Section 14.3.1) |
/data/imports/ |
Uploaded CSV import files during processing, laid out as imports/{import_id}/{table}.csv; deleted when the import finishes, fails or is cancelled; orphans are removed by import-sweep |
Web and worker must mount the same /data volume when running on one host. In the hosted edition /data is a shared network volume or object storage mounted by the platform; the app only uses the filesystem API.
GeoIP resolution order at process start and on file-change: GEOIP_DB_PATH if set; otherwise /data/geoip/current if present; otherwise the bundled file. Auto-update is on when GEOIP_DB_PATH is unset; when GEOIP_DB_PATH is set, auto-update runs only if GEOIP_AUTO_UPDATE=true and the path is writable.
16.3 Self-Hosted Deployment (Docker Compose) #
16.3.1 Compose-only variables (.env next to docker-compose.yml) #
These are read by Compose, not by the application (the application variables are in Section 16.4).
| Variable | Required | Description |
|---|---|---|
QUIETSTAT_DOMAIN |
yes | Public host name served by Caddy, for example analytics.example.com. Must resolve to the server. |
QUIETSTAT_TAG |
no (default latest) |
Image tag of quietstat/quietstat |
POSTGRES_TAG, CLICKHOUSE_TAG, VALKEY_TAG, CADDY_TAG |
yes | Image tags set to the major lines listed in Section 3 |
POSTGRES_PASSWORD |
yes | At least 24 characters, URL-safe (letters, digits, -, _) because it is embedded in DATABASE_URL |
CLICKHOUSE_PASSWORD |
yes | At least 24 characters, URL-safe |
SECRET_KEY_BASE |
yes | Generate with openssl rand -base64 48 | tr '+/' '-_'; passed through to the application |
ACME_EMAIL |
no | Email for certificate expiry notices; omitted means Caddy registers without one |
SMTP_HOST, SMTP_PORT, SMTP_USER, SMTP_PASSWORD, MAILER_FROM |
no | Passed through (Section 16.4) |
16.3.2 docker-compose.yml #
name: quietstat
x-app-env: &app-env
EDITION: selfhosted
BASE_URL: https://${QUIETSTAT_DOMAIN}
SECRET_KEY_BASE: ${SECRET_KEY_BASE}
DATABASE_URL: postgres://quietstat:${POSTGRES_PASSWORD}@postgres:5432/quietstat
CLICKHOUSE_URL: http://quietstat:${CLICKHOUSE_PASSWORD}@clickhouse:8123
CLICKHOUSE_DATABASE: quietstat
VALKEY_URL: redis://valkey:6379
TRUSTED_PROXIES: 172.30.0.0/24
DATA_DIR: /data
SMTP_HOST: ${SMTP_HOST:-}
SMTP_PORT: ${SMTP_PORT:-587}
SMTP_USER: ${SMTP_USER:-}
SMTP_PASSWORD: ${SMTP_PASSWORD:-}
MAILER_FROM: ${MAILER_FROM:-}
LOG_LEVEL: info
x-app: &app
image: quietstat/quietstat:${QUIETSTAT_TAG:-latest}
restart: unless-stopped
read_only: true
tmpfs: [/tmp]
cap_drop: [ALL]
security_opt: ["no-new-privileges:true"]
environment: *app-env
volumes:
- quietstat_data:/data
networks: [default]
services:
migrate:
<<: *app
command: ["migrate"]
restart: "no"
depends_on:
postgres: { condition: service_healthy }
clickhouse: { condition: service_healthy }
web:
<<: *app
command: ["web"]
stop_grace_period: 30s
expose: ["8000"]
depends_on:
migrate: { condition: service_completed_successfully }
valkey: { condition: service_healthy }
healthcheck:
test: ["CMD", "node", "dist/healthcheck.js"]
interval: 15s
timeout: 5s
retries: 5
start_period: 30s
worker:
<<: *app
command: ["worker"]
stop_grace_period: 70s
depends_on:
migrate: { condition: service_completed_successfully }
valkey: { condition: service_healthy }
healthcheck:
test: ["CMD", "node", "dist/healthcheck.js"]
interval: 30s
timeout: 5s
retries: 3
start_period: 30s
postgres:
image: postgres:${POSTGRES_TAG}
restart: unless-stopped
environment:
POSTGRES_USER: quietstat
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: quietstat
volumes:
- pgdata:/var/lib/postgresql
healthcheck:
test: ["CMD-SHELL", "pg_isready -U quietstat -d quietstat"]
interval: 10s
timeout: 5s
retries: 10
start_period: 10s
shm_size: 256mb
clickhouse:
image: clickhouse/clickhouse-server:${CLICKHOUSE_TAG}
restart: unless-stopped
environment:
CLICKHOUSE_DB: quietstat
CLICKHOUSE_USER: quietstat
CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD}
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: "1"
volumes:
- chdata:/var/lib/clickhouse
- chlogs:/var/log/clickhouse-server
- ./clickhouse/config.d/quietstat.xml:/etc/clickhouse-server/config.d/quietstat.xml:ro
- ./backups/clickhouse:/backups
ulimits:
nofile: { soft: 262144, hard: 262144 }
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:8123/ping | grep -q Ok"]
interval: 10s
timeout: 5s
retries: 10
start_period: 20s
valkey:
image: valkey/valkey:${VALKEY_TAG}
restart: unless-stopped
command: ["valkey-server", "--appendonly", "yes", "--appendfsync", "everysec", "--maxmemory-policy", "noeviction"]
volumes:
- valkeydata:/data
healthcheck:
test: ["CMD", "valkey-cli", "ping"]
interval: 10s
timeout: 3s
retries: 10
caddy:
image: caddy:${CADDY_TAG}
restart: unless-stopped
ports:
- "80:80"
- "443:443"
- "443:443/udp"
environment:
QUIETSTAT_DOMAIN: ${QUIETSTAT_DOMAIN}
ACME_EMAIL: ${ACME_EMAIL:-}
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- caddy_data:/data
- caddy_config:/config
depends_on:
web: { condition: service_healthy }
volumes:
pgdata:
chdata:
chlogs:
valkeydata:
quietstat_data:
caddy_data:
caddy_config:
networks:
default:
ipam:
config:
- subnet: 172.30.0.0/24Notes binding the file: the Postgres volume is mounted at /var/lib/postgresql (the parent directory) as required by the current major image layout; TRUSTED_PROXIES contains the Compose network so X-Forwarded-For set by Caddy is trusted and no client can spoof it, because only Caddy reaches web (the web service publishes no host port); Valkey uses AOF so rate-limit and usage-bucket counters survive restarts; ClickHouse and Postgres are not exposed on host ports.
16.3.3 Caddyfile #
{
email {$ACME_EMAIL}
}
{$QUIETSTAT_DOMAIN} {
encode zstd gzip
reverse_proxy web:8000 {
header_up X-Forwarded-For {remote_host}
health_uri /healthz
health_interval 15s
}
log {
output stdout
format filter {
wrap json
fields {
request>remote_ip delete
request>client_ip delete
request>remote_port delete
request>headers>User-Agent delete
request>headers>Cookie delete
request>headers>Authorization delete
request>headers>X-Forwarded-For delete
request>headers>X-Csrf-Token delete
request>uri delete
resp_headers>Set-Cookie delete
}
}
}
}Security headers are set by the application (Section 17.5), not by Caddy. The access log carries no request URI at all, because several paths contain bearer tokens (/share/<token>, /embed/<token>, /invitations/<token>, /unsubscribe/<token>, /api/share/v1/link/<token>); the application logs route patterns instead (Section 17.8). If ACME_EMAIL is empty the global block's email directive is ignored by Caddy.
16.3.4 ClickHouse backup configuration #
./clickhouse/config.d/quietstat.xml (shipped in the repository under deploy/selfhosted/):
<clickhouse>
<backups>
<allowed_path>/backups</allowed_path>
<allowed_disk>default</allowed_disk>
</backups>
<listen_host>0.0.0.0</listen_host>
<logger><level>warning</level></logger>
</clickhouse>Backup commands for self-hosters are in Section 16.8.4.
16.3.5 First-run setup #
cp .env.example .env, fill the variables of Section 16.3.1,docker compose up -d.migrateruns once (Postgres migrations, ClickHouse migrations, retention TTL application, salt bootstrap), thenwebandworkerstart.- On start, while no user row exists,
workergenerates (as a singleton under a pg-boss/advisory lock, so several processes can never create different tokens) a one-time setup token (24 random URL-safe characters; or the value ofSETUP_TOKENwhen set), stores its hash in Valkey (setup:token, no TTL) and logs one line:setup token: <token>(the only place a secret is ever logged; it is deleted from Valkey when setup completes and is useless afterwards). Retrieve it withdocker compose logs worker | grep "setup token". The hosted edition uses the same setup flow on first deploy; the token is supplied throughSETUP_TOKEN_FILE. - Browser:
https://<domain>/setupshows a form: name, email, password (policy per Section 11), setup token.GET /setupreturns the SPA setup route only while there are zero users; afterwards/setupreturns404. There is no first-signup bypass: while no user row exists,POST /api/app/v1/auth/signupreturns403 setup_requiredin every registration mode (Section 11.2), and the account can be created only through the setup route below. POST /api/app/v1/setupbody{name, email, password, setup_token}(anonymous CSRF token, Section 11.7): takes a Postgres advisory lock (pg_advisory_xact_lock(hashtext('bootstrap'))) so two concurrent attempts cannot both succeed, re-checks that no user exists (409 setup_already_completedif one does), creates the first user (email verified by the setup token, no email sent), its personal team, and signs the user in. Wrong token →403 setup_token_invalid; 5 wrong tokens per hour per IP-HMAC (policysetup_token, Section 17.6) →429 rate_limited. Section 11's registration rules apply only after setup.- The setup page shows a checklist after login (dismissible): "Set up SMTP", "Add your first site", "Review registration mode".
- No default credentials exist. A fresh install with no users exposes nothing but
/setup,/healthz,/readyz, the login page and the tracker files.
16.4 Environment Variables (Canonical) #
Every configuration value of the product is an environment variable listed here; no other configuration files exist. Other sections refer to this table and never redefine a variable. Edition column: both, hosted (read only when EDITION=hosted) or selfhosted.
16.4.1 Loading and validation rules #
- A single Zod schema in
packages/shared/src/config.tsparsesprocess.envat process start forweb,workerandmigrate. On failure the process printsinvalid configuration: <VAR_NAME>: <reason>for each problem (variable names and reasons only; never values) and exits with code 78. - Any variable in the "File secrets" list may be supplied as
<NAME>_FILEcontaining a path to a file whose trimmed contents are the value (Docker/Kubernetes secrets). If both forms are set, startup fails. File-capable variables:SECRET_KEY_BASE,SECRET_KEY_BASE_PREVIOUS,DATABASE_URL,CLICKHOUSE_URL,VALKEY_URL,SMTP_PASSWORD,CLICKHOUSE_RO_PASSWORD,STRIPE_SECRET_KEY,STRIPE_WEBHOOK_SECRET,GOOGLE_CLIENT_SECRET,METRICS_TOKEN,SETUP_TOKEN. - Booleans accept exactly
trueorfalse(lower case). Integers are base-10. Lists are comma-separated with optional spaces. Empty string is treated as unset. - Values are read once at start; changing them requires a restart. The configuration is never logged and never returned by any API.
- Hosted-only variables are ignored (with an
infolog naming the variable) whenEDITION=selfhosted. - Cross-variable rules: if
SMTP_HOSTis set thenMAILER_FROMis required; ifSTRIPE_SECRET_KEYis set thenSTRIPE_WEBHOOK_SECRETand all requiredSELLER_*are required;hostedrequiresSMTP_HOST,STRIPE_SECRET_KEY,SUPER_ADMIN_EMAILS,CLICKHOUSE_RO_USER,CLICKHOUSE_RO_PASSWORD,SUPPORT_EMAIL,SECURITY_CONTACT,TERMS_URL,PRIVACY_URLandDPA_URL, refusesREQUIRE_EMAIL_VERIFICATION=falseandSMTP_SECURE=none, and requiresDATABASE_SSL=verify-full, aVALKEY_URLusingrediss://with a password and aCLICKHOUSE_URLusinghttps://;BASE_URLmust have no path or trailing slash;SECRET_KEY_BASEmust be at least 64 characters. - Tuning constants named in other sections (for example
API_RATE_LIMITS,ARGON2_PASSWORD_PARAMS,ARGON2_SHARE_PARAMS,JOURNEY_*,MAX_FUNNEL_STEPS,CODE_PREFIX) are code constants inpackages/shared, not environment variables, and are not configurable.
16.4.2 Core #
| Name | Required | Default | Edition | Description |
|---|---|---|---|---|
PRODUCT_NAME |
no | Quietstat |
both | Display name used in UI, emails, invoices, page titles, tracker docs and snippet comments. The only place the brand name is configured. Must be 1–40 characters, no HTML. |
EDITION |
no | selfhosted |
both | hosted or selfhosted (Section 16.1) |
BASE_URL |
yes | none | both | Public origin including scheme, for example https://analytics.example.com. Used for links in emails, CORS/CSP origins, cookies' Secure flag, Stripe return URLs, OAuth redirect URIs and the tracker's default endpoint. |
NODE_ENV |
no | production |
both | production, development or test. development enables pretty logs and relaxes Secure cookies on http://localhost only. |
HOST |
no | 0.0.0.0 |
both | Bind address of web |
PORT |
no | 8000 |
both | Listen port of web |
SECRET_KEY_BASE |
yes | none | both | Root secret (≥ 64 characters). Keys for cookies signing, CSRF, token HMACs and encryption at rest are derived from it (Section 17.7). Changing it without the rotation procedure invalidates sessions and makes encrypted values unreadable. |
SECRET_KEY_BASE_PREVIOUS |
no | unset | both | Previous root secret during rotation (Section 17.7.4) |
DATA_DIR |
no | /data |
both | Writable data directory (Section 16.2.2) |
SETUP_TOKEN |
no | generated | both | Fixed setup token for the first-run page (Section 16.3.5); generated and logged when unset |
SUPPORT_EMAIL |
hosted: yes | unset | both | Address shown for support and refund requests and in banners. Optional for self-hosted (hides support lines). |
TERMS_URL, PRIVACY_URL, DPA_URL |
hosted: yes (all three) | unset | both | Legal links shown in footer, signup and invoices; omitted from UI when unset |
SECURITY_CONTACT |
hosted: yes | unset | both | Value of the Contact field in security.txt (a mailto: or https: URI). When unset, /.well-known/security.txt returns 404. |
SECURITY_POLICY_URL |
no | unset | both | Policy field of security.txt |
ABUSE_CONTACT_EMAIL |
no | SUPPORT_EMAIL |
hosted | Address for abuse reports shown in the footer |
HOSTING_REGION_LABEL |
no | EU (Germany) |
hosted | Display text naming the hosting region in privacy notices and on invoices; it does not change where data is stored |
DEFAULT_LOCALE |
no | en |
both | UI language (English is the only shipped locale; Section 4 i18n) |
16.4.3 Datastores #
| Name | Required | Default | Edition | Description |
|---|---|---|---|---|
DATABASE_URL |
yes | none | both | Postgres connection string postgres://user:pass@host:5432/db |
DATABASE_POOL_MAX |
no | 20 (web), 10 (worker) |
both | Max connections per process (1–200) |
DATABASE_SSL |
no | prefer |
both | disable, prefer, require or verify-full. Hosted must use verify-full; startup fails otherwise. |
DATABASE_SSL_CA_FILE |
no | unset | both | Path to CA bundle for verify-full |
CLICKHOUSE_URL |
yes | none | both | http(s)://user:pass@host:8123; https:// is mandatory when EDITION=hosted |
CLICKHOUSE_RO_USER |
hosted: yes | unset | both | Read-only ClickHouse user (SELECT only, read-only settings profile with execution-time, memory and row-scan limits) used by the query engine for every dashboard, share and Stats API read. Unset on self-hosted: falls back to the user in CLICKHOUSE_URL. |
CLICKHOUSE_RO_PASSWORD |
with CLICKHOUSE_RO_USER |
unset | both | Password of the read-only user; file-capable like other secrets |
CLICKHOUSE_DATABASE |
no | quietstat |
both | ClickHouse database name |
CLICKHOUSE_REQUEST_TIMEOUT_MS |
no | 30000 |
both | Per-query timeout (1,000–300,000) |
VALKEY_URL |
yes | none | both | redis://[:password@]host:6379[/db] or rediss:// for TLS; EDITION=hosted requires rediss:// with a password |
DATA_RETENTION_MONTHS |
no | 0 |
both | Event retention. 0 keeps data forever; otherwise 1–240 (boot error invalid DATA_RETENTION_MONTHS otherwise). Applied as the ClickHouse TTL (toIntervalMonth) of the events and sessions tables by the migrate command at every deploy (Section 6.6); the imported_* tables are exempt. |
16.4.4 Email #
| Name | Required | Default | Edition | Description |
|---|---|---|---|---|
SMTP_HOST |
hosted: yes | unset | both | SMTP server host. Unset disables all outgoing email (self-hosted only). |
SMTP_PORT |
no | 587 |
both | 1–65535 |
SMTP_SECURE |
no | starttls |
both | starttls, tls (implicit TLS, usually port 465) or none (refused when EDITION=hosted) |
SMTP_USER |
no | unset | both | SMTP username; unset means no authentication |
SMTP_PASSWORD |
when SMTP_USER set |
unset | both | SMTP password |
MAILER_FROM |
when SMTP set | unset | both | From header, Name <address> or bare address |
MAILER_REPLY_TO |
no | unset | both | Reply-To header |
MAILER_FOOTER_ADDRESS |
no | unset | both | Postal address or operator line printed in the footer of every email (required in some jurisdictions for marketing-style mail; optional here) |
EMAIL_MAX_PER_MINUTE |
no | 120 |
both | Global send throttle of the email-send queue (1–6,000); the only throughput limit on outgoing email |
16.4.5 Access, registration and network #
| Name | Required | Default | Edition | Description |
|---|---|---|---|---|
REGISTRATION_MODE |
no | open (hosted), invite_only (selfhosted) |
both | open (anyone can sign up), invite_only (sign up only through an invitation, Section 11) or disabled (no public signup; existing users keep working; invitations to people without an account cannot be accepted, Section 11.2). The first account of a self-hosted install is created only through the setup flow (Section 16.3.5), in every mode. |
REQUIRE_EMAIL_VERIFICATION |
no | true when SMTP_HOST is set, otherwise false |
both | Whether new accounts must verify their email before use. EDITION=hosted refuses false. Accounts created verified because no SMTP is configured never qualify as super admins. |
PASSWORD_BREACH_CHECK |
no | local |
both | local checks passwords against the bundled common-password list only (no outbound call); hibp additionally queries the Have I Been Pwned range API with k-anonymity (Section 11) |
TRUSTED_PROXIES |
no | empty (trust none) | both | Comma-separated CIDRs (IPv4/IPv6) of reverse proxies whose X-Forwarded-For is honored; the client IP is the right-most address in the header that is not itself in this list. Max 64 entries. |
COOKIE_SECURE |
no | true if BASE_URL starts with https:// |
both | Forces the Secure attribute on cookies |
SUPER_ADMIN_EMAILS |
hosted: yes | unset | hosted | Comma-separated emails of operators (Section 15.17.1). Read at process start; ignored on self-hosted installs. |
16.4.6 Tracking, ingestion and data quality #
| Name | Required | Default | Edition | Description |
|---|---|---|---|---|
GEOIP_DB_PATH |
no | unset | both | Path to a MaxMind-compatible City MMDB; resolution order in Section 16.2.2 |
GEOIP_AUTO_UPDATE |
no | true when GEOIP_DB_PATH unset; else false |
both | Run the geoip-update job |
GEOIP_ATTRIBUTION |
no | IP geolocation by DB-IP |
both | Attribution text rendered in the dashboard footer (Section 4.13.1). The app does not detect the database vendor; change this text when you supply a different MMDB. |
GEOIP_UPDATE_URL_TEMPLATE |
no | https://download.db-ip.com/free/dbip-city-lite-{YYYY}-{MM}.mmdb.gz |
both | Download URL; {YYYY} and {MM} are replaced with the current UTC year and month |
LISTS_AUTO_UPDATE |
no | true |
both | Run the lists-refresh job (datacenter CIDRs, spam referrers) |
INGEST_BATCH_ROWS |
no | 5000 |
both | Rows that trigger a ClickHouse batch insert (100–50,000) |
INGEST_FLUSH_INTERVAL_MS |
no | 2000 |
both | Max time before flushing the buffer (200–30,000) |
INGEST_BUFFER_MAX_ROWS |
no | 100000 |
both | In-memory backpressure ceiling for event rows per web process (the session-row buffer is bounded at twice this value); beyond it new events are dropped with 202 and buffer_full counted (Section 8.15.3) |
FX_RATES_URL |
no | https://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.xml |
both | Daily exchange-rate feed for revenue conversion (Section 10) |
16.4.7 Integrations #
| Name | Required | Default | Edition | Description |
|---|---|---|---|---|
GOOGLE_CLIENT_ID |
no | unset | both | OAuth client for the GA4 import (Section 14). When unset, the GA4 import option is hidden. |
GOOGLE_CLIENT_SECRET |
with GOOGLE_CLIENT_ID |
unset | both | OAuth client secret. Redirect URI to register: {BASE_URL}/api/app/v1/integrations/google/callback. |
16.4.8 Hosted: payments and invoices #
| Name | Required | Default | Edition | Description |
|---|---|---|---|---|
STRIPE_SECRET_KEY |
hosted: yes | unset | hosted | Restricted or secret API key (sk_ / rk_); the key mode (live/test) is inferred and shown in /admin |
STRIPE_WEBHOOK_SECRET |
with Stripe | unset | hosted | Signing secret (whsec_) of the endpoint {BASE_URL}/api/webhooks/stripe subscribed to checkout.session.completed, charge.refunded, charge.dispute.created |
STRIPE_TAX_ENABLED |
no | false |
hosted | Enables Stripe Tax on Checkout (Section 15.4) |
SELLER_NAME |
hosted: yes | unset | hosted | Legal entity name on invoices |
SELLER_ADDRESS_LINE1 |
hosted: yes | unset | hosted | Seller address line 1 |
SELLER_ADDRESS_LINE2 |
no | unset | hosted | Seller address line 2 |
SELLER_CITY |
hosted: yes | unset | hosted | City |
SELLER_REGION |
no | unset | hosted | State or region |
SELLER_POSTAL_CODE |
hosted: yes | unset | hosted | Postal code |
SELLER_COUNTRY |
hosted: yes | unset | hosted | ISO 3166-1 alpha-2 |
SELLER_TAX_ID |
no | unset | hosted | VAT or tax ID printed on invoices |
SELLER_EMAIL |
hosted: yes | unset | hosted | Contact email on invoices |
INVOICE_PREFIX |
no | QS |
hosted | 1–6 characters [A-Z0-9] (Section 15.15.1) |
INVOICE_FOOTER_NOTE |
no | unset | hosted | ≤ 300 characters printed in the invoice footer |
REFUND_WINDOW_DAYS |
no | 60 |
hosted | Days after payment during which the operator console accepts a refund without an override (1–365, Section 15.13) |
AUDIT_RETENTION_MONTHS |
no | 24 |
hosted | Age after which audit-prune deletes audit_log rows (6–120). Self-hosted keeps audit rows indefinitely. |
16.4.9 Observability and workers #
| Name | Required | Default | Edition | Description |
|---|---|---|---|---|
LOG_LEVEL |
no | info |
both | fatal, error, warn, info, debug, trace |
LOG_PRETTY |
no | false |
both | Human-readable logs (development only; refused when NODE_ENV=production) |
METRICS_ENABLED |
no | true hosted, false selfhosted |
both | Serve Prometheus metrics on METRICS_PORT |
METRICS_PORT |
no | 9464 |
both | Separate listener, never proxied publicly |
METRICS_TOKEN |
no | unset | both | When set, GET /metrics requires Authorization: Bearer <token> |
OTEL_EXPORTER_OTLP_ENDPOINT |
no | unset | both | OTLP/HTTP endpoint; tracing is off when unset |
OTEL_TRACES_SAMPLE_RATIO |
no | 0.05 |
both | Head sampling ratio 0–1 |
WORKER_CONCURRENCY |
no | 10 |
both | Max concurrent jobs per worker process (1–100) |
16.5 Hosted Topology #
16.5.1 Reference architecture #
All components run in one EU region (three availability zones where the provider offers them; at least two otherwise). Data never leaves the EU region.
| Layer | Design |
|---|---|
| DNS | Authoritative DNS with health-checked failover records for the load balancer |
| Edge | Layer-7 load balancer terminating TLS 1.2 and 1.3 (HTTP/2 and HTTP/3), health check GET /healthz every 5 s (2 failures to drain, 2 successes to restore), idle timeout 60 s, request body limit 1 MiB (the only larger body is the CSV import upload, which uses a dedicated route with a 500 MB limit, Section 17.4), access logs disabled at the load balancer (error logs only, 7-day retention, no client IPs) |
| CDN (optional) | Serves only /js/* (tracker) and SPA static assets from an EU-only edge pool; POST /api/event and all /api/* bypass it |
| Web | Stateless web containers on a container orchestrator, 2 vCPU / 2 GiB each, minimum 3 (spread across zones), maximum 30. Autoscaling on 60% average CPU or p95 POST /api/event latency over 100 ms for 5 minutes; scale-in cooldown 10 minutes |
| Worker | worker containers, 2 vCPU / 2 GiB, minimum 2, maximum 6, autoscaling on pg-boss queue depth over 1,000 waiting jobs for 5 minutes. Cron schedules are singleton through pg-boss, so any number of workers is safe |
| Migrations | One-off job executed before each rollout (Section 16.7) |
| Postgres | Managed Postgres with automated failover (primary plus one synchronous standby in another zone), 4 vCPU / 16 GiB start, storage autoscaling, point-in-time recovery (the salts table is UNLOGGED, so it is excluded from WAL, PITR and replicas, Section 17.9.2), TLS enforced, a connection pooler in transaction mode is not used (pg-boss requires session features) |
| ClickHouse | Self-run, single node at launch on NVMe-backed VM (16 vCPU / 64 GiB, 2 TB), second node added when storage or CPU thresholds in Section 16.11 are crossed by adding a read replica using the same schema; engines are not replicated in v1 (Section 6). Hourly volume snapshots plus backups (Section 16.8) |
| Valkey | Managed or self-run primary with one replica in another zone, AOF everysec, 4 GiB memory, noeviction, TLS and a password (rediss://) |
| Object storage | S3-compatible bucket in the same region for backups, versioned, lifecycle rules per Section 16.8 |
Transactional SMTP relay in the EU, SPF, DKIM and DMARC (p=quarantine) configured on the sending domain |
|
| Secrets | Held by the orchestrator's secret manager, injected as *_FILE mounts (Section 16.4.1) |
Managed versus self-run rationale: Postgres is managed because PITR and failover are commodity; ClickHouse is self-run because managed offerings are priced per query volume and the product's write-heavy workload fits a dedicated node; Valkey is managed because state in it is rebuildable and a failover loses seconds only.
16.5.2 Network rules #
Only the load balancer is reachable from the internet (ports 80 redirect, 443). web and worker reach Postgres, ClickHouse and Valkey over private networking. Outbound allow-list for web/worker: SMTP relay, Stripe (api.stripe.com), Google OAuth and Analytics Data API (oauth2.googleapis.com, analyticsdata.googleapis.com, analyticsadmin.googleapis.com), download.db-ip.com, the feed hosts used by lists-refresh, www.ecb.europa.eu, and the OTLP endpoint. The single exception is the installation check of Section 7.15.2, which fetches the customer's own site: web may open outbound HTTPS to public addresses for it, only through guardedFetch (Section 17.4), and the network layer additionally blocks private, loopback and link-local ranges. Everything else is denied.
16.5.3 Environments #
production, staging (same topology at minimum size, Stripe test mode, separate buckets and databases, real EDITION=hosted), and per-pull-request preview environments using the self-hosted Compose file. Staging never receives production data.
16.6 Background Jobs (Canonical) #
All scheduled jobs are registered by worker at start through pg-boss schedule() (cron expressions in UTC) so they run once cluster-wide. A job's pg-boss queue name equals its job name. The "Lane" column groups queues for concurrency: critical (privacy-critical, at most 2 concurrent per worker), default (general), email (rate-limited by EMAIL_MAX_PER_MINUTE), bulk (imports, exports, purges: at most 2 concurrent per worker). Retry notation limit / delay / backoff where backoff exp means the delay doubles each retry. Timeout is the pg-boss expireInSeconds after which a running job is failed and retried. Jobs that fail all retries are kept in the pg-boss archive for 14 days and surface in the operator Overview (Section 15.17.12) and the alerts in Section 16.9.4.
| Job name | Schedule (UTC) or trigger | Lane | Retries | Timeout | Owner | Edition |
|---|---|---|---|---|---|---|
salt-rotate |
0 0 * * * (also at web start if today's salt is missing) |
critical | 10 / 30 s / exp | 120 s | 8 | both |
salt-expire |
30 0 * * * (deletes the previous salt) |
critical | 10 / 30 s / exp | 120 s | 8 | both |
email-reports-dispatch |
*/15 * * * * (enqueues due email-report-send jobs) |
default | 3 / 30 s / exp | 300 s | 12 | both |
email-report-send |
one per (site, frequency, period key), enqueued by email-reports-dispatch |
5 / 60 s / exp | 300 s | 12 | both | |
spike-check |
* * * * * |
default | 2 / 60 s / none | 300 s | 12 | both |
email-send |
per outgoing email | 8 / 30 s / exp (≈ 2 h) | 120 s | 12 | both | |
usage-reconcile |
*/10 * * * * (rebuilds the per-site usage ledger and the team counters) |
default | 3 / 60 s / exp | 600 s | 15 | hosted |
usage-cap-notify |
*/5 * * * * |
default | 3 / 30 s / exp | 120 s | 15 | hosted |
usage-month-close |
10 0 1 * * |
default | 10 / 300 s / exp | 1800 s | 15 | hosted |
purchase-pending-expire |
5 * * * * |
default | 3 / 60 s / exp | 120 s | 15 | hosted |
stripe-reconcile |
20 4 * * * |
default | 3 / 300 s / exp | 600 s | 15 | hosted |
unlicensed-purge-warn |
0 9 * * * |
default | 3 / 300 s / exp | 600 s | 15 | hosted |
unlicensed-purge |
0 3 * * * |
bulk | 3 / 600 s / exp | 3600 s | 15 | hosted |
fx-rates-fetch |
0 17 * * * (also at worker start when no rates exist) |
default | 5 / 300 s / exp | 60 s | 10 | both |
geoip-update |
15 3 * * * (no-op when the current month's file is installed) |
default | 3 / 600 s / exp | 900 s | 8 | both |
lists-refresh |
0 2 * * 1 (datacenter CIDRs, spam referrers; no-op when LISTS_AUTO_UPDATE=false) |
default | 3 / 600 s / exp | 600 s | 8 | both |
site-purge |
on site delete (scope all: deletes the site's events, sessions and all imported_* rows) and for native-data purges including unlicensed-purge (scope native; imported data is then deleted by import-purge) |
bulk | 8 / 60 s / exp | 3600 s | 6 | both |
site-orphan-reconcile |
45 3 * * * (removes ClickHouse rows whose site_id no longer exists in Postgres, and deletes expired site_domain_reservations) |
bulk | 2 / 300 s / none | 3600 s | 6 | both |
user-anonymize |
0 4 * * * (users with deleted_at older than 30 days) |
default | 3 / 300 s / exp | 900 s | 11 | both |
unverified-user-cleanup |
40 3 * * * (accounts unverified for 14 days) |
default | 2 / 60 s / none | 300 s | 11 | both |
team-purge |
0 * * * * (teams whose 7-day undo window elapsed) |
bulk | 5 / 300 s / exp | 3600 s | 11 | both |
invitation-expire |
10 * * * * (marks invitations expired; deletes them 30 days after expiry; deletes unconfirmed external report and spike recipients older than 7 days) |
default | 2 / 60 s / none | 120 s | 11 | both |
session-cleanup |
20 * * * * (expired auth_sessions and email_tokens; impersonation sessions whose operator is no longer a super admin) |
default | 2 / 60 s / none | 300 s | 11 | both |
export-run |
on export request | bulk | 2 / 60 s / exp | 3600 s | 14 | both |
export-expire |
40 * * * * (marks exports past expires_at expired and deletes their files; hard-deletes expired rows 30 days later) |
default | 2 / 300 s / none | 300 s | 14 | both |
import-run |
on CSV import | bulk | 2 / 60 s / none | 14400 s | 14 | both |
ga4-import-run |
on GA4 import (resumable by date cursor) | bulk | 10 / 300 s / exp | 14400 s | 14 | both |
import-purge |
on import delete, fail or cancel (removes the imported rows of a single import; the imports row is hard-deleted only when its status is deleting, rows of failed or cancelled imports are kept, Section 5.9.1) |
bulk | 5 / 60 s / exp | 3600 s | 14 | both |
import-sweep |
50 * * * * (imports stuck in configuring for over 1 h, GA4 credentials 24 h after a retryable failure, imports stalled over 6 h, orphan upload directories) |
default | 2 / 300 s / none | 600 s | 14 | both |
audit-prune |
30 4 * * 0 |
default | 2 / 300 s / none | 600 s | 11 | hosted |
Rules: (1) each job handler is idempotent; (2) salt-rotate, salt-expire, usage-month-close and unlicensed-purge use pg-boss singleton keys so a retry cannot overlap; (3) all job payloads contain ids only, never email addresses, IPs or user agents; (4) a job that is not applicable to the running edition is not registered; (5) this table is the complete list of registered job names; every job name used anywhere else in this specification appears here, and the Jobs test (Section 18) fails on a registered handler that is missing from it; (6) there is no usage-flush job: the web process flushes counter increments every second (Section 15.9.3) and usage-reconcile persists them.
16.7 Migrations on Deploy #
migrateruns Postgres migrations (Drizzle SQL files, applied in order in a transaction each) and then ClickHouse migrations (SQL files tracked inschema_migrations), then applies the retention TTL, then ensures a salt for today exists. It holds a Postgres advisory lock so concurrent runs wait.- Compose runs
migrateas a one-shot service beforeweb/worker. Hosted runs it as an orchestrator pre-deploy job; the rollout is aborted if it fails. - Compatibility rule (expand/contract): every migration shipped in release N must work with the code of release N−1. Destructive changes (drop column/table, tighten constraint) ship one release after the code stopped using the object, and never in a patch release.
webat start checksmigrate --checksemantics in-process; if migrations are pending,/readyzreturns503with{"error":{"code":"unavailable",...}}andwebdoes not accept events.- ClickHouse migrations are additive only within a major release (new columns with defaults, new tables, new materialized views). Backfills run as
bulkjobs, never inside the migration. - There are no down-migrations. Rollback means redeploying the previous image, which works because of the compatibility rule.
16.8 Backups #
16.8.1 Targets #
| Store | Hosted method | Frequency | Retention |
|---|---|---|---|
| Postgres | Continuous WAL archiving plus daily base backup to object storage; PITR. The salts table is UNLOGGED, so it appears in neither the WAL nor the base backup, and logical dumps exclude its contents (--exclude-table-data=salts) |
WAL continuous; base daily 02:30 UTC | 14 days |
| ClickHouse | Native BACKUP to object storage: full every Sunday 01:00 UTC, incremental daily 01:00 UTC; plus hourly volume snapshots |
As stated | Backups 35 days; snapshots 48 hours |
| Valkey | None (rebuildable; Section 16.12) | ||
| Secrets / config | Orchestrator secret manager with versioning; SECRET_KEY_BASE escrow in a separate vault with two-person access |
On change | All versions |
Backup storage is encrypted at rest with provider-managed keys, versioning on, deletion protected for 7 days, in the same EU region plus a replica bucket in a second EU region. The longest any deleted customer data can survive in a backup is 35 days (Section 17.10).
16.8.2 Restore tests #
A full restore of Postgres and ClickHouse into staging is performed monthly and after any backup tooling change; the run records restore duration and row-count comparison against production for the users, sites, licenses and purchases tables and the per-site event counts of ten sampled sites, and asserts that the restored salts table is empty (a fresh salt is generated at start). A failed test is a P2 incident.
16.8.3 Deletion replay after restore #
After any production restore, the runbook replays deletions: the operator extracts the structured log lines site.deleted, team.purged, user.anonymized, user.deletion_requested, license.data_purged and import.deleted (ids only) from the retained logs for the interval between the restored point and the restore time, and re-runs site-purge, team-purge, user-anonymize, unlicensed-purge (for the listed teams) and import-purge for those ids, and re-applies deleted_at for users who had requested deletion, before re-opening traffic. These log lines are kept for 90 days.
16.8.4 Self-hosted backups #
# Postgres (logical dump)
docker compose exec -T postgres pg_dump -U quietstat -Fc --exclude-table-data=salts quietstat > backups/postgres-$(date +%F).dump
# ClickHouse (native backup into ./backups/clickhouse, mounted at /backups)
docker compose exec -T clickhouse clickhouse-client --user quietstat --password "$CLICKHOUSE_PASSWORD" \
--query "BACKUP DATABASE quietstat TO File('/backups/clickhouse-$(date +%F).zip')"Restore: pg_restore -U quietstat -d quietstat --clean --if-exists into an empty database, and RESTORE DATABASE quietstat FROM File('/backups/<file>.zip'). The Compose documentation in the repository instructs running both daily from cron and copying the files off the host. Valkey need not be backed up. Back up SECRET_KEY_BASE separately from the data backups (a password manager or secret store) and never in the same location as the dumps; without it, TOTP secrets and stored OAuth tokens cannot be decrypted. The dump excludes the contents of the salts table by design; after a restore a fresh salt is generated when web starts.
16.9 Observability #
16.9.1 Metrics #
Prometheus text format at GET /metrics on METRICS_PORT of every process. Names are prefixed quietstat_. Default Node.js process and event-loop metrics are included. Labels never contain IPs, user agents, emails, site domains or team ids (cardinality and privacy), except where listed.
| Metric | Type | Labels | Meaning |
|---|---|---|---|
http_requests_total |
counter | route (pattern), method, status_class |
Requests |
http_request_duration_seconds |
histogram | route, method |
Latency |
ingest_events_total |
counter | result = accepted, invalid, dropped_bot, dropped_datacenter, dropped_exclusion, dropped_unknown_site, dropped_hostname_not_allowed, dropped_spam_referrer, dropped_buffer_full, dropped_cap, dropped_unlicensed, dropped_suspended, rate_limited |
Ingestion outcomes |
ingest_buffer_rows |
gauge | Rows waiting in the process buffer | |
ingest_flush_duration_seconds |
histogram | ClickHouse batch insert time | |
ingest_flush_failures_total |
counter | Failed batch inserts | |
ingest_lag_seconds |
gauge | Age of the oldest buffered row | |
clickhouse_query_duration_seconds |
histogram | family = stats, realtime, export, usage, purge |
Query time |
postgres_pool_in_use / postgres_pool_waiting |
gauge | Pool saturation | |
valkey_errors_total |
counter | Client errors | |
jobs_total |
counter | job, result = completed, failed, retried |
Job runs |
jobs_queue_depth |
gauge | job |
Waiting jobs |
job_duration_seconds |
histogram | job |
Runtime |
email_sent_total |
counter | template, result = sent, failed |
Outgoing email |
stripe_webhook_total |
counter | type, result = ok, ignored, invalid_signature, error |
Webhook outcomes |
usage_teams_by_state |
gauge | state = ok, warning, grace, blocked |
Updated by usage-reconcile |
auth_login_total |
counter | result = success, bad_password, bad_2fa, locked |
Logins |
rate_limited_total |
counter | policy |
Rate-limit hits (policy names in Section 17.6) |
impersonation_blocked_total |
counter | Mutations rejected during operator impersonation (counted, not individually audited, Section 15.17.11) | |
csp_reports_total |
counter | CSP violation reports received (Section 17.5.2) |
16.9.2 Logs #
pino JSON to stdout, one line per record. Mandatory fields: level, time (ISO 8601), msg, service (web/worker/migrate), version, request_id (when in a request), route (pattern), status, duration_ms, account_user_id and team_id when authenticated (the visitor hash user_id is never logged). Never logged: IP addresses, User-Agent, cookies, Authorization, CSRF tokens, request or response bodies, query strings, concrete URLs, email addresses, passwords, tokens, codes (redaction rules in Section 17.8). Retention in hosted: 30 days in the log store; site.deleted, team.purged, user.anonymized, user.deletion_requested, license.data_purged and import.deleted records (ids only) 90 days.
16.9.3 Traces #
OpenTelemetry tracing when OTEL_EXPORTER_OTLP_ENDPOINT is set: spans for HTTP handlers, Postgres queries (statement text without parameter values), ClickHouse queries (query family, no SQL text), Valkey commands (command name only) and job executions. POST /api/event is sampled at one tenth of the configured ratio. Span attributes never contain request URLs with query strings, headers or IPs.
16.9.4 Alerts (hosted) #
| Alert | Condition | Window | Severity | Action |
|---|---|---|---|---|
| Web down | LB healthy targets < 2 | 2 min | P1 | Page on-call |
| Ingestion error rate | 5xx on POST /api/event > 1% |
5 min | P1 | Page |
| Ingestion lag | ingest_lag_seconds > 30 |
5 min | P1 | Page |
| ClickHouse insert failures | ingest_flush_failures_total increase > 5 |
5 min | P1 | Page |
| Postgres unavailable | /readyz failures from Postgres |
2 min | P1 | Page |
| Valkey unavailable | valkey_errors_total rate > 10/s |
2 min | P2 | Ticket and chat |
| p95 dashboard query latency | clickhouse_query_duration_seconds{family="stats"} p95 > 3 s |
15 min | P2 | Ticket |
| Salt rotation missing | No salt-rotate completion by 00:10 UTC |
P1 | Page (privacy-critical) | |
| Salt expiry missing | No salt-expire completion by 00:40 UTC |
P1 | Page | |
| Job failures | Any job with failed increase ≥ 3 in 1 h (except import-run, ga4-import-run) |
1 h | P2 | Ticket |
| Queue backlog | jobs_queue_depth > 1,000 |
10 min | P2 | Ticket |
| Email failures | email_sent_total{result="failed"} ratio > 10% |
30 min | P2 | Ticket |
| Stripe webhook errors | stripe_webhook_total{result="error"} ≥ 3 |
15 min | P1 | Page |
| Invalid webhook signatures | invalid_signature ≥ 20 |
10 min | P3 | Chat (probing) |
| Reconcile stale | No usage-reconcile completion |
30 min | P2 | Ticket |
| Disk | ClickHouse or Postgres disk > 80% (P2) / > 90% (P1) | 10 min | P2/P1 | Ticket / Page |
| Certificates | TLS certificate expiry < 14 days | P2 | Ticket | |
| Backup failure | No successful Postgres base backup in 26 h or ClickHouse backup in 26 h | P1 | Page | |
| Restore test failed | Monthly restore test failed | P2 | Ticket | |
| Error log spike | level=error lines > 50/min |
5 min | P2 | Ticket |
Severity: P1 page immediately; P2 respond within 4 business hours; P3 next business day.
16.10 Upgrades and Zero-Downtime Deploys #
- Versioning: semantic versioning of the image. Patch and minor releases are drop-in. Major releases may contain destructive migrations and list upgrade notes; upgrading across majors requires first reaching the last minor of the previous major.
- Hosted rollout: run
migratejob; rolling update ofwebwithmaxUnavailable: 0andmaxSurge: 25%, each new pod must pass/readyzbefore receiving traffic and old pods drain with the SIGTERM behavior of Section 16.2.1; then rolling update ofworker(running jobs finish or are retried). Rollout pauses automatically when the 5xx rate ofPOST /api/eventexceeds 1% for 2 minutes and rolls back to the previous image. - Self-hosted upgrade:
docker compose pull && docker compose up -d. Themigrateservice runs first;webbriefly restarts (ingestion for a few seconds fails; the tracker does not retry, which is accepted). - Tracker compatibility: the tracker file path and the
/api/eventpayload are backward compatible forever within the product (Section 7); new fields are additive. - Feature flags: none. Behavior changes ship in releases, not toggles.
- Rollback: redeploy the previous image tag (compatible by Section 16.7). Data migrations that a rollback cannot reverse are forbidden by the compatibility rule.
16.11 Capacity Planning #
16.11.1 Workload model #
Maximum billable events per month equal the sum of caps over all active licenses. Example at launch scale: 5,000 Solo (100,000) + 2,000 Pro (500,000) + 500 Agency (2,000,000) = 2.5 billion events per month at 100% utilization. Planning assumes 40% average utilization (1.0 billion per month, about 385 events per second average) and a peak factor of 5 (about 1,900 events per second). Non-billable engagement events add roughly 70% to the stored rows (a pageview typically has one engagement event); capacity numbers below already include them.
| Resource | Rule of thumb |
|---|---|
web |
One 2 vCPU pod handles about 1,500 POST /api/event per second plus dashboard load; plan peak ÷ 1,000 pods, minimum 3 |
| ClickHouse storage | About 90 bytes per stored row compressed (events plus sessions); 1.7 billion rows per month ≈ 150 GB per month, ≈ 1.8 TB per year before TTL |
| ClickHouse memory/CPU | 16 vCPU / 64 GiB sustains the launch workload with p95 stats queries under 1 s for sites up to 50 million events per month; add a second node when sustained CPU > 60% or disk > 70% |
| Postgres | Rows are small; 4 vCPU / 16 GiB handles tens of thousands of accounts; watch connection counts (web pods × pool size must stay below 80% of max_connections) |
| Valkey | Memory ≈ 1 KB per active session key and 200 bytes per team-month counter; 4 GiB is ample for the launch scale |
16.11.2 Self-hosted sizing #
| Monthly events | Server |
|---|---|
| Up to 10 million | 2 vCPU, 4 GiB RAM, 40 GB disk |
| Up to 100 million | 4 vCPU, 16 GiB RAM, 200 GB SSD |
| Up to 1 billion | 8 vCPU, 32 GiB RAM, 1.5 TB NVMe; move ClickHouse to its own host |
16.12 Disaster Recovery #
| Scenario | RPO | RTO | Procedure |
|---|---|---|---|
| Web/worker pod or node loss | 0 (stateless) except the rows still in the buffer: normally at most one flush interval (≤ 2 s); up to 100,000 events during a ClickHouse outage | 2 min (auto-reschedule) | Orchestrator replaces pods |
| Single-zone loss | Rows in the lost pods' buffers (as above) | 10 min | Traffic served from remaining zones; managed Postgres and Valkey fail over |
| Postgres failure or corruption | ≤ 5 minutes (WAL) | 1 hour | Managed failover; else PITR restore to a new instance, update DATABASE_URL secret, roll web/worker |
| ClickHouse node loss | ≤ 1 hour (hourly snapshot); events during the gap are lost | 4 hours | Provision a node from the latest snapshot, replay backups if the snapshot is unusable, repoint CLICKHOUSE_URL. Ingestion buffers up to INGEST_BUFFER_MAX_ROWS (100,000) event rows per pod during a short outage; beyond it new events are dropped with 202 and buffer_full counted (Section 8.15.3) |
| Valkey loss | None required | 15 min | Start empty. Effects: POST /api/event answers 503 with Retry-After: 5 while Valkey is down (Section 8.15.4); dashboard sessions continue (sessions live in Postgres per Section 11), the live visit sessions restart (a visit may split), usage counters rebuild from usage-reconcile within 10 minutes, rate-limit windows reset |
| Accidental deletion by operator | ≤ 5 minutes (Postgres) / ≤ 24 hours (ClickHouse incremental) | 4 hours | Restore to staging, export the affected site rows, import into production by site id; run the deletion replay of Section 16.8.3 for unrelated deletions |
| Region-wide loss | ≤ 24 hours (replica bucket) | 24 hours | Rebuild the stack in the second EU region from infrastructure code and restore from the replica bucket; DNS cutover |
| Stripe outage | n/a | n/a | Checkout shows billing_unavailable; ingestion and dashboards unaffected; stripe-reconcile fulfills missed webhooks afterward |
SECRET_KEY_BASE loss |
n/a | n/a | Sessions and encrypted values (TOTP secrets, OAuth tokens) are unrecoverable: all users must log in again, 2FA is reset by a documented operator procedure after identity verification, GA4 connections are re-authorized. Prevented by escrow (Section 16.8.1) |
The disaster-recovery runbook is exercised once per year as a game day, including a full restore into an empty region.
17. Security, Privacy & Compliance #
17.1 Principles #
- Collect nothing identifying. Raw IP addresses and User-Agent strings are used in memory to compute the daily hash and the location, and are never stored or logged anywhere (Sections 8, 16.9.2, 17.8).
- Deny by default. Every route declares the permission it requires; a route without a declaration fails the route-registration test (Section 18).
- Defense in depth. Validation at the edge (Zod), authorization at the handler, scoping in every query, rate limits per route family, safe headers per surface.
- Secrets only in the environment. No secret is stored in the database in plaintext; credentials stored for users (OAuth tokens, TOTP secrets) are encrypted with keys derived from
SECRET_KEY_BASE(Section 17.7). Documented exceptions, stored in clear because they are revocable bearer values that must be shown again or found by lookup: shared-link tokens (192 bits), report unsubscribe tokens (256 bits), site proxy keys (160 bits) and marketplace codes (60 bits, brute-force limited by Section 15.8.6). - Same rules in both editions. Self-hosted installations get the same headers, limits and redaction. Only the operator-controlled parts (TLS, backups, host hardening) differ.
17.2 Threat Model (STRIDE) #
| Surface / asset | S — Spoofing | T — Tampering | R — Repudiation | I — Information disclosure | D — Denial of service | E — Elevation of privilege |
|---|---|---|---|---|---|---|
Tracker ingestion POST /api/event |
Anyone can post events for any domain. Mitigation: only registered domains are accepted; the per-site hostname allowlist drops events whose URL host is not allowed (reason hostname_not_allowed, not billable, still 202); bot, datacenter and spam-referrer filters; IP exclusions; the X-Quietstat-Client-IP header is trusted only together with the site's valid proxy key (constant-time check; with an invalid key the header is ignored and the request is limited as a normal client). Fake traffic cannot be fully prevented and is documented as a limitation. |
Payload fields validated and size-limited (Section 8); props are length-limited strings; no execution of payload content. | Not applicable (anonymous). | Response is an empty 202 regardless of outcome so the endpoint cannot be used to learn whether a domain exists or is capped. |
Policies event_ingest and event_ingest_proxy (Section 17.6), body limit 32 KB, bounded ingestion buffer with drop-on-full (Section 8.15). |
None: endpoint has no privileges. |
Dashboard app /api/app/v1 |
Session cookie HttpOnly; Secure; SameSite=Lax; passwords argon2id; optional TOTP; login throttling (Section 17.6) |
CSRF header required on all mutating requests; JSON bodies only; strict Zod schemas | Audit log for security-relevant and operator actions; auth events logged with account_user_id |
Resources outside the caller's teams return 404 not_found; error messages never reveal account existence at login or reset |
Rate limits; pagination caps; query timeouts | Permission checks per Section 11; role changes require owner/admin and are audited |
Public APIs /api/v2, /api/v1 |
Bearer API keys, SHA-256 hashed at rest, shown once | Mutations only via Sites API with key scope | Key id recorded in request logs and audit entries | Keys scoped to a team; responses limited to that team's sites | Per-key limits (Section 17.6); query cost limits (Section 13) | Keys cannot exceed the creating role's permissions (Section 11) |
Share links / embeds /api/share/v1 |
192-bit random tokens (Section 12.2.2); optional password (argon2id) with attempt limits | Read-only API; no mutation routes | Link creation and deletion audited | Referrer-Policy: no-referrer; share responses contain only the shared site's data; segment-limited links enforce the fixed filter server-side |
Per-token and per-IP limits | Tokens cannot reach /api/app/v1 |
| Stripe webhook | Signature verification, 300 s tolerance | Raw-body HMAC; replays idempotent (Section 15.6.5) | Event ids logged; purchases/audit rows | No payload logged | 1 MiB body limit; cheap rejection before DB work | Route cannot act except through the three handled event types |
| Operator console | Super-admin allow-list by verified email, mandatory TOTP, step-up re-auth with password and TOTP code | Reason required and audit row in the same transaction as the change | Append-only audit_log; impersonation announced to the user |
Read-only impersonation; no visitor analytics visible in /admin |
Admin rate limit admin_api (Section 17.6) |
Env-based allow-list cannot be edited from the app |
| Workers / jobs | Jobs enqueued only by the application (Postgres access required) | Payloads contain ids only | Job outcomes in pg-boss archive and metrics | No PII in payloads or logs | Concurrency limits; timeouts; retries with backoff | Workers run with the same non-root user and no extra privileges |
| Postgres / ClickHouse / Valkey | Network-isolated, password authentication, TLS in hosted (verify-full for Postgres, rediss:// with a password for Valkey, https:// for ClickHouse; startup fails otherwise, Section 16.4.3) |
Application users have table-level privileges only (no superuser); migrations run with a separate DDL role in hosted | Backups and restore tests (Section 16.8) | Disk encryption; backups encrypted; no raw IP/UA exists to leak | Connection pool caps; query timeouts | Separate credentials per datastore |
| Supply chain | Pinned actions by digest; signed commits for maintainers | Lockfile + frozen installs; provenance attestation | Release SBOM | Secrets scanning in CI | Dependency review gates | No postinstall scripts except an allow-list |
Residual risks accepted and documented to operators: (a) ingestion cannot authenticate the sender, so fake events for a victim's domain are possible and are mitigated by filters, exclusions, the per-site hostname allowlist and, on hosted, the operator action "Grant monthly allowance" after a cap-griefing incident (Section 15.17.4); (b) a malicious site owner can place personal data in custom properties; the UI warns against it (Section 17.9.2); (c) the hash user_id of a given day is stable within that day by design.
17.3 Authentication and Authorization Summary #
Authoritative definitions are in Section 11; this section states cross-cutting obligations.
| Topic | Rule |
|---|---|
| Sessions | Server-side auth_sessions rows; opaque random 256-bit id stored hashed (SHA-256); cookie attributes HttpOnly, Secure (unless COOKIE_SECURE=false), SameSite=Lax, Path=/; cookie name and lifetimes per Section 11. Session id rotation triggers are defined in Section 11.6.3. Operator impersonation uses a separate cookie __Host-qs_imp (30-minute lifetime) and never touches the operator's own session (Section 15.17.11). |
| CSRF | All POST/PUT/PATCH/DELETE under /api/app/v1 require header X-CSRF-Token matching the session-bound token; additionally the Origin header, when present, must equal BASE_URL's origin, else 403 csrf_invalid. Routes that authenticate with a bearer secret are exempt (Section 11.7), as is the unauthenticated, stateless POST /api/app/v1/csp-report. POST /api/app/v1/setup and the other pre-login POST routes use the anonymous CSRF token. |
| Password and 2FA | argon2id (parameters and policy per Section 11); TOTP secrets encrypted (Section 17.7); recovery codes hashed; owners may enforce 2FA for their team per Section 11. |
| Authorization point | One authorize(user, permission, resource) function used by every route; resources are always resolved through the caller's memberships (query scoped by team_id/site_id joins). A failed lookup returns 404, a failed permission on an existing visible resource returns 403. |
| API keys | qs_live_ followed by 43 base64url characters (256 bits), stored as SHA-256 hash with the first 12 characters kept for display; last_used_at updated at most once per 60 s (Section 13.2.1); revocable. Keys created by a user are deleted when that user leaves the team or deletes the account (Section 11.16.5). |
| Super admin | Separate trust level (Section 15.17): the address must be in SUPER_ADMIN_EMAILS, verified by consuming a verification token (the no-SMTP auto-verification of a self-hosted install never qualifies; the variable is ignored there), and TOTP must be enabled (403 admin_2fa_required otherwise). Sensitive actions need step-up with password and TOTP code. Never implicit access to team data. |
| Account enumeration | Login and password-reset responses are identical for existing and unknown emails, with timing equalized by always computing a hash. Signup returns 409 email_taken for an existing address by decision (Section 11.3.1); it is bounded by the signup_ip limit. |
| Lockout | Section 17.6 limits; no permanent lockout (denial-of-service risk); a notification email after 5 failed attempts for an existing account. |
17.4 Input Validation and Output Handling #
| Concern | Rule |
|---|---|
| Schema validation | Every route validates params, query, headers it reads, and body with strict Zod object schemas from packages/shared; unknown keys are rejected with 400 validation_failed; strings are trimmed where the field says so; control characters (U+0000–U+001F except tab/newline in free-text fields) are rejected |
| Body limits | /api/event 32 KB; Stripe webhook 1 MiB; CSV import upload route 500 MB in total (200 MB per file, Section 14.4.1); /api/v2/* and /api/v1/* 256 KB; every other JSON body 256 KB; multipart only on the import upload route. The hosted load balancer admits 1 MiB everywhere except the import upload route (Section 16.5.1) |
| JSON | Max nesting depth 20; duplicate keys rejected; prototype keys (__proto__, constructor, prototype) rejected anywhere in objects (custom property names included) |
| SQL | Postgres via Drizzle parameterized queries only; ClickHouse via typed query parameters ({name:Type}); the query engine (Section 13) builds SQL only from whitelisted dimension/metric identifiers; no string concatenation of user input; dynamic identifiers come from a closed enum |
| XSS | React escapes output; dangerouslySetInnerHTML is forbidden (lint rule); all analytics strings (paths, referrers, UTM values, property values, page titles) are untrusted text; external links built from referrers/URLs use only http:/https: and rel="noopener noreferrer nofollow"; emails use escaped templates |
| CSV injection | CSV exports prefix any cell starting with =, +, -, @, tab or carriage return with a single quote '. The guard applies to every CSV the product produces: site exports, account data exports, audit exports and marketplace code-batch exports (Section 14.2) |
| Open redirects | Redirect targets (next, return_to) must be same-origin relative paths: start with a single /, not // or /\, contain no scheme or backslash; otherwise / is used |
| SSRF and outbound HTTP | All server-side fetches go through one function, guardedFetch in apps/server/src/security/fetch.ts: https only, to port 443 only (plain http is allowed for 127.0.0.1 in tests); the host must be on the fixed allow-list of Section 16.5.2 or be the site domain of an installation check (Section 7.15.2), the only fetch of a user-supplied host; DNS is resolved once and the connection is pinned to the resolved address; private, loopback and link-local results are rejected, including IPv4-mapped and IPv4-compatible IPv6 (::ffff:0:0/96, evaluated as IPv4), NAT64 64:ff9b::/96, 0.0.0.0/8, 192.0.0.0/24, 198.18.0.0/15, 240.0.0.0/4, 255.255.255.255 and documentation ranges; TLS certificates are always verified; redirects are re-validated hop by hop (maximum 3); 10 s timeout; no cookies or credentials are sent |
| Path traversal | Files are addressed by generated ids; DATA_DIR subpaths are built from UUIDs and validated table names (imports/{import_id}/{table}.csv). User-supplied file names are never used in paths, and upload parts whose names fail the validated pattern are rejected before any byte is written (Section 14.4.1) |
| File uploads | Only the CSV import route; size and row limits per Section 14, enforced on bytes actually inflated while streaming; content type and magic-number check; streamed parsing; stored with a generated name and deleted after processing |
| Email header injection and recipients | User-supplied strings used in subjects/names are stripped of CR/LF. Recipient addresses are user accounts, or external addresses added by a user holding email_report.manage or spike.manage; an external address receives nothing until its owner confirms through a one-time link (unconfirmed rows are deleted after 7 days), on hosted adding one requires an active license, and every report and alert email carries a one-click unsubscribe link. Report bodies render visitor-supplied dimension values as plain text, never as links |
| ReDoS | User-supplied regular expressions exist only in Stats API matches filters; they are RE2 (linear time) and executed by ClickHouse, at most 250 characters and 5 per query (Section 13.4.5). Page exclusions use a glob subset compiled to linear-time matchers (Section 8) |
| URL handling | Event URLs parsed with the WHATWG URL parser; only http/https; credentials in URLs dropped; query strings stripped except allow-listed parameters (Section 8) |
| Unicode | Domains normalized to lowercase punycode; text normalized to NFC before storage |
17.5 Security Headers, CSP and CORS #
17.5.1 Headers per surface #
Headers are set by one Fastify hook in apps/server/src/security/headers.ts, selected by route family.
| Surface | Routes | Headers |
|---|---|---|
App (SPA shell and /api/app/*) |
/, /teams/*, /sites/*, /admin/*, /setup, /api/app/* |
Content-Security-Policy (below); Strict-Transport-Security: max-age=63072000; includeSubDomains; preload (hosted) or max-age=31536000; includeSubDomains (self-hosted, only when BASE_URL is https); X-Content-Type-Options: nosniff; Referrer-Policy: strict-origin-when-cross-origin; Permissions-Policy: camera=(), microphone=(), geolocation=(), payment=(), usb=(), interest-cohort=(); Cross-Origin-Opener-Policy: same-origin; Cross-Origin-Resource-Policy: same-origin; X-Frame-Options: DENY; API responses add Cache-Control: no-store |
| Share pages | /share/:token, /api/share/v1/* |
Same as App except: Referrer-Policy: no-referrer; CSP frame-ancestors 'none'; X-Robots-Tag: noindex, nofollow, noarchive |
| Embeds | /embed/:token |
Same as Share pages except: CSP frame-ancestors is computed per link from embed_mode and embed_origins (Section 12.4.3), no X-Frame-Options, Cross-Origin-Resource-Policy: cross-origin; no cookies are set or read on this route |
| Public dashboards | /public/:domain |
Same as Share pages |
| Embed script | /embed.js |
Access-Control-Allow-Origin: *; Cross-Origin-Resource-Policy: cross-origin; X-Content-Type-Options: nosniff; Cache-Control: public, max-age=86400; no Set-Cookie |
| Tracker files | /js/* (tracker script and its source map) |
Access-Control-Allow-Origin: *; Cross-Origin-Resource-Policy: cross-origin; X-Content-Type-Options: nosniff; Content-Type: application/javascript; charset=utf-8; no Set-Cookie; cache headers per Section 7 |
| Event endpoint | POST /api/event |
CORS per Section 17.5.3; Cross-Origin-Resource-Policy: cross-origin; Cache-Control: no-store; no Set-Cookie ever |
| Public API | /api/v2/*, /api/v1/*, /api/openapi.json |
Cache-Control: no-store; X-Content-Type-Options: nosniff; no CORS headers, except GET /api/openapi.json which sends Access-Control-Allow-Origin: *; HSTS as App |
| Static well-known | /.well-known/security.txt, /healthz, /readyz |
X-Content-Type-Options: nosniff; Cache-Control: no-cache |
| Stripe webhook | /api/webhooks/stripe |
Cache-Control: no-store |
17.5.2 Content Security Policy #
App and Share pages (embed differs only in frame-ancestors):
default-src 'none';
script-src 'self';
style-src 'self';
style-src-attr 'unsafe-inline';
img-src 'self' data:;
font-src 'self';
connect-src 'self';
manifest-src 'self';
worker-src 'none';
frame-src 'none';
object-src 'none';
base-uri 'none';
form-action 'self';
frame-ancestors 'none';
upgrade-insecure-requests;
report-to cspNotes: style-src-attr 'unsafe-inline' exists only because the chart library emits inline style attributes; there is no 'unsafe-inline' or 'unsafe-eval' for scripts or style elements; the SPA build emits no inline scripts. The theme bootstrap is the static file /assets/theme-init.js, loaded as a render-blocking <script src> before the app bundle. Runtime configuration is fetched from GET /api/app/v1/bootstrap or inlined as a non-executable <script type="application/json" id="qs-bootstrap"> element, which the browser never executes and which script-src 'self' therefore allows. Source favicons and country flags are served from 'self' (proxied or bundled, Section 9). The Stripe Checkout redirect is a top-level navigation and needs no CSP allowance. The header Reporting-Endpoints: csp="/api/app/v1/csp-report" accompanies it.
CSP reports: POST /api/app/v1/csp-report (unauthenticated, exempt from CSRF because it changes no state; accepts application/csp-report and application/reports+json; body ≤ 8 KB; policy csp_report, Section 17.6; always 204). The handler logs one warn line with directive, blocked_origin (scheme+host only) and document_path (no query), nothing else, and increments quietstat_csp_reports_total.
17.5.3 CORS policy per route family #
| Route family | Access-Control-Allow-Origin |
Methods / headers | Credentials | Max-Age |
|---|---|---|---|---|
POST /api/event |
* |
POST, OPTIONS; allow Content-Type; the tracker uses text/plain and never triggers preflight; JSON clients get a 204 preflight |
No | 86400 |
/js/* |
* |
GET, HEAD |
No | n/a |
/api/app/v1/* |
none (same-origin only). Cross-origin requests with an Origin other than BASE_URL are rejected 403. |
n/a | Cookies only same-origin | n/a |
/api/share/v1/* |
none (the embed page and the API share an origin) | n/a | No cookies used | n/a |
/api/v2/*, /api/v1/* |
none; server-to-server only. OPTIONS returns 204 without CORS headers so browsers block cross-origin calls |
n/a | No | n/a |
GET /api/openapi.json |
* |
GET, OPTIONS |
No | 86400 |
/embed.js |
* |
GET, HEAD |
No | n/a |
/api/webhooks/stripe, /healthz, /readyz, /metrics |
none | n/a | No | n/a |
17.6 Rate Limits (All Limits in the Product) #
This table is the only place where rate-limit values are stated. Other sections name a policy and cite this section. Policy names equal the details.reason values returned with 429 rate_limited.
Implementation: @fastify/rate-limit with a Valkey store. Keys use an HMAC (key derived from SECRET_KEY_BASE, purpose ratelimit) of the client IP, truncated to 16 hex characters, never the raw IP and never the daily salt; user-, team- and key-scoped limits use ids. Windows are fixed windows unless marked "sliding". Exceeding a limit returns 429 rate_limited (plain text rate limited for /api/event) with Retry-After in seconds and the headers X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset (not on /api/event).
Valkey failure policy per family: (1) bearer-key routes (/api/v1, /api/v2) fail closed with 503 unavailable immediately (Section 13.3.3); (2) authentication-sensitive policies (login_*, password_*, reauth_user, admin_reauth, code_redeem_*, share_password_*, setup_token) switch at once to per-process counters with the same limits and fail closed with 503 unavailable after 60 s; (3) everything else, including event_ingest, fails open for 60 s (counted in valkey_errors_total) and then falls back to an in-process limiter of 10 requests/second per IP-HMAC. Ingestion itself answers 503 with Retry-After: 5 while Valkey is down because session state cannot be read (Section 8.15.4).
| Policy name | Scope | Limit | Window | Notes |
|---|---|---|---|---|
event_ingest |
IP | 100 requests | 10 s | POST /api/event; returns 429 plain text. Applies to every request except those carrying a currently valid per-site proxy key |
event_ingest_proxy |
Socket peer address (the proxy) | 2,000 requests | 10 s | POST /api/event requests whose X-Quietstat-Proxy-Key is validated in constant time against the configured keys (Section 8.7). An unknown or invalid key grants nothing: the header is ignored and the request counts under event_ingest |
signup_ip |
IP | 5 signups | 1 h | |
login_ip |
IP | 20 attempts | 15 min | |
login_email |
SHA-256 of the normalized email (exists or not) | 5 failed attempts, then a 15-minute lock | 15 min | Lockout semantics in Section 11.8 |
login_2fa_ip |
IP | 10 attempts | 15 min | |
login_2fa_user |
user | 10 failed attempts | 15 min | TOTP and recovery codes both count; cleared by a successful second step |
password_forgot_email |
SHA-256 of email | 3 requests | 1 h | |
password_forgot_ip |
IP | 10 requests | 1 h | |
password_reset_ip |
IP | 10 attempts | 1 h | |
verify_resend_user |
user | 3 | 1 h | |
verify_token_ip |
IP | 20 | 1 h | |
email_change_user |
user | 5 requests | 24 h | |
reauth_user |
user | 10 failed checks across all step-up endpoints | 15 min | Section 11 |
invitation_create_team |
team | 100 invitations (all kinds) | 24 h | Section 11 |
invitation_resend |
invitation | 1 per minute and 5 per 24 h | rolling | |
invitation_lookup_ip |
IP | 60 | 1 h | GET /invitations/lookup and POST /invitations/decline |
unsubscribe_ip |
IP | 30 | 1 min | Section 12.10 |
share_password |
IP and link | 5 failures | 15 min | Section 12.2.4 |
share_password_link |
link | 30 failures | 1 h | All IPs together |
share_view |
IP and link (IP and site for public dashboards) | 120 requests | 1 min | /api/share/v1/*; each query inside a batch counts as one request |
email_report_external_add |
site | 20 new external recipients | 24 h | Section 12.5.1 |
external_recipient_add |
team | 50 new external recipients | 24 h | Sum over all sites of the team |
email_report_test |
user | 5 | 1 h | Section 12.5.4 |
system_test_email |
user | 5 | 1 h | Operator SMTP test message (Section 12.9.7) |
app_api_user |
user | 600 requests | 1 min | /api/app/v1/* general |
app_api_ip_unauth |
IP | 120 requests | 1 min | Unauthenticated /api/app/v1/* routes |
dashboard_query |
user | 240 queries | 1 min | POST /api/app/v1/sites/{domain}/query and /query/batch; each query inside a batch counts |
tracking_verify |
user | 10 | 1 min | Installation check (Section 7.15.2) |
stats_api_key |
API key | 600 requests | 1 h | POST /api/v2/query (Section 13.3.3) |
stats_api_team |
team | 2,400 requests | 1 h | All Stats API keys of a team together |
stats_concurrency |
API key; team | 5 in flight; 10 in flight | instantaneous | Stats API queries; excess returns 429 with details.reason = "concurrency" |
sites_api_key |
API key | 600 requests | 1 h | /api/v1/sites* (Section 13.3.3) |
api_auth_fail |
IP | 30 failed authentications | 1 min | Bearer routes (Section 13.3.1) |
openapi |
IP | 60 requests | 1 min | GET /api/openapi.json |
site_create |
team | 20 | 1 h | Site creation through UI or API |
export_start |
user | 10 export requests | 24 h | CSV exports (Section 14.3.1) |
import_start |
team | 5 | 24 h | CSV and GA4 imports (Section 14) |
account_data_export |
user | 1 | 24 h | Section 17.9.3 |
checkout_create |
user | 10 | 1 h | Section 15.6.2 |
code_redeem_fail |
user; IP | 10; 30 failures | 1 h sliding | Section 15.8.6 |
code_redeem_success |
user | 5 | 1 h | |
admin_api |
user | 120 requests | 1 min | /api/app/v1/admin/* |
admin_reauth |
user | 5 failures | 15 min | Operator step-up, Section 15.17.2 |
impersonation_start |
operator | 20 sessions | 24 h | Section 15.17.11 |
setup_token |
IP | 5 failures | 1 h | Section 16.3.5 |
csp_report |
IP | 30 | 1 min | |
tracker_files |
IP | 600 | 1 min | /js/* |
webhook_stripe |
none | no limit (signature-verified) | 1 MiB body cap | |
email_send |
global | EMAIL_MAX_PER_MINUTE |
1 min | Queue throttle |
17.7 Secrets Handling and Encryption at Rest #
17.7.1 Key derivation #
All keys derive from SECRET_KEY_BASE with HKDF-SHA-256 (salt quietstat/v1, info = purpose string, 32-byte output). SECRET_KEY_BASE is the single root secret: it derives the AES-256-GCM keys that encrypt TOTP secrets and GA4 OAuth tokens, and the HMAC keys for tokens, rate-limit counters and unlock tokens. It is rotated only with the procedure of Section 17.7.4, which uses SECRET_KEY_BASE_PREVIOUS to decrypt and then re-encrypt every stored value. Purposes: session-cookie-sign, csrf, totp-encryption, oauth-token-encryption, email-token-hmac, ratelimit, unsubscribe-token, share-unlock. Keys are derived at start and held in memory; they are never persisted or logged. The key id kid is the first 8 hex characters of SHA-256 of the derived key.
17.7.2 Encrypted columns #
AES-256-GCM, a fresh random 12-byte nonce per value, 16-byte tag, associated data = "<table>.<column>:<row uuid>". Stored text format: v1.<kid>.<base64url(nonce || ciphertext || tag)>. Every encrypted column is text in this format.
| Data | Column | Purpose key |
|---|---|---|
| TOTP shared secret | users.totp_secret_encrypted |
totp-encryption |
| Google OAuth access and refresh tokens for the GA4 import | imports.oauth_credentials |
oauth-token-encryption |
Not encrypted but hashed: passwords (argon2id), recovery codes, share-link passwords (argon2id), API keys and session ids (SHA-256 of high-entropy values), email verification, password-reset and invitation tokens (HMAC-SHA-256 with email-token-hmac, 1-hour to 7-day lifetimes per Section 11). Stored in clear by design, as revocable bearer values that must be re-displayed or found by a unique index: shared-link tokens (192 bits, Section 12.2.2), report unsubscribe tokens (256 bits), site proxy keys (160 bits, Section 7.15.4) and marketplace codes (Section 15.8.2). Stripe and SMTP credentials exist only in the environment.
17.7.3 Transport and storage #
TLS 1.2 or 1.3 for all external traffic; hosted datastores use TLS (verify-full for Postgres). Disk encryption: provider-managed volume encryption in hosted; self-hosters are advised to enable full-disk encryption. Decrypted secrets are held only in request-scope memory.
17.7.4 Rotation #
- Set
SECRET_KEY_BASE_PREVIOUSto the old value andSECRET_KEY_BASEto the new one; deploy. Reads of encrypted values select the key bykid(current or previous); writes use the current key. CSRF tokens and share unlock tokens signed with the previous key are accepted (verified against both keys) until the previous key is removed. Session cookies and unsubscribe tokens are random values and are unaffected. - Run
migrate --reencrypt-secretsonce: it re-encrypts every encrypted column with the current key in batches of 500 inside transactions and prints counts. - Remove
SECRET_KEY_BASE_PREVIOUSand redeploy. Maximum 2 keys are ever configured. Rotation is required annually (hosted) and on suspected exposure.
17.8 Log Redaction #
Fastify is created with disableRequestLogging: true, so no raw URL is ever logged. pino is configured with redact: { paths, censor: '[REDACTED]', remove: false } and a custom request serializer that emits only: method, route (the route pattern, never the concrete URL), status, duration_ms, request_id, account_user_id, team_id, api_key_id. The field that identifies a dashboard account is named account_user_id so it can never be confused with the visitor hash user_id; the visitor hash user_id and session_id are never logged. Because the serializer is allow-list based, redaction paths are a second layer (pino path syntax supports * for one level only, so nested keys are listed explicitly down to two levels):
req.headers.authorization, req.headers.cookie, req.headers["x-csrf-token"],
req.headers["x-forwarded-for"], req.headers["x-real-ip"], req.headers["cf-connecting-ip"],
req.headers["true-client-ip"], req.headers.forwarded, req.headers["x-quietstat-client-ip"],
req.headers["x-quietstat-proxy-key"], req.headers["x-share-unlock"], req.headers["user-agent"],
req.headers.referer, req.headers.origin, req.ip, req.ips, req.remoteAddress, req.socket,
res.headers["set-cookie"],
*.password, *.*.password, *.password_confirmation, *.current_password, *.totp_code, *.recovery_code,
*.token, *.*.token, *.setup_token, *.code, *.api_key, *.secret, *.client_secret, *.refresh_token,
*.access_token, *.proxy_keyRules: (1) IP and User-Agent are never logged at any level, including debug and trace; (2) request and response bodies are never logged; (3) query strings and concrete URLs are never logged (several paths carry bearer tokens: /share/<token>, /embed/<token>, /invitations/<token>, /unsubscribe/<token>, /api/share/v1/link/<token>); (4) emails are never logged (use account_user_id); (5) caught errors are logged with message and stack but errors from database drivers are scrubbed of parameter values; (6) the allowed secrets in logs are the first-run setup token (Section 16.3.5) and, only on a self-hosted install without SMTP, the action_url field of the email_not_sent line (Section 12.9.6); (7) the reverse proxy removes IP, User-Agent and the whole request URI from its access log (Section 16.3.3), and the hosted load balancer keeps no access logs (Section 16.5.1). A unit test feeds synthetic IPv4/IPv6 addresses, User-Agent strings, tokens and a fixture containing every forbidden key (at depth one and two) through every logging path and asserts that none appear in the output (Section 18).
17.9 Privacy and Compliance #
17.9.1 Roles and legal position #
| Data | Hosted operator's role | Self-hosted operator's role |
|---|---|---|
| Account data (name, email, password hash, 2FA secret, billing details, invoices) | Controller | Controller |
| Visitor analytics data collected for a customer's site | Processor (the customer is the controller) | Controller |
| Billing/payment data processed by Stripe | Stripe acts as independent controller for payment processing; the operator controls invoice records | Not applicable |
Position statement for the product's privacy documentation: the analytics pipeline does not process personal data in the sense of GDPR because no identifier that relates to an identified or identifiable person is stored. This is claimed under the conditions in Section 17.9.2 and is not legal advice; each operator is responsible for their own legal assessment.
17.9.2 Conditions for the "cookie-less, no personal data" claim #
The claim holds only if all conditions are true; the test suite and the release checklist verify each (Section 18):
- The tracker never writes cookies,
localStorage,sessionStorage, IndexedDB, the Cache API or any other device storage, and reads no fingerprinting surface (canvas, WebGL, fonts, plugins, audio, battery, device memory). The only device read is the single opt-out flaglocalStorage.quietstat_ignore(Section 7), which the site owner or the visitor sets to exclude their own visits; the read exists solely to honor that opt-out, and the tracker behaves identically when the flag is absent. - The visitor identifier is the keyed hash of Section 8 with a salt that rotates every 24 hours; the previous salt is deleted by
salt-expireat 00:30 UTC; at most 2 salts exist. Thesaltstable is a PostgresUNLOGGEDtable: it is excluded from the write-ahead log, point-in-time recovery and replicas, and is excluded from logical dumps (pg_dump --exclude-table-data=salts), so no backup ever contains a salt. After a crash or a restore the table is empty and a fresh salt is generated; visitors seen that day may be counted twice, which is documented and accepted. Salts are never exported and never logged. - Raw IP and User-Agent exist only in process memory for the duration of the request and appear in no store, log, queue payload, metric label, trace attribute or backup (Section 17.8).
- Location is limited to country, region and city; browser and OS versions to major.minor; query strings and fragments are dropped except allow-listed parameters.
- No identifier is shared across sites or across days;
user_idhashes include the site id and the day's salt. - Customers do not send personal data in custom properties, page URLs, or the Events API payload. Page paths and referrer paths are stored as sent (query strings removed except allow-listed parameters), so integrators must keep personal data out of URL paths or scrub it with
transformRequest. The product states this at every entry point: the custom property allow-list screen shows the notice "Do not send names, email addresses, user IDs or other personal data in custom properties." and the Events API documentation repeats it. - Retention follows
DATA_RETENTION_MONTHSand the deletion guarantees of Section 17.10.
17.9.3 Data subject requests (account data) #
| Right | How it is served |
|---|---|
| Access / portability | Self-service GET /api/app/v1/account/data-export (policy account_data_export, 1 per 24 h) returns a JSON document with: profile fields, team memberships and roles, own API key metadata (name, prefix, created, last used), own personal segments and annotations, own email report subscriptions, active session list (created time and last-active date; no IPs because none are stored), audit entries where the user is the actor, purchases made by the user (invoice numbers, amounts, dates). Operators can produce the same document with GET /api/app/v1/admin/users/{id}/data-export (step-up required, audited). Impersonation sessions cannot call it. |
| Rectification | Profile editing in settings (Section 11) |
| Erasure | Account deletion in settings: soft delete immediately (all sessions removed; for 30 days a login yields only the restore screen, Section 11.13.3), then anonymization by user-anonymize exactly as Section 11.13.2 (email replaced by deleted-<user id>@deleted.invalid, name set to Deleted user, password hash, TOTP secret, recovery codes, API keys, sessions and tokens deleted, memberships removed; audit_log actor ids and purchases/invoices are retained under legal obligation, Section 17.10). A user who is the sole owner of a team must transfer or delete the team first (Section 11). |
| Restriction / objection | Handled by contacting SUPPORT_EMAIL; the operator may suspend the account (Section 15.17.3) |
| Visitor data subjects | The platform stores no data that can be linked to a person, so access, erasure and portability requests from site visitors cannot be fulfilled with an identifier; the documented answer is that no personal data is held. Site owners can delete a site's data at any time (Section 17.10). |
| Response time | Within 30 days of a verified request; operators track requests in their support system |
17.9.4 DPA and subprocessors (hosted) #
The hosted operator publishes a Data Processing Agreement at DPA_URL, accepted by the customer's acceptance of the Terms (no separate signature needed) and available as a countersigned PDF on request. The DPA states: subject matter and duration (life of the account), nature and purpose (website analytics), categories of data (anonymous aggregate event data; account data of the customer's team members), the processor's obligations, security measures (a summary of Sections 17.3–17.8), assistance with data subject requests, breach notification without undue delay and within 48 hours, return/deletion at termination (Section 17.10), audit rights, and the subprocessor list. The operator gives 30 days' notice of subprocessor changes by email to team owners.
Subprocessor categories the hosted deployment uses (the operator publishes provider names and addresses in the DPA; the product itself needs no provider name):
| Category | Purpose | Personal data involved | Location requirement |
|---|---|---|---|
| Cloud infrastructure (compute, network, managed Postgres, volumes) | Hosting | All stored data | EU region |
| Object storage (backups) | Backups | Encrypted backups | EU, two regions |
| Transactional email relay | Account, report and notification emails | Recipient email, name, email content | EU |
| Stripe (Stripe Payments Europe Ltd for EEA customers) | Payment processing, tax calculation, receipts | Name, email, billing address, card data (held by Stripe only), tax IDs | EU/EEA with Stripe's standard contractual safeguards |
| Content delivery network (optional) | Static tracker and SPA assets | None beyond standard request metadata that the CDN is contractually barred from logging for /js/* |
EU-only pool |
| Error/log storage | Operational logs | account_user_id, team_id only (Section 17.8) |
EU |
No advertising, analytics or tracking service runs on the product's own pages. The hosted dashboard uses its own product (the operator's own site entry) only with the operator's consent.
17.9.5 CCPA and PECR #
- CCPA/CPRA: the operator acts as a service provider; no sale or sharing of personal information; no cross-context behavioral advertising; the privacy policy template lists categories of account data collected and the deletion route of Section 17.9.3. Visitor data contains no personal information as defined, because no identifier is retained that can be linked to a consumer or device.
- PECR / ePrivacy Article 5(3): the tracker writes nothing to the visitor's device and reads exactly one value, the site-owner or visitor opt-out flag (Section 17.9.2 condition 1), which is read solely to honor an opt-out and is therefore not a tracking use of device storage; beyond that it uses only what is technically required to send the request, so the consent requirement does not apply. Operators in jurisdictions that interpret reading browser attributes as device access should add their own assessment; the product has no consent mode.
- Do Not Track / Global Privacy Control: not needed to comply; the tracker does not alter behavior based on them (Section 7).
17.9.6 Outbound connections (transparency) #
Self-hosted installations contact only: the configured SMTP server; download.db-ip.com (GeoIP update, if GEOIP_AUTO_UPDATE is on); the sources used by lists-refresh (if LISTS_AUTO_UPDATE is on); www.ecb.europa.eu (exchange rates, FX_RATES_URL); Google APIs (only after a user connects a GA4 import); and, only when a user runs an installation check, the site domain they entered (Section 7.15.2, through guardedFetch). With GEOIP_AUTO_UPDATE=false and LISTS_AUTO_UPDATE=false only SMTP and the exchange-rate feed remain. The product never sends telemetry, usage statistics, crash reports or license checks to its authors or any third party. Hosted adds Stripe.
17.10 Data Deletion Guarantees and Retention #
| Data | Deletion trigger | Guarantee |
|---|---|---|
Site analytics data (events, sessions, imported_*) |
Site deleted by an owner/admin | Site and settings rows deleted immediately in Postgres; ClickHouse rows removed by site-purge within 24 hours (target 1 hour) with up to 8 retries; the UI shows "Deleting data…" until done; the operation is verified by a count query before the job ends |
| Team | Team deleted | 7-day undo; then team-purge removes all sites' data (as above), members, settings and invitations |
| Unlicensed team | 30 days after revocation | Section 15.14 |
| User account | Deleted by the user | Immediate session revocation; restore-only login for 30 days; anonymization at 30 days (Section 17.9.3) |
| Data older than retention | DATA_RETENTION_MONTHS |
ClickHouse TTL merges remove rows; physical removal can lag up to 7 days after the TTL date (merge_with_ttl_timeout) |
| Exports and uploads | 7 days after completion (exports, expires_at, Section 14.3.1); at job end (imports) |
Files deleted by export-expire / the import jobs |
| Salts | 24 hours (+30 min overlap) | Section 8 |
| Backups | Age out | Postgres 14 days, ClickHouse 35 days (Section 16.8.1); deleted data may survive in backups for at most 35 days and is never restored without the deletion replay of Section 16.8.3. Backups never contain a visitor salt (Section 17.9.2 condition 2) |
| Logs | 30 days | Section 16.9.2 |
| Invoices, purchases, credit notes | Retained for 10 years (tax law) even after account anonymization; they contain billing name, address and tax ID only | Access by super admins only after team deletion |
| Audit log | AUDIT_RETENTION_MONTHS (hosted default 24) |
Section 15.17.10. Rows about licenses, purchases, codes, operator actions and team purges keep their team_id (no foreign key) after a team is purged and are removed only by audit-prune |
| Email relay | Provider retention ≤ 30 days contractually | Subprocessor terms |
17.11 Dependency and Supply-Chain Security #
| Control | Requirement |
|---|---|
| Lockfile | pnpm-lock.yaml committed; CI uses --frozen-lockfile |
| Vulnerability audit | pnpm audit --prod on every pull request and nightly; fail on high or critical without an accepted, dated exception file reviewed monthly |
| Updates | Automated weekly dependency update pull requests; security updates merged within 7 days for critical, 30 days for high |
| Install scripts | onlyBuiltDependencies allow-list in the pnpm configuration (native modules such as argon2 and esbuild only); any new install script needs maintainer approval |
| SBOM | CycloneDX JSON SBOM generated for every release and attached to the release and to the container image as an attestation |
| Image | Base images pinned by digest; image scanned (OS and application layers) in CI and nightly; fail on critical findings that have a fix; images signed (keyless signature) with build provenance attestation; the release job runs on a protected branch only |
| Licenses | CI license check allows MIT, Apache-2.0, BSD-2/3-Clause, ISC, MPL-2.0, 0BSD, CC0-1.0, AGPL-3.0-only, CC-BY-4.0 (data files only) and LGPL-3.0 (dynamic linking only) for dependencies; anything else fails unless approved in a recorded exception |
| CI hardening | GitHub Actions pinned by commit SHA; least-privilege permissions; no secrets exposed to pull requests from forks; secret scanning and push protection enabled |
| Maintainer accounts | Hardware-key two-factor required; signed tags for releases |
| Tracker | Zero runtime dependencies; bundle built reproducibly; the published npm package uses provenance |
| Frontend | No third-party scripts, fonts or analytics loaded by the app (enforced by CSP default-src 'none') |
17.12 security.txt and Vulnerability Disclosure #
GET /.well-known/security.txt returns (RFC 9116, text/plain; charset=utf-8, served by both editions when SECURITY_CONTACT is set):
Contact: <SECURITY_CONTACT>
Expires: <UTC timestamp 180 days after the request, truncated to the day>
Preferred-Languages: en
Canonical: <BASE_URL>/.well-known/security.txt
Policy: <SECURITY_POLICY_URL, line omitted when unset>The repository ships SECURITY.md with the same policy and enables private vulnerability reporting. Policy terms: report by email to the contact or via private reporting; no public disclosure for 90 days or until a fix is released, whichever comes first; safe harbor for good-faith research (no data exfiltration beyond proof, no service degradation, no social engineering, test only your own accounts); out of scope: findings requiring physical access, clickjacking on pages without sensitive actions, missing headers on non-sensitive static files, volumetric DoS, and reports from automated scanners without a proof of impact; no monetary bounty at launch; credit in release notes with consent.
Service levels: acknowledge within 3 business days; triage with severity within 7 days; fix critical within 14 days, high within 30, medium within 90; notify reporters at each step; publish an advisory with CVE for fixed high and critical issues affecting self-hosted installations.
Incident response: hosted operator follows the sequence detect, contain, assess, notify, remediate, review; personal-data breaches affecting account data are notified to the supervisory authority within 72 hours and to affected customers without undue delay; a post-incident review is written within 10 business days.
17.13 Penetration Test Before Launch #
| Item | Requirement |
|---|---|
| Timing | After feature freeze of Milestone M14 (Section 19) and before public launch of the hosted edition |
| Tester | Independent third party with web application and API specialization |
| Scope | Dashboard SPA and /api/app/v1; public APIs; ingestion endpoint and tracker; share and embed pages; Stripe webhook and checkout flow; operator console (with super-admin test credentials); authentication, 2FA, password reset; multi-tenant isolation (cross-team and cross-site access attempts for every role in Section 11); import/export; the Docker Compose self-hosted deployment; infrastructure configuration of the hosted staging environment |
| Method | Grey-box with credentials for each role: owner, admin, editor, viewer, guest viewer, guest editor, super admin; OWASP ASVS Level 2 as checklist |
| Gate | Zero open critical or high findings; medium findings fixed or accepted in writing by the owner with a dated plan; retest of all fixed findings |
| Cadence | Repeated annually and after major architectural changes (new auth method, new public route family) |
| Report handling | Summary (not the full report) shared with customers on request under NDA |
17.14 Abuse Handling (Hosted) #
| Abuse type | Detection | Response |
|---|---|---|
| Customers analyzing phishing, malware or spam sites | Abuse reports to ABUSE_CONTACT_EMAIL; operator review of sites flagged by reports (the product does not scan customer sites) |
Within 24 hours of a credible report: suspend the team (Section 15.17.4), preserve evidence (audit entry), notify the owner by email, delete the site on confirmation; terms of service allow immediate suspension |
| Signup spam / throwaway accounts | Rate limit signup_ip; email verification required before creating sites (no disposable-domain list is kept, Section 11.3.1) |
Unlicensed accounts cannot collect data and are limited to 1 site, so impact is low; the operator may suspend users in bulk from the console |
| Fake traffic to another customer's domain, cap griefing | Bot and datacenter filters; spam referrer list; customers' IP and page exclusions; the per-site hostname allowlist | Documented limitation; the operator can remove spam referrers from the list in a release, and after a griefing incident can grant the victim team extra allowance for the month with the console action "Grant monthly allowance" (Section 15.17.4, audited) |
| Ingestion flooding | event_ingest limit; buffer backpressure; autoscaling; load-balancer rate rules by IP |
Block abusive IP ranges at the load balancer (not in the app, which keeps no IP records) |
| Credential stuffing | login limits, notification emails, optional 2FA enforcement |
Operator can force-logout and suspend accounts |
| Email abuse (invitations, reports) | Invitations limited by invitation_create_team; external report recipients need double opt-in, an active license on hosted, and are limited by email_report_external_add and external_recipient_add; invitation emails include the inviter's name and are plain templates with no custom message field; every report and alert email carries an unsubscribe link (Section 12.5.4) |
Suspend the inviting team |
| Marketplace code leakage | Brute-force protections of Section 15.8.6 | Revoke codes or batches |
| API key abuse | stats_api_key/sites_api_key limits; keys revocable by owners and admins |
Revoke; suspend the team |
| Stripe chargebacks / fraud | charge.dispute.created handler (Section 15.6.6) |
Automatic license revocation and operator alert |
Operator procedure for any action: record a reason in the console (audited), email the affected owner unless law forbids, and review reinstatement within 7 days of a response from the owner.
18. Testing & Quality Strategy #
This section is the single owner of how the product is verified: test layers, tooling, test identifiers, fixtures, CI gates, coverage thresholds and the definition of done. Every other section's behavior is verified by tests that are specified here or listed in the milestone exit criteria in Section 19. Dependency versions for all test tooling are governed by Section 3 and the drift procedure in Section 20.7.
18.1 Principles #
- Tests are part of the deliverable. A feature without its tests is not done (Section 18.16).
- Real dependencies over mocks. PostgreSQL, ClickHouse and Valkey are always real containers in integration and end-to-end tests. Mocks are allowed only for third-party network services (Stripe API, Google OAuth and GA4 APIs, SMTP delivery) and are replaced by local fakes listed in Section 18.4.3.
- Deterministic by construction. No test depends on wall-clock time, randomness, network access, test order or sleeping. Rules in Section 18.2.4.
- Privacy is a tested invariant, not a code-review item: the privacy suite (Section 18.9) runs on every pull request and blocks merge.
- Every test has a stable identifier (for example
ING-07,E2E-23,PRIV-03) that appears at the start of its title. Milestone exit criteria in Section 19 reference these identifiers. The mapping from milestone to identifiers is the machine-readable filetests/milestones.json, andpnpm verify:milestone <Mn>runs it (Section 18.2.3). - Specification-derived expected values. Where this specification states a number (limits, thresholds, prices, tier caps), the test asserts that number literally. Tests never import the constant they are checking.
- Tests are never weakened to pass. A failing test is fixed in the product, or, if the test contradicts this specification, the test is corrected and the correction is recorded in
DECISIONS.md(Section 20.3).
18.2 Test pyramid, layout and conventions #
18.2.1 Pyramid per package #
Target proportions by test count across the repository: unit about 70%, integration about 20%, browser end-to-end about 10%. The table lists what each package must contain.
| Package / app | Unit (Vitest, no I/O) | Integration (Vitest + real containers) | Browser (Playwright) | Other |
|---|---|---|---|---|
packages/shared |
Every Zod schema (valid and invalid tables), constants, money and time helpers, Clock and Rng implementations, error-code registry |
none | none | Schema-versus-database drift tests run from apps/server (Section 18.3.4) |
packages/query |
Golden SQL tests, validation, filter and metric builders, time-range and timezone resolution, injection corpus | Execution of the golden corpus against ClickHouse with fixture data and oracle comparison (Section 18.4.4) | none | Mutation testing nightly (Section 18.15.3) |
packages/tracker |
Option parsing, URL normalization, payload building, event dispatch with fake fetch/sendBeacon, SPA and hash-route logic using a DOM implementation |
none | Real-browser matrix and ingestion end-to-end (Sections 18.5 and 18.7) | Size-budget and forbidden-API checks (Section 18.7.3), npm pack test |
apps/server |
Route handlers with fake dependencies, permission matrix, ingestion enrichment functions, privacy engine (hash, salts, sessions), channel classifier, bot filter, exclusions, auth primitives, CSV parsers and writers, license-code generator, usage-cap state machine, email template rendering, Stripe event mapping | Every route family against real stores (Section 18.4.2), ingestion pipeline, auth flows, teams, sharing, API keys, imports and exports, licensing and webhooks | Covered by Section 18.6 flows | Security suite (Section 18.10), privacy suite (Section 18.9) |
apps/worker |
Job handlers with fake dependencies, schedule definitions, spike-detection math, report assembly | Every job in the job schedule of Section 16 run against real stores and a local SMTP sink | Covered by Section 18.6 flows that trigger jobs | Job-schedule completeness test (Section 18.4.2) |
apps/web |
Pure logic (URL state to filter serialization, formatters, query-builder hooks, permission-aware UI helpers, i18n catalog completeness) and component tests with Testing Library | none | All flows in Section 18.6, accessibility scans, responsive smoke | Bundle-size budget (Section 18.14.2) |
integrations/wordpress |
PHPUnit unit tests per settings class and renderer | PHPUnit with the WordPress test suite; browser test in a WordPress instance started with wp-env |
one Playwright test (Section 18.7.4) | Plugin Check run, PHP matrix |
integrations/gtm |
Template tests executed by an in-repo harness (Section 18.7.5) | none | none | Manual import check in the launch checklist (Section 20.10) |
18.2.2 Repository layout for tests #
Test support code is not a workspace package. It lives in the root tests/ directory:
| Path | Content |
|---|---|
tests/support/ |
Container bootstrap, per-file database isolation helpers, eventually() polling helper, app factory wrapper, HTTP client with CSRF handling, e-mail sink client, ClickHouse and Postgres fixture inserters, normalized event reader fetchEvents(siteId) and fetchSessions(siteId) (they hide ClickHouse column naming from tests so a column rename is a one-file change), fake Stripe/Google servers |
tests/support/generator/ |
Seeded fixture-data generator and independent oracle (Section 18.11) |
tests/fixtures/ |
Static fixtures: stripe/*.json webhook payloads, csv/ import files (valid and malformed), ga4/ recorded response bodies, mmdb/ the synthetic GeoIP test database, db/ previous-release database dumps (Section 18.12) |
tests/ingestion/ |
Playwright specs for tracker-to-ClickHouse scenarios ING-* |
tests/e2e/ |
Playwright specs for user flows E2E-* |
tests/privacy/ |
Vitest integration specs PRIV-* |
tests/security/ |
Vitest integration and Playwright specs SEC-* |
tests/load/ |
k6 scripts LOAD-* |
tests/migrations/ |
Vitest specs MIG-* |
tests/stripe/ |
Vitest specs STR-* |
tests/milestones.json |
Map from milestone id (M1 to M14) to the list of test identifiers that must pass at that milestone |
Unit tests are colocated with source as *.test.ts. Integration tests are colocated as *.int.test.ts. Golden files live next to the query tests in packages/query/test/golden/.
18.2.3 Root scripts #
All scripts exist from milestone M1 (those that have nothing to run yet exit with success and print "no tests").
| Script | Action |
|---|---|
pnpm lint |
ESLint (with the custom rules in Section 18.9.6) and Prettier check over the whole repository |
pnpm typecheck |
tsc -b in strict mode across all TypeScript packages |
pnpm test:unit |
Vitest project unit with coverage |
pnpm test:int |
Vitest project integration (starts containers) |
pnpm test:ingestion |
Playwright project ingestion |
pnpm test:e2e |
Playwright projects e2e-chromium, e2e-firefox, e2e-webkit |
pnpm test:privacy |
Vitest project privacy |
pnpm test:security |
Vitest project security plus the Playwright security specs |
pnpm test:stripe |
Vitest specs in tests/stripe/ |
pnpm test:migrations |
Vitest specs in tests/migrations/ |
pnpm test:load |
k6 scripts (--smoke runs the pull-request smoke subset) |
pnpm test:wp |
PHPUnit for the WordPress plugin |
pnpm test:gtm |
GTM template test harness |
pnpm check:size |
Tracker size budget and forbidden-API scan (Section 18.7.3) plus dashboard bundle budget |
pnpm check:conventions |
Route-registry completeness, JSON field snake_case in Zod schemas, error-code registry, schema snapshot (Section 18.9.5) |
pnpm check:licenses |
Dependency license allow-list and per-package license header check |
pnpm seed:fixtures |
Fixture generator CLI (Section 18.11) |
pnpm verify:milestone <Mn> |
Runs every test identifier mapped in tests/milestones.json for milestones M1 through Mn, then pnpm lint, pnpm typecheck, pnpm check:size, pnpm check:conventions; prints a pass/fail table by identifier; exits non-zero if any identifier is missing, skipped or failing |
pnpm ci |
The full pull-request pipeline of Section 18.14 run locally (excludes the nightly-only stages) |
verify:milestone fails when an identifier listed in the manifest has no test with that identifier at the start of a test title. This makes the exit criteria of Section 19 executable.
Three variables are read only by test tooling under tests/ and are not application configuration: TEST_SERVICES (Section 18.4.1), UPDATE_GOLDEN (Section 18.3.1) and TZ. They are not rows of the table in Section 16.4, and the environment-variable scan of REL-03 covers only apps/ and packages/, where no code reads them.
18.2.4 Determinism rules #
- Time. All server and worker code obtains time through the
Clockinterface exported frompackages/shared(now(): Date,monotonicMs(): number). The production implementation reads the system clock. Tests inject a fake clock that only advances when told. Direct calls toDate.now(),new Date()without arguments andperformance.now()are forbidden outside the productionClockimplementation (enforced by the lint rule in Section 18.9.6). - Randomness. All identifiers and salts that are not database-generated come from the
Rnginterface (bytes(n: number): Uint8Array,u64(): bigint) exported frompackages/shared. Tests inject a seededRng(xoshiro128** seeded from a 32-bit integer).Math.random()is forbidden in server, worker and tracker code except inside the productionRngimplementation. - Application factory.
apps/serverexportsbuildApp(config, deps)andapps/workerexportsbuildWorker(config, deps).configis the validated configuration object defined by the config table in Section 16.4;depsoptionally overridesclock,rng,stripe,mailer,geoandfetch(theguardedFetchoutbound HTTP client of Section 17.4). Integration tests call these factories in-process. Playwright tests start the compiled server as a separate process through the PlaywrightwebServeroption, with configuration supplied through the environment variables of Section 16.4. - Polling instead of sleeping.
eventually(fn, { timeoutMs = 10000, intervalMs = 100 })retries an assertion until it passes or times out.setTimeout-based waits longer than 50 ms are forbidden in tests. The ingestion buffer flushes everyINGEST_FLUSH_INTERVAL_MS(2 s) orINGEST_BATCH_ROWS(5,000 rows), Section 16.4, so the default 10 s timeout covers one full flush with margin. - Process timezone. Test processes run with
TZ=UTC. Site-timezone behavior is tested explicitly with sites whosetimezonevalues areUTC,Europe/Berlin,America/New_York,Asia/Kolkata,Pacific/AucklandandAustralia/Lord_Howe. - Isolation. Each test file owns its own Postgres database, ClickHouse database and Valkey logical database (Section 18.4.1). Tests within a file create their own team and site through helpers with unique domains (
t-<random>.test). Tests never assert on global counts. - Fixed anchor date. Fixture data is generated relative to the anchor instant
2026-03-15T12:00:00Z. Dashboard and report assertions use explicit custom date ranges inside the fixture window (2026-02-15to2026-03-14inclusive in the site timezone), never "today" or "last 30 days", except in the dedicated relative-range tests that use the fake clock in integration tests. - No network. Test processes have no outbound internet access in CI. Any attempt to reach a non-local host fails the test (
guardedFetchof Section 17.4 is replaced in tests by one that throws on non-allow-listed hosts). - Flake policy. CI retries Playwright tests twice; a test that passes only on retry is reported as "flaky" in the job summary. Vitest never retries. A flaky test must be fixed in the same pull request that first shows it. A test may be tagged
@quarantineonly with an entry inDECISIONS.mdnaming the cause; CI fails if more than 3 tests are quarantined or if any quarantined test is tagged on a privacy, security or Stripe test.
18.3 Unit tests #
18.3.1 Query engine golden tests (packages/query) #
The query engine converts a validated query JSON (Section 13) plus a site context (site UUID, timezone, imported-data flag) into a parameterized ClickHouse SQL statement and parameter map. Golden tests pin the exact SQL.
Mechanics.
- Case files:
packages/query/test/cases/*.json, each an object{ "name": "...", "site": { "id": "<uuid>", "timezone": "..." }, "query": { ... }, "now": "<ISO instant>" }. - Output files:
packages/query/test/golden/<name>.sqlholds every compiled statement of the case in execution order (Section 13.5 produces one to four statements per query), each preceded by a comment line-- role: <role>that names the statement's role, with parameter placeholders such as{p1:String};<name>.params.jsonis an array with one entry per statement, in the same order, mapping parameter names to values and ClickHouse types. - A case passes when both generated outputs equal the golden files byte for byte.
UPDATE_GOLDEN=1 pnpm test:unitrewrites golden files; the pull request diff of golden files is reviewed like code, and the pull request description lists which goldens changed and why. - Golden files are never edited by hand.
- A test asserts that the case directory and golden directory contain exactly the same names (no orphans).
Required corpus. At least one case for each item (every enumerated value in Section 13 and Section 9 must appear in at least one case):
| Area | Required cases |
|---|---|
| Metrics | Each of: visitors, visits, pageviews, views_per_visit, bounce_rate, visit_duration, events, scroll_depth, time_on_page, conversion_rate, group_conversion_rate, total_revenue, average_revenue, percentage, exit_rate alone; all metrics together on an event-level dimension; all metrics together on a session-level dimension |
| Dimensions | Every dimension in Section 13 alone, in pairs with a time dimension, and with a goal filter. visit:device (user-agent class desktop, mobile, tablet, and (none) for the empty value) and visit:screen_size (mobile, tablet, laptop, desktop, (none)) are separate cases that read the device and screen_size columns of Section 6 respectively. Custom-property breakdowns use the meta.key/meta.value expressions of Section 13.5.3, including the (none) bucket |
| Time | Every date-range keyword in Section 13; explicit ranges in each of the six test timezones; ranges spanning DST start and end for Europe/Berlin and America/New_York; a Australia/Lord_Howe 30-minute DST shift; a day with 23 hours and a day with 25 hours; time:hour, time:day, time:week, time:month bucketing; week bucketing that starts a partial week; range end in the future; leap day 2028-02-29 |
| Filters | Every operator in Section 13 on every filterable dimension class (string, number, set); negation; contains with special characters (%, _, \); nested and/or/not up to the maximum depth, and the case one level beyond (validation error); empty is list (validation error); filters on custom properties (compiled to the meta.key/meta.value parallel-array expressions of Section 13.5.3, never to a map lookup, including the (none) case) with up to 3 property conditions on a goal |
| Comparison | previous_period, year_over_year, custom comparison range, comparison combined with time series and with breakdown |
| Order and pagination | Order by each metric and dimension ascending and descending; multiple order keys; limit 1, 10,000 and 10,001 (validation error); offset beyond result size |
| Revenue | Revenue metrics (the reporting-currency columns revenue_reporting_amount and revenue_reporting_currency of Section 6 are the source) with a single currency, multiple currencies, no revenue events; currency breakdown |
| Funnels | Ordered, optional-step and strict-order funnels with 2, 5 and the maximum number of steps; step defined by page, by event, by goal; comparison; revenue per step (query shape owned by Section 10) |
| Journeys | The journey query shape owned by Section 10 with maximum depth |
| Imported data | Same queries with include.imports = true where imported tables are merged, and with imports excluded; imported rows are used only for dates strictly earlier than the site's first native day (Section 13.4.10 rule 2), including a case whose comparison range straddles that day; dimensions not present in imported tables, among them visit:screen_size (the engine must omit imported rows and report meta.imports_skip_reason per Section 13); the column-mapping cases of Section 13.4.10 for visit_duration, entrances, exits and total_time_on_page |
| Segments | A query filtered by a saved segment expanded into its filter tree |
| Realtime | The realtime query shape of Section 9 |
| Sites | Single site through executeStatsQuery; consolidated all-sites query through executeConsolidatedQuery of Section 13.5.1 (site lists of 1 and of 100 sites with different timezones; per-site statements grouped by site_id and summed in the application) |
Safety tests (same package, not golden):
| ID | Assertion |
|---|---|
| Q-SAFE-1 | For every golden case, the SQL text contains no literal user-supplied string or number: every value appears only in the parameter map |
| Q-SAFE-2 | An injection corpus (at least 40 strings, including ' OR 1=1 --, "; DROP TABLE events; --, \\, \0, Unicode confusables, very long strings of 2,000 and 2,001 characters, ClickHouse parameter syntax such as {x:String}) is used as filter values, property keys, property values and order_by entries; for each, the engine either produces SQL whose text is identical to the benign counterpart (value only in parameters) or rejects with a validation error |
| Q-SAFE-3 | Dimension, metric and property-key identifiers are mapped through an allow-list; an unknown dimension yields a validation error that lists the allowed values |
| Q-SAFE-4 | Every generated statement contains a site-scoping predicate on the site UUID (site_id equality; a consolidated query is compiled as one statement per site, each scoped to that site's UUID). A test fails if any golden SQL lacks it |
| Q-SAFE-5 | Every executed statement carries the ClickHouse settings readonly, max_execution_time, max_memory_usage, max_rows_to_group_by and max_result_rows with the values of Section 13.4.11 (asserted from the engine's settings object in the unit run and from the system.query_log settings of the executed statement in the integration run) |
18.3.2 Channel and source classifier table tests (apps/server) #
The classification rules (referrer source normalization, UTM handling, channel assignment, AI-assistant detection, precedence) are owned by Section 8. The test is one table-driven file channels.cases.ts exporting rows of { referrer, utm_source, utm_medium, utm_campaign, url, expectedChannel, expectedSource } consumed by one test body. A second assertion fails the build if any of the 18 channels (Direct, Organic Search, Paid Search, Organic Social, Paid Social, Email, Affiliates, Display, Organic Video, Paid Video, Cross-network, Organic Shopping, Paid Shopping, Audio, SMS, Mobile Push Notifications, Referral, AI Assistants) has fewer than 6 rows.
Required rows (the expected values are those defined by Section 8; the inputs below are mandatory):
| Group | Mandatory inputs |
|---|---|
| Direct | No referrer and no UTM; empty-string referrer; referrer equal to the site's own domain (internal navigation is ignored); referrer equal to a subdomain of the own domain; android-app:// referrer for an unknown package |
| AI Assistants | Referrers chatgpt.com, chat.openai.com, claude.ai, perplexity.ai, gemini.google.com, copilot.microsoft.com, chat.deepseek.com, grok.com, meta.ai, chat.mistral.ai, phind.com, you.com; with www. prefix; with path and query on the referrer; utm_source=chatgpt.com, utm_source=claude.ai, utm_source=perplexity; AI referrer combined with utm_medium=cpc (precedence per Section 8); an upper-case host (ChatGPT.com) |
| Organic Search | google.com, google.de, google.co.uk, bing.com, duckduckgo.com, search.yahoo.com, ecosia.org, yandex.ru, baidu.com, search.brave.com; Google with utm_medium=organic |
| Paid Search | utm_medium=cpc with utm_source=google; utm_medium=ppc; utm_medium=paidsearch; a gclid landing parameter alone is not a channel signal unless Section 8 says so |
| Organic Social | facebook.com, l.facebook.com, t.co, x.com, linkedin.com, lnkd.in, reddit.com, news.ycombinator.com (per Section 8 table), instagram.com, pinterest.com, mastodon.social; utm_medium=social |
| Paid Social | utm_medium=paid_social, utm_medium=paidsocial, utm_medium=cpc with utm_source=facebook |
utm_medium=email, utm_medium=e-mail, utm_source=newsletter with utm_medium=email; mail.google.com referrer |
|
| Affiliates, Display, Organic Video, Paid Video, Cross-network, Organic Shopping, Paid Shopping, Audio, SMS, Mobile Push Notifications | At least 6 rows each exercising the exact utm_medium and utm_source tokens listed for that channel in Section 8, including mixed case and surrounding whitespace |
| Referral | Unknown external host; host with port; IDN host in punycode and in Unicode; IPv4 literal host; localhost |
| Normalization | Strip www., m., mobile. where Section 8 says so; lowercase; trailing dot; android-app://com.google.android.googlequicksearchbox mapped to Google; very long referrer (2,001 characters) is truncated or dropped per Section 8; malformed URL; javascript: and data: schemes (dropped, treated as Direct) |
| Spam | Hosts in the known spam-referrer list produce a dropped event (asserted in the bot-filter tests of Section 18.3.5) |
18.3.3 Visitor hash, salt rotation and session tests (apps/server) #
Hash function (user_id).
| ID | Assertion |
|---|---|
| H-1 | The SipHash-2-4 implementation reproduces the 64 reference test vectors published with the algorithm (key bytes 00 01 02 ... 0f, messages of length 0 to 63 whose bytes are 00 01 02 ...); the empty-message vector is 0x726fdb47dd0e0e31 |
| H-2 | user_id is returned as an unsigned 64-bit value represented as a decimal string at the ClickHouse boundary and as bigint in code; a value above 2^53 round-trips without precision loss |
| H-3 | The message is site_id as 16 raw UUID bytes (big-endian, as in the canonical text form without hyphens) followed by the client IP bytes followed by the User-Agent UTF-8 bytes; a test vector computed independently (in a small script checked into tests/fixtures/) fixes the expected user_id for three inputs |
| H-4 | Changing any single one of site, IP, User-Agent or salt changes the result; the same inputs give the same result across 1,000 repeated calls |
| H-5 | IPv4 and the IPv4-mapped IPv6 form of the same address (::ffff:203.0.113.5) hash identically; two different IPv6 addresses in the same /64 hash differently (the product hashes the full address) |
| H-6 | Empty User-Agent is rejected by the bot filter before hashing (hash function is never called with an empty User-Agent in the pipeline); the function itself accepts it |
| H-7 | The IP and User-Agent are never kept in the event object after the hash is computed: a test walks the enriched event object recursively and asserts neither string appears as any key or value |
Salt lifecycle (uses the fake clock and a real Postgres, in *.int.test.ts):
| ID | Assertion |
|---|---|
| S-1 | With the clock at D 00:00:00 UTC, running the salt-rotate job (Section 16.6) creates exactly one new salts row with valid_from = D and a 16-byte random salt |
| S-2 | Running salt-rotate twice for the same date creates one row (idempotent through the unique valid_from); two salt-rotate jobs started concurrently also create one row |
| S-3 | From 00:00:00 to 00:29:59.999 ingestion computes both candidate user_id values; when a live session exists under the previous-salt value, the event continues that session and the stored user_id is the previous-salt value |
| S-4 | In the same window, when no session exists under the previous-salt value, the stored user_id is the current-salt value and a new session starts |
| S-5 | At 00:30:00 the salt-expire job deletes the previous salt row; a SELECT count(*) FROM salts never exceeds 2 at any sampled instant during a simulated 3-day run with sampling every minute |
| S-6 | An in-memory salt cache refreshes within 60 s: after rotation and clock advance of 61 s, a web instance that cached the old salt uses the new one |
| S-7 | A web instance started at 00:10 UTC loads both salts; started at 00:31 UTC loads one; a web instance that finds no salt for the current date at start runs salt-rotate itself (Section 16.6) |
| S-8 | A session that started at 23:50 and continues at 00:05 the next day keeps the same session_id and user_id (previous-salt continuation); a session that started at 23:50 and continues at 00:40 (gap above 30 minutes) is a new session with the current-salt user_id |
| S-9 | The same visitor on day D and day D+1 produces different user_id values; across 1,000 simulated visitors no user_id repeats across days |
| S-10 | salt-rotate refuses to run (and raises an alert log line) when the latest salt is dated in the future; the missed-day case (no job ran for 2 days) creates the salt for the current date only and deletes salts older than the previous one |
| S-11 | The salts table is UNLOGGED (pg_class.relpersistence = 'u'), so it is absent from WAL, point-in-time recovery and replicas, and the output of pg_dump --exclude-table-data=salts (the invocation the backup job of Section 16 uses) contains no row of it |
| S-12 | After a simulated crash recovery (the salts table emptied, as PostgreSQL does for unlogged tables), the next event makes the web instance generate a fresh salt for the current date and ingestion continues; visitors already counted that day may be counted again, which is the documented consequence and is asserted as such (no error, no event lost) |
Sessions:
| ID | Assertion |
|---|---|
| SE-1 | A gap of 29 min 59.999 s between two events of the same user_id and site continues the session; a gap of exactly 30 min 0 s starts a new session |
| SE-2 | Different sites with the same visitor inputs produce different user_id and independent sessions |
| SE-3 | session_id is a random UInt64 from the injected Rng, never derived from user_id |
| SE-4 | A session with one pageview and no interactive custom event is a bounce; a session with one pageview and one engagement event is still a bounce; one pageview plus one interactive custom event is not |
| SE-5 | Concurrent events from the same visitor handled by two web instances produce one session (the Valkey lookup uses the atomic operation specified in Section 8); test runs 200 parallel events and asserts exactly one session row chain |
| SE-6 | An engagement event for a visitor with no live session is dropped silently (202) and creates no session |
18.3.4 Zod schema tests (packages/shared and consumers) #
- Table-driven validity tests. Every exported Zod schema has a test file with arrays
valid: [...]andinvalid: [{ input, path, messageIncludes }]. A meta-test enumerates all exports ofpackages/shared/src/schemasand fails if any schema lacks a corresponding test file. - Event payload boundaries (wire format defined in Section 7.6.1, validation rules in Section 8.4):
| Field | Accepted | Rejected |
|---|---|---|
n |
1 and 120 characters; pageview; engagement |
empty; 121 characters; non-string |
u |
absolute http or https URL of 2,000 characters | relative URL; 2,001 characters; ftp://; javascript: |
d |
one domain; 10 comma-separated domains; WWW.Example.COM:8080 (accepted by validation and resolved as example.com by the normalization rule of Section 18.3.5) |
empty; 11 domains; a domain with spaces; empty list item (a.com,,b.com) |
r |
null; absent; a URL string |
non-string non-null |
w |
1 and 100,000; null |
(never rejected) 0, negative, non-integer and string values are ignored and stored as unknown width |
h |
0, 1, absent |
2; string |
p |
30 keys; key of 300 characters; value of 2,000 characters; number and boolean values (stored as strings); a null or empty value (drops the key) |
31 keys; key of 301 characters; value of 2,001 characters; nested objects |
$ |
{amount: "12.50", currency: "EUR"}; {amount: 12.5, currency: "USD"}; {amount: 0, currency: "JPY"} |
negative amount; non-numeric amount; currency not 3 upper-case letters; unknown ISO 4217 code; amount with more decimals than the currency allows or more than 3 decimal places |
i |
true, false, absent |
non-boolean |
sd |
0, 100 | -1, 101, 1.5, string |
e |
0 and the maximum in Section 8 | negative; non-integer |
| Body | valid JSON as text/plain; valid JSON as application/json; 32,768 bytes |
32,769 bytes (413); invalid JSON; JSON array; empty body |
- Error shape. Every API validation failure test asserts the envelope of Section 4.4: status 400,
error.code = "validation_failed",details.fields[]withpathandmessage, and arequest_id. - Schema versus database drift (integration): for every Postgres enum-like column (
role,status,kind, and each column defined with a CHECK constraint in Section 5) the test asserts that the Zod enum inpackages/sharedand the database constraint contain the same set of values. For ClickHouseLowCardinality(String)columns with fixed vocabularies, the same check applies to the shared constants. - Response schemas. Every public API response (Section 13) and every dashboard API response (Sections 9 to 12, 14, 15) is validated in the integration tests against a Zod response schema exported from
packages/shared; a response containing an undeclared field fails the test (strict objects). - Unit coverage of
snake_case.pnpm check:conventionswalks every request and response schema and fails if any property name does not match^[a-z][a-z0-9]*(_[a-z0-9]+)*$; the single exceptions are the tracker payload keys defined in Section 7.6.1 (n,u,d,r,w,h,p,$,i,sd,e).
18.3.5 Other required unit suites #
| Suite | Required assertions |
|---|---|
| URL and path normalization | Query string removed from stored pathname; UTM and ref/source parameters extracted into dedicated fields; fragment dropped unless hash routing flag is set; trailing slash policy per Section 8; percent-encoding normalization; pathname of 2,000 characters; non-ASCII paths; // collapse; path-only / |
| User-Agent parsing | Browser and OS versions truncated to major.minor (Chrome 126.0.6478.61 becomes 126.0); device is desktop, mobile or tablet, and smart TVs, game consoles, wearables and embedded devices store '' (Section 8.10.2); an unknown UA yields empty strings in every parsed column; a table of at least 60 real User-Agent strings across the browsers and operating systems shown in the dashboard |
| Screen-size class | screen_size is derived from the payload width w: mobile below 576, tablet from 576 to 991, laptop from 992 to 1439, desktop from 1440, '' when w is absent; boundary widths 575, 576, 991, 992, 1439 and 1440 are asserted, and the value is independent of the device class (a desktop UA with w = 400 stores device = desktop, screen_size = mobile) |
| Domain normalization | A site domain is stored lowercase without scheme, port, path or leading www.: input www.Example.com and input example.com:8080 are stored as example.com, and input containing a scheme or a path is rejected with validation_failed. Ingestion strips a leading www. and any port from each d entry before the lookup, so WWW.Example.COM:8443 and example.com resolve to the same site. Uniqueness is installation-wide on the normalized domain: creating www.example.com when example.com exists, in the same or any other team, returns 409 domain_taken |
| Hostname allowlist | site_settings.allowed_hostnames empty accepts the URL hostnames equal to the site domain or any of its subdomains (blog.example.com for example.com, with a leading www. ignored) and rejects every other host; a non-empty list accepts only what Section 8.6 defines for its entries; comparison is case-insensitive; a rejected event is dropped with reason hostname_not_allowed, answers 202 and is not billable |
| Client IP resolution | The TRUSTED_PROXIES right-to-left walk with chains of 1, 2 and 5 hops and a spoofed left-most entry; an untrusted peer's X-Forwarded-For is ignored entirely. Per-site proxy key (Section 8.7.2): X-Quietstat-Client-IP is honored only when X-Quietstat-Proxy-Key equals that site's proxy_key under a constant-time comparison (a test replaces the comparator with a spy and asserts it is the only comparison path); with a missing or invalid key the header is ignored and the request is rate-limited as a normal client under the event_ingest policy of Section 17.6, with no elevated limit; a key valid for site A grants nothing for site B in a multi-site d list |
| Bot filter | isbot matches; empty or missing UA; headless markers (HeadlessChrome, PhantomJS, Puppeteer-style tokens as listed in Section 8); datacenter CIDR membership (IPv4 and IPv6, boundary addresses: first, last, one before, one after each tested range); spam-referrer list hit; per-site datacenter toggle off lets datacenter IPs through while UA-bot filtering stays on; every drop increments the per-site per-day Valkey counter exactly once |
| Exclusions | IP exclusion exact IPv4, IPv6, CIDR; wildcard page exclusion patterns (/admin/*, /*/preview, exact path) with the matching semantics of Section 8; case sensitivity; interaction with multi-site d lists; exclusion applied before hashing |
| Geo lookup | Synthetic MMDB fixture (Section 18.4.3): country-only record, country plus region plus city, no record, IPv6 record; output limited to country_code, subdivision1_code and city_geoname_id; latitude and longitude never copied to the event |
| Referrer spam, self-referral | Covered by Section 18.3.2 rows |
| Permission matrix | A generated test reads the permission matrix from Section 11 (encoded in packages/shared as a typed table) and, for every action and every role (owner, admin, editor, viewer, guest_viewer, guest_editor, no membership, anonymous), asserts the helper can(role, action) result against a literal copy of the matrix in the test file. The literal copy includes these rows of the Section 11 matrix, which are the ones other sections depend on: billing.view (owner and admin only), billing.manage (owner only), export actions allowed for every role including both guest roles, import actions (site.import.manage) owner and admin only, and API-key actions owner and admin only |
| Passwords and 2FA | argon2id parameters match Section 11; verification of a known hash; TOTP against the RFC 6238 reference vectors (SHA-1, 30 s step, 8 digits for the vector secret 12345678901234567890: times 59, 1111111109, 1111111111, 1234567890, 2000000000, 20000000000); window tolerance of one step each side; replay of a used code within its window is rejected; recovery codes are single use and compared in constant time |
| Money | Revenue amounts in 0-decimal (JPY), 2-decimal (USD, EUR) and 3-decimal (BHD) currencies are accepted and kept as decimals with at most 3 places, matching the Decimal(18,3) column of Section 6, and a fourth decimal place is rejected; conversion to the site's reporting currency (Section 8.14) and rounding follow the rule stated in Section 4; license prices are integers in minor units (5,900, 14,900 and 34,900); formatting short and long strings from Section 4; amounts above 2^31 |
| Time ranges | Resolution of every keyword range, week start, month boundaries, site timezone, DST |
| CSV | Writer escapes quotes, commas, newlines, and neutralizes formula injection (a cell beginning with =, +, -, @, tab or carriage return is prefixed with '); parser handles BOM, CRLF and LF, quoted newlines, ragged rows, duplicate headers, extreme widths |
| GA4 mapping | Each GA4 report dimension and metric maps to the target imported_* table column; unmapped values go to the documented fallback; date handling with property timezone |
| License codes | Format QS-XXXX-XXXX-XXXX; Crockford base32 alphabet excludes I, L, O, U; generation of 100,000 codes has no duplicates; case-insensitive and hyphen-tolerant parsing; 0/O and 1/I/L normalization on input |
| Usage-cap state machine | Per Section 15.9.2, for a team cap C with warn_at = ceil(0.8C) and hard_at = floor(1.1C) + extra_allowance (extra_allowance is 0 unless the operator granted a monthly allowance): count N < warn_at is ok; warn_at <= N < C is warning; C <= N < hard_at is grace; N >= hard_at is blocked, and ingestion accepts only while N < hard_at. Test points: warn_at - 1, warn_at, C - 1, C, hard_at - 1, hard_at for each of the three tiers (C = 100,000; 500,000; 2,000,000, so hard_at = 110,000; 550,000; 2,200,000) for a non-round cap (C = 12,345) that exercises the ceil and floor, and with extra_allowance = 5,000 on a team at the old hard_at (it moves from blocked to grace at once and hard_at rises by exactly 5,000); email-once semantics per Section 15.9.5; month rollover on 00:00:00 UTC on the 1st; an upgrade or an additional code immediately lifts blocked and re-evaluates the warned_* flags |
| E-mail templates | Every template in the inventory of Section 12.8 renders with fixture data to an HTML part with inline CSS only (no <style> blocks, no remote images other than those Section 12 allows) and a non-empty plain-text part; no unresolved placeholder markers; every link is absolute and uses the configured public URL; snapshots stored as files; subject lines are at most 150 characters (Section 12.9.1). The one suite grows with the milestones: it covers the templates owned by Sections 11 and 12 at M10, adds export_ready, export_failed, import_completed and import_failed (Section 14) at M11, and adds usage_cap_80, usage_cap_100, license_activated, license_upgraded, license_revoked, refund_processed, purchase_auto_refunded, unlicensed_purge_warning, unlicensed_purge_completed, impersonation_notice and operator_alert (Section 15) at M13 |
| i18n catalog | Every UI string key used in apps/web exists in the default catalog and no catalog key is unused (the mechanism is owned by Section 4) |
| Web unit | Filter state to URL query serialization round trip for every filter type; time-range keyboard shortcut mapping; number, duration and percentage formatters; comparison delta color and sign logic; permission-aware navigation visibility per role |
Component tests (Vitest with Testing Library and a DOM implementation) cover: chart empty state, "no data yet" onboarding panel, filter chips, segment dialog validation messages, role-restricted buttons hidden for viewer. They do not duplicate Playwright flows.
18.4 Integration tests #
18.4.1 Infrastructure #
| Concern | Decision |
|---|---|
| Container orchestration | Testcontainers for Node, started once per Vitest run in a global setup file. Images match the major lines in Section 3 for PostgreSQL, ClickHouse and Valkey. Image tags are pinned in tests/support/images.ts to the major line only |
| Local reuse | Environment variable TEST_SERVICES=compose makes the global setup skip container creation and connect to the services started by docker-compose.test.yml at the repository root (the same three images plus the e-mail sink and the Stripe fake). Default is testcontainers. CI uses testcontainers |
| Postgres isolation | Global setup creates database template_migrated, runs all Postgres migrations (Section 5) into it, then marks it as a template. Each test file runs CREATE DATABASE test_<n> TEMPLATE template_migrated and drops it after the file finishes |
| ClickHouse isolation | Global setup runs the in-repo ClickHouse migration runner (Section 6) against database template_migrated. Each test file creates database test_<n> and runs the migration runner against it (the runner accepts the target database name). Tables are always referenced through the connection's default database; production code never hard-codes a database name |
| Valkey isolation | Each Vitest worker uses logical database number equal to its worker index (maximum 8 workers, so databases 0 to 7); the harness flushes that database before each file |
| Services under test | buildApp and buildWorker (Section 18.2.4) are started in-process with the file's databases and the fakes below |
| Parallelism | Vitest maxWorkers = 4 in CI, maxWorkers = 8 locally. Files run in parallel; tests inside a file run sequentially |
| Timeouts | Test timeout 30 s, hook timeout 120 s; the global setup timeout is 300 s |
| Logs | Container logs are collected and attached to the CI job on failure |
18.4.2 Required integration suites #
| Suite | Content |
|---|---|
| Route registry completeness | The server exposes a route registry (method, path pattern, auth kind, required permission, and the readOnly flag that marks read endpoints served by POST, such as the dashboard query routes). The test asserts every registered route has an authorization test: for each route, an anonymous call yields 401 (or the public behavior defined in its owning section), a call with a user lacking permission yields the status defined by the permission policy of Section 11, and a call with a permitted role succeeds. The suite iterates the registry, so a new route without a test fails the build |
| Health | GET /healthz returns 200 with the database containers stopped; GET /readyz returns 200 with all three stores up and 503 (error envelope code unavailable) when each of Postgres, ClickHouse and Valkey is paused in turn (three tests) |
| Auth | Signup (duplicate e-mail returns 409 email_taken, Section 11.3.1), e-mail verification, login (a 2FA account receives 401 two_factor_required with a challenge token), logout, session expiry, session revocation, password reset, change e-mail, 2FA enrol, login with TOTP, recovery code use, account deletion with soft delete (the next login yields only the restore-only session of Section 11.13.3) and the 30-day user-anonymize job (fake clock) |
| Setup bootstrap (self-hosted) | With zero users, POST /api/app/v1/auth/signup returns 403 setup_required in every REGISTRATION_MODE and creates nothing. POST /api/app/v1/setup with a wrong token fails and counts against the setup_token policy of Section 17.6; with the correct one-time token (SETUP_TOKEN, or the generated value read from the worker log as Section 16.3.5 specifies) it creates the verified owner and the personal team, and two concurrent requests with the same token create exactly one account. After setup the registration rules of Section 11.2 apply and the setup route is closed |
| Teams | Personal team creation at signup, invitations of each kind (team_member, site_guest, ownership_transfer, site_transfer: create, accept, expire, revoke, re-send), role changes, last-owner protection, ownership transfer, member removal (the API keys that member created are deleted, Section 11.16.5), guest assignment per site, team soft delete with 7-day undo window (events of its sites are dropped during the window and requests with the team's API keys return 404 not_found) and the team-purge job's final purge |
| Sites | Create (domain validation and normalization per Section 18.3.5, duplicates on the normalized domain returning 409 domain_taken, timezone default, script_id and proxy_key generation), update, site-limit enforcement on hosted (409 site_limit_reached), delete with the asynchronous site-purge job (which also enqueues import-purge for every import of the site), verification that no rows remain in any ClickHouse table for that site_id after the jobs run, and the 30-day domain reservation of Section 5.5.5 (the same team can add the deleted domain again; another team receives 409 domain_taken until the reservation expires) |
| Settings | Goals, property allow-list, funnels, segments, annotations, exclusions, shared links, e-mail report subscriptions, API keys, and the site_settings columns (tracker toggles, datacenter_filter_enabled, allowed_hostnames, public_dashboard, reporting_currency): CRUD, uniqueness and cascade behavior from Section 5, and every numeric limit of the Section 5.16 table asserted at its literal value with 409 limit_reached and details.limit_name one past it |
| Ingestion pipeline | The full POST /api/event path against real stores: accepted pageview, custom event, engagement, revenue; all drop reasons (bot, datacenter, spam referrer, excluded IP, excluded page, unknown domain, hostname_not_allowed) return 202 and write nothing, and the hosted-only reasons (unlicensed team, cap exceeded at hard_at) are asserted the same way but registered in tests/milestones.json under M13, because the cap and license stubs of Section 19.1 accept everything until then; 400 reasons return plain-text bodies; 413 at 32,769 bytes; 429 on the 101st request in 10 s from one IP; CORS headers on OPTIONS and POST (Access-Control-Allow-Origin: *); X-Forwarded-For honored only when the socket peer is inside TRUSTED_PROXIES and ignored otherwise; the per-site proxy key (X-Quietstat-Client-IP with a valid X-Quietstat-Proxy-Key is honored, an invalid or missing key leaves the header ignored and the request rate-limited as a normal client with no elevated limit); IPv6 client addresses; d entries with a leading www. or a port resolve to the normalized domain; multi-site d lists create one event per domain with a distinct user_id per site |
| Buffer and failure modes | Behavior per Section 8 when ClickHouse is unreachable during a flush (retry, bounded buffer, drop policy), when Valkey is unreachable (fail-open or fail-closed as Section 8 decides), when Postgres is unreachable (site lookups served from cache for the TTL in Section 8). The suite pauses each container and asserts the documented behavior and recovery |
| Query execution | The golden corpus of Section 18.3.1 executed against ClickHouse seeded with the basic and ecommerce fixture profiles; every result compared with the oracle (Section 18.4.4); the same queries through the dashboard routes POST /api/app/v1/sites/{domain}/query and /query/batch (domain URL-encoded) and through POST /api/app/v1/consolidated/query, with identical results; each query of a /query/batch counts as one request toward its rate-limit policy (Section 17.6) |
| Stats API | POST /api/v2/query with API key: success shape (results, meta, query), every validation error, key scopes, revoked key, key of another team's site (not found), rate limit headers and 429 with Retry-After (policies of Section 17.6, header semantics of Section 13.3.3), no CORS headers on a foreign Origin |
| Sites API | /api/v1/sites create, list with cursor pagination (limit default 50, max 200, 201 is rejected), get, update, delete, with key permissions |
| Realtime | Events ingested are visible in the realtime report within 2 s of the buffer flush; visitors older than the realtime window drop out (fake clock) |
| Goals and revenue | Goal matching rules (event name, pageview path pattern, scroll depth threshold), property narrowing with up to 3 properties (a 4th is a validation error), revenue totals across currencies, conversion rate denominators |
| Funnels and journeys | Ordered, optional and strict funnels against hand-built event sequences covering: skipped step, repeated step, out-of-order step, steps across sessions (not matched), step satisfied after session timeout, optional step missing, strict order with an intervening page, comparison period; journeys top-N truncation and depth limit |
| Sharing | Shared link with and without password (hashed at rest), segment-limited link (filters cannot be removed by the client; the lock also applies to every mirrored endpoint including filter suggestions, realtime, funnels and journeys, and a viewer request carrying a segment filter is rejected with validation_failed), the include_imports option of the link (default true: viewers may switch imported data off; false forces imports off and a viewer request cannot turn them on), embed token behavior, revoked link returns 404, brute-force protection on link passwords (Section 17.6) |
| Reports and notifications | Worker runs the e-mail report jobs of Section 16.6 for weekly and monthly schedules in several site timezones; the spike-check job (every minute) fires once per spike and respects cooldown (Section 12); an external recipient added to a report or alert receives nothing until it confirms through the link in its notice e-mail, and unconfirmed recipients are deleted after 7 days; every e-mail lands in the SMTP sink with both HTML and text parts |
| Imports and exports | CSV export of every report (files expire 7 days after completion through the export-expire job, Section 14.3.1; a download after the requester lost access returns 404); CSV import of valid and malformed fixtures (limits, duplicate dates, wrong columns, huge file just under and just over the limit of Section 14); GA4 import against the fake Google server (OAuth exchange with a state bound to the session user, pagination, quota error retry, partial failure, cancellation, resume); imported data appears in queries with include.imports |
| Licensing and caps | Code redemption (valid, used, revoked, wrong format returning 422 code_invalid, stacking 1, 2 and 3 codes, a 4th rejected with 409 code_stack_limit_reached), Stripe-driven purchase (Section 18.13), usage counting with fake clock across month boundary, 80% and 100% e-mails sent once, the drop at hard_at = floor(1.1C) + extra_allowance (Section 15.9.2), upgrade lifts block, unlicensed team behavior, the unlicensed-purge job's 30-day purge of unlicensed data |
| Operator console | Super admin access via SUPER_ADMIN_EMAILS, a non-super-admin gets 404 not_found, step-up re-authentication with password and TOTP for the sensitive actions of Section 15.17.2, code batch generation and CSV export, refund action through the Stripe fake, read-only impersonation (mutating calls rejected with impersonation_read_only; every impersonation start, end and expiry audited in audit_log as admin.impersonation_started, admin.impersonation_ended and admin.impersonation_expired; rejected writes counted by a metric, not individually audited) |
| Jobs | The jobs table in Section 16.6 is read by a test that asserts: every listed job has a registered handler; every handler is executed once against real stores and completes; no handler exists that is not listed; every job name is kebab-case (^[a-z0-9]+(-[a-z0-9]+)*$); the registered schedules equal the table, in particular salt-rotate at 00:00 UTC, salt-expire at 00:30 UTC and spike-check every minute |
| Audit log | Every action that the catalogue in Section 11.20.2 marks as audited writes an audit_log row with the columns of Section 11.20.1 (actor, target, action, reason where required) and no secrets; mutations made with an API key store the key id as the actor |
| Edition switch | With EDITION=selfhosted (default), licensing routes and /admin routes return 404, ingestion never consults caps or licenses, no billing UI assets are referenced; with EDITION=hosted they are active |
18.4.3 Local fakes for third parties #
| Dependency | Fake | Notes |
|---|---|---|
| SMTP | A Mailpit container, queried through its HTTP API | tests/support/mail.ts exposes waitForMail({to, subject}) and clear() |
| Stripe API | The official Stripe mock server container for API calls (Checkout session create, retrieve, refund create) plus in-process webhook delivery signed with a test signing secret | The Stripe client is built by one factory that accepts a host, port and protocol override, so tests point it at the mock |
| Google OAuth and GA4 APIs | A small in-repo HTTP server in tests/support/fake-google.ts that implements the token endpoint and the report endpoints used by Section 14, replaying bodies from tests/fixtures/ga4/ |
Supports injecting quota errors, 401 refresh and truncated pages |
| GeoIP | A synthetic MMDB generated by tests/support/make-mmdb.ts (a minimal in-repo MMDB writer following the public MMDB format) containing records for reserved documentation ranges: 203.0.113.0/24 (country DE, region DE-BE, GeoNames city 2950159), 198.51.100.0/24 (country US, region US-CA, city 5391959), 192.0.2.0/24 (country only FR), 2001:db8::/32 (country JP, region JP-13, city 1850147) |
The generated file is written to a temporary directory and committed as tests/fixtures/mmdb/test-city.mmdb for reproducibility |
| Datacenter list | A fixture CIDR list containing 198.18.0.0/15 as the "datacenter" range for tests |
Production list per Section 8 |
| Outbound HTTP | guardedFetch of Section 17.4 is replaced in tests by one that allows only 127.0.0.1 fakes (the guard's own behavior is tested separately by SEC-18 with the real implementation) |
18.4.4 Oracle (differential) testing #
The fixture generator (Section 18.11) writes, alongside the rows it inserts, an expected.json computed by an independent reference implementation that works on plain in-memory arrays and does not import packages/query. For each of the fixed query set (all metrics by every dimension for the fixed range, time series by day and month, top 10 of each report, goal conversion, funnel conversions, revenue by source), integration tests execute the query through the real engine and ClickHouse and assert equality with the oracle (exact for counts and sums; absolute tolerance 0.01 percentage points for rates; durations exact in whole seconds). A mismatch fails the test and prints the first differing row.
18.5 Ingestion end-to-end tests (tracker in a headless browser to ClickHouse) #
Setup. A Playwright project ingestion starts: the compiled server with EDITION=selfhosted, TRUSTED_PROXIES set to the loopback range, the synthetic MMDB, fixture CIDR list and real containers; and a small static fixture site server (tests/support/fixture-site/) that serves HTML pages from a second origin. The fixture site includes pages for: plain page; SPA page using history.pushState and replaceState; hash-routed page; outbound link; download links; a form; a page returning HTTP 404 with the content the tracker's 404 detection expects (Section 7); a long scrollable page; a page that loads the tracker with each option combination. The Playwright browser is launched with host-resolver rules that map the reserved .test fixture hostnames (the site domain, its subdomains such as blog.<domain> and a foreign host) to the fixture server, so that one server can answer on several hostnames.
Browser identity. Headless Chromium advertises HeadlessChrome, which the bot filter correctly drops. Therefore every context in this project sets a realistic userAgent (a current desktop Chrome string, a current Firefox string and a current Safari string for the matching engines). Test ING-27 uses the default headless User-Agent and asserts the event is dropped. To make the server see a chosen client IP, tests intercept **/api/event with page.route and continue the request with an added X-Forwarded-For header; this avoids CORS preflight because interception happens after the browser's CORS decision.
Assertion pattern. After the browser action, eventually polls fetchEvents(siteId) and fetchSessions(siteId) (Section 18.2.2) and asserts the normalized rows. Column naming is per Section 6.
| ID | Scenario | Assertions (all rows are read from ClickHouse) | Milestone |
|---|---|---|---|
| ING-01 | Plain page load | One pageview; pathname /; site UUID matches; session row exists with one pageview; user_id non-zero |
M4 |
| ING-02 | SPA pushState navigation to /a then /b |
Three pageviews total in order, same session_id; no duplicate for the initial load |
M4 |
| ING-03 | replaceState and popstate |
replaceState with same path produces no event; popstate back produces one event with the previous path |
M4 |
| ING-04 | Hash routing disabled | Navigating to /#/x produces no additional pageview; stored pathname has no fragment |
M4 |
| ING-05 | Hash routing enabled (h=1 per Section 7 option) |
Each hashchange produces a pageview whose stored pathname includes the hash route |
M4 |
| ING-06 | Query string handling | URL /p?utm_source=news&utm_medium=email&utm_campaign=c1&secret=1 stores pathname /p, UTM fields populated, secret nowhere in any column |
M4 |
| ING-07 | Referrer capture | Navigation from fixture origin A to B stores referrer source for A and the Referral channel; same-site referrer is ignored | M4 |
| ING-08 | Outbound link click | One Outbound Link: Click event whose prop url is the target URL; navigation proceeds; event delivered (beacon) even though the page unloads |
M4 |
| ING-09 | File download click | One File Download event with prop url per configured extension from Section 7.5.6; non-configured extension produces none |
M4 |
| ING-10 | Form submission | One Form: Submission event with prop path (and form_id when the form has an id); form field values appear nowhere in stored rows |
M4 |
| ING-11 | 404 page | A 404 event with prop path equal to the requested path |
M4 |
| ING-12 | Scroll depth | Scrolling to 25%, 50%, 100% results in a final engagement event whose sd is the maximum reached (100); the event is non-interactive |
M4 |
| ING-13 | Engagement time | Page foreground for about 3 s then navigation away yields an engagement event whose e is between 2,000 and 4,500 ms; hidden-tab time is excluded |
M4 |
| ING-14 | Custom event with props | qs('Signup', {props:{plan:'pro'}}) stores event name Signup and prop plan=pro |
M4 |
| ING-15 | Revenue event | $ is stored as revenue_amount (Decimal(18,3), exactly as sent, at most 3 decimal places) and revenue_currency; revenue_reporting_amount and revenue_reporting_currency hold the conversion into the site's reporting_currency (equal to the sent values when the currencies match, and computed with the fixture exchange rates when they differ) |
M4 |
| ING-16 | Manual pageviews mode | With automatic pageviews disabled, no pageview is sent until the manual call; custom url override is stored |
M4 |
| ING-17 | transformRequest hook |
Returning null drops the event; returning a modified payload sends the modification |
M4 |
| ING-18 | localhost capture |
Default: events from localhost are not sent; with capture enabled they are sent |
M4 |
| ING-19 | Multi-domain d list |
Two domains produce two events, each under its own site UUID, with different user_id |
M4 |
| ING-20 | Proxy endpoint option | With endpoint set to a path on the fixture site proxied to the server, events arrive; the browser makes no request to the default endpoint |
M4 |
| ING-21 | Cookie-free | After a full session, document.cookie is empty, localStorage and sessionStorage are empty, the response to /api/event has no Set-Cookie header, and the browser context holds zero cookies |
M4 |
| ING-22 | Bounce semantics | One pageview with an engagement event leaves the session a bounce; a second pageview does not | M4 |
| ING-23 | Session continuity | Two visits 31 minutes apart (fake time through the integration suite S/SE tests; in the browser, via server-side seeded session TTL) produce two sessions | M4 |
| ING-24 | Geo and device enrichment | With injected X-Forwarded-For: 203.0.113.9, the Chrome UA and a 1,280-pixel-wide window: country_code DE, subdivision1_code DE-BE, city_geoname_id 2950159, browser Chrome with major.minor version only, OS set, device desktop, screen_size laptop; no raw IP or UA string in any column |
M4 |
| ING-25 | Exclusions | IP exclusion for 203.0.113.9 and page exclusion for /private/* each drop their events |
M4 |
| ING-26 | Unknown domain | An event for a domain not registered returns 202 and writes nothing | M4 |
| ING-27 | Headless User-Agent | The default HeadlessChrome User-Agent produces no rows and increments the filtered-traffic counter |
M4 |
| ING-28 | Tracker via npm package | A page that imports @quietstat/tracker (built bundle) behaves as ING-01, ING-02 and ING-14 |
M12 |
| ING-29 | Domain normalization | A page whose tracker is configured with the domain www.<site-domain>:8443 stores its events under the site registered as <site-domain> (no unknown-domain drop) |
M4 |
| ING-30 | Hostname allowlist | With allowed_hostnames empty, a page served from blog.<site-domain> is accepted and a page served from a foreign host that names the site domain in d is dropped with reason hostname_not_allowed (202, no row, not counted as billable usage); after setting allowed_hostnames to the single entry app.<site-domain>, the blog. page is dropped and app. is accepted |
M4 |
| ING-31 | Proxy key | A fixture proxy forwards events with X-Quietstat-Client-IP: 203.0.113.9 and the site's X-Quietstat-Proxy-Key: the event carries the DE location of that address; with a wrong key the same proxy's events carry no location (the header is ignored and the proxy's own address is used) |
M4 |
18.6 Playwright end-to-end user flows #
18.6.1 Projects, data and conventions #
| Item | Decision |
|---|---|
| Projects | e2e-chromium runs every flow. e2e-firefox and e2e-webkit run the flows marked "all browsers" below. A mobile project (Chromium with a 390x844 viewport and touch) runs the flows marked "mobile" |
| Server | Compiled server and worker run as separate processes (webServer option) with containers from the Playwright global setup, EDITION=selfhosted for the project e2e-* and a second project set e2e-hosted-* (Chromium only) with EDITION=hosted, the Stripe fake and the e-mail sink |
| Data | Each test creates its own team, user and site through API helpers (never through the UI unless the flow is about creating them), then inserts fixture events with the generator (basic, ecommerce or agency profile) anchored per Section 18.2.4 rule 7 |
| Flows involving e-mail read messages from the SMTP sink | |
| Worker triggers | Flows needing a scheduled job enqueue it through a test helper that sends the pg-boss job directly |
| Accessibility | Every flow ends with an axe scan (@axe-core/playwright) of the final page; zero violations of impact serious or critical; Section 4 defines the accessibility target |
| Artifacts | Trace on first retry, screenshot on failure, no video |
| Selectors | Role and label selectors (getByRole, getByLabel) and data-testid only where no accessible name exists; text content matching UI copy uses the i18n catalog keys, not literals |
18.6.2 Flow catalog #
| ID | Flow and mandatory assertions | Edition | Browsers | Milestone |
|---|---|---|---|---|
| E2E-01 | Signup with e-mail and password, receive verification e-mail, follow link, land on onboarding; password policy errors shown inline; duplicate e-mail shows the email_taken message inline (Section 11.3.1) |
both | all | M6 |
| E2E-02 | Login, logout, wrong password error, lockout message after repeated failures per Section 11, session persists on reload | both | all | M6 |
| E2E-03 | Password reset request, e-mail, new password, old sessions revoked | both | chromium | M6 |
| E2E-04 | Enable TOTP 2FA (scan secret displayed as text, enter code), receive recovery codes, log out, log in with TOTP, log in with a recovery code once (second use fails), disable 2FA | both | chromium | M6 |
| E2E-05 | Add first site: domain validation errors (a pasted URL is rejected, www.Example.com is shown as stored example.com), timezone preselected from browser, snippet shown with copy button, "waiting for first pageview" state, trigger a pageview from the fixture site, state turns to "receiving data" without reload |
both | all | M6 |
| E2E-06 | Self-hosted first run on empty database: the app redirects to the setup page, the signup route refuses (setup_required), the page accepts the one-time setup token (read from the worker log as Section 16.3.5 specifies) and creates the first account, and no billing or license UI appears anywhere in navigation |
selfhosted | chromium | M6 |
| E2E-07 | Dashboard overview with the basic profile: top stats values equal oracle, chart renders with accessible summary, switching every time range (including custom range and Last 24 Hours) updates numbers and URL, keyboard shortcuts of Section 9 |
both | all | M7 |
| E2E-08 | Period comparison toggle: previous period and year-over-year, deltas displayed with correct sign and color classes | both | chromium | M7 |
| E2E-09 | Reports tabs: sources, channels (all 18 channel labels appear for the basic profile), campaigns, pages (path and full URL toggle), entry pages, exit pages, countries, regions, cities, browsers, operating systems, devices (desktop, mobile, tablet) and screen sizes (mobile, tablet, laptop, desktop) as two separate reports; the AI Assistants channel shows ChatGPT, Claude, Perplexity entries |
both | chromium | M7 |
| E2E-10 | Filters: add filters for source, page, location, device; combine two filters; remove one; reload keeps filters via URL; invalid filter combination shows the message from Section 9 | both | all | M7 |
| E2E-11 | Saved segments: save current filters as a segment, rename, apply, delete; personal segment hidden from another user; editor creates a site-wide segment | both | chromium | M7 |
| E2E-12 | Realtime view: visitors counter increments after a browser sends events, sources and pages lists update, counter drops after the window passes (fake data timestamps) | both | chromium | M7 |
| E2E-13 | Annotations: create on a chart date, personal and shared types, edit, delete, visibility per role | both | chromium | M7 |
| E2E-14 | Consolidated all-sites view and sites list: sorting by traffic and by domain, pinning and unpinning a site (the 20-pin limit returns 409 limit_reached), shared team scope, empty state when no sites |
both | chromium | M7 |
| E2E-15 | Site settings: general (rename domain rules, timezone change effect on next load), IP and page exclusions saved and effective (ties to ING-25), datacenter-filter toggle, hostname allowlist (an event from a hostname outside the list is not counted, ties to ING-30), Filtered traffic stat shown | both | chromium | M6 |
| E2E-16 | Goals: create a goal of each type (event, pageview, scroll); narrow by 3 properties and see a 4th rejected; goal conversion appears in reports and filters |
both | chromium | M8 |
| E2E-17 | Custom property allow-list management and property breakdown report | both | chromium | M8 |
| E2E-18 | Revenue: ecommerce profile, revenue goal, total revenue and average revenue tiles, revenue by source, multi-currency display with short and long formats |
both | chromium | M8 |
| E2E-19 | Funnels: create ordered funnel with 4 steps from mixed pages and events, view drop-off; edit to optional step; switch to strict order and see changed numbers; period comparison; revenue per step; delete | both | chromium | M9 |
| E2E-20 | User journeys: Sankey renders for the fixture, node click filters, depth control, empty state | both | chromium | M9 |
| E2E-21 | Team management: create team, invite by e-mail, accept in a second browser context, role change, remove member, last-owner protection message, ownership transfer, guest access to one site | both | chromium | M6 |
| E2E-22 | Viewer restrictions: viewer sees dashboards, cannot see settings routes (404 or redirect per Section 11), can create a personal segment and subscribe self to a report | both | chromium | M6 |
| E2E-23 | Team deletion with 7-day undo: delete, banner with undo, undo restores everything | both | chromium | M6 |
| E2E-24 | API keys: create (secret displayed once), use it with the request fixture against POST /api/v2/query (available because M5 has merged before M6 does) and /api/v1/sites, revoke and see 401 |
both | chromium | M6 |
| E2E-25 | Shared link: create, open in a fresh context without cookies, see read-only dashboard; password-protected link prompts and rejects wrong password; segment-limited link cannot widen filters; revoke returns not-found page | both | all | M10 |
| E2E-26 | Embed: a third-party fixture page embeds the dashboard iframe via the snippet from the UI; renders; the main dashboard URL refuses framing | both | chromium | M10 |
| E2E-27 | Scheduled e-mail report: subscribe self, enqueue job, e-mail arrives with HTML and text, unsubscribe link works without login | both | chromium | M10 |
| E2E-28 | Traffic spike notification: seeded baseline plus burst, enqueue detection job, e-mail arrives, second run within cooldown sends none | both | chromium | M10 |
| E2E-29 | CSV export: download of each report, header row, formula-injection prefix applied for a seeded page named =HYPERLINK(...) |
both | chromium | M11 |
| E2E-30 | CSV import: upload a valid fixture, progress, imported data appears with an "imported" indicator and date range; invalid file shows row-level errors; cancel; delete import | both | chromium | M11 |
| E2E-31 | GA4 import via the Google fake: connect, pick property, choose date range, run, completion e-mail, data visible | both | chromium | M11 |
| E2E-32 | Snippet and setup guides: snippet panel shows the script tag with the site domain, proxy guide, and each framework guide of Section 7 renders; copy buttons work | both | chromium | M12 |
| E2E-33 | WordPress plugin installed in a wp-env site: setting the domain makes the page source contain the tracker and one pageview reaches ClickHouse (Section 18.7.4) |
selfhosted | chromium | M12 |
| E2E-34 | Hosted: unlicensed team banner, one site limit, events dropped; redeem a code for Tier 1 (solo); banner disappears; usage meter shows 0 of 100,000; site limit now 5 |
hosted | chromium | M13 |
| E2E-35 | Hosted purchase via Stripe Checkout fake: choose tier, redirect to fake checkout, deliver signed webhook, return to app, license active, invoice listed and PDF downloadable with number <INVOICE_PREFIX>-<year>-<6 digits> (default prefix QS) |
hosted | chromium | M13 |
| E2E-36 | Hosted upgrade: Tier 1 (solo) to Tier 2 (pro) charges 90 USD difference (149 minus 59); Tier 2 to Tier 3 (agency) charges 200 USD; Tier 1 to Tier 3 charges 290 USD; limits update |
hosted | chromium | M13 |
| E2E-37 | Hosted usage states with a 100,000-event cap: seeded usage_monthly counts of 79,999, 80,000, 99,999, 100,000, 109,999 and 110,000 show respectively no banner, warning, warning, grace, grace and blocked; owners and admins see the banners with figures, editors and viewers see only the figure-free text of Section 15.12.2 |
hosted | chromium | M13 |
| E2E-38 | Operator console: super admin lists users, teams, licenses; generates a code batch and downloads the CSV; refunds a purchase inside the refund window (REFUND_WINDOW_DAYS, default 60) and is refused after it unless the override with a reason is used; impersonates read-only (mutating buttons disabled, API calls rejected) |
hosted | chromium | M13 |
| E2E-39 | Account settings: change e-mail with confirmation, change password, appearance (light, dark, system), session list and revocation, delete account (soft delete; the next login shows only the restore screen of Section 11.13.3) | both | chromium | M6 |
| E2E-40 | Responsive smoke: overview, reports and settings render without horizontal scroll at 390 px width; navigation reachable by touch | both | mobile | M7 |
| E2E-41 | Footer attribution shows the GEOIP_ATTRIBUTION text (default IP geolocation by DB-IP) with the bundled database; with GEOIP_ATTRIBUTION set to another value the footer shows that value |
selfhosted | chromium | M7 |
The "all browsers" flows run in Chromium, Firefox and WebKit. Total runtime of the pull-request pipeline must stay within the budget in Section 18.14.
18.7 Tracker, WordPress plugin and GTM template verification #
18.7.1 Browser matrix #
| Layer | Matrix |
|---|---|
| Playwright engines | Chromium, Firefox and WebKit at the versions bundled with the installed Playwright release, desktop viewports |
| Device emulation | One mobile Chromium profile (Android) and one mobile WebKit profile (iPhone), used for ING-01, ING-02, ING-12, ING-13, ING-21; the emulated viewport widths also assert the screen_size value of ING-24's rule |
| Syntax floor | The bundle is parsed by esbuild with the target declared in Section 7; a test fails if the bundle contains syntax newer than that target |
| Scenarios | ING-01 to ING-27 and ING-29 to ING-31 run in all three engines; ING-08 (beacon on unload) and ING-13 (visibility) are the engine-sensitive ones and are never skipped |
18.7.2 Tracker unit tests #
With a DOM implementation and fake fetch, sendBeacon and timers: option parsing for every option in Section 7 (valid, invalid, defaults); payload field mapping (n u d r w h p $ i sd e); event queue ordering; no throw on sendBeacon absence; no throw when localStorage access is blocked (the tracker only reads the opt-out flag inside try); idempotent double-load guard; SSR import (import under plain Node without window does not throw); the exported public API surface of the npm package matches a stored API snapshot (packages/tracker/api-snapshot.json).
18.7.3 Size budget and forbidden API check (pnpm check:size) #
- The script builds every tracker variant defined in Section 7 (core and each extension build) with esbuild, measures gzip (level 9) and brotli (quality 11) byte sizes of the minified output, and compares them with the budgets of Section 7.8.1, which the script reads from
packages/tracker/size-budget.json: the core is at most 1,500 bytes gzip, each module has its own budget (tagged 650, outbound 450, downloads 400, forms 300, notfound 260, engagement 800) and the genericscript.jsis at most 4,800 bytes. A test asserts thatsize-budget.jsoncontains an entry for every variant and no extras. - The script fails if any variant exceeds its budget. It also prints the difference against the same measurement on the
mainbranch and fails when any variant grows by more than 100 bytes (gzip) relative tomainunless the pull request description contains the linesize-increase-approvedand the increase is recorded inDECISIONS.md. - Forbidden-token scan on every built variant:
document.cookie,localStorage.setItem,localStorage.removeItem,localStorage.clear,sessionStorage,indexedDB,openDatabase,toDataURL,getImageData,RTCPeerConnection,AudioContext,fingerprint,navigator.plugins,navigator.mediaDevices. Any match fails, except exactly onelocalStorage.getItem("quietstat_ignore")occurrence (the opt-out flag read of Section 17.9.2). - The dashboard bundle check in Section 18.14.2 runs in the same script invocation.
18.7.4 WordPress plugin #
| Test | Detail |
|---|---|
| PHPUnit unit | Every setting in Section 7 has tests for sanitization (valid, invalid, empty), default value, and the rendered script tag attributes; output escaping (esc_attr, esc_url) is asserted with hostile values such as "><script> |
| PHPUnit with the WordPress test suite | Activation, deactivation, uninstall (options removed), settings page capability check (only administrators), nonce verification on save, role-based exclusion of logged-in roles if Section 7 defines it, multisite activation |
| Matrix | The lowest PHP version in Section 7 and the newest stable PHP, each against the newest WordPress release supported in Section 7 and the previous major WordPress release |
| Plugin Check | The WordPress Plugin Check tool runs with zero errors; warnings listed in integrations/wordpress/plugin-check-allowlist.txt with a reason per line |
| Browser | wp-env starts WordPress with the plugin; Playwright sets the domain in the settings page, loads the home page, asserts the script tag, and asserts that the pageview reaches ClickHouse (E2E-33) |
| Packaging | The release zip contains only the files listed in a manifest test; license header is GPL-2.0-or-later; readme headers parse |
18.7.5 GTM template #
The template file keeps its own test scenarios in its tests section. integrations/gtm/test/run-template-tests.ts is an in-repo harness that implements the sandboxed JavaScript API functions used by the template (injectScript, setInWindow, callInWindow, copyFromWindow, queryPermission, logToConsole, getUrl, createQueue, makeNumber, makeString) as fakes and runs every scenario. Required scenarios: script injected once with the configured domain; each option maps to the right attribute; custom event tag fires the tracker call with props; revenue fields; missing domain triggers failure; permissions declared in the template match the API calls actually made (a static check compares the two lists). The template file license header is Apache-2.0. A manual import check is part of the launch checklist in Section 20.10 because it requires a Google Tag Manager workspace.
18.7.6 Tracker payload conformance tests (TRK-*) #
These tests run the real tracker bundle in the three Playwright engines against the recording fake receiver of tests/support (no server, no databases), so the tracker can be verified before the ingestion pipeline exists. They run in the Playwright project tracker (pnpm test:ingestion -- --project tracker).
| ID | Assertion | Milestone |
|---|---|---|
| TRK-01 | A page load posts exactly one body with n = "pageview", absolute u, d equal to the configured domain, w equal to the window width, r null or the referrer; Content-Type is text/plain; no custom request headers are sent (so no CORS preflight occurs) |
M3 |
| TRK-02 | pushState navigation posts one pageview per distinct URL; replaceState to the same URL posts none; popstate posts one |
M3 |
| TRK-03 | Hash routing option off: no event on hashchange; on: one event per change and h = 1 |
M3 |
| TRK-04 | Outbound link (Outbound Link: Click, prop url), file download (File Download, prop url; each configured extension and one unconfigured), form submission (Form: Submission, prop path) and 404 (404, prop path) each post the event names and props defined in Section 7 |
M3 |
| TRK-05 | Scroll to 100% and leaving the page posts one engagement event with i = false, sd equal to the maximum depth reached and e equal to foreground time; hidden-tab time excluded |
M3 |
| TRK-06 | Custom event with up to 30 props and a revenue object posts p and $ exactly as passed; a 31st prop is dropped or rejected exactly as Section 7 defines, with a console message when logging is enabled |
M3 |
| TRK-07 | Options: manual pageview mode, transformRequest returning a modified payload and null, localhost capture off and on, endpoint override, multi-domain d list with 10 domains |
M3 |
| TRK-08 | Cookie-free: after the full set of interactions document.cookie is empty, localStorage and sessionStorage are empty, and no cookie exists in the browser context |
M3 |
| TRK-09 | pnpm check:size passes for every variant (the core at most 1,500 bytes gzip) and the forbidden-token scan passes |
M3 |
| TRK-10 | The packed npm tarball installs in a clean project; ESM import, CommonJS require and type resolution succeed; importing under plain Node does not throw; the public API snapshot matches | M3 |
18.8 Load tests (k6) #
18.8.1 Environment and tooling #
Load tests run with the k6 binary against a compose stack defined in tests/load/compose.yml with this reference hardware (containers with CPU and memory limits): one web instance 4 vCPU and 8 GiB; ClickHouse 8 vCPU and 32 GiB; Postgres 2 vCPU and 4 GiB; Valkey 1 vCPU and 1 GiB; k6 on a separate host or with 4 dedicated vCPU. Scripts live in tests/load/*.js; every script declares its thresholds so that the k6 exit code is non-zero when a target is missed. The fixture generator (Section 18.11) provisions datasets. All load tests use EDITION=selfhosted except where noted.
Traffic realism for ingestion tests: 50,000 distinct client IPs sent through X-Forwarded-For (the per-IP limit of 100 requests per 10 s, policy event_ingest of Section 17.6, would otherwise throttle the test; 5,000 events/s needs at least 500 distinct IPs; the k6 host is inside TRUSTED_PROXIES of the load stack so that the header is honored), 200 sites, payload mix 70% pageview, 20% custom events with 3 props, 5% revenue, 5% engagement, realistic User-Agent pool of 40 strings, payload bodies sent as text/plain.
18.8.2 Scenarios and targets #
| ID | Scenario | Pass criteria |
|---|---|---|
| LOAD-01 | Sustained ingestion on one web instance: constant arrival rate of 5,000 events/s for 10 minutes |
POST /api/event p95 under 50 ms and p99 under 150 ms; at least 99.9% of responses are 202; zero 5xx; after the test, the row count in events equals accepted count minus counted drops within 0%, verified after the buffer drains (at most 10 s); event-loop lag p99 under 100 ms; resident memory growth during the run under 10% after the first minute; CPU under 85% on average |
| LOAD-02 | Horizontal scale: three web instances behind a round-robin balancer at 15,000 events/s for 5 minutes against one ClickHouse node |
Same latency and loss criteria as LOAD-01; throughput scales at least 90% of linear |
| LOAD-03 | Spike: one instance, 15,000 events/s for 60 seconds after 5,000 events/s steady | No process crash or restart; resident memory under 1.5 GiB (limit 2 GiB for this scenario); within 30 s after the spike ends, p95 returns under 50 ms; any data dropped under overload is limited to the policy in Section 8 and counted in a metric |
| LOAD-04 | ClickHouse outage: pause the ClickHouse container for 60 s during 2,000 events/s | readyz returns 503 while paused; web process stays up; after resume, buffered events are inserted; total loss is no more than the bound stated in the failure-mode policy of Section 8 |
| LOAD-05 | Per-IP limit: one IP sends 150 requests within 10 s | Requests 1 to 100 return 202; requests 101 and beyond return 429 with a Retry-After header; after the window the IP is accepted again |
| LOAD-06 | Soak: 1,000 events/s for 2 hours (weekly run) | Memory growth under 10% across hours 1 to 2, no file-descriptor growth over 5%, event-loop lag p99 under 100 ms, zero 5xx |
| LOAD-07 | Mixed: 2,000 events/s ingestion plus 20 virtual users cycling the dashboard query set of LOAD-09 on dataset D1 | Ingestion p95 under 75 ms; every dashboard query p95 within 1.5 times its D1 target in Section 18.8.3 |
| LOAD-08 | Tracker file delivery: 2,000 requests/s for 2 minutes for the tracker script path from Section 7 | p95 under 20 ms; response includes the caching headers of Section 7; 100% 200 or 304 |
| LOAD-09 | Dashboard query set on dataset D1 with 20 virtual users for 5 minutes | Per-query p95 targets in Section 18.8.3 (D1 column); error rate 0 |
| LOAD-10 | Dashboard query set on dataset D2 with 20 virtual users for 5 minutes | Per-query p95 targets (D2 column); error rate 0 |
| LOAD-11 | Consolidated all-sites view on dataset D3 (100 sites) with 10 virtual users | p95 under 800 ms; the sites list p95 under 400 ms |
| LOAD-12 | Stats API concurrency: 50 virtual users on POST /api/v2/query with a rotating set of 10 query shapes on D1, rate limits configured high |
p95 under 1.5 times the matching dashboard target; no 5xx; 429 behavior verified separately under the limits of Section 13 |
| LOAD-13 | Pull-request smoke (--smoke): 1,000 events/s for 30 seconds on a minimal stack and a 1M-event dataset query set with 5 virtual users for 30 seconds |
p95 for ingestion under 100 ms; all D1 query targets multiplied by 3 (smoke environment is smaller); zero 5xx |
18.8.3 Datasets and dashboard query targets #
| Dataset | Content | Generator profile |
|---|---|---|
| D1 | One site, 30 days, 1,000,000 events (about 600,000 pageviews, 150,000 sessions) | --scale medium-1m |
| D2 | One site, 12 months, 50,000,000 events | --scale large-50m |
| D3 | 100 sites, 30 days, 200,000,000 events total with a Zipf distribution of traffic across sites | --scale agency-200m |
| Query | Definition | p95 on D1 | p95 on D2 |
|---|---|---|---|
| Q1 | Top stats: visitors, visits, pageviews, views_per_visit, bounce_rate, visit_duration, last 30 days |
100 ms | 300 ms |
| Q2 | Time series by day, last 30 days, all top-stat metrics | 100 ms | 300 ms |
| Q3 | Time series by month, last 12 months | 150 ms | 500 ms |
| Q4 | Top pages, limit 100 | 150 ms | 400 ms |
| Q5 | Top sources with one page filter | 200 ms | 600 ms |
| Q6 | Countries plus regions plus cities | 150 ms | 400 ms |
| Q7 | Devices, browsers, operating systems, screen sizes (four queries in parallel; slowest counted) | 150 ms | 400 ms |
| Q8 | Realtime report | 50 ms | 100 ms |
| Q9 | Goal conversion with two property filters | 250 ms | 800 ms |
| Q10 | Funnel with 5 steps, last 30 days | 500 ms | 1,500 ms |
| Q11 | User journeys, depth 4, top 50 paths | 600 ms | 2,000 ms |
| Q12 | Revenue breakdown by source | 250 ms | 800 ms |
| Q13 | Q1 plus comparison to previous period | 200 ms | 600 ms |
| Q14 | Full dashboard load: all overview widgets requested in parallel, slowest request | 1,500 ms | 3,000 ms |
Targets are server-side response time measured by k6 (including network on the same host). A target missed by less than 10% in a single run is retried once; two consecutive misses fail.
18.8.4 Where load tests run #
LOAD-13 runs in every pull-request pipeline. LOAD-01, LOAD-05, LOAD-08 to LOAD-12 run nightly. LOAD-02 to LOAD-04 and LOAD-07 run weekly and before every release. LOAD-06 runs weekly. A release requires the most recent nightly and weekly runs to be green and no older than 7 days at tag time.
18.9 Privacy tests #
The privacy suite proves that raw client IP addresses and User-Agent strings are never persisted or logged, and that the other privacy properties of Section 8 and Section 17 hold. It runs in pnpm test:privacy and in every pull-request pipeline.
18.9.1 Canary method #
- A canary client uses addresses and User-Agent values that cannot occur naturally: IPv4
203.0.113.77, a second IPv4198.51.100.201, IPv62001:db8:feed:beef::7, and the User-AgentMozilla/5.0 (X11; Linux x86_64) QSCANARY-<32 random hex> Chrome/126.0.0.0 Safari/537.36. - For every address the test computes all textual and numeric forms: dotted and compressed forms, uppercase and lowercase hex, expanded IPv6, 32-bit and 128-bit integer forms in decimal and hex, base64 of the raw bytes, and the reverse-pointer form. The canary User-Agent is searched as the full string, as the
QSCANARY-token, and as its base64 and URL-encoded forms. - The test drives traffic through every path that handles a client request (list in 18.9.2).
- After a forced buffer flush and job runs, the test dumps every data store and log stream (18.9.3) and asserts zero occurrences of any form.
18.9.2 Traffic paths exercised #
Accepted pageview and custom event; engagement event; revenue event; multi-domain event; event dropped for bot, datacenter, spam referrer, excluded IP, excluded page, unknown domain, hostname_not_allowed, unlicensed team (hosted; registered in tests/milestones.json under M13), cap at hard_at (hosted; registered under M13); events carrying X-Quietstat-Client-IP with a valid and with an invalid X-Quietstat-Proxy-Key; malformed JSON (400); oversized body (413); per-IP rate limit (429); handler error forced through a fault-injection dependency (500); OPTIONS preflight; requests through a trusted proxy with X-Forwarded-For chains of 1, 2 and 5 hops; requests from an untrusted peer that spoof X-Forwarded-For; dashboard login from the canary address (the dashboard auth flow is allowed to use IP for rate limiting in memory or Valkey only for the duration of the window defined in Section 17, never persisted, so for the dashboard paths the scan covers every store except the Valkey rate-limit keys, which are scanned for the raw IP form and must contain only hashed identifiers per Section 17).
18.9.3 Stores and streams scanned #
| Store | Method |
|---|---|
| ClickHouse | For every table returned by SHOW TABLES in the test database, SELECT * FORMAT JSONEachRow, scanned as text; plus system.query_log, system.text_log, system.part_log filtered to the test database with query logging enabled in the test server config |
| PostgreSQL | pg_dump --data-only --inserts of the test database including the pgboss schema; every table scanned as text; the server runs with log_statement = all and its log is scanned |
| Valkey | SCAN over every key in the worker's logical database, then DUMP (or type-specific reads for strings, hashes, sets, sorted sets, lists, streams) of every value, scanned as text, keys included |
| Application logs | stdout and stderr of the server and worker processes captured at the most verbose level (trace) for the whole run, including logs for forced 500 errors and the startup banner |
| HTTP responses | Headers and bodies of every response in the run |
| E-mails | Every message in the SMTP sink |
| Files | Any file written to the temporary directory and the data directory by the server and worker (export files, import staging, PDF output) |
18.9.4 Test catalog #
| ID | Assertion | Milestone |
|---|---|---|
| PRIV-01 | The canary scan across all stores in 18.9.3 after the traffic of 18.9.2 finds zero occurrences of any canary form | M4 |
| PRIV-02 | Same scan with the pino log level set to trace and with a thrown error in the request handler: logs contain no IP, no User-Agent and no x-forwarded-for or x-quietstat-client-ip value; the default Fastify request log serializer is replaced so remoteAddress, req.ip and req.headers['user-agent'] are never emitted |
M4 |
| PRIV-03 | Schema snapshot: a snapshot test of every column name in Postgres (information_schema.columns) and ClickHouse (system.columns) is stored in tests/privacy/schema-snapshot.json. The test fails on any difference. A second test fails if any column whose entire name equals one of ip, ip_address, client_ip, remote_addr, user_agent, ua, ua_raw, lat, latitude, lon, lng, longitude, fingerprint, cookie exists (whole-name match, so legitimate names such as ip_exclusions or a country_code column do not trigger it) |
M2 |
| PRIV-04 | Location fields contain only: country matching ^[A-Z]{2}$, region matching ^[A-Z]{2}-[A-Z0-9]{1,3}$, city as an unsigned integer GeoNames id; the fixture MMDB record with latitude and longitude produces no coordinates anywhere |
M4 |
| PRIV-05 | All browser and OS version columns match ^\d+(\.\d+)?$ or are empty |
M4 |
| PRIV-06 | No stored pathname contains ? or # (except hash-route events, where the fragment route is stored per Section 8); the query parameter secret=1 of ING-06 appears nowhere in any store |
M4 |
| PRIV-07 | The same canary client on two consecutive UTC days (fake clock) produces two different user_id values; on two different sites on the same day, two different values |
M4 |
| PRIV-08 | After the 00:30 UTC salt-expire run, the previous salt's 16 bytes (hex and base64) no longer appear in the salts table; at no sampled moment does salts have more than 2 rows. The table is UNLOGGED (relpersistence = 'u'), so no salt reaches write-ahead logs, point-in-time recovery or replicas, and a pg_dump --exclude-table-data=salts of the database contains neither the current nor any earlier salt in any encoding (S-11 and S-12 assert the table properties and the post-crash behavior; PRIV-16 asserts it on the real backup procedure) |
M4 |
| PRIV-09 | No response from /api/event or from the tracker file sets a cookie (Set-Cookie), and no response from any route outside the dashboard authentication and share-password flows sets a cookie |
M4 |
| PRIV-10 | The pino redaction configuration contains exactly the paths required by Section 17, and a unit test logs an object containing each sensitive key (password, token, authorization, cookie, x-csrf-token, x-forwarded-for, user-agent, email, API keys and share passwords) and asserts redaction |
M1 |
| PRIV-11 | The custom lint rules of 18.9.6 reject a fixture file that logs request.ip |
M1 |
| PRIV-12 | An export file, the account data export (GET /api/app/v1/account/data-export), an import staging file and a PDF invoice contain no canary forms and no visitor-level identifiers (user_id, session_id do not appear in any export) |
M11 |
| PRIV-13 | Deleting a site removes every row for its site_id in all ClickHouse tables including sessions and imported tables, and the Valkey keys for that site |
M6 |
| PRIV-14 | The tracker never writes cookies, local storage or session storage and reads only the opt-out flag of Section 17.9.2 (also covered by ING-21 in three engines) and never sends any identifier other than those in the payload definition of Section 7.6.1 | M4 |
| PRIV-15 | Valkey keys written by ingestion contain only user_id, session_id and site UUID based identifiers and expire within the session TTL of Section 8 |
M4 |
| PRIV-16 | The backup procedure of Section 16 (the shipped script and its documented invocation), run against the basic profile while a current and a previous salt exist, produces archives in which neither salt (hex and base64) nor any row of salts appears; restoring them into a scratch database yields an empty salts table that the next web start repopulates |
M14 |
| PRIV-17 | The shipped Caddyfile of Section 16.3.3 is checked statically: no client address, no User-Agent and no request URI is logged | M14 |
18.9.5 Schema snapshot discipline #
Adding a column to any table requires updating tests/privacy/schema-snapshot.json in the same pull request and adding a DECISIONS.md entry that states the purpose of the column and confirms it holds no client IP, User-Agent string, precise location or persistent identifier. pnpm check:conventions runs the snapshot comparison.
18.9.6 Custom lint rules #
Implemented as ESLint rules in tools/eslint-rules/ and enabled for apps/* and packages/*:
| Rule | Behavior |
|---|---|
qs/no-client-identifier-logging |
Error when request.ip, request.ips, request.socket.remoteAddress, request.headers['user-agent'], request.headers['x-forwarded-for'] or request.headers['x-quietstat-client-ip'] (any casing) or a variable assigned from them is passed to any logger method or interpolated into an error message |
qs/no-direct-time |
Error on Date.now(), argument-less new Date() and performance.now() outside packages/shared/src/clock.ts |
qs/no-math-random |
Error on Math.random() outside packages/shared/src/rng.ts and the load-test scripts |
qs/no-raw-sql-interpolation |
Error on template literals containing expressions inside any string passed to ClickHouse query methods or Postgres raw SQL helpers, except through the approved tagged helpers of packages/query and Drizzle's sql tag |
qs/no-pending-marker-comments |
Error on comments containing the markers FIXME, XXX, HACK and the common four-letter "to do" marker |
qs/no-focused-tests |
Error on .only in any test file |
qs/product-name-constant |
Error on a string literal equal to the product name outside PRODUCT_NAME definition (Section 16.4) and test files |
18.10 Security tests #
Security tests live in tests/security/ and run in pnpm test:security on every pull request, except the items marked "nightly". Controls themselves are specified in Section 17 and Section 11; this section specifies the tests.
| ID | Area | Assertion | Milestone |
|---|---|---|---|
| SEC-01 | Authentication required | Every route in the registry whose auth kind is not public returns 401 without credentials, with the error envelope of Section 4.4 | M6 |
| SEC-02 | Authorization matrix | For every route and every role (owner, admin, editor, viewer, guest_viewer, guest_editor, member of another team, anonymous) the response status equals the entry generated from the permission matrix of Section 11 |
M6 |
| SEC-03 | Object-level access (IDOR) | For every resource type with an identifier in the path or body (site, goal, funnel, segment, annotation, shared link, API key, invitation, import, export, report subscription, license, purchase), a user of team A requesting a resource of team B receives the not-found or forbidden status chosen by the policy in Section 11; no response body reveals existence | M6 |
| SEC-04 | CSRF | Each mutating dashboard route (POST, PUT, PATCH, DELETE under /api/app/v1) called with a valid session cookie but without X-CSRF-Token returns 403 csrf_invalid; with a wrong token returns 403 csrf_invalid; with a valid token succeeds. The routes that Section 11.7 lists as exempt (the CSP report endpoint, and the OAuth callback that is protected by its single-use state instead) are asserted by their own rule; the unauthenticated setup and login routes require the anonymous CSRF token. GET routes never mutate state, except that OAuth callback |
M6 |
| SEC-05 | Cookie attributes | Session and CSRF cookies carry HttpOnly (session), Secure (when the public URL is https), SameSite and path as specified in Section 11; session cookie value is not guessable (at least 128 bits of entropy, checked by length and distribution test over 1,000 samples) |
M6 |
| SEC-06 | Session fixation and rotation | Login issues a new session identifier and invalidates any pre-login identifier; password change and 2FA enable revoke other sessions as Section 11 specifies; logout deletes the server-side session | M6 |
| SEC-07 | Brute-force limits | Login, signup, password-reset request, 2FA verification, share-link password and invitation acceptance each return 429 with Retry-After after the limits of the policies in Section 17.6; limits are keyed per account and per client without persisting raw IP (Section 17) |
M6 |
| SEC-08 | Account enumeration | Login with an unknown e-mail and password reset for an unknown e-mail produce responses identical in status and body shape to the known-account case, and identical within 150 ms in median response time over 50 samples each. Signup with an existing e-mail is the documented exception: it returns 409 email_taken (decision of Section 11.3.1), bounded by the signup limit of Section 17.6, and the test asserts exactly that |
M6 |
| SEC-09 | Password storage | users.password_hash begins with $argon2id$; two users with the same password have different hashes; the plaintext never appears in logs or the database |
M6 |
| SEC-10 | API key storage | api_keys holds only the SHA-256 hash (a 32-byte bytea) and a non-secret display prefix (the first 12 characters, Section 13.2.1); the full key is returned exactly once on creation; a revoked or deleted key fails immediately; a key presented with an invalid prefix takes no longer than a valid-prefix miss (no timing oracle, measured with 200 samples and a 20% tolerance) |
M6 |
| SEC-11 | Token storage | E-mail and invitation tokens are stored only as keyed digests (Section 17.7); share tokens (192-bit) and license codes are stored as issued, by design, and are found through a unique index; a miss costs the same as a hit (license-code responses are padded, Section 15.8.6); reuse of a consumed or expired token fails with the codes of Section 11 | M6 |
| SEC-12 | TOTP and recovery codes | A TOTP code cannot be reused within its window; recovery codes are single use; 2FA secrets are encrypted at rest (the database column does not contain a valid base32 secret); disabling 2FA requires a current password | M6 |
| SEC-13 | SQL injection (Postgres) | An injection corpus of at least 40 strings is sent into every string field of every dashboard route (names, domains, search parameters, cursors, sort keys); no response is 500, no corpus string alters query behavior (verified by comparing result counts with a benign value) | M6 |
| SEC-14 | SQL injection (ClickHouse) | The corpus of Q-SAFE-2 is sent through POST /api/v2/query, the dashboard query routes (single-site, batch and consolidated) and the shared-link query routes; no 500, no data from other sites returned, and after the run the ClickHouse tables are unchanged |
M5 |
| SEC-15 | Cross-site scripting (stored) | Hostile values ("><img src=x onerror=window.__xss=1>, <script>window.__xss=1</script>, javascript:window.__xss=1, SVG payloads) are stored as site names, goal names, funnel step names, segment names, annotation text, custom property keys and values, page paths, referrer hosts, UTM values and team names; Playwright opens every page that renders them and asserts window.__xss is undefined and the text appears escaped |
M7 |
| SEC-16 | Content Security Policy and headers | The dashboard HTML contains no executable inline script: its only inline <script> element is the non-executable <script type="application/json" id="qs-bootstrap"> bootstrap data, and the theme bootstrap is the external file /assets/theme-init.js loaded before the app bundle. Dashboard HTML responses carry the headers of Section 17 (Content-Security-Policy without unsafe-inline for scripts, X-Content-Type-Options: nosniff, Referrer-Policy, Strict-Transport-Security when https, Permissions-Policy); dashboard routes send frame-ancestors 'none'; share and embed routes send the frame policy of Section 12.4.3 (frame-ancestors computed per link from its embed mode and allowed origins); Playwright asserts no CSP violation is reported during E2E-07 |
M7 |
| SEC-17 | CORS | POST /api/event and the tracker file answer with Access-Control-Allow-Origin: *; GET /api/openapi.json answers with * and no credentials; every other route family (/api/app/v1, /api/v1, /api/v2) returns no Access-Control-Allow-Origin header for a foreign Origin and rejects its preflight (Section 17.5.3); /api/share/v1 follows Section 12 |
M6 |
| SEC-18 | SSRF | Every server-side outbound request path (GA4 and Google OAuth calls, the installation-verification request of Section 7.15.2, any other URL verification or webhook the product performs per Section 17) is tested with targets http://127.0.0.1, http://[::1], http://[::ffff:127.0.0.1], http://0.0.0.0, http://10.0.0.1, http://172.16.0.1, http://192.168.0.1, http://169.254.169.254, a NAT64 address (64:ff9b::a00:1), a hostname resolving to a private address, a redirect to a private address, and a DNS-rebinding fake resolver that returns a public address then a private one; all are blocked by guardedFetch of Section 17.4 (real implementation, not the test double) |
M11 |
| SEC-19 | Open redirect | Login next, checkout return and OAuth return parameters accept only same-origin relative paths and the configured public URL; //evil.test, https://evil.test, /\evil.test and encoded variants are rejected |
M6 |
| SEC-20 | Path traversal and downloads | Export, invoice and account data export (GET /api/app/v1/account/data-export) download routes accept only identifiers owned by the caller; ../ and absolute paths in any filename input are rejected; downloads set Content-Disposition: attachment and X-Content-Type-Options: nosniff |
M11 |
| SEC-21 | Upload handling | CSV import rejects: a file above the size limit in Section 14 (and accepts one byte below), a non-CSV content type with a CSV extension, a binary file, a file with more columns than allowed, a zip bomb-style repeated-row file hitting the row limit, and a file with a null byte; the rejected upload leaves no staging file behind | M11 |
| SEC-22 | CSV and formula injection | Exports neutralize cells beginning with =, +, -, @, tab and carriage return (also covered by the CSV unit tests and E2E-29) |
M11 |
| SEC-23 | Request size and shape | Dashboard and API routes reject bodies above the limit of Section 4 with 413; JSON with duplicate keys, a prototype-pollution payload (__proto__, constructor.prototype) and deeply nested objects (depth above 20) is rejected with 400 and does not alter object prototypes |
M6 |
| SEC-24 | Rate limits | API key routes return rate-limit headers and 429 per the policies of Section 17.6 (header semantics of Section 13.3.3); rate-limit counters live in Valkey and survive a web restart; with Valkey paused, the bearer-key routes (/api/v1, /api/v2) fail closed with 503 unavailable while the ingestion rate limiter fails open (Section 17.6) |
M5 |
| SEC-25 | Secrets and logs | After running the whole integration suite with sentinel secrets (database password, SMTP password, Stripe secret key, webhook secret, SECRET_KEY_BASE, a site proxy key), a scan of all captured logs and HTTP responses finds none (the one-time setup token is excluded: it is printed once by design, Section 16.3.5); the bootstrap endpoint GET /api/app/v1/bootstrap never returns secret values |
M1 |
| SEC-26 | Webhook authenticity | Stripe webhook route rejects unsigned, mis-signed and stale requests (detail in Section 18.13) | M13 |
| SEC-27 | Impersonation safety | Read-only impersonation blocks mutating requests with 403 impersonation_read_only, is visibly marked in the UI, expires after the lifetime in Section 15 and writes audit_log rows admin.impersonation_started, admin.impersonation_ended and admin.impersonation_expired; blocked mutations are counted by the rejected-write counter of Section 15.17.11 and are not individually audited |
M13 |
| SEC-28 | Operator console isolation | /admin routes return 404 for non-super-admins (not 403), including users whose e-mail differs from a SUPER_ADMIN_EMAILS entry by a plus-suffix (a case-only variant is the same address, because addresses are lowercased at signup and compared case-insensitively, Section 15.17.1), and for every user when EDITION=selfhosted |
M13 |
| SEC-29 | Dependency and image scanning | pnpm audit --prod reports zero high or critical advisories; container image scan reports zero critical and zero high findings that have a fix; the gate and the exceptions file (security/exceptions.json, each entry with advisory id, justification and an expiry date at most 90 days ahead) are enforced in CI (Section 18.14) |
M1 |
| SEC-30 | Secret scanning and license check | A secret scanner over the git history and working tree reports nothing; pnpm check:licenses confirms every production dependency license is in the allow-list (MIT, Apache-2.0, BSD-2-Clause, BSD-3-Clause, ISC, 0BSD, CC0-1.0, CC-BY-4.0 for data, MPL-2.0, LGPL-3.0 only when dynamically linked, AGPL-3.0-only for first-party code) and every first-party package has the license declared in Section 3; the allow-list is the one printed in Section 17.11 |
M1 |
| SEC-31 | Dynamic scan (nightly) | An OWASP ZAP baseline scan runs against the compose stack with a seeded team; zero medium or higher alerts that are not listed in security/zap-ignore.txt with a justification |
M14 |
| SEC-32 | Permission escalation | A viewer or editor cannot raise their own role, cannot create an invitation with a role above their own, an admin cannot change or remove the owner, and the last owner cannot leave; each attempt returns the status of Section 11 and leaves the data unchanged |
M6 |
| SEC-33 | Setup-token-only bootstrap | With zero users, POST /api/app/v1/auth/signup returns 403 setup_required in every REGISTRATION_MODE and creates no user; no request sequence other than POST /api/app/v1/setup with the one-time token creates the first account; a second setup attempt after success is refused |
M6 |
| SEC-34 | Segment-locked share link | A viewer of a segment-limited shared link cannot widen the data: filter-suggestion, realtime, funnel and journey mirrors apply the locked filters; a request that carries a segment filter (even a guessed segment UUID) is rejected with validation_failed; include_imports cannot add rows outside the segment |
M10 |
| SEC-35 | Archive bomb | A zip whose headers lie about sizes and whose entry inflates beyond the per-entry limit of Section 14.4.1, a zip with more than 50 entries, a zip whose total uncompressed size exceeds 2 GB and a zip with an inflate ratio above 100:1 are rejected with 422 while streaming, with no file left on disk; entries with absolute names, .., symlinks, duplicate names or encryption are rejected |
M11 |
| SEC-36 | Upload file names | Upload part names containing ../, absolute paths or separators never reach the file-store path (the stored path is imports/{import_id}/{file_id}.csv, built only from the server-generated import and file identifiers), and parts whose names fail the filename pattern of Section 14.4.1 are rejected before any byte is written |
M11 |
| SEC-37 | Impersonation request scope | During read-only impersonation the dashboard query routes flagged readOnly in the route registry (POST /query and /query/batch) succeed and every other POST, PUT, PATCH and DELETE returns 403 impersonation_read_only; the impersonation session does not appear in the target's session list, cannot be revoked by the target and does not count toward the target's session cap; a removed super admin's impersonation session is rejected on its next request |
M13 |
| SEC-38 | Operator step-up | The step-up re-authentication of Section 15.17.2 requires password and, for accounts with 2FA (all operator accounts), a current TOTP code; granting a license, resetting 2FA, downloading user data, audit export, force logout and suspension each fail without a fresh step-up | M13 |
| SEC-39 | OAuth state binding | A GA4 OAuth callback whose state was issued to another user's session, or that is replayed, is rejected without creating an imports row and the authorization code is discarded |
M11 |
| SEC-40 | Export download after access loss | An export file downloaded by a user who was removed from the team, or whose role no longer allows site.export_csv, after requesting it returns 404 |
M11 |
| SEC-41 | Operator CSV exports | The audit-log export and the code-batch export neutralize cells beginning with =, +, -, @, tab and carriage return, with team names, batch names and e-mail addresses as hostile inputs |
M13 |
18.11 Fixture data generator #
tests/support/generator/ provides a deterministic generator used by integration, E2E and load tests. It is both a library and the CLI pnpm seed:fixtures.
18.11.1 CLI #
| Flag | Default | Meaning |
|---|---|---|
--profile |
basic |
One of empty, basic, ecommerce, agency, capped, legacy-import, all-channels |
--seed |
1 |
32-bit integer seed for the xoshiro128** generator; same flags and seed always produce byte-identical output |
--scale |
small |
small (about 50,000 events), medium-1m, large-50m, agency-200m |
--anchor |
2026-03-15T12:00:00Z |
The instant treated as "now" for all timestamps |
--days |
30 |
Number of days before the anchor to fill (up to 730) |
--timezone |
UTC |
Site timezone for the primary site |
--sites |
profile-dependent | Number of sites |
--clickhouse-url, --postgres-url |
CLICKHOUSE_URL and DATABASE_URL of Section 16.4 |
Targets |
--out |
none | Writes expected.json (oracle answers) to this directory |
--reset |
false | Deletes all rows for the generated site UUIDs before inserting |
Identifiers (team, user, site UUIDs) are derived from the seed so they are stable between runs. Rows are inserted into ClickHouse in batches of 100,000 through the client's native insert path (throughput target at least 500,000 rows/s on the reference ClickHouse node), and Postgres rows are created through repository functions (not raw SQL) so constraints are exercised.
18.11.2 Data model of the generator #
- Visitors and sessions. Visitor
user_idvalues are unique per site per day. Sessions per day follow a weekday-weighted rate (weekdays 1.0, Saturday 0.6, Sunday 0.55) with a slow linear trend of plus or minus 20% across the window and a Poisson-distributed daily noise. Pageviews per session follow a geometric distribution (mean 2.4) with 45% of sessions being single-page bounces. Gaps between events inside a session are log-normal with a median of 25 s and a maximum of 25 minutes; sessions never contain a gap of 30 minutes or more. - Pages. 40 paths per site with a Zipf exponent of 1.1; entry pages and exit pages derived from session order; 3% of pageviews are 404 pages, each also emitting a
404event with proppath. - Sources. A source table with at least one source for every one of the 18 channels (
all-channelsguarantees at least 50 sessions per channel); AI assistants have all 12 hosts listed in Section 18.3.2; UTM campaigns: 12 campaign names across mediums. - Locations. 30 countries with a Zipf distribution, regions and cities for the top 10; no coordinates.
- Devices.
devicevaluesdesktop55%,mobile40%,tablet5%; 8 browsers and 6 operating systems withmajor.minorversions; window widths spread so that all fourscreen_sizeclasses occur (mobile35%,tablet5%,laptop25%,desktop35%), independent of thedeviceclass. - Events. Per session: 12% outbound click (
Outbound Link: Click, propurl), 6% file download (File Download, propurl), 4% form submission (Form: Submission, proppath), 8% custom eventSignupwith propplanin (free,pro,team), 10% custom eventAdd to cartwith propsproductandcategory, and 40% of pageview sessions emit anengagementevent withsdbetween 10 and 100 andebetween 500 ms and 180,000 ms. - Revenue (
ecommerceprofile). 3% of sessions emitPurchasewith revenue; currencies USD (60%), EUR (25%), GBP (10%), JPY (4%, zero decimals), BHD (1%, three decimals); amounts log-normal with a median of 40 in major units, generated with the decimals of the currency (0, 2 or 3) and stored asrevenue_amountandrevenue_currency;revenue_reporting_amountandrevenue_reporting_currencyare computed with a fixed exchange-rate table shipped with the generator (tests/support/generator/fx.json) into the site's reporting currencyUSD, and the oracle uses the same table; propscouponandcountry. - Funnels. A built-in funnel path (
/pricingto/signupto eventSignupto/welcome) with step-to-step continuation probabilities 0.5, 0.4 and 0.8, plus 5% of visitors that skip the second step and 3% that visit steps out of order, so optional and strict modes yield different numbers. - Imported data (
legacy-import). Rows in eachimported_*table for the 60 days before the first native event, with a one-day overlap to test the merge boundary (the overlap day must be served from native data only, because imported rows are used only for dates strictly earlier than the first native day, Section 13.4.10 rule 2). - Bots. Not generated as events. Separate fixtures produce raw HTTP traffic for the drop tests.
- Timezone stress.
--timezoneaccepts the six test timezones; the generator places 8% of events within 30 minutes of local midnight to test bucketing. agencyprofile. One team with 25 sites (generated onsmallscale), 1 owner, 2 admins, 3 editors, 4 viewers and 3 guests, each guest bound to one site, 6 shared links, 4 segments, 10 annotations and 3 e-mail report subscriptions.cappedprofile. Six hosted teams with Tier 1 (solo) licenses (cap 100,000) andusage_monthlyrows withbillable_eventsof 79,999, 80,000, 99,999, 100,000, 109,999 and 110,000 for the current UTC month of the anchor (the statesok,warning,warning,grace,graceandblockedof Section 15.9.2), plus one unlicensed team.emptyprofile. Team and user with no sites.
18.11.3 Oracle #
The same run writes expected.json containing, for the primary site and the fixed range 2026-02-15 to 2026-03-14 in the site timezone: all metrics for every dimension value, daily and monthly time series, goal conversions for the built-in goals (Signup of type event, /welcome of type pageview, scroll 75% of type scroll), funnel step counts in ordered, optional and strict modes, revenue totals per currency and per source, and journeys top 20. The oracle code is in tests/support/generator/oracle.ts, uses only arrays and maps, and is tested by a small hand-computed fixture of 40 events whose expected values are written out literally in the test.
18.12 Migration tests #
Migrations are forward-only: no down-migrations exist. Recovery from a bad release is restore from backup (Section 16). Schema code lives in packages/shared/src/db/schema and the migration files in packages/shared/migrations/postgres and packages/shared/migrations/clickhouse; the tests below read them from there. The Postgres runner is Drizzle's migrator driven by the migrate command of the image; the ClickHouse runner is the in-repo SQL runner with the schema_migrations table (Section 6).
| ID | Test | Assertion | Milestone |
|---|---|---|---|
| MIG-01 | Postgres from empty | Applying all migrations to an empty database succeeds; the resulting schema equals the Drizzle schema definition in packages/shared/src/db/schema (running the Drizzle generator reports no pending changes); all tables listed in Section 5 exist; all foreign keys and indexes in Section 5 exist (checked from pg_catalog); the salts table is UNLOGGED |
M2 |
| MIG-02 | ClickHouse from empty | Applying all migrations succeeds; every table in Section 6 exists with the engine, ORDER BY, PARTITION BY and TTL clause from Section 6 (compared to a snapshot of system.tables columns engine, sorting_key, partition_key, and the TTL expression in create_table_query); schema_migrations lists every file with its checksum |
M2 |
| MIG-03 | Idempotent re-run | Running migrate a second time applies nothing and exits 0 for both stores |
M2 |
| MIG-04 | Concurrent runners | Two migrate processes started at the same instant on an empty database result in each Postgres and ClickHouse migration applied exactly once; neither process fails with a partial schema; the second waits or no-ops according to the locking approach of Sections 5 and 6 |
M2 |
| MIG-05 | Checksum drift | Editing an already-applied ClickHouse migration file makes the runner refuse to start with a clear message naming the file; same for Postgres journal hash mismatch | M2 |
| MIG-06 | Seed data | After migration, license_tiers contains exactly three rows: code solo (price 5,900 minor units, 5 sites, 100,000 events per month), code pro (14,900, 25, 500,000) and code agency (34,900, 100, 2,000,000), all in USD; other seeded rows required by Section 5 exist; seeding is idempotent |
M2 |
| MIG-07 | Upgrade with data | Starting from each fixture dump in tests/fixtures/db/ (one directory per previous release tag, created by the release pipeline, containing a Postgres dump and a ClickHouse backup produced from the basic and ecommerce profiles), apply the current migrations; assert row counts and checksums of every non-derived table are unchanged, queries return the oracle answers, and the application boots and readyz returns 200. When no previous release exists (first release), the test is skipped with an explicit passing placeholder that is removed at the first tag |
M14 |
| MIG-08 | Previous image compatibility | In the release pipeline only: start the previous release's web image against the database migrated by the new release; ingest a pageview and run Q1; both succeed. This enforces expand-only migrations within a minor release (new columns nullable or defaulted, no drops or renames); destructive changes are allowed only in a major release and must be listed in the release notes |
M14 |
| MIG-09 | Backfill migrations | Any ClickHouse migration that populates a materialized view or table on existing data is tested on 1,000,000 rows and completes under 60 s; the test asserts results equal a from-scratch computation | M2 |
| MIG-10 | Rollback documentation test | A test asserts that the restore procedure documented in Section 16 is executable: dump the basic profile, drop the databases, restore, and run the oracle comparison |
M14 |
| MIG-11 | Migration ordering and naming | Postgres and ClickHouse migration files (in the two directories above) are numbered with 4 digits and strictly increasing; names match ^\d{4}_[a-z0-9_]+\.(sql|ts)$; no gaps; no two files share a number |
M2 |
| MIG-12 | Job schema | pg-boss creates its own schema on first start of the worker (pgboss); the migrate command does not touch it; restart of the worker is idempotent |
M2 |
18.13 Stripe webhook and payment tests (tests/stripe/) #
These tests cover the hosted edition only (EDITION=hosted). The payment flow, event handling, idempotency and invoice numbering rules are owned by Section 15; this section fixes the fixtures and assertions.
18.13.1 Fixtures #
tests/fixtures/stripe/ holds one JSON file per scenario, containing a complete event object as the Stripe API delivers it. A helper signs the exact raw bytes with the Stripe test signing algorithm (t=<timestamp>,v1=<hmac>) using a test webhook secret. Each checkout fixture is used together with a pending purchases row created through the checkout-creation route against the Stripe mock, and the fixture's session id is rewritten to that row's stripe_checkout_session_id: the purchase is resolved by session id, and the metadata in the fixtures is advisory only. Only the three event types Section 15.6.5 handles (checkout.session.completed, charge.refunded, charge.dispute.created) change state. Required fixtures, one per tier where noted:
| Fixture | Event type | Content |
|---|---|---|
checkout_completed_tier1.json, _tier2.json, _tier3.json |
checkout.session.completed |
Paid session, payment mode, USD amounts 5,900 / 14,900 / 34,900, metadata with team id, tier and purchase kind new |
checkout_completed_upgrade_1_to_2.json, _2_to_3.json, _1_to_3.json |
checkout.session.completed |
Metadata kind upgrade with amounts 9,000 / 20,000 / 29,000 (the price differences) |
checkout_async_succeeded.json, checkout_async_failed.json |
checkout.session.async_payment_succeeded, checkout.session.async_payment_failed |
Delayed-payment events; delayed methods are disabled (Section 15.4), so these types are unhandled and must be acknowledged and ignored |
checkout_expired.json |
checkout.session.expired |
Abandoned checkout; an unhandled type, acknowledged and ignored |
charge_refunded_full.json, charge_refunded_partial.json |
charge.refunded |
Full refund (amount_refunded equal to the total) revokes the license; a partial refund follows Section 15.6.6 |
charge_dispute_created.json |
charge.dispute.created |
A dispute on a paid purchase |
payment_unpaid.json |
checkout.session.completed |
payment_status of unpaid (no license) |
unhandled_event.json |
any event type not handled by Section 15 | Must be acknowledged and ignored |
checkout_tax.json |
checkout.session.completed |
Totals including tax lines (tax is not part of the tier price; amount_minor, tax_minor and total_minor are stored as Section 15.6.4 step 4 defines; the license is granted for the tier of the pending purchase) |
checkout_unknown_session.json |
checkout.session.completed |
A session id that matches no purchase (Section 15.6.4 step 1: ignored, HTTP 200) |
18.13.2 Test catalog #
| ID | Assertion | Milestone |
|---|---|---|
| STR-01 | A valid signed checkout_completed_tier1 moves the pending purchases row to paid, creates one licenses row with status active for the team (tier solo, source='stripe', credit_minor 5,900), allocates an invoice number and queues the license_activated e-mail; response status 200 |
M13 |
| STR-02 | The same for tiers 2 and 3 (pro, agency); site limit and monthly cap read from license_tiers become 25 / 500,000 and 100 / 2,000,000 |
M13 |
| STR-03 | Missing Stripe-Signature returns 400 and changes nothing; invalid signature returns 400; signature computed over re-serialized JSON (different whitespace than the bytes sent) returns 400 (proves the raw body is verified) |
M13 |
| STR-04 | Timestamp older than the tolerance (300 s) returns 400; timestamp 301 s old fails and 299 s old passes | M13 |
| STR-05 | Replaying the same event id 5 times, sequentially, yields one paid purchase, one license and one invoice number, and 200 each time; 10 parallel deliveries of the same event id also yield exactly one paid purchase and one license | M13 |
| STR-06 | Upgrade fixtures raise the license tier without creating a second active license; the amounts paid equal 9,000 / 20,000 / 29,000 minor units and credit_minor becomes the target tier's price; an upgrade event for a team whose current tier is not the purchase's from_tier takes the automatic duplicate-refund path of Section 15.6.4 step 5 (purchase refunded, no license change, purchase_auto_refunded and operator_alert e-mails) |
M13 |
| STR-07 | checkout.session.completed with payment_status = unpaid grants nothing and leaves the purchase pending; checkout.session.async_payment_succeeded and checkout.session.async_payment_failed return 200 and change nothing (unhandled types) |
M13 |
| STR-08 | checkout.session.expired returns 200 and changes nothing; the pending purchase becomes expired only through the purchase-pending-expire job or through replacement by a newer checkout (Section 15.6.2 step 2) |
M13 |
| STR-09 | A charge.refunded whose amount_refunded reaches the purchase total marks the purchase refunded, allocates a credit note and revokes the license (refunded, purge_after 30 days ahead); the team enters the unlicensed state of Section 15 (ingestion dropped, dashboards readable, unlicensed-purge scheduled). The event is applied regardless of the purchase's age, because the REFUND_WINDOW_DAYS window (default 60) is a rule of the operator console only (422 refund_window_closed without the override), not of the webhook handler |
M13 |
| STR-10 | A partial refund sets the purchase to partially_refunded and issues no credit note; the license is unchanged and never left inconsistent (Section 15.6.6) |
M13 |
| STR-11 | Out-of-order delivery: a refund event arriving before its completed event is handled as Section 15.6.5 defines (a retryable response, then redelivery) and ends in the same final state as ordered delivery | M13 |
| STR-12 | The purchase is resolved by stripe_checkout_session_id, never by metadata: a fixture with wrong or missing metadata still fulfills the pending purchase it belongs to, and checkout_unknown_session returns 200 with no change. A deleted team, or a team that already holds an active license when a new purchase completes, takes the automatic duplicate-refund path of Section 15.6.4 step 5 (200 to Stripe, purchase refunded, no license, no invoice number allocated) |
M13 |
| STR-13 | Unhandled event types return 200 and write nothing | M13 |
| STR-14 | Invoice numbering: 50 parallel successful purchases receive 50 distinct invoice numbers in the format of Section 15.15.1 (<INVOICE_PREFIX>-<year>-<6 digits>, default prefix QS) that are consecutive, with no duplicates and no gaps; the sequence restarts at 000001 in a new UTC year (fake clock across New Year); an auto-refunded duplicate or an expired purchase does not consume a number |
M13 |
| STR-15 | The invoice PDF for a purchase is generated server-side, parses as a PDF, and contains the invoice number, the tier name, the amount, the currency, the seller details from the SELLER_* configuration of Section 16.4 and the bill-to name and e-mail; the PDF route requires billing.manage (owner only): an admin and every other role get 403, and a purchase of another team gets 404; the invoice list route requires billing.view (owner and admin) |
M13 |
| STR-16 | Checkout creation calls the Stripe API (through the mock) with mode=payment, the correct price for the tier, inline price_data for the tier price (no Stripe Price or Product object is referenced), automatic_tax enabled only when STRIPE_TAX_ENABLED of Section 16.4 is true, success and cancel URLs on the public URL, and metadata with team id and tier; a team that already holds an active license cannot create a checkout (409 license_already_active); creating a second checkout replaces the first pending purchase |
M13 |
| STR-17 | License code redemption concurrency: two requests redeeming the same code simultaneously: exactly one succeeds and the loser receives 422 code_invalid (the same response as for a used code, Section 15.8.4); the stacking limit of 3 codes per team is enforced under concurrency |
M13 |
| STR-18 | Usage metering under concurrency: 1,000 parallel ingested events on a team at 99,950 of 100,000 events produce a final usage_monthly count within the tolerance stated in Section 15.9.3 (the metering uses Valkey counters reconciled from ClickHouse by the usage-reconcile job); on a team at 109,900 events with extra_allowance = 0, 1,000 parallel events never push the accepted total beyond hard_at (110,000) plus the in-flight overshoot bound of Section 15.9.3 |
M13 |
| STR-19 | The guarantee of Section 15.6.4 holds: a paid checkout always ends with an active license or an automatic refund, never neither: a signed checkout.session.completed with payment_status = paid for a purchase already expired (replaced by a newer checkout, or expired by purchase-pending-expire) is fulfilled when the team has no active license and auto-refunded when it has one; the stripe-reconcile job does the same for completed sessions whose purchase is pending or expired; creating a second checkout while the first session is already complete fulfills the first instead of expiring it |
M13 |
| STR-20 | Two different codes redeemed in parallel by a team holding 2 units: exactly one succeeds (3 units) and the other receives 409 code_stack_limit_reached; two different codes redeemed in parallel by an unlicensed team create exactly one license holding 2 units, with no 500 response |
M13 |
| STR-21 | A duplicate charge.refunded delivery, and the charge.refunded that Stripe emits for the automatic refund of STR-06 or STR-12, allocate no second credit note, apply no license effect twice and, for a purchase without a license or invoice number, allocate no credit note at all |
M13 |
| STR-22 | Upgrade chains refund in reverse order: after solo to pro to agency, the console refuses to refund an earlier purchase while a later non-refunded upgrade exists on the license (422 refund_upgrade_first); a refund of an earlier purchase made directly in Stripe never silently discards a later paid upgrade, and ends in the state Section 15.13.3 defines with an operator_alert e-mail |
M13 |
| STR-23 | A charge.dispute.created revokes the license (dispute) and drops ingestion without starting the 30-day purge clock: unlicensed-purge skips the license while the dispute is open; reinstating restores service, and recording the dispute as lost starts the 30-day clock |
M13 |
| STR-24 | Deleting and recreating a site, or transferring a busy site to another team, does not lower the current month's usage of the team that accepted the events (usage is reconciled per site by the usage-reconcile job, Section 15.9.4) and does not move earlier events to the new team |
M13 |
18.14 CI pipeline, stages and gates #
18.14.1 Platform and principles #
The pipeline is defined for GitHub Actions in .github/workflows/ci.yml and .github/workflows/nightly.yml. Every stage runs a pnpm script from Section 18.2.3, so the same stages run locally with pnpm ci and port to any other CI system. Branch policy: trunk-based development on main; main is protected; milestone work happens on branches named m<N>-<slug> (Section 20.5) merged by squash after all required stages pass. Release tags and release jobs follow Section 4.12.2. Dependencies and Docker layers are cached by lockfile hash. Wall-clock target for the pull-request pipeline is 25 minutes, achieved by parallel jobs and sharding (integration tests in 4 shards, Playwright in 6 shards).
18.14.2 Stages #
| # | Stage | Commands | Required to merge |
|---|---|---|---|
| 1 | Install | pnpm install --frozen-lockfile |
yes |
| 2 | Static checks | pnpm lint, pnpm typecheck, pnpm check:conventions, pnpm check:licenses, secret scan, pnpm audit --prod gate (SEC-29, SEC-30) |
yes |
| 3 | Unit tests with coverage | pnpm test:unit (all packages, coverage thresholds of Section 18.15) |
yes |
| 4 | Build | pnpm build for every package and app; Docker image build for web, worker and migrate commands (one image); pnpm check:size: tracker budgets (Section 18.7.3) and dashboard bundle budget: initial route JavaScript at most 300 KiB gzip, every lazily loaded route chunk at most 150 KiB gzip, no source maps in the production image |
yes |
| 5 | Integration tests | pnpm test:int in 4 shards, then pnpm test:privacy, pnpm test:security, pnpm test:stripe, pnpm test:migrations |
yes |
| 6 | Ingestion end-to-end | pnpm test:ingestion in Chromium, Firefox, WebKit |
yes |
| 7 | Browser end-to-end | pnpm test:e2e (e2e-chromium, e2e-firefox, e2e-webkit, mobile, and the hosted projects) |
yes |
| 8 | Integrations | pnpm test:wp (PHP matrix per Section 18.7.4), pnpm test:gtm, npm package pack and import test (ESM, CommonJS and type resolution from a clean temporary project that installs the packed tarball) |
yes |
| 9 | Image scan and SBOM | Container scan (SEC-29), software bill of materials produced and attached | yes |
| 10 | Load smoke | pnpm test:load --smoke (LOAD-13) |
yes |
| 11 | Milestone verification | pnpm verify:milestone <current milestone> on branches named m<N>-* |
yes on milestone branches |
Gates summary: no skipped tests (a lint rule fails the build on .skip, pending-test declarations and xit outside the quarantine mechanism of Section 18.2.4 rule 9), no focused tests, no unhandled promise rejections in test output, no console error output in Playwright runs, zero TypeScript errors, zero ESLint errors (warnings not allowed: --max-warnings 0), coverage thresholds met, size budgets met.
18.14.3 Nightly and weekly pipeline #
| Cadence | Jobs |
|---|---|
| Nightly | LOAD-01, LOAD-05, LOAD-08 to LOAD-12 (Section 18.8.4); ZAP baseline (SEC-31); full browser matrix of ING and E2E with 0 retries; mutation testing (Section 18.15.3); the dependency drift canary: in a throwaway job, install the latest stable of every dependency (pnpm update --latest without writing the lockfile to the branch), run stages 2 to 5, and report which packages break and which move to a new major line; the canary never blocks merges and opens a tracking note in the job summary |
| Weekly | LOAD-02 to LOAD-04, LOAD-06, LOAD-07; GeoIP freshness check (the bundled database must be no older than 45 days at release time, since the upstream database is updated monthly) |
Release (tag v*) |
Stages 1 to 10 plus MIG-07, MIG-08, MIG-10; multi-architecture image build (amd64 and arm64) pushed as quietstat/quietstat with the version tag and latest only for non-pre-release tags; npm publish of @quietstat/tracker with provenance; WordPress plugin zip and GTM template attached to the release; release notes assembled from the CHANGELOG.md entry; a check that the latest weekly and nightly load runs are green and no older than 7 days |
18.14.4 Required status and failure handling #
A failing stage blocks merge. Failure artifacts (container logs, Playwright traces, golden diffs, k6 summaries) are uploaded for 14 days. Executors never merge with a failing required stage and never disable a stage to make a pipeline pass.
18.15 Coverage thresholds and mutation testing #
18.15.1 Thresholds #
Coverage uses the V8 provider of Vitest. Thresholds are enforced by configuration in each package's Vitest config, so a drop fails stage 3 or stage 5.
| Scope | Lines | Branches | Functions | Source of measurement |
|---|---|---|---|---|
packages/shared |
95% | 90% | 95% | unit |
packages/query |
95% | 92% | 95% | unit (golden and safety tests) |
packages/tracker |
95% | 90% | 95% | unit plus browser coverage merged |
apps/server overall |
85% | 80% | 85% | unit and integration, merged |
apps/server privacy engine (hash, salt, session, visitor identification modules) |
100% | 100% | 100% | unit and integration, merged |
apps/server channel classifier, bot filter, exclusions, usage-cap state machine, permission helpers, license-code generator |
100% | 95% | 100% | unit |
apps/server auth, Stripe handlers, ingestion pipeline |
95% | 90% | 95% | merged |
apps/worker |
85% | 75% | 85% | merged |
apps/web unit and component |
70% | 60% | 70% | unit; the browser flows cover the rest |
integrations/wordpress |
85% lines | n/a | n/a | PHPUnit with a coverage driver |
integrations/gtm |
every scenario passes and every line of the template JavaScript is executed by at least one scenario | harness |
Per-file floor: no non-generated source file may have below 60% line coverage, except files listed in coverage-exclusions.json (generated code, type-only files, entrypoints that only call main()), each with a one-line reason.
18.15.2 Rules #
Coverage is never raised by tests that call code without asserting results; the mutation run below checks this. Coverage thresholds are never lowered in a pull request except by an explicit entry in DECISIONS.md that names the file and reason.
18.15.3 Mutation testing #
Nightly, Stryker runs on packages/query, the privacy engine modules, the channel classifier, the usage-cap state machine and the permission helpers. The mutation score thresholds are 85% for the privacy engine and usage-cap state machine, 80% for the query engine, classifier and permission helpers. A release requires the latest nightly mutation run to meet them; pull requests are not blocked by it.
18.16 Definition of done #
18.16.1 A change (pull request) is done when #
- It implements only scope that a section of this specification assigns, and each acceptance statement in the owning section has a test (unit, integration or browser) or is covered by a named test identifier.
- Every new route is in the route registry with its auth kind and permission, and the registry-driven tests pass (Section 18.4.2).
- Every new API field is
snake_case, has a Zod schema inpackages/shared, and appears in the response schema tests. - Every new table or column is in a migration, in the schema snapshot of Section 18.9.5, and in
DECISIONS.mdif it holds anything derived from visitors. - Every new user-facing string is in the message catalog (Section 4) and every new page passes the axe scan.
- All required CI stages in Section 18.14.2 are green, coverage thresholds are met, and no test was skipped, focused, weakened or quarantined without a
DECISIONS.mdentry. - Documentation within the repository is updated:
README.mdfor operator-visible behavior,.env.examplefor every new environment variable (names and defaults per Section 16.4),CHANGELOG.mdfor user-visible changes. - No pending-work marker comments remain, and no placeholder values for secrets are committed.
18.16.2 A milestone is done when #
The exit criteria in Section 19 for that milestone pass, pnpm verify:milestone <Mn> exits 0, the branch is merged to main, and the progress record in DECISIONS.md (Section 20.5) lists the command output summary. The project-level definition of done is in Section 20.9.
19. Milestones & Execution Plan #
This section sequences the build into 14 milestones, from an empty repository to launch. Each milestone states its scope (by owning section), dependencies, deliverables and exit criteria. Exit criteria are executable: each is a command or a test identifier from Section 18, and pnpm verify:milestone <Mn> (Section 18.2.3) runs all of them. A milestone is complete only when every exit criterion passes on a clean checkout of the merged branch.
19.1 Rules that apply to every milestone #
- Ownership. The milestone builds exactly what the listed sections define. Anything a section assigns to a later milestone is not built early, except the stubs listed below.
- Test identifiers. Each exit criterion cites test identifiers defined in Section 18 (
ING-,E2E-,PRIV-,SEC-,STR-,MIG-,LOAD-,TRK-, plus the named unit suites). The identifiers are listed intests/milestones.jsonunder the milestone that introduces them. A test introduced in milestone Mn must remain green in every later milestone. - Stubs allowed before their owner milestone. Only these interfaces may be stubbed early; each stub has an interface and a test double, and the real implementation replaces it in the named milestone:
| Stub | Interface | Stub behavior | Replaced in |
|---|---|---|---|
| Ingestion decision | decideIngest(teamId, nowMs) from apps/server/src/licensing/entitlement.ts (Section 15.3.4) |
Always returns 'accept' and records no usage |
M13 |
| Entitlement | getEntitlement(teamId) from the same module |
In selfhosted returns state: 'active' with unlimited sites and event cap; in hosted before M13 returns state: 'active' with the Agency limits (100 sites, 2,000,000 events per month) for every team |
M13 |
| Mailer | Mailer.send(message) |
SMTP via Nodemailer from M6; before M6 a no-op that records messages in memory | M6 |
| API key creation | Repository function createApiKey(teamId, ...) used by test helpers |
Direct repository call; no HTTP route | M6 (routes) |
| Query features of later milestones | Engine rejects the unsupported query shapes with validation_failed and the message <feature> queries are not available |
Rejection | M8, M9, M11 |
- Branch and merge. One branch per milestone (
m<N>-<slug>, Section 20.5), sub-branches allowed for parallel tracks, squash-merged intomainwhen exit criteria pass. - Effort weights are relative sizes, not schedules: S = 1, M = 2, L = 3, XL = 5 units.
- Seam discipline for parallel work. Changes to
packages/shared(schemas, constants, error codes) are merged first in small branches so parallel tracks do not conflict; each track owns its directories exclusively as listed in Section 19.3. - Cases that need a later implementation. A test whose behavior needs the real license and cap implementation is registered in
tests/milestones.jsonunder M13 even when it lives in a suite that otherwise belongs to an earlier milestone. These are the unlicensed-team and cap-at-hard_atdrop cases of the Ingestion pipeline suite (Section 18.4.2) and ofPRIV-01; every other case of those suites stays with its own milestone.
19.2 Milestone overview #
| ID | Name | Builds (owning sections) | Depends on | Track | Size |
|---|---|---|---|---|---|
| M1 | Repository scaffold, CI and foundations | 3, 4 (envelopes, errors, config), 16.4 (config loader), 17 (logging redaction, guardedFetch of 17.4), 18 |
none | Foundation | L |
| M2 | Data schemas and migrations | 5, 6, 4 (shared schemas), 18.11 (generator v1) | M1 | Data | L |
| M3 | Tracking script and npm package core | 7 (tracker), 13 (payload) | M1, M2 (payload schema only) | Tracker | M |
| M4 | Ingestion and privacy engine | 8, 16 (salt-rotate and salt-expire jobs, worker bootstrap), 7 (script delivery) |
M2, M3 | Data | XL |
| M5 | Query engine and Stats API | 13 (query engine, Stats API, rate limits), 9 (metric definitions) | M2, M4 | Data | XL |
| M6 | Accounts, teams, sites and app shell | 11, 4 (app shell, design system, i18n), 12 (auth e-mails), 13 (API keys, Sites API), 9 (sites list, site settings) | M2 to start; M4 and M5 to merge (E2E-05, E2E-15, PRIV-13 need ingestion; E2E-24 needs the Stats API) | App | XL |
| M7 | Dashboard core | 9 | M5, M6 | App | XL |
| M8 | Goals, custom properties and revenue | 10 (goals, properties, revenue) | M7 | Conversions | L |
| M9 | Funnels and user journeys | 10 (funnels, journeys) | M8 | Conversions | L |
| M10 | Sharing, embeds, e-mail reports and notifications | 12 | M7 | Reporting | L |
| M11 | Data import and export | 14, 17 (guardedFetch use) |
M5, M6, M7 (the dashboard export and import controls) | Data portability | L |
| M12 | Integrations | 7 (WordPress, GTM, npm release, proxy and framework guides, snippet UI) | M3, M6 | Integrations | L |
| M13 | Licensing, payments and operator console | 15, 12 (cap e-mails), 16 (hosted topology items) | M4, M6 | Hosted | XL |
| M14 | Hardening, release and launch | 16, 17, 18 (full gates), 20 | all | Release | L |
19.3 Dependency graph, critical path and parallel tracks #
Execution waves (a milestone starts when every milestone in its "Depends on" column has merged, except where Section 19.4.6 says otherwise):
| Wave | Milestones that can run concurrently | Predecessors that must have merged |
|---|---|---|
| 0 | M1 | none |
| 1 | M2 | M1 |
| 2 | M3, M6 (start) | M3 needs M1; M6 needs M2 |
| 3 | M4 | M2, M3 |
| 4 | M5; M6 (merge) | M5 needs M4; M6 merges after M4 and M5 |
| 5 | M7; M12; M13 | M7 needs M5 and M6; M12 needs M3 and M6; M13 needs M4 and M6 |
| 6 | M8; M10; M11 | M8 needs M7; M10 needs M7; M11 needs M5, M6 and M7 |
| 7 | M9 | M8 |
| 8 | M14 | M1 to M13 |
Critical path: M1, M2, M4, M5, M7, M8, M9, M14. Every milestone on it is XL or L, and M4, M5 and M7 are the longest. Work on the critical path must never wait for a non-critical milestone.
Parallel tracks. After M2 merges, up to four tracks proceed concurrently:
| Track | Milestones | Owns directories | Joins when |
|---|---|---|---|
| Data plane | M4, M5, then M11 | apps/server/src/ingestion, apps/server/src/privacy, packages/query, apps/worker/src/jobs/salts*, apps/server/src/stats-api |
M7 needs M5; M11 needs M7 |
| Tracker | M3, then M12 | packages/tracker, integrations/* |
M4 needs the M3 tracker for ING tests; M12 needs M6 |
| Application | M6, then M7, M8, M9, M10 | apps/web, apps/server/src/{auth,teams,sites,dashboard,sharing,reports}, apps/worker/src/jobs/{reports,spikes} |
M7 needs M5 |
| Hosted | M13 (after M4 and M6) | apps/server/src/licensing, apps/server/src/admin, apps/web/src/admin, apps/worker/src/jobs/{usage,licenses} |
M14 |
An AI executor working serially follows this order: M1, M2, M3, M4, M5, M6, M7, M8, M9, M10, M11, M12, M13, M14. That order respects every dependency and keeps the critical path first.
19.4 Milestone specifications #
19.4.1 M1 Repository scaffold, CI and foundations #
- Scope. Monorepo and tooling (Section 3 layout), shared conventions of Section 4 that every later milestone uses, configuration loader (Section 16.4), logging and redaction (Section 17), test infrastructure and CI (Section 18).
- Depends on. Nothing.
- Deliverables.
- pnpm workspace with
apps/server,apps/worker,apps/web,packages/shared,packages/tracker,packages/query,integrations/wordpress,integrations/gtm; strict TypeScript project references; ESLint with all custom rules of Section 18.9.6; Prettier; per-package license files and headers per Section 3. packages/shared:ClockandRng(Section 18.2.4), error-code registry and the error envelope of Section 4.4, success and list envelope helpers, pagination helpers (cursor, limit default 50, maximum 200), thePRODUCT_NAMEconstant (Section 16.4), money and time helpers of Section 4.apps/server:buildApp(config, deps); request-id generation (req_prefix); error handler producing the envelope; route registry (method, path, auth kind, permission);GET /healthz;GET /readyzchecking Postgres, ClickHouse and Valkey; pino logger with the redaction paths of Section 17 and a replaced request serializer that never emits client address or User-Agent; the guarded outbound HTTP clientguardedFetchof Section 17.4 (https only, blocks private, loopback and link-local targets, revalidates after redirects and DNS resolution).apps/worker:buildWorker(config, deps)that starts and stops cleanly (job handlers arrive with their features).- Config loader: parses every variable in the config table of Section 16.4 with Zod, applies defaults, fails fast listing every invalid variable, never prints secret values.
- One Dockerfile producing the image
quietstat/quietstatwith commandsweb,worker,migrateandcli(Section 16.2.1; themigratecommand runs zero migrations at this point and exits 0, andclihas no subcommands yet);docker-compose.dev.ymlanddocker-compose.test.ymlwith Postgres, ClickHouse, Valkey, SMTP sink and Stripe mock. - Test infrastructure of Section 18.4.1 and 18.4.3 (container bootstrap, per-file isolation,
eventually, mail client), the scripts of Section 18.2.3,tests/milestones.json,verify:milestone. - CI workflows of Section 18.14,
.env.examplegenerated from the config table (a test asserts equality),README.md,DECISIONS.md,CHANGELOG.md.
- pnpm workspace with
- Exit criteria.
- On a clean clone:
pnpm install --frozen-lockfile && pnpm ciexits 0. - The Health integration suite (Section 18.4.2) passes, including the 503 cases for each store.
PRIV-10,PRIV-11,SEC-25,SEC-29,SEC-30pass.- Unit tests for the error envelope, pagination cursors,
Clock,Rng, config loader (every variable: valid, invalid, default) andguardedFetch(private and loopback blocked, redirect to private blocked, DNS rebinding fake blocked) pass. docker buildsucceeds anddocker run <image> webanswersGET /healthzwith 200 when the stores are reachable.- A test asserts
.env.examplecontains exactly the variable names of Section 16.4.
- On a clean clone:
19.4.2 M2 Data schemas and migrations #
- Scope. Postgres data model (Section 5), ClickHouse store (Section 6), shared Zod schemas for entities and the ingestion payload, fixture generator version 1.
- Depends on. M1.
- Deliverables.
- Drizzle schema for every table in Section 5 (code in
packages/shared/src/db/schema) with indexes, constraints, foreign-key behavior and seed data (license_tierswith codessolo,pro,agencyand prices 5,900 / 14,900 / 34,900 minor units), thesaltstable createdUNLOGGED, migrations in order (files inpackages/shared/migrations/postgres),migratecommand applying Postgres migrations then ClickHouse migrations with advisory locking. - ClickHouse migration runner with
schema_migrations, and migrations (files inpackages/shared/migrations/clickhouse) for every table, materialized view and TTL in Section 6, including theimported_*tables. - Zod schemas in
packages/sharedfor every entity, request and response object defined in later sections as they become known at this milestone: the ingestion payload (Section 13), the pagination and envelope types, and the query JSON (Section 13); entity schemas for later sections are added in their own milestones. - Generator version 1: profiles
emptyandbasic, scalesmall, ClickHouse inserts, oracle for top stats and breakdowns, determinism test. - Schema snapshot file (
tests/privacy/schema-snapshot.json).
- Drizzle schema for every table in Section 5 (code in
- Exit criteria.
MIG-01toMIG-06,MIG-09,MIG-11,MIG-12,PRIV-03pass; the Zod meta-test of Section 18.3.4 item 1 passes; schema drift tests of Section 18.3.4 item 4 pass; generator run twice with the same seed produces identicalexpected.jsonand identical ClickHouse content checksums; the oracle self-test on the 40-event hand-computed fixture passes.
19.4.3 M3 Tracking script and npm package core #
- Scope. Section 7 tracker behavior and options, build, size budget, npm package core. Not in this milestone: WordPress plugin, GTM template, proxy and framework guides, snippet UI (M12).
- Depends on. M1; the payload schema from M2.
- Deliverables.
packages/trackersource: init options, automatic pageviews, SPA (History API) and hash-routing navigation, outbound link, file download, form submission, 404 tracking, custom events with props and revenue, scroll depth, engagement time, manual pageview mode,transformRequest, localhost handling, proxyendpoint, multi-domaind, logging toggle.- esbuild builds for every variant in Section 7;
size-budget.jsonwith the budgets of Section 7.8.1 (core at most 1,500 bytes gzip, one budget per module, generic script at most 4,800 bytes); thepnpm check:sizescript; forbidden-token scan. - npm package
@quietstat/tracker(ESM, CommonJS, type declarations, API snapshot), MIT license. - A recording fake receiver in
tests/supportthat stores the posted bodies, and a static fixture site (tests/support/fixture-site/). - Tracker conformance tests
TRK-01toTRK-10(Section 18.7.6).
- Exit criteria.
pnpm test:unit --filter @quietstat/trackermeets the coverage thresholds of Section 18.15;TRK-01toTRK-10pass in Chromium, Firefox and WebKit;pnpm check:sizeexits 0 with the core under 1,500 bytes gzip;npm packtarball installs into a clean temporary project and imports under Node (SSR safe), ESM and CommonJS.
19.4.4 M4 Ingestion and privacy engine #
- Scope. Section 8 in full: endpoint, validation, site registry lookup, bot, datacenter and exclusion filters, enrichment (UA, GeoIP, referrer and channel), visitor hash and daily salts, sessions, realtime counters, buffering and batch insert; tracker script delivery from the server (Section 7); the
salt-rotateandsalt-expirejobs and worker bootstrap with pg-boss (Section 16). - Depends on. M2, M3.
- Deliverables.
POST /api/eventwith CORS*, content typestext/plainandapplication/json, status codes 202, 400 (plain text reason), 413 and 429, and the per-IP rate limit of 100 requests per 10 seconds (policyevent_ingest, Section 17.6).- Client IP resolution with
TRUSTED_PROXIESand the per-site proxy key (X-Quietstat-Client-IPhonored only with a validX-Quietstat-Proxy-Key; an invalid key means the header is ignored and the request is rate-limited as a normal client); privacy engine modules (SipHash-2-4, salt cache with 60 s refresh, dual-salt window 00:00 to 00:30 UTC, session lookup in Valkey). - Enrichment: User-Agent parsing with
major.minorversions, GeoIP lookup (bundled DB-IP database andGEOIP_DB_PATHoverride), referrer and channel classification including AI assistants, query-string and fragment handling. - Bot and datacenter filter with Valkey drop counters; IP and page exclusion evaluation from the Postgres tables; domain normalization (leading
www.and port stripped before lookup); theallowed_hostnamescheck with drop reasonhostname_not_allowed; thedecideIngeststub call (Section 19.1); site-registry cache. - ClickHouse writer: in-process buffer, batch insert every
INGEST_FLUSH_INTERVAL_MS(2 s) orINGEST_BATCH_ROWS(5,000 rows), failure-mode behavior of Section 8;sessionsrows; realtime counters in Valkey. apps/workerboot with pg-boss and the dailysalt-rotatejob (00:00 UTC) andsalt-expirejob (00:30 UTC), scheduled per Section 16.6, and thefx-rates-fetch,geoip-updateandlists-refreshjobs (the exchange rates feed the converted reporting amounts ofING-15); thesaltstable isUNLOGGEDand excluded from dumps.- Server route for the tracker script file with the caching headers of Section 7.
- k6 scripts
LOAD-01,LOAD-05,LOAD-08and the compose stack for load tests.
- Exit criteria. Unit suites
H-1toH-7,S-1toS-10,SE-1toSE-6, channel table tests (Section 18.3.2), bot filter, exclusions, geo, URL and UA suites pass with the 100% thresholds of Section 18.15 for the privacy engine;S-11andS-12pass;ING-01toING-27andING-29toING-31pass in all three engines;PRIV-01,PRIV-02,PRIV-04toPRIV-09,PRIV-14,PRIV-15pass, with the hosted-only drop cases ofPRIV-01moved to M13 (Section 19.1 rule 7); the ingestion integration suite (excluding its hosted-only unlicensed-team and cap drop cases, which move to M13) and the buffer-failure suite (Section 18.4.2) pass;LOAD-01meets its criteria on the reference environment (5,000 events/s on onewebinstance),LOAD-05andLOAD-08pass.
19.4.5 M5 Query engine and Stats API #
- Scope. The query engine (
packages/query) and the public Stats API (POST /api/v2/query) with API-key authentication and rate limits (Section 13); metric definitions of Section 9. - Depends on. M2, M4.
- Deliverables.
- Engine: query validation (Zod), metrics, dimensions, filters with all operators, time ranges and comparison, ordering and pagination, site timezone bucketing, realtime shape, server-side ClickHouse limits (
Q-SAFE-5); unsupported later shapes rejected per Section 19.1 stubs. - Stats API route, bearer authentication against
api_keys(key hash lookup), rate limit headers and429. - Golden corpus for the shapes available now, orphan check,
UPDATE_GOLDENmechanism; safety suiteQ-SAFE-1toQ-SAFE-5. - Oracle extended to all metrics and dimensions; generator profiles
all-channelsand scalemedium-1m. - Dashboard-independent response schemas in
packages/shared. - k6 scripts
LOAD-09,LOAD-10andLOAD-12for the query shapes that exist (Q1 to Q8 and Q13).
- Engine: query validation (Zod), metrics, dimensions, filters with all operators, time ranges and comparison, ordering and pagination, site timezone bucketing, realtime shape, server-side ClickHouse limits (
- Exit criteria. Golden tests and
Q-SAFE-*pass with coverage per Section 18.15; the oracle differential suite passes for every metric-by-dimension combination on thebasicandall-channelsprofiles; the Stats API integration suite passes;SEC-14andSEC-24pass;LOAD-09andLOAD-10meet the Q1 to Q8 and Q13 targets of Section 18.8.3 andLOAD-12meets its criterion.
19.4.6 M6 Accounts, teams, sites and app shell #
- Scope. Section 11 in full; the application shell, navigation, design system foundations, i18n and accessibility baseline of Section 4; site management and site settings (Sections 9 and 5); API key management and the Sites API (Section 13); auth e-mails (templates per Section 12); the first-run setup flow for self-hosted (Section 16.3.5); the user, team and site lifecycle jobs of Section 16.6 (
session-cleanup,invitation-expire,unverified-user-cleanup,user-anonymize,team-purge,site-purge,site-orphan-reconcile). This milestone may be split into a backend branch (m6-accounts-api) and a web branch (m6-app-shell) that proceed in parallel after the shared schemas are merged. - Depends on. M2 to start. The work runs in parallel with M3 to M5, but the milestone cannot be merged before M4 and M5 have merged:
E2E-05,E2E-15andPRIV-13need working ingestion, andE2E-24callsPOST /api/v2/query, which M5 delivers. - Deliverables.
- Signup, e-mail verification, login, logout, session management, password reset, change e-mail, TOTP 2FA with recovery codes, account deletion (soft delete, 30-day anonymization job), CSRF protection, brute-force limits (Sections 11 and 17).
- Teams, memberships, invitations (create, accept, expire, revoke), the six roles and the permission matrix as a typed table,
can(role, action)helper, guest access per site, ownership transfer, team deletion with 7-day undo, personal team on signup. - Sites: create, list, update, delete (asynchronous
site-purgejob), site settings (domain with normalization, timezone, datacenter filter toggle,allowed_hostnames, tracker toggles, IP and page exclusions, filtered-traffic stat), site-limit hook throughgetEntitlement(Section 19.1). - API keys (create once-shown secret, list, revoke) and Sites API
/api/v1/siteswith cursor pagination. apps/web: Vite and React scaffold, router, TanStack Query client, design system primitives, app shell (runtime configuration fromGET /api/app/v1/bootstrap, the external/assets/theme-init.jstheme bootstrap, no inline scripts), navigation, theme switch, i18n catalog, auth screens, account settings, team and site settings screens, onboarding with basic snippet display and "waiting for first pageview" state, empty states.- Mailer with SMTP and the auth e-mail templates; audit-log writer; the jobs for token cleanup, user anonymization, team purge and site data purge.
- Self-hosted first run: there is no first-signup bypass. While no user exists, signup returns
403 setup_requiredand the first account is created only throughPOST /api/app/v1/setupwith the one-time setup token (Section 16.3.5).
- Exit criteria.
E2E-01toE2E-06,E2E-15,E2E-21toE2E-24,E2E-39pass;SEC-01toSEC-13,SEC-17,SEC-19,SEC-23,SEC-32,SEC-33pass;PRIV-13passes; the permission-matrix unit suite, the route-registry completeness suite and the Auth, Teams, Sites and Settings integration suites (Section 18.4.2) pass; password and 2FA unit suites pass.
19.4.7 M7 Dashboard core #
- Scope. Section 9: single-page dashboard, all reports, filters, segments, annotations, realtime view, comparison, consolidated view, and dashboard API routes under
/api/app/v1that call the query engine. - Depends on. M5, M6.
- Deliverables.
- Top-stats tiles, time-series chart with accessible summary, interval selection, time-range picker with every range, custom range, Last 24 Hours and keyboard shortcuts, period comparison.
- Reports: sources, channels (all 18), campaigns (UTM), pages (path and full URL), entry pages, exit pages, countries, regions, cities, browsers, operating systems, devices (
visit:device), screen sizes (visit:screen_size, a separate report), AI assistant traffic. - Filters (combinable, URL-persisted) and saved segments (personal and site-wide), annotations, realtime view, consolidated all-sites view and sites list sorting, footer attribution for the bundled GeoIP database, responsive layout.
- Golden-corpus additions for segment-expanded and consolidated queries; generator scale additions needed for UI tests.
- Exit criteria.
E2E-07toE2E-14,E2E-40,E2E-41pass (browser projects as listed in Section 18.6.2);SEC-15andSEC-16pass; web unit and component suites meet thresholds; the dashboard bundle budget of Section 18.14.2 passes; the Realtime and Query-execution integration suites pass.
19.4.8 M8 Goals, custom properties and revenue #
- Scope. Section 10 for goals (custom event, pageview, scroll depth), property narrowing (up to 3), custom property allow-list, revenue values and currency, ecommerce reports.
- Depends on. M7.
- Deliverables. Goal CRUD, goal matching in the query engine, conversion metrics (
conversion_rate,group_conversion_rate) in reports, property allow-list and breakdown, revenue goals,total_revenue,average_revenue, revenue by source, multi-currency formatting, generator profileecommerce, golden additions for goal and revenue queries, oracle additions, k6 queries Q9 and Q12 inLOAD-09andLOAD-10. - Exit criteria.
E2E-16toE2E-18pass; the Goals and revenue integration suite passes; golden and oracle suites including goals and revenue pass; money unit suite passes;LOAD-09andLOAD-10meet Q9 and Q12 targets.
19.4.9 M9 Funnels and user journeys #
- Scope. Section 10 for funnels (ordered, optional-step, strict-order; steps from pages, events or goals; period comparison; revenue per step) and the user journeys visual map.
- Depends on. M8.
- Deliverables. Funnel and funnel-step CRUD, funnel query shape in the engine, comparison, revenue per step, journeys query and Sankey visualization with
d3-sankey, generator funnel data, golden and oracle additions, k6 queries Q10 and Q11. - Exit criteria.
E2E-19andE2E-20pass; the Funnels and journeys integration suite passes (all mandatory sequences of Section 18.4.2); golden and oracle suites for funnels and journeys pass;LOAD-09andLOAD-10meet Q10 and Q11 targets, so the full Q1 to Q14 set passes.
19.4.10 M10 Sharing, embeds, e-mail reports and notifications #
- Scope. Section 12 in full, including the complete e-mail template inventory.
- Depends on. M7.
- Deliverables. Shared links (optional password, fixed-segment limit, revoke),
/api/share/v1routes, embeddable dashboards with the frame policy of Section 12, scheduled e-mail reports (weekly, monthly, per Section 12) and the worker jobs, traffic spike detection and notification job with cooldown, unsubscribe links, template inventory with HTML and plain-text parts, subscriptions UI. - Exit criteria.
E2E-25toE2E-28pass; the Sharing and Reports and notifications integration suites pass;SEC-34passes; the e-mail template unit suite covers every template whose owning section is 11 or 12 (M11 adds the Section 14 templates and M13 adds the Section 15 templates to the same suite); the Jobs integration suite passes for the jobs introduced here.
19.4.11 M11 Data import and export #
- Scope. Section 14: CSV export, CSV import (Plausible-compatible format), GA4 import; imported data merged into queries.
- Depends on. M5, M6, M7: the export and import controls and the imported-data indicators that
E2E-29toE2E-31exercise are part of the dashboard built in M7. - Deliverables. Export routes and jobs (
export-run,export-expirewith the 7-day file lifetime of Section 14.3.1) for every report, CSV writer with formula neutralization; CSV importer with validation and staging; GA4 OAuth flow and report fetching viaguardedFetch; imports and exports tables workflow with progress, cancel, delete; import-aware query engine (include.imports; imported rows only before the first native day); the jobsimport-run,ga4-import-run,import-purgeandimport-sweep; import UI; thelegacy-importgenerator profile; golden and oracle additions for imported data. - Exit criteria.
E2E-29toE2E-31pass;SEC-18,SEC-20toSEC-22,SEC-35,SEC-36,SEC-39,SEC-40pass;PRIV-12passes; the Imports and exports integration suite passes against the Google fake; CSV and GA4 mapping unit suites pass.
19.4.12 M12 Integrations #
- Scope. Section 7 remainder: WordPress plugin, Google Tag Manager template, npm package release readiness, proxy endpoint guides, setup guides for common frameworks inside the app, snippet UI.
- Depends on. M3, M6.
- Deliverables.
integrations/wordpress(GPL-2.0-or-later) with settings page, exclusions and packaging;integrations/gtmtemplate (Apache-2.0) with test scenarios and harness; snippet panel with script-tag, npm and proxy variants; in-app setup guides for each framework or platform listed in Section 7;ING-28. - Exit criteria.
E2E-32andE2E-33andING-28pass;pnpm test:wppasses on the PHP and WordPress matrix with Plugin Check clean;pnpm test:gtmpasses; the npm tarball test passes; the release zip manifest test passes.
19.4.13 M13 Licensing, payments and operator console #
- Scope. Section 15 in full (hosted edition only): tiers, Stripe Checkout, webhooks, license codes and batches, stacking, upgrades, refunds, usage metering and caps, banners, cap e-mails, unlicensed state, invoices, operator console.
- Depends on. M4, M6.
- Deliverables. Real
getEntitlementanddecideIngestimplementations replacing the stubs of Section 19.1; Stripe client factory, checkout creation, webhook route with raw-body verification and idempotency;purchases,licenses,license_codes,license_code_batches,usage_monthlyworkflows; usage counters (Valkey) and the jobsusage-reconcile,usage-cap-notifyandusage-month-close; 80% and 100% e-mails; the drop athard_at; the jobsaudit-prune,purchase-pending-expire,stripe-reconcile,unlicensed-purge-warnandunlicensed-purge; invoice numbering and PDF generation;/adminconsole UI and API (users, teams, licenses, code batches with CSV export, purchases and refunds, usage, read-only impersonation); site-limit enforcement (site_limit_reached); the edition switch behavior of Section 18.4.2. - Exit criteria.
STR-01toSTR-24pass;E2E-34toE2E-38pass in the hosted projects;SEC-26toSEC-28,SEC-37,SEC-38,SEC-41pass; the hosted-only drop cases of the Ingestion pipeline suite and ofPRIV-01(unlicensed team, cap athard_at) pass; the Licensing and caps, Operator console and Edition switch integration suites pass; the usage-cap state machine and license-code unit suites pass; withEDITION=selfhosted, a test asserts no billing UI, no license routes and no cap checks (Section 18.4.2).
19.4.14 M14 Hardening, release and launch #
- Scope. Operations and release (Section 16: backups, observability, upgrades, hosted topology, compose for self-hosting), security review (Section 17), performance verification, full test matrix, documentation, release pipeline.
- Depends on. M1 to M13.
- Deliverables. Production
docker-compose.ymlfor self-hosting and example hosted topology files; backup and restore scripts and their documentation (the backup excludes the data ofsalts); metrics and log configuration of Section 16; the full release pipeline of Section 18.14.3; performance tuning to meet all load targets; remaining weekly and nightly jobs;security/exceptions.json; operator and user documentation inside the repository (README.md,docs/self-hosting.md,docs/hosted-operations.md,docs/upgrading.md); the smoke scripts below. - Exit criteria.
pnpm ciand the nightly and weekly pipelines are green on the release candidate commit: all identifiers intests/milestones.jsonpass, coverage thresholds (Section 18.15.1) and mutation thresholds (Section 18.15.3) are met.- All load criteria
LOAD-01toLOAD-13pass on the reference environment, including 5,000 events/s on onewebinstance and every dashboard query p95 target of Section 18.8.3. SEC-31(ZAP baseline) passes;MIG-07,MIG-08,MIG-10pass (the first release uses the placeholder rule ofMIG-07);PRIV-16andPRIV-17pass.REL-01(self-hosted smoke): from an empty directory containing only the release compose file and.env,docker compose up -dbrings all services healthy;GET /readyzreturns 200 within 120 seconds; creating the first account, creating a site, sending one event withcurland querying visitors through the Stats API returns 1 visitor.REL-02(hosted smoke): withEDITION=hosted, the Stripe mock and SMTP sink, an end-to-end script signs up, buys Tier 1 through the fake checkout, ingests an event and reads it back.REL-03(documentation integrity):.env.example, the configuration table of Section 16.4 anddocs/self-hosting.mdlist the same variable names; every environment variable that code underapps/andpackages/reads is in the table (a static scan of the source).REL-04(licensing and attribution): every first-party package has the license of Section 3; the production image containsLICENSEand a third-party notices file including the DB-IP attribution text; the dashboard footer shows the attribution.REL-05(job schedule): the Jobs integration suite proves every job in the table of Section 16.6 exists, runs and appears with its documented schedule.- The launch checklist in Section 20.10 is complete, every item with evidence recorded.
19.5 Risks and mitigations during execution #
| Risk | Where | Mitigation |
|---|---|---|
| ClickHouse SQL for funnels and journeys is slow | M9 | Build the engine's funnel query using the approach in Section 10; verify Q10 and Q11 on D2 before UI work; if a target is missed, add a projection or materialized view through a new migration, and record it in DECISIONS.md |
| Daily salt rotation edge cases cause visitor discontinuity | M4 | The S-series tests with the fake clock are written before the implementation (Section 20.5) |
| Dependency drift breaks the build | any | Procedure in Section 20.7; nightly drift canary (Section 18.14.3) |
| Flaky browser tests | M7 onward | Isolation rules of Section 18.2.4; poll with eventually; never use fixed sleeps |
Parallel tracks conflict on packages/shared |
M3 to M6 | Seam discipline (Section 19.1 item 6) |
| Stripe behavior differs from the mock | M13 | Fixtures follow Stripe's documented event shapes; the launch checklist includes a live-mode purchase and refund before announcing (Section 20.10) |
| Load targets missed on the reference environment | M4, M5, M14 | Profile first (event-loop lag, ClickHouse query log), then fix; targets are never edited, a deviation requires a DECISIONS.md entry and owner confirmation noted for the launch checklist |
20. Executor Instructions #
This section tells an AI coding agent how to build the product from this document alone, without asking questions. It is binding for every executor, including executors that work on a single milestone.
20.1 Operating model #
You are building the whole product described in this document in a new repository. You work milestone by milestone (Section 19), prove each milestone with the tests of Section 18, and record every judgment call in DECISIONS.md. You never wait for a human answer: the document contains a decision for every question, and where it is silent, Section 20.3 rule 4 tells you how to decide. The product name is the constant PRODUCT_NAME (Section 16.4); the working name is Quietstat and appears as a literal only in package scopes, image names and file names that Section 3 fixes.
20.2 Reading order and precedence #
20.2.1 Reading order #
Read in this order before writing code. Re-read the owning section of whatever you are about to build.
- Section 1 (decisions, defaults, owner customization questions) and Section 2 (scope, out of scope, glossary): the contract.
- Section 3 (stack, versions, repository layout, flows) and Section 4 (conventions: names, IDs, envelopes, errors, pagination, time, money).
- Section 16.4 (configuration table) and Section 18 (how everything will be tested).
- Sections 5 and 6 (data model), then Section 8 (ingestion and privacy), because they govern the data plane.
- Section 19 (your milestone plan). From here, for each milestone read only the sections it lists in "Builds", plus Sections 4, 11 and 17 whenever you add a route.
- Section 9 before any dashboard work; Sections 10 to 15 as their milestones come up; Section 17 before the first route, and again at M14.
20.2.2 Which section wins #
Each concern has exactly one owning section. When two sections appear to disagree, the owning section wins and you record the conflict in DECISIONS.md.
| Concern | Owner |
|---|---|
| Dependency version lines, repository layout, process model | Section 3 |
| Naming, IDs, response and error envelopes, pagination, time and money rules, UI design system, accessibility, app shell | Section 4 |
| Postgres tables, columns, constraints, seed data | Section 5 |
| ClickHouse tables, engines, ordering, TTL, imported tables | Section 6 |
| Tracker behavior, options, size budget, WordPress, GTM, npm, proxy and framework guides | Section 7 |
| Ingestion processing, visitor hash and salts, sessions, enrichment, channels, bot and exclusion filters | Section 8 |
| Metric definitions, reports, filters, segments, annotations, realtime | Section 9 |
| Goals, properties, funnels, journeys, revenue | Section 10 |
| Authentication, teams, roles, permission matrix | Section 11 |
| Sharing, embeds, e-mail reports, spike notifications, e-mail template inventory | Section 12 |
| Stats API, Events API, Sites API, API keys, query JSON and the query engine | Section 13 |
| Import and export | Section 14 |
| Error code catalogue | Section 4.4.3 |
| Row-count service limits | Section 5.16 |
| Environment variables | Section 16.4 |
| Background jobs (names, schedules, retries) | Section 16.6 |
| E-mail template inventory | Section 12.8 |
| Licensing, payments, caps, invoices, operator console | Section 15 |
| Editions, deployment, environment variables, jobs schedule, backups, observability | Section 16 |
| Security and privacy controls, compliance, and all rate limits (the table in 17.6) | Section 17 |
| Tests, CI, coverage, test identifiers | Section 18 |
| Milestone order and exit criteria | Section 19 |
Within a section, a normative table or numbered rule wins over an illustrative example. If a number appears in two places in the same section, the table wins.
20.3 Ground rules #
- Canonical sections win. Follow the owning section exactly: names of tables, columns, routes, environment variables, roles, tiers, limits, error codes and test identifiers are used verbatim.
- Never invent scope. Build exactly what the sections specify. Do not add features, endpoints, settings, tables, analytics of your own, telemetry, update checks or third-party calls that the document does not define. The excluded items stay excluded: recurring subscriptions and any monthly or annual billing, free trials, Google Search Console, Looker Studio, Slack reports, SSO, mobile SDKs, feature gating between tiers, marketing site and documentation-site copy, nonprofit or education discounts, session replay, heatmaps, A/B testing, data-residency settings. If a library offers an extra capability, do not expose it.
- Clean-room. Plausible Analytics is the product being matched in features, nothing else. Do not copy its source code, assets, names, wording or trademarks. Write original code and original UI text.
- Ask-never policy. You never ask a human a question and never stop for an answer. When the document is silent or ambiguous, decide in this order and stop at the first rule that yields an answer:
- the owning section;
- the defaults in Section 1;
- the conventions in Section 4;
- the option that adds the least scope and preserves the privacy rules (no cookies, no raw IP or User-Agent stored, no persistent identifiers) and the feature-parity target.
Record every decision made under rule 3 or 4 in
DECISIONS.md(format in 20.3.1). Never leave a decision open, a pending-work marker comment, or a placeholder value.
- Tests are the contract. Do not delete, skip, weaken or quarantine a test to make a milestone pass. If a test contradicts this document, fix the test and record why (Section 18.1 item 7).
- Privacy invariants are never traded off. If a change makes
PRIV-*tests fail, the change is wrong. - Secrets. Never commit secrets, never print them, never place real credentials in tests, fixtures, logs or
.env.example. Generate development secrets locally withopenssl rand -hex 32and keep them in an untracked.env. - Dependencies. Use only the libraries named in Section 3. Adding any other runtime dependency requires a
DECISIONS.mdentry that states the reason, the license (it must be in the allow-list ofSEC-30) and why an existing dependency or about 50 lines of code do not suffice. Development-only tooling (test helpers, linters) needs no entry if its license is in the allow-list. - Licenses and headers. Core application code is AGPL-3.0-only; the tracker script and npm package MIT; the WordPress plugin GPL-2.0-or-later; the GTM template Apache-2.0 (Section 3). Every first-party package carries its license file and the header check of
pnpm check:licensespasses. - Commits. Small, focused commits with Conventional Commit messages (
feat(query): ...,test(privacy): ...,fix(ingest): ...). Never commit generated build output,.env, or large fixtures above 5 MiB (the dump fixtures ofMIG-07are generated by the release pipeline and stored outside git history through the release assets or by the artifact mechanism of the CI platform). - Idempotent, forward-only data changes. Never edit an applied migration; add a new one (
MIG-05). - Do not modify the specification. If you believe the document is wrong, implement the owning section as written unless it is impossible, record the concern as a
DECISIONS.mdentry of typeconcern, and continue.
20.3.1 DECISIONS.md #
DECISIONS.md lives at the repository root. It has three parts: a header with the product constant values and the owner-customization answers of Section 1 (the Section 1 defaults, unless the owner has replaced them in Section 1 before the build), the decision log, and the milestone log.
Decision entry template (append only; never rewrite old entries; correct by adding a new entry that supersedes):
### D-0001 <short title>
- Type: decision | deviation | concern | dependency | quarantine | size-increase | coverage-change
- Milestone: M<N>
- Sections: <numbers of the sections involved>
- Context: <what the document leaves open or what blocked you, in 1 to 3 sentences>
- Options considered: <list>
- Decision: <what you did>
- Consequence: <what it affects; tests added or changed>Milestone log entry (added when a milestone merges):
### M<N> <name> - merged <date, UTC>
- Branch: m<N>-<slug>; merge commit: <hash>
- `pnpm verify:milestone M<N>`: pass; identifiers run: <count>
- Notable decisions: D-xxxx, D-yyyy
- Open items carried forward: none | <list, each with the milestone that will close it>A milestone may not be logged as merged while "Open items carried forward" lists anything that its own exit criteria cover.
20.4 Environment setup #
Perform these steps once, at the start, in this order. Each step has a check; do not proceed until it passes.
| # | Step | Command or action | Check |
|---|---|---|---|
| 1 | Operating system and resources | A Linux or macOS machine with at least 4 CPU cores, 16 GiB of memory and 30 GiB of free disk (ClickHouse and the load tests need it) | nproc, free -g or the platform equivalent |
| 2 | Git | Install Git; create an empty directory for the repository; git init -b main |
git status works |
| 3 | Node.js | Install the Node.js release line named in Section 3 (an LTS line) from the official distribution or a version manager | node --version prints the Section 3 major |
| 4 | pnpm | corepack enable then corepack prepare pnpm@latest --activate; confirm the major line named in Section 3 (apply Section 20.7 if the major differs) |
pnpm --version |
| 5 | Docker | Docker Engine with the Compose plugin (v2 command docker compose), able to run containers without root prompts |
docker compose version, docker run --rm hello-world |
| 6 | Browsers for tests | After M1 creates the workspace: pnpm exec playwright install --with-deps chromium firefox webkit |
pnpm exec playwright --version |
| 7 | k6 | Install the k6 binary from its official release (needed from M4; the smoke subset runs from M1 on in CI) | k6 version |
| 8 | PHP and Composer | Needed from M12 only: the lowest and the newest stable PHP versions named in Section 7 (a container image is acceptable), Composer, and Docker for wp-env |
php --version, composer --version |
| 9 | Repository bootstrap | Execute milestone M1 (Section 19.4.1) | pnpm install --frozen-lockfile && pnpm ci exits 0 |
| 10 | Local services | cp .env.example .env; replace every value Section 16.4 marks as a secret with generated values (openssl rand -hex 32); docker compose -f docker-compose.dev.yml up -d |
docker compose -f docker-compose.dev.yml ps shows all services healthy |
| 11 | Migrate | pnpm migrate (runs the image's migrate command logic locally: Postgres then ClickHouse migrations) |
exits 0; running it again applies nothing |
| 12 | Run the stack | pnpm dev starts server, worker and web dev server concurrently |
curl -fsS http://localhost:<server port from Section 16.4>/readyz returns 200 |
| 13 | GeoIP data | Obtain the bundled DB-IP database by the procedure defined in Section 8 and place it where the configuration of Section 16.4 expects it. Tests use the synthetic database of Section 18.4.3 and never need the real file | server start logs that the geo database loaded (without logging any visitor data) |
| 14 | DECISIONS.md | Create DECISIONS.md with the header and the empty decision and milestone logs (Section 20.3.1) |
file exists, committed |
Network access is needed only for installing dependencies, pulling container images and downloading the browsers and the GeoIP database. After setup, all tests run offline (Section 18.2.4 rule 8).
20.5 Per-milestone workflow #
Repeat this loop for each milestone in the order of Section 19.3 (serial order M1 to M14).
- Load context. Read the milestone entry in Section 19.4 and every section in its "Builds" column. List the test identifiers it must turn green (Section 19.4 exit criteria and
tests/milestones.json). - Branch. From an up-to-date
main:git switch -c m<N>-<slug>with the slug from the milestone name in lowercase hyphenated words (for examplem4-ingestion-privacy). Parallel sub-branches are namedm<N>-<slug>-<part>. - Register tests. Add the milestone's test identifiers to
tests/milestones.jsonfirst (empty tests that fail with "not implemented" are allowed only for the first commit of the branch). From this pointpnpm verify:milestone M<N>shows a red list that you will drive to green. - Tests first where the behavior is exactly specified. Write the tests before the implementation for: hash, salt and session behavior (
H-,S-,SE-); channel classifier tables; Zod schema tables; permission matrix; usage-cap state machine; license-code generator; CSV writer and parser; money; migrations (MIG-); Stripe fixtures (STR-); privacy tests (PRIV-); the query engine case files (write thecases/*.json, implement, then generate the golden files and read each generated SQL file line by line to verify site scoping, parameterization and semantics before accepting it; never accept goldens unread). For screens and flows, implement first, then write theE2E-specs against the working UI, then harden them. - Implement in vertical slices. One slice is one route, job or screen from storage to UI with its tests. After each slice run
pnpm lint && pnpm typecheck && pnpm test:unit, and the integration project filtered to the files you touched. - Commit often. Commit each green slice. Update
.env.example,README.mdandCHANGELOG.mdin the same commit that changes behavior (Section 18.16.1 item 7). - Record decisions as you make them (Section 20.3.1), not afterwards.
- Full local verification. Before opening the merge:
pnpm ciandpnpm verify:milestone M<N>. Both must exit 0 with no skipped tests. - Merge. Squash-merge into
mainwith the messagefeat(m<N>): <milestone name>; wait for the CI pipeline of Section 18.14 onmainto be green; if it is red, fix forward immediately before starting anything else. - Log. Add the milestone log entry to
DECISIONS.md(Section 20.3.1) and commit it directly tomainwithdocs: log milestone M<N>. Tagm<N>-done.
When a test fails. Read the failure; check the owning section for the expected behavior; fix the product code. After three failed attempts with three different hypotheses, re-read the entire owning section and the related sections in Section 20.2.2, then continue. Do not retry unchanged code, do not add sleeps, and do not loosen assertions. If after reading the sections the test itself is wrong (for example a typo in a literal), correct the test and record a deviation entry.
When the machine cannot run something (no Docker, no browser): fix the environment per Section 20.4; if truly impossible, record a concern entry, run everything else, and mark the milestone log entry as "verified partially" listing the identifiers not run. A milestone with unrun identifiers is not complete, so this is a last resort that blocks the final launch checklist until the identifiers have been run somewhere.
20.6 Verifying a milestone #
- Run
pnpm verify:milestone M<N>. It runs all identifiers of milestonesM1toM<N>(earlier tests must stay green), the lint, type, size and convention checks, and prints a table of identifier, status and duration. Expected result: every rowpass, exit code 0. - Confirm coverage thresholds (Section 18.15) in the unit and integration output for the packages you touched.
- Manual evidence for UI milestones (M6 to M13): start the stack (
pnpm devwith thebasicorecommerceprofile loaded throughpnpm seed:fixtures --profile basic --out artifacts/seed), open the pages you built in a browser through Playwright, and save screenshots of each main screen in light and dark themes and at 390 px width intoartifacts/m<N>/(an untracked directory). Check that each screen shows its empty state with--profile empty, its loading state (throttled network) and its error state (stop the server). - For data milestones (M4, M5), run the load scripts named in the exit criteria against the compose stack of
tests/load/and store the k6 summary files inartifacts/m<N>/. - The milestone is verified only when the exit criteria listed in Section 19.4 are all true, not when most are.
20.7 Dependency version drift #
Section 3 states a known-good floor of major lines. At build time, for every dependency:
- Install the current stable release:
pnpm add <package>@latest(or the ecosystem equivalent: container image tag of the same major line, Composer for PHP development tools). - Read the resolved version. If its major line equals the line in Section 3, keep it. The lockfile records the exact version; do not copy exact versions into this document or into prose.
- If the resolved major line is newer than Section 3's line, do not adopt it silently. Install the newest release inside the Section 3 line instead (
pnpm add <package>@<major>), and add adependencyentry toDECISIONS.mdnaming the package, the line in use and the newer line that exists. Adopt the newer major only in one case: a security advisory affecting the product has no fix inside the older line. Then adopt it, migrate the code, runpnpm ci, and record the decision. - For packages whose Section 3 line is below 1.0 (Drizzle ORM, drizzle-kit, iovalkey, argon2, d3-sankey, esbuild), the minor version counts as the major line: declare the dependency with a tilde range on that minor and apply rules 2 and 3 to the minor.
- If Section 3's line is newer than anything the registry offers (the registry has not published it), use the newest stable release and record a
dependencyentry. - If a Section 3 package has been removed, renamed, or deprecated without a successor line, substitute the closest maintained package with the same license class, keep the call sites behind a thin adapter module so later replacement is local, and record the substitution.
- If a newer release inside the same major line fails the test suite, pin the previous working version in the lockfile with
pnpm.overridesor an exact specifier and record why. - Container images (PostgreSQL, ClickHouse, Valkey) use the major line tags of Section 3; the ClickHouse tag is the long-term-support line named there. The Node.js base image of the Dockerfile uses the Node.js line of Section 3.
- Always commit the lockfile. CI uses
--frozen-lockfile. The nightly drift canary (Section 18.14.3) reports which dependencies have moved to a new major line; treat the report as information, not a task.
20.8 External services, credentials and fakes #
You never receive production credentials and never ask for them. Build and verify against the fakes of Section 18.4.3. The owner supplies the real values at launch; list each of them in the final report under "Owner actions".
| External dependency | Used for | What you use while building | What the owner supplies at launch |
|---|---|---|---|
| Stripe (hosted edition) | Checkout, webhooks, refunds, receipts | Stripe mock container, signed fixtures | Live keys, webhook signing secret, the Stripe Tax decision (STRIPE_TAX_ENABLED), and registration of the webhook endpoint with the three event types Section 15.6.5 handles. No Stripe products or prices are created, because checkout uses inline pricing (Section 15.4) |
| SMTP provider | All e-mail | SMTP sink container | SMTP host, credentials, sender address, SPF, DKIM, DMARC records |
| Google OAuth and GA4 APIs | GA4 import | In-repo fake Google server | A Google Cloud OAuth client with the Analytics read scope enabled, consent screen configuration and any verification Google requires, redirect URI |
| GeoIP database | Location lookup | Synthetic MMDB for tests; real DB-IP Lite file for manual runs | Nothing for the bundled database; an optional commercial MaxMind-format file through GEOIP_DB_PATH |
| Container registry | Publishing quietstat/quietstat |
Local image build | Registry credentials for the release pipeline |
| npm registry | Publishing @quietstat/tracker |
npm pack and dry-run publish |
npm account, scope ownership and publish token |
| WordPress.org plugin directory | Plugin distribution | The zip artifact and Plugin Check | Directory submission by the owner |
| Google Tag Manager Community Template Gallery | Template distribution | The template file and the in-repo harness | Gallery submission by the owner |
| DNS, TLS, hosting, backups storage | Hosted and self-hosted operations | Local compose | Domains, certificates, infrastructure accounts, backup bucket credentials |
| Legal documents (terms, privacy policy, data processing agreement, refund policy) | Hosted launch | Not part of the product | The owner publishes them; marketing and documentation-site copy is out of scope |
20.9 Definition of done for the whole project #
The project is done when all of the following are true and each is evidenced in the final report (Section 20.11).
- All 14 milestones are merged and logged in
DECISIONS.md, andpnpm verify:milestone M14exits 0 on the release candidate. - The full CI pipeline of Section 18.14 (pull-request, nightly, weekly and release stages) is green on the release candidate commit, with coverage thresholds (Section 18.15.1), mutation thresholds (Section 18.15.3) and all load criteria (
LOAD-01toLOAD-13) met. REL-01toREL-05(Section 19.4.14) pass.- Every feature in the contract has working, tested behavior, as mapped below.
- There are no unresolved markers: a repository scan finds no pending-work marker comments, no open-decision placeholders, and no skipped, focused or quarantined tests that lack a
DECISIONS.mdentry. README.md,.env.example,CHANGELOG.md,docs/self-hosting.md,docs/hosted-operations.md,docs/upgrading.mdandDECISIONS.mdexist and agree with the code (REL-03).- The launch checklist of Section 20.10 is complete except the items explicitly marked "owner action", each of which is listed in the final report.
Feature coverage map (the final report must show this table with test evidence):
| # | Feature of the contract | Milestones | Main test identifiers |
|---|---|---|---|
| 1 | Cookie-free tracking script and event types | M3, M4, M12 | TRK-01 to TRK-10, ING-01 to ING-31 |
| 2 | Privacy-preserving visitor counting | M4, M14 | H-*, S-*, SE-*, PRIV-01 to PRIV-17 |
| 3 | Single-page stats dashboard and reports | M5, M7 | E2E-07 to E2E-14, E2E-40, E2E-41, golden and oracle suites |
| 4 | Filters and saved segments | M7 | E2E-10, E2E-11 |
| 5 | Goals, custom events, properties | M8 | E2E-16, E2E-17 |
| 6 | Funnels and user journeys | M9 | E2E-19, E2E-20 |
| 7 | Ecommerce revenue reporting | M8 | E2E-18 |
| 8 | Accounts, teams, roles | M6 | E2E-01 to E2E-04, E2E-21 to E2E-23, E2E-39, SEC-01 to SEC-13, SEC-32, SEC-33 |
| 9 | Sharing, embeds, e-mail reports, spike notifications | M10 | E2E-25 to E2E-28, SEC-34 |
| 10 | Stats API and Events API | M4, M5, M6 | Stats API and Sites API integration suites, E2E-24, SEC-14, SEC-24, LOAD-12 |
| 11 | CSV export, CSV import, GA4 import | M11 | E2E-29 to E2E-31, SEC-18, SEC-20 to SEC-22, SEC-35, SEC-36, SEC-39, SEC-40 |
| 12 | WordPress, GTM, npm, script tag, proxy, setup guides | M3, M12 | E2E-32, E2E-33, ING-20, ING-28, pnpm test:wp, pnpm test:gtm |
| 13 | Traffic quality: GeoIP, bot and datacenter filter, exclusions, hostname allowlist | M4, M6 | ING-24 to ING-27, ING-30, E2E-15, bot filter and exclusion suites |
| 14 | Lifetime deal purchase and licensing | M13 | STR-01 to STR-24, E2E-34 to E2E-38, SEC-26 to SEC-28, SEC-37, SEC-38, SEC-41 |
| 15 | Hosted and self-hosted editions | M1, M13, M14 | Edition switch suite, E2E-06, REL-01, REL-02 |
20.10 Final launch checklist #
Complete every item. For each, record the evidence in the final report: the command run and its result, or a screenshot path. Items marked (owner) need a credential or an external account only the owner has; implement and verify everything around them against fakes, then list the item under "Owner actions".
20.10.1 Hosted edition #
- Production configuration passes the config loader with
EDITION=hosted; every secret in Section 16.4 is set from a secret store and none is committed. - Infrastructure in an EU region: PostgreSQL, ClickHouse, Valkey, at least 2
webinstances, at least 1worker, TLS with HSTS, as the topology in Section 16 describes. (owner) TRUSTED_PROXIESequals the real proxy or load-balancer ranges; a request with a spoofedX-Forwarded-Forfrom outside those ranges is ignored (verified by a request and a check that the storeduser_iddiffers from the spoofed-address hash).- Reverse-proxy and load-balancer access logs for
/api/eventdo not retain the client IP or User-Agent (disabled or anonymized as Section 17 requires), and no request URI is logged at all (PRIV-17run against the production configuration); the configuration is shown as evidence. pnpm migrateran on production;license_tiersholdssolo5,900 minor units / 5 sites / 100,000 events,pro14,900 / 25 / 500,000 andagency34,900 / 100 / 2,000,000 (MIG-06query run against production).- Stripe live mode configured: the webhook endpoint registered with exactly the three event types Section 15.6.5 handles, the signing secret set,
STRIPE_TAX_ENABLEDset as the owner decided, and the Stripe Dashboard setting that e-mails customers about successful payments switched on. No Stripe products or prices are created (inline pricing, Section 15.4). (owner) - A real purchase of Tier 1 completed with a live card produces an active license, the receipt e-mail, the in-app invoice with a correctly numbered PDF; a refund inside the 60-day window from the operator console revokes the license and puts the team in the unlicensed state; a second purchase restores service. (owner)
- A code batch is generated for the marketplace, exported as CSV, one code redeemed in production and then revoked from the operator console.
- Usage-cap behavior verified on a staging team: e-mails at 80% and 100%, drops beyond 110%, banner states, month rollover (
E2E-37and the Licensing and caps suite). - SMTP provider configured and every template in the Section 12 inventory delivered to a real mailbox and rendered correctly in a web mail client and a desktop client; SPF, DKIM and DMARC pass. (owner)
- The bundled GeoIP database is present, no older than 45 days, and the dashboard footer shows "IP geolocation by DB-IP".
SUPER_ADMIN_EMAILSset; operator accounts have 2FA enabled;/adminreturns 404 to other accounts.- Backups run on the schedule of Section 16; a restore into a scratch environment completes and the oracle comparison on a seeded site passes.
- Monitoring and alerts of Section 16 are active: readiness failures, 5xx rate, ingestion buffer backlog, ClickHouse disk usage, job failures, certificate expiry. (owner)
- Load sign-off on a production-like environment:
LOAD-01(5,000 events/s perwebinstance) and the dashboard targets of Section 18.8.3. - Privacy sign-off:
PRIV-01executed against the production-like stack at the most verbose log level finds zero occurrences; production log level is confirmed as nottrace. - Security sign-off:
SEC-29andSEC-31clean on the release candidate image; secrets used during testing rotated. - Production smoke test: sign up, redeem or buy, add a site, install the snippet on a real page, see the visit in realtime, run a Stats API query with a new API key.
- Rollback plan prepared: previous image tag retained, database backup taken immediately before the release migration.
- Legal documents (terms, privacy policy, data processing agreement, refund policy of
REFUND_WINDOW_DAYSdays, 60 by default) are published by the owner outside this product. (owner) - The penetration test of Section 17.13 is complete with zero open critical or high findings. (owner)
- The Google OAuth consent screen is verified for the Analytics read scope used by the GA4 import (Section 14.5.1). (owner)
SECRET_KEY_BASEis escrowed with two-person access, stored separately from the data backups (Section 16.8.1). (owner)
20.10.2 Self-hosted edition #
- The image
quietstat/quietstatis built for amd64 and arm64, tagged with the release version, andlatestfor non-pre-release tags;docker pullworks from a clean machine. (owner: registry credentials) - The release assets contain
docker-compose.yml,.env.exampleand the self-hosting guide; the defaultEDITIONisselfhosted; the dashboard shows no billing, license or usage UI and the license routes return 404 (Edition switch suite). REL-01passes on a clean virtual machine with only Docker installed, and the time fromdocker compose up -dto the first visible event is recorded (target: under 10 minutes including reading the guide steps).- The bundled GeoIP database loads; a MaxMind-format database supplied through
GEOIP_DB_PATHis verified by loading a test file; footer attribution behaves as inE2E-41. - Upgrade from the previous release is documented and passes
MIG-07andMIG-08; backup and restore instructions are executed once by hand and recorded. - First-run account creation works as in
E2E-06; the application makes no outbound network calls other than SMTP, the Google APIs after a GA4 connect,download.db-ip.com(geoip-update), the hosts of thelists-refreshjob andwww.ecb.europa.eu(exchange rates), as Section 17.9.6 lists them; withGEOIP_AUTO_UPDATE=falseandLISTS_AUTO_UPDATE=falseonly SMTP and the exchange-rate feed remain. Verified by running the stack with outbound traffic blocked except these hosts and exercising the dashboard and ingestion. - The AGPL-3.0-only license text and third-party notices (including the DB-IP attribution) are in the image and the repository; the deployment guide tells operators who modify the code about the AGPL network-use obligation.
- The WordPress plugin zip passes Plugin Check and
E2E-33; submission to the directory is the owner's step. (owner) - The GTM template is imported into a real Google Tag Manager workspace, previewed on a test page, and an event arrives in a running instance. (owner: needs a GTM account)
npm publish --dry-runfor@quietstat/trackerlists exactly the intended files; the real publish with provenance is the owner's step. (owner)
20.10.3 Both editions #
- The release candidate commit is green in every stage of Section 18.14, including the latest nightly and weekly runs no older than 7 days.
CHANGELOG.mdhas the release entry, the version is taggedv<version>, and release notes list any destructive migration (none are allowed outside a major release).PRODUCT_NAMEholds the final product name the owner chose in Section 1 (the working name otherwise), and a repository scan finds no hard-coded product name outside the places Section 3 fixes.- All answers to the owner-customization questions of Section 1 are recorded in the header of
DECISIONS.md(defaults where the owner gave none). - A repository scan finds no secrets, no real e-mail addresses other than documentation placeholders, no pending-work marker comments.
20.11 Final report #
When all milestones are done, produce one report (a Markdown file at docs/build-report.md, also given as your final message) with:
- The result of
pnpm verify:milestone M14and the pipeline run references. - The feature coverage map of Section 20.9 filled with test identifiers and pass status.
- The table of load results against the targets of Section 18.8.
- The list of
DECISIONS.mdentries of typedeviation,concernanddependency, one line each. - The launch checklist of Section 20.10 with evidence for each item, and a separate list headed "Owner actions" containing every item marked (owner) and the external accounts and credentials listed in Section 20.8.
- Known limitations: only items that this document itself defines as limits (for example caps and size limits); any other limitation counts as an unfinished item and must be resolved before reporting completion.
Generated at GenerateSpecs.com — one idea in, one buildable spec out.
Licensed under CC BY 4.0. Use it for anything — just credit GenerateSpecs.com.