Automated Evaluation, Accessibility (a11y) & Performance Linter Suite Specification

Automated Evaluation, Accessibility (a11y) & Performance Linter Suite Specification

Document Status: Living True Specification (Single Source of Truth)
Authority: Tooling, Governance, Quality Assurance & CI Pipeline
Related Epics & Issues: #22 (Umbrella Spec: Automated Evaluation Suite), #314 (Edge Linting & TypeScript), #315 (Axe-Core a11y Scanner), #316 (Lighthouse CI Performance Budgeting), #317 (Schema.org JSON-LD Validator), #318 (Author True Spec)
Related Documents: ARCHITECTURAL_LINTERS.md, CLIENT_APPLICATION_ANATOMY.md, CAPABILITY_PACKAGE_ANATOMY.md, HIGH_LEVEL_DESIGN.md, PRD.md

---

1. Executive Summary & Objective

In SiteSwarm, applications are built using an agent-first development model: AI coding agents synthesize 100% bespoke, un-themed frontends for each small business client rather than dragging and dropping components from a generic UI widget library.

While this delivers distinctive, tailor-made visual identities for local bakeries, craft studios, and service providers, it creates a unique engineering challenge: the platform must reliably inspect, audit, and quality-gate large volumes of newly generated application code across heterogeneous layouts.

The Automated Evaluation, Accessibility (a11y) & Performance Linter Suite serves as the automated quality gatekeeper across the platform. It enforces four non-negotiable standards before any pull request can deploy to staging or production:

1. Edge Runtime Safety & Static Code Integrity: Zero prohibited Node.js runtime calls in edge routes, strict TypeScript adherence, and tight client JS payload budgets.
2. Automated Accessibility (a11y) Enforcement: Zero critical or serious WCAG 2.1 AA violations across all built pages and viewports.
3. Lighthouse CI Performance Budgeting: Hard gate requiring a mobile Performance score $\ge 95$, sub-second First Contentful Paint (FCP), and Cumulative Layout Shift (CLS) $< 0.05$.
4. Semantic HTML & Structured Data (SEO) Validation: Valid Schema.org JSON-LD syntax, complete LocalBusiness metadata, and valid heading structures.

flowchart TD
    PR[Pull Request Opened / Updated] --> Gate1[Gate 1: Edge Runtime & AST Linters\npnpm run lint:arch & pnpm run lint:edge]
    Gate1 --> Gate2[Gate 2: Static Build & Typecheck\npnpm run build & pnpm run check]
    Gate2 --> Gate3[Gate 3: Automated a11y Scanner\n@axe-core/playwright WCAG 2.1 AA]
    Gate3 --> Gate4[Gate 4: Lighthouse CI & Schema Validator\nMobile >= 95 & Schema.org JSON-LD]
    Gate4 --> Pass{All 4 Gates 100% Green?}
    Pass -- Yes --> Deploy[Eligible for Ephemeral Preview / Staging]
    Pass -- No --> Block[Hard CI Failure & Diagnostic Escalation]

---

2. The Four Pillar Automated Quality Gates

2.1 Gate 1: Cloudflare Edge Runtime Safety & AST Linters

Client applications running on Cloudflare Workers operate within the V8 isolate-based runtime (workerd), which does not provide standard Node.js runtime primitives (such as the raw filesystem, native C++ addons, or uncapped memory allocations). Furthermore, per ADR-0003, public client applications must adhere to static-first edge delivery.

Quality Standards & Invariants:

1. Prohibited Node.js Builtins: AST linters (arch/no-node-builtins-in-edge-routes) forbid importing Node-specific modules (node:fs, node:child_process, node:cluster, node:os) inside client application edge handlers.
2. Strict Environment Contract: Client code is forbidden from reading unmanaged process.env globals; runtime configuration must be accessed strictly through typed Cloudflare environment bindings (context.locals.runtime.env or @siteswarm/governance contracts).
3. Payload Budget Threshold:
   • Initial client-side JavaScript payload: $\le 50\text{ KB}$ gzipped.
   • Initial client-side CSS payload: $\le 40\text{ KB}$ gzipped.
   • Monolithic runtime bundling of admin/CMS logic in public client frontends is strictly forbidden (enforced by ADR-0003 Worker Splitting).

---

2.2 Gate 2: Automated Accessibility (a11y) Verification

Because AI coding agents author unique markup for each client without a centralized component library, automated accessibility scanning must run directly against the built DOM of every client application route.

Quality Standards & Invariants:

1. Target Standard: Strict conformance to WCAG 2.1 Level AA.
2. Scanning Engine: Powered by @axe-core/playwright executing headless browser audits across mobile (390x844) and desktop (1440x900) viewports.
3. Zero-Tolerance Violations (Hard CI Blocks):
   • Color Contrast: Text and images of text must have a contrast ratio of at least 4.5:1 (3:1 for large text).
   • Alternative Text: 100% of raster images (<img>, <Picture />) must include meaningful, non-empty alt attributes (or explicit alt="" aria-hidden="true" for purely decorative elements).
   • Accessible Interactive Controls: All buttons, links, and form inputs must have accessible names (aria-label, visible text, or associated <label>).
   • Keyboard Navigation & Focus Traps: Interactive modals, hamburger navigation menus, and drawers must trap focus when opened and release focus upon Esc or close actions.
   • Form Field Associations: Form inputs must have programmatically associated <label for="..."> tags.
4. CI Enforcement: The presence of any critical or serious violation results in an immediate non-zero exit code, blocking pull request merge.

---

2.3 Gate 3: Lighthouse CI Performance & Bundle Budgeting

Small business clients rely on instant mobile load times to capture local customers searching for operating hours, menus, contact numbers, and driving directions. SiteSwarm mandates enterprise-grade Core Web Vitals across all client properties.

Quality Standards & Invariants:

1. Auditing Harness: Executed via @lhci/cli simulating a mid-tier mobile device on a throttled 4G network connection.
2. Threshold Matrix:

Metric	Target Threshold	CI Enforcement Severity
Lighthouse Mobile Performance	$\ge 95$	Hard Error (Blocks Merge)
First Contentful Paint (FCP)	$\le 1.0\text{s}$	Hard Error (Blocks Merge)
Largest Contentful Paint (LCP)	$\le 1.8\text{s}$	Hard Error (Blocks Merge)
Cumulative Layout Shift (CLS)	$\le 0.05$	Hard Error (Blocks Merge)
Total Blocking Time (TBT)	$\le 150\text{ms}$	Hard Error (Blocks Merge)
Lighthouse SEO Score	$= 100$	Hard Error (Blocks Merge)
Lighthouse Best Practices	$\ge 95$	Hard Error (Blocks Merge)

3. Layout Shift Immunity: All image tags must declare explicit width and height attributes or use the platform’s @siteswarm/components <Picture /> component with aspect-ratio preservation to guarantee zero layout shifts.

---

2.4 Gate 4: Semantic HTML & Structured Data (SEO) Validation

Local business discovery depends heavily on search engine knowledge graph extraction and rich snippets. SiteSwarm enforces strict semantic HTML hierarchy and Schema.org structured data validity.

Quality Standards & Invariants:

1. Heading Hierarchy: Each page must possess exactly one <h1> element, followed by logical descending hierarchy (<h2> $\to$ <h3>). Skipping levels (e.g. <h1> immediately followed by <h4>) is flagged as a lint error.
2. Structured Data Syntax & Semantics:
   • All client marketing templates must output valid JSON-LD adhering to the Schema.org vocabulary.
   • Mandatory LocalBusiness properties:
     • @context: "https://schema.org"
     • @type: Specific sub-type (e.g., "Bakery", "Restaurant", "ProfessionalService", "LocalBusiness").
     • name: Matches client business name declared in swarm.config.ts.
     • address: Full postal address structure (PostalAddress with streetAddress, addressLocality, addressRegion, postalCode).
     • telephone: E.164 or formatted local telephone number.
     • openingHoursSpecification: Structured schedule specifying days of week and opening/closing hours.
3. OpenGraph & Twitter Card Metadata: Pages must declare OpenGraph title, description, URL, and a responsive edge-generated social card image (og:image) conforming to @siteswarm/seo contracts.

---

3. Enforcement Mechanics & CI Pipeline Integration

Quality gate enforcement is integrated at multiple levels of the developer and agent feedback loop:

+-------------------------------------------------------------+
| Level 1: Local Developer / Agent Loop                       |
|   pnpm run lint:arch        (AST architectural rules)       |
|   pnpm run check            (TypeScript typechecking)       |
|   pnpm run test             (Unit & integration test suites)|
+-------------------------------------------------------------+
                              |
                              v
+-------------------------------------------------------------+
| Level 2: GitHub Actions Pull Request Gate (Fast-Pass & App) |
|   .github/workflows/ci.yml                                  |
|   - Change detection isolates modified apps and packages.   |
|   - Headless Playwright / Axe-core executes a11y suite.     |
|   - LHCI runner spins up static preview server.             |
|   - Schema.org validator checks compiled dist/*.html.       |
+-------------------------------------------------------------+
                              |
                              v
+-------------------------------------------------------------+
| Level 3: Gated Staging & Production Deployment               |
|   - Zero-pending / zero-failing CI invariant enforced.      |
|   - Merged to main -> Deploys to Cloudflare edge.           |
+-------------------------------------------------------------+

3.1 Failure Escalation & Remediation

When an evaluation check fails:

1. The CI runner outputs machine-readable JSON and human-readable markdown summaries detailing the exact DOM selector, violation rule, and remediation instruction.
2. For AI coding agents running pm ship or pm fleet, the agent inspects the diagnostic failure log, applies targeted remediation inside the dedicated wt worktree, and pushes a corrective commit without human intervention.

---

4. False-Positive Management & Strict Waiver Escalation Procedure

In rare scenarios, legitimate design constraints may conflict with static evaluation thresholds (for example, an artistic retro brutalist aesthetic requiring unusual color contrasts, or an experimental client interaction).

To maintain strict security and accessibility governance while preventing developer gridlock, waivers are governed under the following protocol:

1. Waiver Declaration: Waivers must never be silent inline comments (e.g. <!-- axe-ignore -->). They must be declared programmatically in the client application’s swarm.config.ts:

   export default defineAppConfig({
     name: "bakery",
     waivers: [
       {
         ruleId: "a11y/color-contrast",
         target: "src/components/BrutalistBanner.astro",
         reason: "Artistic monochrome styling approved by client lead in contract addendum B",
         expiresAt: "2026-12-31",
         approvedBy: "jacobmiller22"
       }
     ]
   });

2. Mandatory Expiration Date: All waivers are time-bounded. When expiresAt passes, CI automatically treats the waiver as invalid and blocks the build until renewed or remediated.
3. Architectural Review: Introducing a waiver requires review from the Technical Lead / TPM.

---

5. Implementation Roadmap & Sub-Ticket Matrix

Transitioning this specification into production execution is sequenced across the following shovel-ready sub-tickets:

Sub-Ticket	Focus Area	Scope & Deliverables	Priority
#314	Edge Runtime Safety & ESLint	Shared ESLint & TypeScript configuration enforcing Cloudflare Workers runtime invariants (banning Node builtins, raw process.env) and payload size limits.	priority:high
#315	Axe-Core a11y Scanner	Automated @axe-core/playwright accessibility runner in CI testing built client pages for WCAG 2.1 AA compliance.	priority:high
#316	Lighthouse CI Budgeting	GitHub Actions @lhci/cli integration enforcing mobile Performance $\ge 95$, FCP $< 1.0\text{s}$, and CLS $< 0.05$.	priority:high
#317	Schema.org Structured Data Validator	AST / HTML parser validating Schema.org LocalBusiness JSON-LD syntax, mandatory fields, and @siteswarm/seo integration.	priority:medium

---

6. Living Specification Invariants

1. The Single Source of Truth Invariant: This document is the sole authority governing accessibility, performance budgeting, structured data, and runtime evaluation standards.
2. The Zero-Degradation Invariant: New client onboardings and feature additions must never downgrade fleet-wide performance or accessibility scores.
3. The Automated Gate Invariant: No code reaches production without passing all four quality gates in continuous integration.