Stage 01
Build Crawl Engine
Implemented sitemap discovery, Crawlee orchestration, Playwright browsing, exclusions, source tracking, bounded crawling, and repeatable desktop and mobile sessions.
Output / Browser crawl engine
Back to portfolio
Draft case study2026
Web QA Tooling
Website QA Crawler.
Built a browser QA crawler that checks five sitemap classes across two viewports, classifies render health, captures evidence, and exports structured reports for repeatable validation.
TypeScriptCrawleePlaywrightaxe-coreWeb QA
Explore the rebuild
Sitemaps
5 crawl classes
Viewports
2 QA sizes
Tests
32 automated tests
Reports
JSON CSV HTML
The System Problem
A CRM rebuild starts by identifying the connected failures underneath the visible symptoms.
Root cause / 01
Page QA needed repeatable browser evidence instead of manual spot checks
Manual page QA was slow to repeat across large site sections and inconsistent when rendering problems, responsive layouts, forms, assets, accessibility, or network behavior changed between runs. A URL list alone could not prove whether the browser actually rendered the page correctly.
The crawler therefore needed to separate real defects from blocked or degraded rendering. The render-integrity layer tracks document, CSS, scripts, fonts, images, backgrounds, iframes, XHR, and fetch activity before deciding which audits are trustworthy enough to run.
Audit signals
What the system was telling us.
01
Five sitemap classes feed the crawler
02
Desktop and mobile viewports are tested
03
Render health gates downstream audits
04
Screenshots preserve visual evidence
05
Thirty-two tests validate crawler behavior
Rebuild Sequence
The work moved from diagnosis to architecture, implementation, and governance—in that order.
Stage 02
Add QA Modules
Added checks for metadata, content, accessibility, assets, links, layout, forms, network behavior, responsive rendering, and interactive components systematically.
Output / Modular QA checks
Stage 03
Detect Render Health
Classified pages as healthy, degraded, blocked, or failed by tracking critical resources and visible browser state before running dependent audits.
Output / Render diagnostic layer
Stage 04
Package Evidence
Generated JSON, CSV, HTML, screenshots, render diagnostics, tests, fixtures, and staged crawl commands for review and repeatable local execution.
Output / Structured QA reports
Target System Architecture
Four connected layers turned the portal from a collection of tools into an operating system.
System spine
Target-state sequence
Good automation and reporting sit on top of a governed data and process model—not the other way around.
Layer 01
Crawl Layer
Sitemap-defined URLs are visited in a real browser with bounded discovery, exclusions, source tracking, and repeatable session behavior.
Layer 02
Render Layer
Resource loading and visible DOM state determine whether a page is healthy enough for downstream visual and functional checks.
Layer 03
Audit Layer
Independent modules systematically inspect accessibility, content, assets, links, layout, forms, metadata, network behavior, and responsive issues across pages.
Layer 04
Evidence Layer
Findings are written to machine-readable reports, human-readable HTML, screenshots, and render diagnostics for structured review and technical handoff.
Qualitative Outcomes
The engagement focused on structural improvement, so the strongest results are clearer operations—not invented vanity percentages.
SystemBeforeAfter rebuild
Coverage
Manual spot checks missed broad patterns
Five sitemap classes automated
Rendering
Blocked pages could create false defects
Render health now gates audits
Evidence
Issues depended on manual screenshots
Evidence captured during crawl
Repeatability
QA varied between individual reviews
Automated tests and reports
What Was Delivered
A reusable case study should make the work tangible without exposing confidential client data.
Deliverable 01
Multi-sitemap crawl engine
Deliverable 02
Desktop and mobile QA
Deliverable 03
Accessibility audit module
Deliverable 04
Render health classifier
Deliverable 05
JSON CSV HTML reports
Deliverable 06
Screenshot evidence capture
System note / final
Architecture before automation.
The hardest part was not detecting more issues; it was knowing when not to trust a result. A browser blocked by Cloudflare can generate hundreds of technically correct but useless findings. Render health became the gatekeeper so the tool could distinguish a broken page from a broken audit session.
Client-specific URLs, credentials, screenshots, and private crawl evidence are excluded from public examples.