An enterprise AI platform that improves itself
Sanitized work sample by Rich Sadowski. Written for a mid-size B2B SaaS company. The company name, people, internal documents and dates are removed.
A shared, governed platform where every department builds AI agents. Most designs stop at automating the work. This one also automates finding what to improve, and a person decides every change.
Don't just automate the work. Automate the improvement.
Every run leaves evidence. Scheduled reviews turn that evidence into proposals. A person approves each one, and only tested changes ship.
01The improvement cascade
Four scheduled reviews. Each looks back further than the one below it. Evidence climbs from nightly to quarterly, and direction comes back down as new rules.
Nightly
looks back 26 hours
Reads
Yesterday's runs, quality scores, blocked requests, edits people made
Proposes
Fixes to individual skills and new test cases
Reads
Nightly reports, routing mistakes, search misses, cost by skill
Proposes
Fixes for patterns that repeat across skills and teams
Monthly
looks back 35 days
Reads
Weekly reports, drift from the release baseline, business outcomes
Proposes
Threshold changes, model choices, stale knowledge to retire
Quarterly
looks back 95 days
Reads
Monthly reports and the improvement log, not raw data
Proposes
Agents to scale or retire, policy changes, and an audit of whether past changes helped
evidence
climbs
direction
descends
Propose onlyNo loop changes anything. Each run ends in a numbered list a person marks accept, defer or reject.
Tested releaseAccepted changes ship only after passing a fixed test set. Rejected items do not move up.
Audits itselfThe quarterly loop checks for changes later reverted and problems fixed twice.
02The platform
Six layers. Every request from a person or an agent passes the gateway, so access, privacy, quality and cost are controlled in one place.
1
Where people workChat, the AI assistant, office suite and dashboards. No new app to learn.2
Gateway and guardrailsChecks who is asking, removes personal data, blocks prompt attacks, routes to the right model.3
Agents and skillsBuilt from approved templates. Each agent has a named owner and an off switch.4
Context and memoryOnly what this person is allowed to see, assembled per request.5
ModelsCommercial models under zero or bounded data retention, kept in-region.6
Data layerBusiness data landed once, described once, authorized once.
IdentityOne source for people, groups and agents
ObserveEvery step traced and scored
CostBudgets by department
03Key decisions
- D1One door for every AI callA single gateway fronts all tools and models. No team wires up its own.
- D2Agents act as a personAgents carry the requesting person's identity and permissions. No shared service accounts.
- D3Authorize data onceAccess rules live in the data layer, not in each agent or endpoint.
- D4Agents ask for numbersAgents query defined business metrics. They do not write their own SQL or paste rows into prompts.
- D5A person approves every writeChanges to systems of record and any customer contact need human approval.
- D6Improvement by review onlySkills change only through the cascade and a tested release, never on their own.
04Top risks
- R1Prompt injectionInstructions hidden in web pages, tickets or tool output. Untrusted content is labeled, screened, and cannot trigger writes without approval.
- R2Data leaving its regionSome bundled AI features process data outside the customer's region. Each is checked before regulated data reaches it.
- R3Retention driftSome AI tools keep chats and memory indefinitely by default. Retention is set deliberately for each tool.
- R4Vendor churnAI vendors rename, merge and retire products within months. Open protocols keep each component replaceable.
- R5Reviewers grading themselvesLoops that read only their own scores can reinforce their own mistakes. The quarterly loop checks against business outcomes.
05Rollout
Months 1–3
Foundations
- Enterprise AI plan with central identity
- Guardrail service
- Tracing and logging
Months 4–5
Gateway and data
- Tool gateway and allowlist
- Governed data and metrics
- Release pipeline with test gate
Months 6–8
First department
- One agent, one workflow
- Nightly and weekly loops
- Go or no-go review
Months 9–12
Scale
- Templates for other teams
- Department budgets
- Monthly and quarterly loops
Rich Sadowski · richsadowski.com