We built what most teams skip: data contracts, monitoring from day one, four agents that actually coordinated.

The situation

The Product Compass ran a 2-week AI Agent Buildathon with three tracks. Track 3 was multi-agent coordination: build a system of at least three autonomous agents working together, demonstrate real coordination, and ship something that works.

I chose Track 3. The domain: job search automation. The product: CareerAgent, a system that discovers jobs matching a user's profile, generates tailored CVs and cover letters, and learns writing style preferences from user feedback over time.

It had to actually work: real job matching, real CV generation, real preference learning from user edits.

The architecture

We built four specialized agents, each with a single responsibility and clear data contracts between them.

"Multi-agent systems fail on misaligned data contracts. Ours nearly did, mid-sprint."

We hit integration failures mid-sprint when the Composer expected a field the Matcher wasn't providing, and when the Adapter was writing to a different storage location than the Matcher was reading from. The fix was a strict 6-tab data schema with documented read and write permissions per agent. That schema became the source of truth for everything.

Observability from day one

You can't debug what you can't see, and you can't trust what you can't measure. I built three layers of observability into the architecture before writing a single agent workflow.

The Think tool captured agent reasoning during decision-making, making black-box LLM choices visible and debuggable. Automated evals ran after every execution: routing validation for the Orchestrator, confidence thresholds for the Adapter's pattern extraction. Every agent execution logged to a Google Sheets dashboard with timestamp, routing decision, tools called, patterns found, and pass or fail.

Human-in-the-loop validation handled the high-stakes output: no AI-generated CV reached a real job application without user review. The user's edits fed directly back into the Adapter's learning loop.

The crisis pivot

On day 11, Google API rate limits hit during testing and the architecture couldn't continue on the original stack.

We evaluated four options: new Google account, full Supabase migration, hybrid approach, or scope reduction. The decision: migrate to Lovable and Supabase. Parallel development while keeping the agent logic intact. Two days to migrate, three days unblocked to finish.

The agent logic stayed intact. The pivot reduced risk by keeping the most complex part unchanged while solving the infrastructure problem with better tooling.

The outcomes

We finished 5th out of 25 competing projects, with a 95% success rate across 47 test requests, 100% orchestrator routing accuracy, and 83% Adapter pattern extraction success. The live product is deployed and accessible.

"Monitoring is how you know an AI system actually works."

The gallery of all 36+ teams from the buildathon is live at The Product Compass Gallery. The live demo runs at nextstepcareer.lovable.app.

My contribution

This was a team of 3. My areas: problem definition, PRD, business case, and system architecture. I designed the data contracts and the observability layer, and built the Orchestrator and Adapter agents hands-on. The overall architecture was a collaborative decision.

My teammates handled the Matcher and Composer agent implementation, workflow integrations, and parts of the frontend. The crisis pivot on Day 11 was a joint call.

What this taught me

Working with agents changed how I think about teams. Guardrails, context, memory, observability, and fallbacks aren't just engineering patterns: they're the same things that make any team function. Without memory every interaction starts from scratch, without observability nothing is debuggable, and without fallbacks one failure cascades through the rest.

The Adapter made this concrete. A test user edited one of the generated CVs, swapping passive voice for active and cutting a sentence in half. The next CV the Adapter generated reflected that preference without being asked again. That's how a good new hire responds to their first round of feedback: show them once, and they run with it.

Data contracts are the actual foundation of multi-agent systems, more than the agent logic itself. Every integration failure we hit mid-sprint came from misaligned expectations about data formats and storage locations, which meant documenting contracts with required fields, read and write permissions, and examples, and testing integration before anything else.

Observability built from the start changes how fast you can debug: the Think tool, automated evals, and decision logs turned black-box LLM behavior into something transparent and fixable, and retrofitting that after the fact would have cost days.

Architecture decisions under real constraints require tradeoffs you can defend. Google Sheets was pragmatic for a 2-week buildathon: visual, fast, zero setup, and not production-grade, which we knew going in. Knowing which tradeoffs are acceptable in context, and which ones create compounding debt, is the actual skill.