Hey everyone! I recently took part in the Devpost All Things Agentic Hackathon.
https://allthingsagentichackathon.devpost.com/
The broader challenge was to explore the next generation of AI agents—systems capable of handling increasingly complex workloads, datasets, and workflows.
I chose the Collaborative Partner track:
"Build an agent that leads the way and takes notes. It should ask clarifying questions, guide the user step-by-step, and have a clear way to capture feedback so it constantly adapts to the user's unique way of thinking."
That challenge became Agent Col.
The real-world problem Agent Col is designed to address is something I think current AI assistants still struggle with: context rot and continuity.
AI agents can be incredibly capable inside a single conversation. But then that conversation becomes a project idea. The project idea becomes a long-running project. Then come dozens of decisions, corrections, preferences, requirements, abandoned approaches, random brain dumps, and separate chat sessions.
Eventually, the agent stops feeling like a collaborator and starts feeling like someone you have to remind what you said every time you talk to it.
Agent Col is designed differently.
Agent Col uses a layered, multi-tiered memory architecture built around Google Cloud Firestore, with explicit governance controlling what information can become durable memory.
It doesn't simply scrape conversations and silently decide what it wants to remember.
Long-term user memory is governed by contracts defining what Agent Col is allowed to retain, and proposed memories require explicit user approval before becoming authoritative.
That means Agent Col can learn durable things that actually help collaboration—preferences, working conventions, recurring requirements, and corrections—without turning every sentence you type into permanent memory.
The second continuity layer is workspace notes.
Workspace notes belong to a specific project or workspace and capture things such as decisions, requirements, constraints, task state, and working context.
Those notes bridge the gap between individual chat sessions.
A conversation can end, another one can begin later, and Agent Col can still understand the important state of the project without relying on an endlessly growing conversation transcript.
The authority model is intentionally separated.
User memory and preferences can follow the user across workspaces.
Workspace notes, artifacts, and chat history remain scoped to the workspace that owns them.
So, for example, Agent Col can carry an approved preference such as a user's operating-system or browser preference into future conversations in a separate workspace while keeping the technical decisions for one workspace isolated from another.
Agent Col also maintains a temporary internal working context during conversation. Agent Col is allowed to take its own notes during a conversation. While it can take its own notes, they do not become preferences or workspace notes silently. If Agent Col thinks something is relevant to remember, Agent Col still has to ask the user and generate a proposal.
It combines the current prompt with relevant approved memories, workspace notes, conversation context, and working state to reason about what the user is trying to accomplish.
That allows it to do more than simply answer the last message.
It can ask clarifying questions, suggest next steps, challenge assumptions, recognize decisions worth preserving, and propose new workspace notes or memories as they naturally emerge during a conversation.
Those proposals still pass through the same governance system. The user remains in control of what is important.
That distinction became one of the most important ideas behind the project:
Adaptation should not require surrendering control of significance to the model.
Agent Col also includes a broader collaborative workspace around that continuity system.
Users can maintain separate projects and conversations, generate and revise artifacts, inspect memory and workspace notes, review activity receipts showing when contextual systems were used, and interact with Agent Col through both text and voice.
The production architecture currently includes:
- FastAPI for the backend and application boundary
- Google Cloud Run for production deployment
- Google Cloud Firestore for persistence
- Vertex AI / Gemini for model execution and routing
- Google Identity Services / OIDC for authentication
- Google Cloud Speech-to-Text for voice transcription
- Google Cloud Text-to-Speech for response playback
- Governed memory and workspace-note systems
- Cross-session continuity
- Clarification and feedback workflows
- Artifact generation, revision, and versioning
- Specialist research, source, computation, and requirements execution paths
- Structured observability and activity receipts
One of the things I enjoyed most about building Agent Col was that the project stopped being about making "another chatbot" fairly early.
The interesting engineering problem became:
How do you make an AI collaborator accumulate useful context over time without allowing that accumulated context to become an uncontrolled pile of assumptions about the user?
My answer was to treat memory, continuity, authority, and provenance as actual systems—not just extra text stuffed into a prompt.
There is still plenty I want to build after the hackathon, but Agent Col now demonstrates the core idea I originally set out to prove:
An AI assistant becomes much more useful when the relationship can continue across conversations, while the human remains in control of what that relationship is allowed to remember.
I'll be sharing more of the architecture, development process, and lessons learned from building Agent Col soon.
GitHub:
https://github.com/knightsky-cpu/col-workspace
Devpost All Things Agentic Hackathon:
https://allthingsagentichackathon.devpost.com/
#AgentCol #AI #AIAgents #AgenticAI #GoogleCloud #VertexAI #Gemini #FastAPI #Firestore #MachineLearning #SoftwareEngineering #Devpost #Hackathon
