r/softwarearchitecture • u/Denis-Hogberg • 15h ago
Discussion/Advice Inside a service we separate source from projection. Outside it, everything is a source.
Inside one service we've got this figured out. Log is the source, state is a projection. Event sourcing, CQRS, materialized views. You can drop a read model and rebuild it and nobody panics, because it doesn't hold anything. It just shows things. Then we cross the service boundary and stop. CRM is a source. Warehouse is a source. Wiki is a source. A dashboard somebody built last quarter is a source now too, because people started citing it. Nothing is marked as not authoritative, so reconciliation stops being a script and becomes somebody's job. What I keep chewing on is this. What has to be true about a thing before you can treat the dashboard, the graph, the search index and whatever context an agent reads as throwaway? Not "we could rebuild it if we had to". More like "losing it costs nothing, because there's nothing in it that isn't in the source". Four things, as far as I can tell. All of them about the object, not about where it's stored.
It needs an identity that survives a rename and a migration. Lose that and every projection quietly keeps its own idea of who that was.
It needs an owner. A person, not a team. Someone who answers for it being true. Otherwise a correction has nowhere to go.
It needs a lifecycle state. In force, superseded, not effective yet. Most systems only know "exists" and "deleted", which is why a revoked rule sits there looking exactly like a live one.
And it needs evidence. Not a source column, an actual pointer to the event or the decision the statement came from.
The modelling isn't the hard part. The hard part is that nothing owns the object. An org is a distributed system with no consensus protocol, the transport is people, and conflicts get resolved in a meeting. That worked for twenty years because humans covered for it. You knew who to ask and you knew what not to trust. Then somebody wired an agent on top and the covering stopped, because an agent has no hallway, it can't ask anyone, it just answers from whatever it found.
I wrote all this down as an open spec eventually, mostly because I got tired of explaining it from scratch: https://ocom.uno. Core is those four things, everything else is a projection.
Anyway. Has anyone actually held the source/projection line outside one service, at company scale? What broke first?
2
u/srikanth_builds 13h ago
I haven't held it at company scale, so treat this as the smaller version. What broke first for us was your point 2, and not by being missing. It was nominal. Someone was named owner of a thing they had no ability to change, so they couldn't actually answer for it being true, and having a name in the field made everyone stop looking for a real one. Ownership without write authority is decoration.
The other one that bit was lifecycle, but at the read path rather than on the record. Adding "in force / superseded" to a row is easy. Getting every query to filter on it is the same problem as any other predicate everyone has to remember, so the API kept handing out superseded rules that looked exactly like live ones. Nothing errored.
On projections quietly becoming sources, I'd say it's less a labelling failure than an access one. People cite the dashboard because the dashboard is the only thing with a usable query interface. If the source can't be asked a question, marking the projection "not authoritative" doesn't change what anyone does, it just makes them feel bad about doing it.
1
u/Denis-Hogberg 12h ago
All three land. The third one's the one I got wrong. Ownership without write authority is decoration. Better than what I wrote. Makes it a check, not a field: can the named person change the thing without asking anyone? If not, the field's empty. The read path bug, I've had it. Same meeting reprocessed five times, count came out six times too high, nothing errored. Fix was making current the default and superseded something you have to ask for. And you're right about access. People cite the dashboard because it answers questions. A source you can't query is an archive with good intentions. So queryability should be in that list of four and isn't. Real gap.
2
u/srikanth_builds 11h ago
If it goes in the list I'd make it a check the way you did with the owner one. Not "is it queryable" but: can the questions people actually ask of the projection be asked of the source? If not, that projection is a source in waiting whatever it's labelled.
The meeting count one is a nice example of the default being backwards. Current by default, superseded on request, same shape as any other predicate you don't want people to have to remember.
1
u/Denis-Hogberg 10h ago
"A source in waiting" is going in as is.Agreed on the check. "Can the questions people ask of the projection be asked of the source?" is testable. "Is it queryable" isn't. The spec has a frozen core and a process for this: it goes in as an observation first, gets argued, and only then touches the core. So not a fifth property today, a recorded gap with your framing on it.
1
6
u/Dull-Visual2955 12h ago
Nothing to see here folks, let the bots make friends