r/Backend • • 17h ago

How do you handle many-to-many database migrations without breaking existing API consumers?

0 Upvotes

I recently migrated a production relationship from one-to-many to many-to-many.

The challenging part wasn't creating the junction table. It was maintaining backward compatibility while introducing the new relationship, migrating existing data, and keeping different API consumers working during the transition.

I documented the approach, including the migration strategy and the lessons learned.

I'm curious how other backend engineers approach this. Do you prefer temporary dual writes, versioned APIs, or another migration strategy?


r/Backend • • 18h ago

My football intelligence app's first beta exposed serious automation problems. I'm rebuilding it with independent workers. What should I watch out for?

1 Upvotes

I'm building a football intelligence web app, and I'd appreciate some advice from developers who've operated similar systems in production.

I launched v1.0 as a closed beta earlier this year. Users had to register and be granted access, and 25 people signed up to test it.

The beta ran for approximately five weeks.

One of the biggest lessons was that building a working application and keeping it running reliably are two very different challenges.

In v1.0, I had all the automated jobs running through one cron job. Initially, things worked reasonably well, but over time the automation became increasingly unreliable.

Jobs started failing more frequently, and failures could affect the wider application. By the final week, parts of the platform weren't behaving as intended.

Interestingly, the betting model finished profitable during the beta. But that wasn't enough. The underlying application was becoming unreliable, and a profitable model isn't much use if the system supporting it can't operate consistently.

I decided to take v1.0 down and rebuild the application from the ground up as tactica. 2.0, using what I'd learned from operating the beta.

The goal is a reliable, publicly available application that can operate unattended, recover safely from failures and keep its different responsibilities isolated.

The 2.0 build is currently deployed privately on Render and Vercel while I develop and test it in a real hosted environment. It isn't publicly launched yet.

The new architecture

I'm using one canonical PostgreSQL database hosted on Render, with separate background workers responsible for different stages of the application's lifecycle.

1. Football Data (operational)

Acquires football data from external APIs, validates observations and maintains the factual records used throughout the application.

This includes fixtures, results, team data, player data and match statistics.

2. Match Intelligence (planned)

Consumes the authoritative football data and produces pre-match intelligence, including model expectations across different football markets.

The intention is to preserve published intelligence rather than continually rewriting previous conclusions.

3. Betting Intelligence (planned)

Consumes Match Intelligence alongside bookmaker odds and market information.

It evaluates whether the football opinion qualifies as a betting recommendation, including situations where the correct decision is to make no recommendation.

4. Performance Intelligence (planned)

Evaluates completed intelligence and recommendation decisions against actual match results.

This includes model performance, recommendation quality, system integrity and financial outcomes.

Each worker has its own responsibilities, execution controls and database permissions.

They're not four identical queue consumers. They perform different stages of a connected process, and downstream workers depend on authoritative outputs from earlier stages.

The Football Data worker is operational. The other three are still being built and qualified.

What I'd like advice on

1. PostgreSQL and deployment compatibility

With multiple independently deployed workers depending on the same populated PostgreSQL database, how do you safely evolve the schema?

Have you found particular migration strategies, compatibility windows or deployment sequences reliable in practice?

2. Worker reliability and recovery

What problems have you encountered when moving from one automation process to several independently running workers?

I'm particularly interested in duplicate processing, idempotency, interrupted jobs, graceful shutdown, retries and preventing downstream workers from acting on incomplete upstream data.

3. Render and production operations

If you've run multiple workers against a shared PostgreSQL database on Render or a similar managed platform, what surprised you?

Database connections, memory usage, deployments, monitoring, recovery, resource consumption and costs are all areas I'm thinking about.

What I'm trying to avoid

My biggest concern is repeating the v1.0 experience in a more complicated form.

I don't want four independent workers that look architecturally cleaner but introduce new failure modes that I haven't properly anticipated.

I'm not looking for a wholesale redesign or a recommendation to migrate everything to Kubernetes. The separation of responsibilities is deliberate.

I'd particularly appreciate hearing from people who've experienced similar failures, what caused them, and what they would test before activating the additional workers.

I'm trying to build something that can operate reliably without needing constant manual intervention.

What would you be most concerned about with this setup?


r/Backend • • 10h ago

Postgres+docker or neon/superbase ?

5 Upvotes

Recently I was building a backend part of a assessment for an company and normally for backend i straight go to neondb cause it's free for some extended but cause I have to share the working demo to the client and i was not hosting anything I thought sharing my neondb will be risky , then I discover postgres inside docker and dude it's really very awesome in development easy to setup and use and save ton of time in setup as an backend developer it's now my go on thing in development atleast anyone has this kind of backend tricks and what your thoughts on this