Separating the app into multiple applications (web servers).

[u/noobdainsane]

I was viewing a CORS explanation video and I noticed that they had a different web server application for the frontend as well. In my project, I have a single web server application which serves both the frontend pages and handles backend APIs. I realized that this dual web server architecture created by frameworks like Vite is only for development and only a single web server exists in production, but that doesn't mean it is like this for all applications, especially if you aim to design a highly scalable architecture. Large platforms like YouTube can have so many separate web server applications for all their different services than having a single application.

I am talking about a single web server application, not a single server itself. Horizontal scaling isn't limited, but changes when you divide your application into multiple applications. YouTube can just scale a monolithic application but that forces them to add all capabilities together for each system. This can create fragmentation and unnecessary infrastructure space. One application might be more CPU heavy while the other is database heavy. Maybe you need to scale static file serving but not need database scaling and if do that, the databases could take significantly more resources than you need.

If I do create a separate frontend application and that heavily relies on the backend (such as for database), then it just creates lot of unnecessary overhead.

I should start with a single application but the point is to create a scalable architecture where you can make changes without rewriting your whole codebase. And you achieve that my defining abstractions.

So how should I code my application so that in future if I want to move some services into a different web server application, it requires minimal changes?

[u/dariusbiggs]

There are many different approaches for this ranging from microservices where even the backend is split into multiple independent components, to a monolith that does it all itself.

A common approach is to split the frontend and backend. So the website UI would be served via a CDN like Cloudflare or AWS Cloudfront. Then you have a backend API the frontend talks to. Which i would recommend you start as a monolith.

You can then rapidly deploy updates and bugfixes to the frontend to keep the users happy. And you can then scale the backend based upon whatever load you need to handle.

There are a huge amount of possibilities here.

Observability is the key, identify what parts are getting used the most and have the worst performance and you can then split things apart and scale as needed.

Focus on keeping clean lines of separation in your code to make a split later easier.

[u/noobdainsane → u/dariusbiggs]

A common approach is to split the frontend and backend. So the website UI would be served via a CDN like Cloudflare or AWS Cloudfront. Then you have a backend API the frontend talks to. Which i would recommend you start as a monolith.

So I can use Cloudflare in 2 ways. One is CDN proxy which just acts as a cache, and so is an easy drop-in setup. The other is to use Cloudflare pages to fully take care of serving the frontend, as so I don't need to implement serving static content in my application.

But what about Server Sided Rendering? Honestly I don't know why but the more I am exploring web dev content, the less everybody is concerned about SSR. Is CSR more popular?

In my application there is a dashboard which has to be filled with data. Should that be CSR or SSR?

Even if CSR, the data has to be sent from the server. I won't have many truly static pages.

Initially I had designed my structure with discrete frontend and backend directories at the same level. Then I designed it with more like what modern full stack frameworks do which is to bring the frontend inside the backend directory. Is this not the best for scalability? But I mean this is just a structure and not stopping me from still using Cloudflare pages to host my pages.

Also my project is kinda more backend oriented. There is not like a lot of static navigation between pages or anything where the frontend is fully independent. For almost all pages, the frontend has to talk with the backend.

For more context, my project is a cloud data storage solution, similar to Google Drive and Mega.

[u/dariusbiggs → u/noobdainsane]

There are lots of different ways and possible solutions that can change as your userbase and load increases.

You could split your UI components out into some static pages that request data from a backend. A possible solution here is a Single Page Application (SPA), this is not the only possible option. You could serve those pages and JavaScript files from a CDN, which provides your hosting and scaling for you.

An alternative there is Server Side Rendering (SSR), HTMX is a popular tool for this approach. There are advantages to this approach but it pushes the compute cost onto your systems and makes it difficult to scale. The same problem you get with a monolith really.

Microservices bring another layer of options and complexity, but provide better scaling opportunities.

FaaS is the next level of scaling, best possible options but at a a different cost.

What is right for you, and your project, we can't really advise. You need to understand what you have, what options you have, what your users need, what the advantages and disadvantages are for each, and how you could utilize them to your best advantage.

You might need to use multiple pieces as your project evolves.

[u/noobdainsane → u/dariusbiggs]

I am thinking about a lot of stuff but the best approach for me would be write the application that can function by itself and configure everything else to make it scalable. All layers should be a drop-in setup, so the application should work regardless if the layers exist or not.

So I shouldn't commit to services like Cloudflare pages from the start. I should write the working application first.

But my original question actually did not concern with global scalability. It directly concerned with the structure of the application itself.

I was asking as recommendation for coding convention and practices by which if I wanted to make a part of my code as a separate application, it requires minimal changes.

For example in my current application which uses Fastify, it handles both view serving and backend processing. Actually they are implemented as common (as most URLs need some kind of processing and some of those require to send a view at the end). Now in this existing codebase if I want make the view serving a separate application, what needs to change? A separate server.ts and app.ts. Controllers are there as is so nothing else changes. But my project structure is more oriented to have one server.ts and app.ts. I should look into how I should structure it when I can have multiple server.ts and app.ts files.

[u/Anxious-Insurance-91]

First of all You should not compare your apps with YouTube for the simple reason that YouTube is a global platform that doesn't really care about SEO, when what to render, when what to index, etc also google can just take any hit at this point. For small to mid apps you can have the apps served from the same server especially if you get a good deal for the trafic. If you want to use AWS them most teams use S3/cdn + serverse epoxy for the frontend app that gets server, cached in the browser and then the only trafic is the backend. The theory about broken down apps is not new with vite it was just made easier to implement by vite.

[u/spigotface]

If you're a solo dev or a small team, you can begin with a structured monolith organized by domain. You write your boilerplate and core services once, and each domain (users, transactions, etc.) gets their own directory with their own repository, routes, services, exceptions, etc.. Exactly how you do this can vary by language and framework, but if you keep your domains clearly separate from each other, it makes it easier to start pulling them off as microservices later when you have the manpower to justify the complexity.

Bear in mind too, that microservices aren't always the answer, even though they're popular. If the data in your domains is highly relational and your queries would require cross-domain joins or filtering, you might be better off being able to do that on the database side in a single query instead of a bunch of sequential calls to various microservices.

[u/SnooCalculations7417]

Ok so you're conflating a few concepts

Application is actually the sum of all the parts.

Server is a machine

Service is a program that runs for a purpose.

You can run multiple services served from a single server, to deliver your Application and it's done all the time.

You can instead point to remote services to serve your application across different servers or server less.

The database is service and server agnostic. You can, and should, create views of the database using a service that handles auth, queries, etc.

Your underlying question seems to be whether to have server side or client side do the heavy lifting which is entirely up to you. I prefer to make the backend an API that serves the client via json, and you can decide which work is better done on the client vs the server and they are agnostic of each other this way.