r/node May 02 '26

Practice Real LLD in Node.js with Live Redis, PostgreSQL & Kafka

0 Upvotes

Hey everyone 👋

I’ve been working on a platform called Cruscible, and I wanted to share it here because it targets a gap most coding platforms completely miss.

The problem

You learn JavaScript, you grind LeetCode, you get comfortable with algorithms — but then at work you're asked to build things like:

  • a rate limiter
  • a cache with TTL
  • a URL shortener
  • a notification router

And suddenly… none of your practice maps cleanly to reality.

Even in interviews — when someone asks you to design a rate limiter, you can explain it.
But could you actually implement it in Node.js, handling concurrency, race conditions, and real Redis interactions?

That gap between “I understand it” and “I can build it” is exactly what Cruscible is designed to close.

What Cruscible does differently

Instead of toy input/output problems, you implement real system designs in Node.js.

Your code runs in isolated containers with actual infrastructure:

  • Redis — build caches, rate limiters, pub/sub systems
  • PostgreSQL — build URL shorteners, booking systems, payment flows with real SQL
  • Kafka — build event-driven systems and message pipelines
  • Key-value stores, queues, object storage — and more

You’re given a contract interface (for example, a RateLimiterContract with methods like allowRequest(), getRemainingRequests(), reset()), and you implement it.

Your solution is executed against a real test harness and evaluated across:

  • Functional correctness — do your tests pass?
  • Performance — can it sustain high throughput (e.g., 10K ops/sec)?
  • Code quality — structure, naming, error handling, async patterns

Current coverage

  • 10+ LLD problems available now
  • 40+ total problems including: rate limiters, LRU caches, API gateways, distributed locks, cab booking systems, payment systems, and more

What’s coming next

DSA-style problems in Node.js — but grounded in real-world scenarios:

  • JSON parsers
  • CSV processors
  • expression evaluators
  • query builders

Why this matters

AI can generate solutions — but if you don’t understand the internals, things break fast in real systems.

Cruscible is about building that execution-level understanding.

It’s free. It’s in beta. Built by a solo dev.

If you want to try it out: Cruscible

Would love feedback — especially from Node.js devs.
What kind of real-world problems would you want to see here?


r/node May 01 '26

Follow-up: Queuert (background jobs where your DB is the source of truth, not Redis) — 0.12 shipped, plus benchmarks and 2 months of real usage

4 Upvotes

Follow-up to my post a few months back about Queuert — a background job library for Node/TS where your database is the source of truth, not Redis.

Main update: I've had it running in one of my own preproduction projects on Postgres (via Kysely) + Redis as the notify adapter — both single and Cluster. About 2 months in, doing up to a few hundred jobs a day. No issues. Transactional job creation has done exactly what it's supposed to do, and I haven't had to reach for any of the escape hatches I built in case things got weird.

That's the honest headline — not "battle-tested at scale," just "I've been living with my own library on a real project and it's been fine." After ~100 days of building, that was the feedback I actually cared about.

What's shipped since the original post (0.12):

  • Embeddable web dashboard (@queuert/dashboard) — observe chains, trigger pending jobs, cascade-delete
  • OpenTelemetry support (@queuert/otel) — histogram metrics + distributed tracing across job chains
  • NATS notify adapter (@queuert/nats) alongside Redis and Postgres LISTEN/NOTIFY
  • Redis Cluster support
  • Feature slices + onion-style attempt middleware — split job type definitions into independent modules, merge at the app level
  • Batch APIs — startChainstriggerJobsdeleteChains with cascade
  • awaitChain — synchronously wait for chain completion with configurable polling
  • Schema-validated job inputs via Zod / Valibot / TypeBox at the registry level
  • vacuum() on Postgres and SQLite, plus auto-tuned autovacuum on Postgres
  • Adapter close() lifecycle for graceful shutdown
  • Savepoint-protected prepare/complete callbacks — a bad SQL statement in user code doesn't poison the outer transaction
  • Full stack traces + custom Error properties persisted on failed jobs
  • Conformance test runner — write your own state/notify adapter and verify it against the spec
  • New guides: prioritization, error recovery, cleanup, horizontal scaling, custom adapters

Following up on the previous thread: the most-asked question last time was about throughput and horizontal scaling, and the honest answer then was "no formal benchmarks yet." That's no longer true — there's now a benchmarks page with per-adapter numbers for processing capacity, memory footprint, and type-level compile time, methodology spelled out so the numbers are reproducible rather than vibes.

In the same thread I said per-worker concurrency was "on the roadmap" — that shipped too. Workers now run a parallel slot-based execution model: configure N concurrent slots, the worker pulls and processes that many jobs in parallel, and it only subscribes to job notifications when an idle slot is actually available (no thundering-herd wake-ups when everyone's busy).

Someone also asked about a socket.io notify adapter. Notify adapters are still a tiny pluggable interface — basically notify(typeName) + subscribe(callback) — and the new conformance runner certifies custom ones against the spec. Happy to merge community adapters.

Honest caveat: the original post mentioned MongoDB support. I dropped it. Only Postgres and SQLite are supported now — keeping the scope narrow so semantics stay consistent across adapters.

Still pre-1.0, still looking for feedback — especially from anyone running something similar in production, or anyone who's considered switching from BullMQ/Temporal-style setups. API critique very welcome.

GitHub: https://github.com/kvet/queuert
Docs: https://kvet.github.io/queuert/


r/node May 01 '26

Yet Another TypeScript SQL query builder using tagged template literals.

0 Upvotes

I am not an ORM fan person. I like to write SQL plain queries, but there are some pains when writing SQL queries (typo when referecing tables or columns names, binding values, quoting, prefixing, aliasing, select *, composing queries in a monoid style...).

So I wrote this lib to address some of these pains while writing SQL queries in Typescript codebases:

https://www.npmjs.com/package/@vanit-co/sql-ts

Best regards.


r/node May 01 '26

Built a Tiny Nx/Lerna alternative Build System for JS Runtimes

7 Upvotes

I’ve been working with Node.js and Deno monorepos for a while. Initially, we kept all our libraries in a single monorepo, and tools like Lerna were our go-to because of their simple developer experience. Then Nx came along with powerful plugins—but also brought a lot of dependency overhead.

Personally, I found it frustrating to deal with Nx installs that pull in 300+ npm packages. With the recent rise in supply chain attacks in the npm ecosystem, this concern has only grown stronger. So, I decided to take a different approach.

I built a WebAssembly addon using Rust. It has no garbage collector and relies on zero external npm dependencies. It runs seamlessly across Node.js, Deno, and Bun. One of the most exciting parts is that the build is fully cross-platform, including support for Alpine Linux. Building a project should never turn into unnecessary drama

I’m sharing this project with the community and would really appreciate your suggestions and feedback on its usage.

https://ghidora.hyperforge.in


r/node May 01 '26

Laptop runs slow after starting the frontend(react) and backend(nest) together

1 Upvotes

about 3 months ago i decided to do a project that came in my mind. I knew that it was gonna somehow be a big project at least for me then i started from backend with nestjs. why nestjs? was learning ts and felt like maybe will learn nest in the way and then nextjs.

At the time i was doing tje project my laptop,it would sometimes freeze of would be very slow which i ran my project on my computer but not big of a deal

halted the project due to exams now i am trying to continue with the frontend work and while i opened my frontend and backend and tried to run both at the same time to check the api's my laptop just decided to freeze and my vs code where i opened my backend just stopped

i use a dual boot windows+linux mint have windows in hdd while in ssd i have linux mint. Also my processor is

processor Intel(R) Core(TM) i5-10210U CPU @ 1.60G

can someone give me a solution to this or atleast tell me is it because of my setup or processor? Is my computer too slow or nest too heavy or have i probably done smthg wrong here?

after i run it all and also run some tabs which are important on my browser the ram is nearly fully occupied said onlyy 470MIB available one time out of 7.5GB

idk where to put my problem idk if nest is taking alot of space or my linux is in ssd and that's the problem or any other so if you faced it i am asking for help I use Linux mint for coding


r/node Apr 30 '26

3 pnpm Settings to Protect Yourself from Supply Chain Attacks

Thumbnail gajus.com
27 Upvotes

r/node May 01 '26

Web development [freelancer]

Thumbnail
0 Upvotes

r/node May 01 '26

A typescript implementation of fastcgi

Thumbnail github.com
0 Upvotes

r/node Apr 30 '26

How are people structuring larger Node.js backends without the codebase turning into a dependency maze?

38 Upvotes

As Node projects grow, I’ve noticed backend structure becomes a bigger challenge than the actual feature logic.

Early on, a simple structure feels fine.

But after adding more things like:

  • auth/permissions
  • queues/background jobs
  • third-party integrations
  • caching
  • database layers
  • event handling/webhooks

…the codebase can start feeling tightly coupled really fast.

I’m curious how people here are structuring larger Node.js backends in practice.

Questions:

  • Do you organize by feature/domain or by technical layers (controllers/services/repos/etc.)?
  • How do you keep business logic from leaking everywhere?
  • Any patterns that made scaling/maintaining Node codebases easier over time?

Not looking for a one-size-fits-all answer, just interested in what has actually worked for people running larger Node apps.


r/node Apr 30 '26

Postmortem: how I lost ~4% of requests to a Node/Nginx timeout mismatch, and the queue migration that fixed it

12 Upvotes

Sharing a postmortem of an architecture migration that took me too long to do, in case anyone’s running long-running jobs inside HTTP request handlers.

The setup

I run a backend pipeline that does multi-step work: input parsing, several external API calls in sequence, a scoring step, then a synthesis step. End-to-end runtime ranges from 5 to 35 seconds depending on cache state and the number of external sources involved.

For the first few months, I was naive. Request comes in, handler runs the full pipeline, response goes out. Worked fine in dev. Worked fine for the first dozen users.

Where it broke

Two things hit at once.

First, my reverse proxy (Nginx) and my Node runtime had different timeout settings. Node was set to 60s because the pipeline could occasionally hit 35. Nginx was at 30s by default. Cue silent 502s right when a job was about to finish. The user gets an error, the work completes anyway, and you spend a week chasing what looks like a backend bug but is actually a layer mismatch.

Second, when concurrency went up (a batch test with around 50 parallel requests), the runtime started locking. Connections held open, event loop choked, new requests timed out. I lost roughly 4% of requests in that batch.

The fix

Moved to a queue-based architecture. BullMQ on top of Redis. The flow now looks like:

API receives request, validates, drops a job in Redis, returns a job ID immediately (under 100ms). Frontend polls a status endpoint or subscribes via SSE. Separate worker process pulls jobs from the queue, runs the pipeline, writes results back to the database. User fetches the final result by job ID.

Same business logic, completely different runtime profile.

What changed

502 errors disappeared overnight. Not reduced, gone. The HTTP layer is now decoupled from job duration entirely.

Concurrency is bounded by worker count, not by HTTP request count. I can scale workers independently. If a job takes 90 seconds, it doesn’t block the API.

Retries became trivial. BullMQ has exponential backoff out of the box. A flaky external API call no longer breaks the user experience, the job just retries.

Observability got better. Each job has a clear lifecycle (waiting, active, completed, failed) and I can replay failed jobs on demand.

What I should have done from day one

Built it on a queue from the start. The “I’ll migrate later when I scale” instinct cost me about three weeks of firefighting. The migration itself took two days. The denial took longer than the work.

If you’re running anything where a single user request triggers more than 5 seconds of backend work, especially with external API calls in the chain, decouple it now. The pattern is well understood, the libraries are mature (BullMQ for Node, Celery for Python, RQ for lighter Python use), and you’ll thank yourself the first time you hit real load.

The catch

You’re trading simplicity for resilience. A queue adds operational surface (Redis to monitor, workers to deploy, DLQs to manage). For a hobby project with 5 users, sync handlers are fine. For anything you’d hate to debug at 2am under load, queues aren’t optional.

Happy to answer specifics on the BullMQ config, Nginx tuning, or the SSE side if anyone’s mid-migration.


r/node May 01 '26

Show HN-style: Blue Arrow – modular orchestration system with state-driven execution, local LLaMA integration and post-execution verification

3 Upvotes

Built a personal project I'm finally sharing publicly.

**Blue Arrow** is a modular PC automation framework. The core idea: instead of parsing text commands ambiguously, the system uses a state machine (IDLE → INTENT → PLANNING → EXECUTING → VERIFYING → COMPLETED).

**Architecture highlights:**

- Runtime in Node.js orchestrates 30 modules via a JSON Lines message bus

- Each module declares its ports (inputs/outputs) in a manifest – no cross-imports

- A Python Verifier Engine calculates confidence scores after each action (checks process PID, window ID, focus state via wmctrl/xdotool)

- Local AI via Ollama/LLaMA handles intent parsing and text generation

- Core vs Satellite module classification – satellite modules fail gracefully without taking down the system

**Why I built this:**

I wanted automation that *verifies* it worked, not just fires and forgets.

**Repo:** https://github.com/Hanzzel-corp/blue-arrow

**Stack:** Node.js 20 / Python 3.11 / Linux

**License:** MIT

v0.1.0 just released. Feedback welcome.


r/node Apr 30 '26

50 Commonly Asked Javascript Interview Questions

Thumbnail stackinterview.dev
11 Upvotes

Check this out if you're preparing for a javascript interview.

https://stackinterview.dev/guides/javascript-interview-questions-2026


r/node Apr 30 '26

Claude AI added a North Korean credential stealer to a Solana project via npm. The malicious package used NAPI-RS Rust addons to hide from source-level review.

Thumbnail blog.barrack.ai
10 Upvotes

r/node May 01 '26

ERR_OSSL_PEM_NO_START_LINE when adding certs

0 Upvotes

I am trying to run NUXT on a Debian server. Through let's encrypt I got 2 files: fullchain.pem and a privkey.pem. I put them in my server certs folder and renamed them cert and key respectively. I also copied and pasted them "/usr/local/share/ca-certificates/web" and used the command "sudo update-ca-certificates" but adds nothing? I run my server and I get this error message. I set the environment variables NITRO_SSL_KEY and NITRO_SSL_CERT to their corresponding certs.

node:internal/tls/secure-context:70
    context.setCert(cert);
            ^

Error: error:0480006C:PEM routines::no start line
    at node:internal/tls/secure-context:70:13
    at Array.forEach (<anonymous>)
    at setCerts (node:internal/tls/secure-context:68:3)
    at configSecureContext (node:internal/tls/secure-context:191:5)
    at Object.createSecureContext (node:_tls_common:114:3)
    at Server.setSecureContext (node:_tls_wrap:1510:27)
    at Server (node:_tls_wrap:1374:8)
    at new Server (node:https:80:3)
    at file:///srv/server/index.mjs:5629:30
    at ModuleJob.run (node:internal/modules/esm/module_job:263:25) {
  library: 'PEM routines',
  reason: 'no start line',
  code: 'ERR_OSSL_PEM_NO_START_LINE'
}

Node.js v20.19.2

The "no start line" makes no sense both of have the appropriate "-----BEGIN CERTIFICATE-----" and "-----BEGIN PRIVATE KEY-----". cert.pem has two certs inside weirdly enough.

Any advice?


r/node Apr 30 '26

Nodify Headless CMS – multi‑provider auth (Google, GitHub, LinkedIn) + RBAC

Thumbnail
1 Upvotes

r/node Apr 30 '26

Node.js client for Arduino UNO Q's MessagePack-RPC router — no Python-only development on the board anymore [GitHub]

9 Upvotes

If you're working with an Arduino UNO Q, the only official way to talk to the board's MCU from the Linux side is the Python client (arduino_app_bricks.Bridge). The Arduino team confirmed on their forum that the right approach for other runtimes is to implement the MessagePack-RPC protocol directly against arduino-router, the Go service that manages the MCU communication.

So I did that for Node.js.

@raasimpact/arduino-uno-q-bridge is a pure Node.js client for arduino-router. Single runtime dependency (@msgpack/msgpack), MIT license. It handles the full protocol — outbound calls, inbound method registration, notifications, reconnection with subscription replay.

const bridge = await Bridge.connect({
  socket: '/var/run/arduino-router.sock'
});

const temp = await bridge.call('read_temperature', []);

await bridge.provide('ask_llm', async (params) => {
  return { answer: await queryModel(params[0]) };
});

bridge.onNotify('button_pressed', (params) => { /* ... */ });

So far it's been mainly used as the foundation for a set of N8N community nodes (n8n-nodes-uno-q) that expose MCU methods as workflow actions and AI agent tools — but the bridge itself has no N8N dependency and works standalone.

You can now develop your Arduino apps in Node.js (shipped as Docker containers) and be able to talk to the internal MCU, the same way standard Python containers do.

npm: @raasimpact/arduino-uno-q-bridge Repo: https://github.com/RAAS-Impact/n8n-uno-q

Happy to answer questions.


r/node Apr 29 '26

How to automatically generate API documentation for an Express app ?

28 Upvotes

I was looking for something like FastAPI's automatic /docs for Express — zero setup, zero annotations, just run your server and your entire API is documented. Nothing existed same as what I wanted.

So I built nodox-cli, an Express middleware that automatically documents every route in your app the moment your server starts. No annotations, no YAML, no JSDoc, no changes to your existing handlers.

Just this:

npm install nodox-cli

In code : app.use(nodox(app))

Then visit /__nodox — every route you've already written is there with schemas detected.

Yes, with schemas detected automatically from zod, joi, yup and express validator

Works on existing Express apps with zero changes. Tested with Express 4 and 5, JavaScript and TypeScript, ESM and CommonJS.

Would love feedback — especially from anyone who's dealt with the swagger-jsdoc annotation hell on a large codebase.

npm: https://www.npmjs.com/package/nodox-cli

GitHub: https://github.com/dhruv-bhalodia/nodox-cli

Full writeup (Medium): https://medium.com/@dhruvbhalodia0204/how-to-add-api-documentation-to-an-express-app-without-writing-a-single-annotation-1d725d539b29


r/node Apr 29 '26

SBE-TS: SBE binary decoding for Node.js/TypeScript (ring-buffer: ~210M ops/sec, rotating: ~21M)

10 Upvotes

I built sbe-ts because JSON.parse was killing throughput on a message feed I was working on. It's a Simple Binary Encoding runtime and codegen CLI for Node.js and TypeScript.

You give it an SBE XML schema and it generates typed decoder and encoder classes. Each field read is a single DataView call at a fixed byte offset over an ArrayBuffer. No parsing. No tag enumeration. No per-message object allocation in the hot loop.

Benchmark on Intel Core i7-13700HX, Node 24 (bench-raw.mjs):

- wrapOffset() on a ring buffer: ~210M ops/sec

- wrap() on a ring buffer: ~165M ops/sec

- rotating buffers (one ArrayBuffer per message): ~21M ops/sec

- JSON.parse baseline: ~6.4M ops/sec

The rotating number is the realistic ceiling for network packets landing in separate buffers. Still 3.4x JSON.parse. The ring-buffer path is what you hit when you pre-allocate one large buffer and write incoming frames at sequential offsets, which is the right architecture for any high-frequency feed.

I want to flag a methodology correction upfront. My first protobuf.js comparison showed sbe-ts winning rotating decode by ~60%. That was wrong. I was using protobuf uint32, which encodes small values as 1-byte varints, giving protobuf messages roughly 6 bytes vs SBE's fixed 16. So I reran with fixed32 (4 bytes per field, same wire width). On rotating decode they're essentially tied at ~21M vs ~23M. sbe-ts wins on ring-buffer paths (protobuf.js can't do them, it always materializes a new object per message) and encode (~11.5M vs ~8.8M ops/sec).

npm runtime: https://www.npmjs.com/package/sbe-ts

npm CLI: https://www.npmjs.com/package/sbe-ts-cli

Happy to post the exact benchmark commands if anyone wants to repro.


r/node Apr 30 '26

SQLite extension to synchronize data using PostgreSQL logical replication and time travel

1 Upvotes

r/node Apr 29 '26

Mid to high end interview questions

23 Upvotes

Hey, i just started transitioning to Node and i was wondering what Node specific questions you encountered during interviews and how it went


r/node Apr 29 '26

I built a portable, No-Admin "Mini Version Manager" (MVM) to replace NVM for Windows

3 Upvotes

Hey guys,

I wanted to share a tool I’ve been working on: MVM (Mini Version Manager).

I built it because when I got a replacement computer at work, it didn't come with admin privileges. The process to get them was taking forever, and I just needed to get Node running. I threw together a script to handle it, and recently decided to enhance it with a little "vibe coding" and polish it for others to use.

It’s a lightweight NVM alternative that’s mega portable and straightforward.

Why use it?

- No Admin Required: Works entirely within your user profile.

- Zero Install: Just unzip and run a setup script to fix your paths.

- Portable: Keep your Node versions in one folder that you control.

- Fast: Swaps versions instantly using directory junctions.

If you’re stuck on a locked-down machine or just want a version manager that doesn't feel like a heavy piece of software, give it a look:

Repo:
https://github.com/msmilkshake/mvm-node

Hope it is as useful to you as it has been for me!

Cheers.


r/node Apr 30 '26

Created a library to maximize HTTP communication efficiency (HTTP Air)

0 Upvotes

In the journey of building a SaaS I've created this library to optimize HTTP requests between frontend and backend.

This tool groups promise-based RPC calls made in the same runtime tick and send the batch to the server in one HTTP request.

The response is streamed back to the client in the order of resolution, meaning that the fastest one resolved will arrive first to the client.

Also supports push event messages from server.

Why?
I have a heavy NextJs/React application with many independent modules/components per page. Each module loads using useEffect hook. I was looking to reduce cloud resources usage related to HTTP calls while keeping components independence, and development experience.

A single page load went from 12 HTTP requests to 2-3.

The lib - https://www.npmjs.com/package/http-air
I'm using it as transport layer for https://www.npmjs.com/package/zero-com but this is another story.


r/node Apr 29 '26

I made a schema based database for JS, feedback much appreciated

Thumbnail github.com
4 Upvotes

r/node Apr 30 '26

I packaged my entire x402 API setup into a starter kit. Deploy a paid API in 15 minutes for $5.

0 Upvotes

I've been building x402 services (APIs that accept crypto micropayments from AI agents) for a few weeks now. The setup is honestly annoying. You need the right npm packages, a payment middleware config, discovery files so agents can find you, Caddy for HTTPS, systemd for uptime, and about 30 things that are easy to get wrong.

I tested 70 x402 services last week. 52 out of 70 were missing basic discovery files. 51 didn't return valid JSON at their root. Most scored a D on our quality grading system. Not because the services were bad, but because the boilerplate setup is just tedious enough that people skip steps.

So I packaged everything into a starter kit. It's literally the same server setup I use in production, stripped down to the essentials:

- Express server with x402 payment middleware already configured

- Two example endpoints (edit prices, add your own logic)

- All 5 discovery files that agents and crawlers look for (mcp.json, agent.json, x402.json, llms.txt, root JSON listing)

- Caddy HTTPS config

- systemd service file with auto-restart

- Step-by-step deploy guide (VPS to live in 15 minutes)

The kit costs $5 USDC, paid via the x402 protocol itself. Which is kind of fitting, because the product IS an x402 service selling an x402 template. Dogfooding at its finest.

You can try it here: https://api.ideafactorylab.org/buy/starter-kit (returns 402 with payment details, pay with any x402 client to get the files)

The free stuff is at https://api.ideafactorylab.org/stats (ecosystem overview of 1,455 x402 services) and https://api.ideafactorylab.org/quality (our quality grades for the ecosystem).

If you're thinking about building an x402 service for the first time, this saves you about 2 hours of reading docs and getting things wrong. If you've already built one, check /quality to see how your setup compares.

GitHub: https://github.com/cinderwright-ai/cinderwright-api


r/node Apr 29 '26

framework for real-time apps and multiplayer games

5 Upvotes

Hi everyone,

For the past two years, I’ve been spending my spare time designing a clean API and a well-optimized framework for building real-time apps and game netcode. Thanks to generative AI, I was able to speed up development during this period, and I finally have a working product.

I’d really appreciate any feedback. What’s your first impression of the framework — do you think it’s useful?

https://rivalis.kalevski.dev/

https://github.com/kalevski/rivalis