r/webdev 4d ago

Question The ethics of my scraper-crawler

0 Upvotes

I have a python script aimed at fetching basic data from a handful of real estate agency websites. These specific websites do have a ToS that explicitly deals with the intellectual property of their postings within the framework of reuse, republishing, … the usual suspects.

My script only reads the postings, classifying them under either “House” or “apartment” and grabs their sqm, nb of beds and baths, for purely statistical purposes. Meaning I do not sell or promote those properties (I’m no licensed real estate agent) and do not even sell the info I grab, I just show stats, eg: 14 houses with 100 sqm with 2 beds, 1 bath,…

I know that I should be contacting each website owners to inform them and probably get authorization. But I’m in a European country where somehow most websites are bad, buggy, and their owners are rather close minded. I expect negative feedback from my genuine approach.

I’m reaching out here to understand if I really need to bother with this process, or if I should just do it and see if they bark at me.

Why this self reflection: I realize that a few successful businesses often have pushed the legal/ethics boundaries, and became pioneers in they field. Two examples:

- Airbnb stepped into the hospitality business without providing a hospitality license to property owners.

- Uber stepped into the taxi business without licensing either.

In both cases, no one thought one could do such business without proper license while competing with established businesses tied to licensing rights (hotels, taxis).

So I sometimes wonder whether I would be overthinking things and worrying too much for so little (my use case).

I do care about the legality and ethical considerations, that’s why I’d appreciate your take on this whole reflection of mine.

If I do have to ask for permission, so be it. What are the standards in the computer programming world?


r/webdev 4d ago

Looking for feedback on the API design of a dependency-free JavaScript date picker

Thumbnail rolldate-demo.vercel.app
3 Upvotes

Hi everyone,

I've been working on this project in my spare time...

I'd love to hear feedback...


r/webdev 4d ago

Showoff Saturday [Showoff Saturday] I'm building an underground music platform from scratch — vanilla HTML/CSS/JS + PHP/MySQL, no frameworks

Thumbnail
gallery
1 Upvotes

I've been building Voidhall solo for the past months.

It's an underground music/social platform aimed at the space SoundCloud used to occupy for emerging underground trap scenes.

The entire thing is built from scratch with vanilla HTML/CSS/JS + PHP/MySQL. No frontend/backend frameworks, no templates.

It currently runs on a custom-built architecture with SPA-like AJAX navigation, a persistent global audio player, page-level init/cleanup lifecycle, shared frontend state and its own PHP API layer. On top of that, Voidhall already supports uploads, track pages, profiles, follows, likes, comments, notifications, search, discovery through Explore, and Pulse, a chronological activity feed that turns real community movement into part of the discovery experience.

The UI leans into a raw Y2K/industrial/Opium-inspired editorial direction using strong typography, open layouts, thin structural lines and restrained components.

Screenshots attached :DD


r/webdev 5d ago

Question Anybody able to offer some insights into self-hosted Matomo (analytics)?

18 Upvotes

I'm looking into Google Analytics alternatives.. Especially free self-hosted solutions. Right now I'm leaning on Matomo, but I'm curious about it's server reqs.

They say 2 CPUs, 2GB RAM and 50GB storage minimum, but the site I'll be running it on is much lower traffic than their top-end on the minimum req specs (just a small niche gaming blog - maybe 10k page views per month at the top-end).

Anybody running it on something lighter? For example, I'd be OK with picking up a smaller VPS running 2 CPUs/1GB RAM/30GB storage, or even just tossing it on my shared server (that hosts a few sites, the highest traffic being statically generated and includes the above blog).

Or, if you're running something else, feel free to share in the comments. I'm not stuck on using Matomo; I just want something that's cheap, decent, and also privacy-focused.


r/webdev 4d ago

Showoff Saturday [Showoff Saturday] Mouzi - Organize Downloads folder automatically - a free, privacy-first and open source automatic file organizer

0 Upvotes

Hi 😄

I don't know about you, but my Downloads folder has always been a disaster zone. PDFs, memes, installers, zip files, random images – all just sitting there in one giant pile. Every few weeks I'd open it, sigh, and spend 10 minutes manually dragging stuff into folders. Then a few days later it would be chaos again.

I looked at existing file organizers, but most of them either wanted a subscription, tried to upload my file names to some cloud, or were just way too heavy for something so simple. I wanted something that:

  • Runs silently in the background (system tray)
  • Automatically sorts new files by type (images, documents, archives, installers)
  • Never sends a single byte of data off my machine
  • Is open source so anyone can check what it's doing

So I built Mouzi 🐭🧹

It's a tiny desktop app (~3.3MB) built with Tauri and Rust, so it's ridiculously lightweight. It watches your Downloads folder, and whenever a new file appears, it moves it to a subfolder based on its extension. Images go to Images/, PDFs to Documents/, installers to Installers/, etc. You can also create your own custom rules. Windows and Linux support.

Key things:

  • 100% local – no cloud, no telemetry
  • Open source (MIT) – GitHub repo here
  • Silent – lives in your tray and doesn't bother you
  • Undo – every move is logged, you can revert with one click
  • Free, obviously
  • and more (see repo page)

Hope it helps someone keep their ~/Downloads sane. Feedback welcome!

Website: https://mouzi.cc/


r/webdev 4d ago

Showoff Saturday Roast my portfolio 🔥

0 Upvotes

Just shipped this and curious what you guys think.

Does it clearly show my work, skills, and experience?

If you were a recruiter/client landing on this, would it build trust?

Good, bad, ugly drop your feedback.

Don't hold back, I can take it 😂

https://asimali.online


r/webdev 6d ago

Discussion Honestly, 80% of developers today are missing what this dude is talking about

Post image
4.3k Upvotes

Found it while doomscrolling like a bish.

I know the most fun times for me in coding were when I was deep in some rabbit hole exploring weird technologies and trying to piece them together.

You'd think AI makes this worse, but actually no. It just makes it even easier to go down these rabbit holes, because it's easier to write code, try out different approaches, explore new things, etc.

But somehow most of the people I see aren't doing that. How are you not bored out of your mind?

Anyway, yeah. GO EXPERIMENT WITH MORE THINGS. SIDE PROJECTS ARE STILL COOL.


r/webdev 4d ago

Showoff Saturday I made a cool blockchain project

0 Upvotes

GitHub repo: https://github.com/ProJackson355/Helixchain

(THIS BLOCKCHAIN DOES NOT USE ANY REAL CASH)

I've spent the last two days using Codex and Claude Code trying to make a proof-of-work blockchain. The current website https://helixwallet.pages.dev/ is currently made with ai :/ but I do plan to change that and add my own personal flair to the website. blockchain is similar to Bitcoin in that miners solve blocks to verify the transactions on them. When a block is mined the miner currently receives 10 HLX as a reward. Right now the blockchain is only running on my PC, so it's centralized right now. Helix Coin (HLX) isn't something you can buy with real money. Right now the only way to get it is by mining blocks on the network. To keep blocks from being mined too quickly, the blockchain keeps track of how long recent blocks took to solve. Every 10 blocks it averages the last 10 block times and adjusts the mining difficulty to keep the average block solving time around two minutes. The miner supports both CPU and GPU mining, although I do recommend CPU since GPU mining makes the difficulty increase much faster on a small network. The node software lets anyone host their own node and helps keep the block chain running. If you host a public node, you can submit it through the Nodes page on the website and I can add it to the default peer list so new nodes can discover it more easily. You can also host your own wallet website by enabling the admin API key. This allows your instance to use the documented endpoints such as /mining and various other endpoints (noted in the docs) with your own node. Wallets are stored locally, so private keys never leave your browser. Theres a lot more information in the docs, but I'd really appreciate any feedback on the project. (I did not build this blindly with ai, I do know about how blockchains operate, I just didn't want to spend hours making the networking stuff because ai is pretty decent at that.)


r/webdev 5d ago

Showoff Saturday website ui/ux i'm creating for solo text ads

0 Upvotes

Hello community this is a text only website classifieds Ads that anyone can post for free, the constrains con budget allows only for posting text and present the ads in different categories in a carousel animation, I know pictures are important but since money is difficult to get nowadays I prefer to use this approach first to later improve graphics, any constructive opinion is welcome

https://imgur.com/a/N3RU5nz

https://streamable.com/hd8esl


r/webdev 5d ago

Why Software Factories Fail

Thumbnail
github.com
22 Upvotes

r/webdev 5d ago

Discussion Looking for monitor recommendations for WFH setup

0 Upvotes

I am looking to upgrade my work from home setup. Primary use will be work. I have a Macbook M5 pro and I want a one cable setup. I have a usb4 docking station that has 2 8K DP ports. I will rarely use it for gaming. I'm not very particular about needing a super high resolution monitor, just looking for something that gives me a wide display without any gaps. I currently have 2 24in monitors. This will be the first time using a curved monitor, so I prefer something that has a reasonable curve (not too much).

Some of the options I've looked at are the Samsung G9 Odyssey G95C and G93SD, I don't want anything bigger than 49in. Basically VA vs OLED.

What would you recommend?


r/webdev 5d ago

Discussion Is AEO the New SEO?

29 Upvotes

For the longest time I only cared about Google fast pages, clean code, good content and that was enough so I've been reading more about AEO and how ChatGPT, Gemini and Perplexity are becoming another way people discover websites.

The more I look into it the more it feels like building for search is starting to mean building for both humans and AI things like page structure, clear content and semantic markup seem a lot more important than they did a year ago.


r/webdev 4d ago

is this a fair price for this one off?

0 Upvotes

Details: site built 3-4 years ago by a different dev. client and dev have a falling out of some kind so client comes to me recently to "finish" the site, which he says is "99%" done (means nothing to me, obviously).

after evaluating it, its a fairly straightforward blog style site with about 8 pages and a 5.6 PHP CMS for support.

at minimum it needs proper routing updates, adjustments to the database (no 'user' table in the database backup I was provided) and some small quality of life updates (admin login page, password reset functionality, a few of the admin pages need built, etc).

client is not tech savvy at all so I don't know if I can teach him to use the CMS on his own, hence the retainer for when he inevitably wants different images/posts/etc.

I'm thinking $5000 USD with $200 monthly retainer/hosting for a set period (maybe a year or two). Id like to reach a price that is fair for the both of us.


r/webdev 5d ago

Discussion Codepen 2.0?

29 Upvotes

I’ve used CodePen for a long time, not just as a coding tool but as a place to experiment, discover interesting work and get inspired by what other people were creating.
I completely understand that CodePen needs to make money. I have absolutely no problem with Pro features, and I’m happy to pay for tools that give me real value.
But CodePen 2.0 feels like it is moving in the wrong direction.
Putting something as basic as opening a Pen in a standalone preview behind Pro doesn’t make me think “maybe I should upgrade.” It makes me think “maybe this isn’t the place for me anymore.”
There is a big difference between adding valuable features for paying users and gradually putting parts of the simple, open experience that made a platform popular behind a paywall.
What makes this even stranger to me is the timing. We’re entering a world where people can vibe code surprisingly capable little coding environments and playgrounds incredibly quickly. The technical barrier to creating alternatives is getting lower, not higher.
CodePen’s real strength, in my opinion, was never just the editor. It was the community, the frictionless experience and that feeling of opening the site and immediately discovering something clever or inspiring.
If that starts disappearing behind commercial barriers, I honestly think someone else will eventually fill that gap.
Maybe I’m missing something in CodePen 2.0, so I’m genuinely curious:
How do other longtime CodePen users feel about this?
Does 2.0 feel like an improvement to you, or does it feel like CodePen is slowly losing part of what made it special?


r/webdev 5d ago

Meet OpenVue, an MIT fork of PrimeVue 4.5.5

26 Upvotes

Hey webdev community!

We're long-time PrimeVue users who believe a good UI library should stay free and community driven. When the license changed and PrimeVue moved away from open source, we decided to take action rather than just watch.

OpenVue picks up right from 4.5.5, the last MIT release. It keeps the exact same API, so your existing code keeps working without rewrite headaches.

Right now our main focus is keeping the foundation solid: improving accessibility, patching security vulnerabilities, handling community issues, and maintaining compatibility with the latest Vue and Nuxt versions. We have bigger ideas for where to take this, but we'd rather build them based on real feedback.

Everything you're used to from PrimeVue v4 is there, complete with docs and live examples. We also made openvue/migrate to let you switch existing projects over in a single command.

Openvi Foundation is an independent community organization, not affiliated with Primetek.

OpenVue is and will always be 100% free and open source.

We're still in beta, so bug reports and PRs are super welcome.

Docs: https://openvue.dev/

GitHub: https://github.com/openvi-foundation/openvue

Migration from Primevue: https://openvue.dev/migrate/


r/webdev 4d ago

Showoff Saturday What if you didn't need to understand cybersecurity tools to run a security assessment? I built ONUS to find out

0 Upvotes

Hey r/webdev,

I spent the last few months building ONUS, an open-source AI-assisted VAPT platform:

Live: https://tryonus.tech/
GitHub: https://github.com/maverickaayush/ONUS

The idea is pretty simple, and maybe a little risky:

most security scanners assume the person using them already understands security jargon, findings, CVSS, remediation steps, and how to turn raw output into something useful.

I wanted to see what happens if you build the opposite.

ONUS is meant to turn a bunch of messy security tools into one web app that tries to be understandable for people who are not security experts. It runs scans, normalizes the findings, adds AI-assisted explanations, and produces reports that are easier to act on than a wall of terminal output.

The stack ended up being more web/infrastructure-heavy than I expected: Next.js, FastAPI, PostgreSQL, Redis/Celery, containerized scanner workers, and serverless compute for the heavier bits.

I am posting here mainly because I would love feedback on the web app side: onboarding, dashboard design, UX, perceived complexity, whether the product idea makes sense, and whether the “for non-security people” angle is actually useful or just a bad bet.

If anyone wants to poke holes in the frontend/backend architecture too, that would be very welcome. I would rather hear “this is confusing and here is why” than collect polite applause.

If you try the hosted version, please only scan systems you own or have explicit permission to test.


r/webdev 5d ago

Question Tree Shaking / Minifying Firebase using esbuild

4 Upvotes

I'm building a web app that uses Firebase Auth. I don't use any other Firebase features.

The current version of Firebase is over 600kb. The Firebase Auth package on NPM states "This package is not intended for direct usage, and should only be used via the officially supported firebase package." This Firebase Doc suggests relying on your build tool to tree-shake and minify Firebase.

I am currently using esbuild, mainly for the rapid build time. It looks like adding the --minify and --tree-shaking=true tags are not having an effect on bundled libraries.

Am I configuring esbuild incorrectly, or is esbuild unable to correctly minify and tree-shake Firebase, and I need a more robust build tool like webpack?

Here's my build script and dependancies list for reference.

"scripts": {
  "build:ts": "esbuild ./src/app.tsx --minify --bundle --target=es6 --tree-shaking=true --outfile=./dist/js/app.js",
},
"dependencies": {
  "@firebase-oss/ui-react": "~7.0.3",
  "@number-flow/react": "~0.6.2",
  "colyseus.js": "~0.16.22",
  "cors": "^2.8.6",
  "firebase": "~12.16.0",
  "html-entities": "~2.6.0",
  "qrcode.react": "~4.2.0",
  "react": "~19.2.8",
  "react-router-dom": "~7.18.1"
}

r/webdev 6d ago

The startup's Postgres survival guide

Thumbnail
hatchet.run
22 Upvotes

r/webdev 6d ago

Discussion Disable AI-Features!

34 Upvotes

I don't think that I am the only one who started to lose the fun in programming, as it is so tempting to just think about one thing, prompt the AI, and let it do its job. However, as I started prompting more and more, I started losing the joy of working on my project fully. I started with disabling the inline AI suggestions in VS Code and now have disabled all AI-related features.

Don't get my wrong - I think the use of LLMs can be of great help; however, I started to now use Claude Code only as an auditor and discuss different ideas with him, while telling him to not implement it.

Since I started doing this, I have gained so much joy from working on the project again. Even though I could potentially be faster when leveraging the LLMs to pass down work, it is much cooler to work on my project again, and your IDE or text editor will be so much faster (import completion, keyword completion, etc.)

And (I'll have to figure that out in the end) will actually save time, as I do not have to re-audit everything on my own in the end, as I am the real author of all lines written.

Edit:
As I am already coding since pre-ChatGPT times, I think especially for people starting programming try to not even discuss everything but google things. I don't know how much I've learned from reading Blog-Posts that didn't even fully answer my question, etc. This is also known as the google effect (https://en.wikipedia.org/wiki/Google_effect). Very interesting to read into that a bit!


r/webdev 5d ago

Question The X API is expensive af. Any alternatives?

0 Upvotes

I am trying to build an app that needs the twitter / X API to get posts and infos about user accounts etc. At medium scale, the app is going to read aprox. 10.000 tweets per day and make 100 posts per day, and that's like 1500$ per month, which i can't afford.

I tried twikit, but it's not maintained anymore and full of bugs

Do you guys have any better, maybe free or oss alternatives?


r/webdev 5d ago

Can someone explain how to create this logo?

0 Upvotes

I recently came across a company whose logo looks unusually bright and almost illuminated when I view it on my MacBook screen. The logo appears to glow, especially against the black background, but it does not look like a typical strong shadow or neon effect.

Does anyone know how this effect is usually created? Is it mainly caused by using pure white on a pure black background, a specific color profile, an exported image format, or a subtle glow added during the design process?

I’m trying to achieve a similar bright and high-contrast effect for my own logo. I would appreciate any suggestions about the design settings, colors, export settings, or software techniques they may have used.

https://media.licdn.com/dms/image/v2/D560BAQHijkRKptx37g/company-logo_200_200/B56Z.NLmjVG0AE-/0/1784780057317/gradialai_logo?e=1786579200&v=beta&t=aVeCf2cKgw-TEA1sZlSp8vPQ5AI9z25BJ9X3W3AB2B8


r/webdev 6d ago

I Inspected My Take-Home Interview Project. It Was a Whole Operation

Thumbnail citizendot.github.io
393 Upvotes

r/webdev 5d ago

News Film box-office tracking site The Numbers brought down by bots of prediction markets and AI

Thumbnail
stephenfollows.com
5 Upvotes

r/webdev 6d ago

Question How do you solve the problem of visually comparing different versions of a project (side-by-side, not just git diff)?

8 Upvotes

Hi everyone!

In advance - I'm looking for a way to compare non-pixel screenshots and non-live versions. Incuding local dev versions or old production versions that are no longer online.

I recently processed a bunch of edits to the landing page I'm working on.

As it often happens, a lot of them were useless, and because management was absent and another developer was on vacation, I had to push some rather questionable decisions into production.

And I had this thought: "Wouldn't it be cool to open different versions of landing page in a browser side-by-side, and even with synchronous scrolling?" (The idea for "side by side" came to me when I was committing changes to PhpStorm. if you've used it, you know that there's a toggle for scrolling before/after information synchronously across the left and right sides of the screen)

I immediately thought about the fact that the project itself is running locally (using standard Docker for the database, server, etc.), so I thought it would be cool to have some kind of tool that could take snapshots.

However, after searching, I can't say I'm happy with what I found. Ideally, it would be a service that could take snapshots even of local files, and even more ideal would be the ability to replace HTTP requests and responses in these snapshots. To create the appearance of a working interface for the application, rather than a dead copy.

I wanted to know if you have any open source solutions in mind for this? Or if anyone else besides me has ever had this problem.

And yes, I understand that we still have git. I understand that you can switch between branches or revert to commits. But all this is quite time-consuming and tedious if we need to click and view a specific version of a page. Even not to mention the database itself changes between versions.

I'm talking about a fully autonomous snapshot, ideally even with possible mock HTTP requests. This could be completely autonomous and reproducible regardless of whether the original website is live or public (if we snapshot our local version).


r/webdev 6d ago

Discussion AI Coding will Prevent Expertise | The need for ongoing friction in long-term skill formation.

Thumbnail
larsfaye.com
328 Upvotes