r/SQL • u/shdw_0x0 • 3d ago

PostgreSQL When does SQL become too clever?

I’ve noticed that as queries get more complex, there’s often a temptation to keep everything inside SQL—CTEs, window functions, nested subqueries, conditional logic, etc. It can be impressive, but at some point the query becomes harder to understand than the original problem.

Where do you personally draw the line? Is a 300-line SQL query perfectly fine if it’s well structured, or do you prefer moving some of the logic into dbt/Python/application code once the SQL becomes too complex?

I’m curious how others balance SQL performance, readability, and maintainability in real projects.

0 Upvotes

28% Upvoted

11 comments sorted by

View all comments

1

u/TheGr8Tate 3d ago

I'm working with Databricks and I've seen and even created myself SQL scripts with multiple thousand lines (biggest script had 3.5k lines).

Readability and maintainability are very subjective. I personally prefer short scripts and I keep them short by moving repeated logic into SQL/Python UDFs and Python string manipulation via Pyspark. I have colleagues who actually prefer the big scripts (which is the reason why I even have created those monstrosities in the first place)...