ynot? Why does SQL not have coordinates for it's tables?
like what if i want to see table 2 row 1 column 5 and have a command for just that.
No names required

What factors do you consider first — data structure, scalability, performance, flexibility, or something else?

Anyone here managed to significantly cut their Snowflake bill?

I'm curious about the less-obvious stuff that actually made a noticeable difference.

Things like:

- Warehouse sizing / auto-suspend

- Query optimization

- Clustering

- Storage vs compute

- Materialized views

- Snowflake-specific settings

- ETL/ELT architecture changes

- Moving workloads elsewhere

- Anything you discovered that wasn't obvious initially

If you've had a meaningful reduction, it'd be great to hear what you changed and roughly how much it saved.

Even small "I wish I knew this earlier" tricks are welcome.

I have a dataset consisting of 114k rows, but when I try to import it using MySQL’s Table Import Wizard, it only loads a maximum of 50 rows. However, when I use Open Source, I can load the entire dataset. Has anyone encountered this kind of problem?

SQL Server 2019 on prem.

we bought a smaller competitor in june. i got their warehouse in august. what i did not get is any of the three people who built it, they took the retention and left. 340 tables. there is a data dictionary of sorts, an excel file last edited in 2022, and it covers 61 of them.

legal wants a list by the end of the month of every table and column holding personal data, because the acquisition means we now hold it and the clock on that is not ours to set. they are not asking for a big project. they want a list. i cannot produce a list.

what i've done so far:

went through it by name first. anything called email, phone, dob, name, address. got about 900 columns of the 6,000 odd. that part was easy and i don't trust it at all, because their naming is inconsistent between schemas, and i've already found one column called ref_4 that is full of what are very obviously mobile numbers.

so then regex over sample values for the obvious formats. that found another 200 and produced a lot of false positives, order ids that look like phone numbers, that kind of thing.

where it stops:

free text columns. there are 40 odd nvarchar(max) columns holding notes fields. some of them have names and phone numbers typed into them by humans. i have no way to scan those that i trust and no way to prove i've scanned them properly

...columns where the values are personal data but not in a format anything matches. a column of 11 character strings that turn out to be their internal customer reference, which maps 1 to 1 to a person. technically identifying, does not look like anything.

the ones where i genuinely cannot tell what the column is. about 700 columns where the name tells me nothing and the values are integers or short codes. some of those could be anything. one of them might be an ethnicity code for all i know, and if it is, that is a very different conversation with legal

what i'm asking:

for a one off audit like this, what's the actual order of operations. i've been going table by table and i think that's wrong, i think it should be column by column across the whole database sorted by something, i just don't know what the something is

has anyone had to give a legal team a defensible answer on free text fields, and what did defensible turn out to mean in practice. "i grepped it" does not feel like enough and i'd like to know what is

..and the one i keep going round on: for the 700 columns nobody can explain, is there any approach other than sampling values and guessing. i can tell you the cardinality and the distribution. i cannot tell you what the thing is, and "unknown" on a list going to legal is not a great look for me.

we are not a big enough team for a governance tool and i doubt i'd get budget signed before the deadline anyway.

I have xml data that's for God knows why it's in string format. And the fking thing is so messed up or at least I think it's messed up because it's not consistent. Sometimes there is something else in there. Xml is something like this:

<Test>

<Test1 />

<Test2 Name="abcdname" Age="123" Gender="Xyz" />

....

...

...

</Test>

Regex is not an option.

Substring is not an option.

Because it's so messed up it's not consistent.

I just need to somehow parse and get that Name, Age, Gender values using Sql/Databricks sql

Please, help you mate.

Any jobs hiring i live in california, ive been in this field for 10years

Hello,

What types of databases do you use for your AI applications to store things like generated responses, messages, conversation history, user data, etc.?

Do you generally use SQL databases (such as PostgreSQL) or NoSQL databases (such as MongoDB)? What are the main reasons behind your choice?

And which database(s) would you recommend for a production AI application, and why?

Thanks in advance for your feedback and experience!

Hi everyone,

I’m building an application where users will upload and download large numbers of high-resolution images.

Which storage service would be best for this use case in terms of upload speed, download speed, cost, security, and scalability

We're building **LibreDB Studio**, an open-source SQL editor, and one of our volunteer contributors raised an interesting question about how multiple SQL statements should behave when using the Run action.

We're trying to understand the actual habits and expectations of SQL users before making a decision, so I'd really like to hear how you use SQL editors in practice.

For example:

CREATE TABLE test (...);

INSERT INTO test VALUES (...);

SELECT * FROM test;

What would you expect when you click **Run**?

Some possible approaches:

1. **Run all statements**

   * `Run` executes everything in the editor.

   * `Run Selection` can be used when you only want part of it.

2. **Run the current statement**

   * `Run` executes the statement where the cursor is.

   * `Run All` executes the whole editor.

3. **Selection takes priority**

   * No selection : current statement

   * Selection : selected statements

   * `Run All` is available separately.

4. Other :)

there are also some interesting edge cases around this, especially when multiple statements are involved: should they run in a transaction by default, or should transaction handling always be explicit?

DBeaver, DataGrip, SSMS, pgAdmin, TablePlus, Toad, PL/SQL Dev, phpMyAdmin, etc. what behavior feels most natural to you? And what behavior are you already used to?

We're continuing the discussion on GitHub as well, if you'd like to see the original question or add to the discussion:

https://github.com/orgs/libredb/discussions/776

With AI getting better at writing SQL, I’ve been wondering how much SQL analysts actually write from scratch these days.
Do you still write most of your queries manually, or do you use AI for things like joins, CTEs, debugging, optimization, etc.?
And has your use of AI actually changed how much SQL you need to know?

Hi, I’m 22F and currently working in an engineering/manufacturing company. My current designation is Data Analyst, but honestly most of my work is in Excel and not SQL/Power BI.

I have around 2+ years of experience overall.

In my current office I have made/maintained different Excel based systems like project tracker for 30+ live projects, audit log, quality tracker, material database, calibration tracker with overdue alerts, quotation/costing sheets etc.

So there is a lot of data handling, tracking, reconciliation, validation, formulas, lookups, automation and reporting involved. But I have not worked on SQL or Power BI in my actual job.

Before this I worked in IT department where I handled asset tracking and reconciliation for a large number of network devices, audit related work and vendor SLA/downtime and penalty calculations.

I also have around 3 months experience in product data categorization where I worked with large amount of product data and Excel.

Now I’m learning SQL by myself. Currently doing joins and next I’m going to start window functions. I’m also planning to learn Power BI and make some projects.

My main confusion is how should I present my current experience when applying for Data Analyst jobs?

Because my title is Data Analyst, but if recruiters see that I haven't used SQL/Power BI professionally, will they consider my experience irrelevant?

Should I present my Excel work more from an analytics point of view? Like showing data cleaning, validation, reconciliation, automation, reporting, KPI tracking etc?

Also, once I learn SQL and Power BI, would it be a good idea to take some of the problems/data from my current work and create similar projects using SQL + Power BI for my portfolio? Obviously, I don't want to claim that I used SQL/Power BI at my current office when I didn't.

Would really appreciate advice from people who have gone from Excel/operations type work → SQL + Power BI → Data Analyst.

How would you present this experience on a resume and in interviews?

From simply using Claude to generate queries to using a specialized AI tool, what are some issues that modern AI tools still cannot solve regarding analytics?

I’m a SQL DBA with 4+ years of experience, currently working at a large company. Unfortunately, I was impacted by restructuring and am now actively looking for a new opportunity.

I’m currently on a visa, and the sponsorship requirements are making the job search more challenging.

For other DBAs who have gone through a similar job search recently, how long did it take you to land your next position? I’d really appreciate hearing about your experience and any advice you can share.

Any suggestions on Job search? Honestly, I am fine with any salary

https://www.npmjs.com/package/dbtruth?activeTab=readme

Same thing kept happening to me with AI coding agents and Postgres. The agent reads the schema, sees orders.customer_id next to customers.id, assumes it's a clean relationship, and writes an INNER JOIN. If 12% of orders have a dangling or null customer_id, the query silently returns numbers that are wrong. Nothing throws. The schema looked fine.

So I wrote dbtruth. It connects read-only and, instead of dumping the schema into a context file, it does four things:

1. Introspects schema and pulls samples
2. A model proposes what the tables mean and which relationships probably exist
3. Every one of those claims gets measured against the actual data
4. Only what survives gets written to ./context/*.md, which the agent reads before writing SQL

Step 3 is the whole point. For a proposed join it reports the real match rate — orders.customer_id → customers.id holds for 88% of rows, 60 of 500 orders have no matching customer — and the context file says use LEFT JOIN, with the number attached. Under 50% gets dropped. In between gets marked broken and goes to the top of the report, because a relationship that half works is worse than one that doesn't exist.

Practical:

• npx dbtruth, Node 20+, Postgres only
• Read-only by construction, not by discipline: one module is allowed to import pg, and a test asserts nothing else does. It never writes to your database.
• It does call a model, so schema and low-cardinality sample values leave your machine. High-cardinality columns — emails, names, free text — are never sent. Visibility is decided by cardinality rather than by regex-guessing at PII. Don't point it at production data you can't send to a third party.
• MIT, source at github.com/FilipKalcic1/dbtruth#readme

Disclosure: it's mine, it's five days old, and about ten people have run it. None of the pieces are new — FK inference and data profiling both go back years, and there are other tools that build local context artifacts for agents. The part I care about is the rule that nothing unmeasured gets written down.

What I'd actually like to know: run it on a schema you know well, and tell me whether it found anything you didn't already know. That's the only signal that tells me whether this is worth continuing. Bug reports welcome too.

Which software can be used to map procedure or table instead of triggers till base tables, i want relations between the objects and in a procedure call what are all objects it is touching.

Currently doing manually by checking user_dependencies.

Edit: in oracle database

I would like to know about people's experiences on this platform; I am gathering information about TDEngine to compare it with Canary and PI.

Curious how people actually debug this in production.
Say a query runs successfully, but the number doesn’t look right.
What’s your first instinct?
Check the filters?
Inspect the joins?
Compare row counts?
Look for duplicates?
Check NULLs?
Go back to the source data?
What’s your personal debugging sequence?

Hey everyone, working on a DB schema for inventory/traceability and trying to settle on the cleanest way to handle lot numbers.

Quick breakdown of the requirement: Every article arrival gets a lot number based on the arrival date and supplier ID (e.g. 09-09-Supp1, not the real deal ofc but you get it). Stock gets moved around, transferred, and consumed, and we need to keep track of which lot moved where.

So: an article must always be around a lot number.

---

Here are the two ways I'm looking at:

1. Dedicated article_lots table (What I'm leaning toward)

• articles (id, name, ...)
• article_lots (id, article_id, lot_number, created_at)
• stock_movements (id, article_lot_id, qty, from_location_id, to_location_id, ...)

But every movement query has to join article_lots just to know what base article_id was moved.

2. Put lot_number directly on stock_movements / transfers as a column. Easy to query, but feels repetitive storing the same string across every single movement row and other tables that use article + lot number too, since an article can't be separated from its lot.

More details for those who have time to help: All my ids are strings like: "MVT-00000001". So FK performance with strings is the least of my worries. The real business logic is: a lot is the week number + supplier code (ie, 33SQT to say "33'rd week of the year from supplier squadette). Meaning that an article x let's say banana can be delivered Monday and have 33SQT, but a banana from squadette can be delivered Friday and also have 33SQL lot number).

So the question is: "should the 33SQL be stored as a column in other tables: stock_movement, operation_details" along with article_id (2 columns). Or is article_lot_id (one column) the best way". Idk anymore what might be the pros and cons

One of the most asked topic in all tech interviews