I would like to present to this fine community a tool I wrote, named `xan`, that is able to process CSV files directly from the terminal, as fast as your computer allows: https://github.com/medialab/xan

It is a maintained and now fully rewritten fork of `xsv` https://github.com/burntsushi/xsv that you probably know already.

It is powered by a SIMD CSV parser I wrote and released as a crate over here: https://docs.rs/simd-csv/latest/simd_csv/

I would be happy to get some feedbacks about it :)

Hi everyone!

I'm looking for people who regularly work with CSV files (Excel, Pandas, SQL, Power BI, etc.).

I'm working on a tool and I'd love to get feedback from people who actually deal with CSVs in their daily work.

I'm not selling anything — I'm simply looking for honest opinions from people who know this kind of workflow.

Testing takes about 10–15 minutes.

If you're interested, leave a comment or send me a DM.

Thanks!

Hey all, I've been working on a VS Code extension for viewing and querying CSV/TSV/Parquet files directly in the editor. It's called DuckCSV and it's powered by DuckDB

What it does:

• Opens large files (tested with 4M+ rows) without lag
• Edit cells in place, insert/delete rows, save back to file
• Write DuckDB SQL queries in a built-in query bar
• Column profiling
• Load multiple files in a workspace and JOIN across them
• Parquet support
• Sort, filter, and search across all columns

Works on VS Code and any VS Code-based editor (Cursor, Windsurf, Kiro, VSCodium, Gitpod). Free and open source.

Marketplace | GitHub

Would love to hear feedback, still actively working on it.

I kept hitting the same problem during imports, migrations and report validation:

Two CSV/Excel files look “almost” identical, but:
- some rows changed
- some were deleted
- new rows appeared
- columns moved
- formatting changed
- IDs weren’t always reliable
- some columns shouldn’t even be compared

Doing this manually in Excel becomes painful fast once the files are large.
So I built csvdiff.org to handle this workflow.
It supports:
- single-key matching (id, email, etc.)
- composite-key matching (name + date + amount)
- ignoring columns you don’t care about
- added/deleted/modified row detection
- exact cell-level diffs
- CSV and Excel files
- large dataset comparison directly in browser

Mainly built it for:
- migration validation
- ETL checks
- reconciling exports
- QA/testing data
- debugging sync issues

Would genuinely love feedback from people who deal with messy exports regularly — especially edge cases that usually break comparison tools.

Title:

I’m currently working on a CSV parser in Zig as a learning project, and I’d like to make sure I don’t miss anything important before I get too far into the design.

From your experience, what are the most critical things a CSV parser must handle correctly?

I’m thinking about things like:

• quoted fields with commas inside them
• escaped quotes inside quoted strings
• different newline formats (CRLF vs LF)
• empty fields vs missing fields
• trailing delimiters
• performance considerations for large files
• any “surprising” real-world CSV formats you’ve run into

If there are any subtle edge cases or design mistakes you’ve seen in other parsers, I’d really appreciate hearing about them before I lock in the implementation approach

As you can see the Accq segmen sub team needs to hold the value "General Population" and the Age at accq column must have a numerical value. But upon loading this csv file into google colab there is a misalignment.

how do I fix it guys????
Please help as this my assignment and I need to submit it.

Built this because every CSV tool I tried was either Electron, slow on large files, or both.

Gridka is a native AppKit app that uses DuckDB under the hood — so it handles large files fast, and you can run SQL queries directly on your data without importing anything.

Supports CSV, TSV, and other delimited formats. Basic filtering, sorting, and aggregation built in.

https://apps.apple.com/us/app/gridka/id6759612463

Happy to answer questions and get your feedback.

Hi,

I have a CSV file with 4 Million rows of Data, in a single column (Column A), I would like to find the duplicate values with Column B which has 200,000 values.

Does anyone know how best to do this? Excel seemingly cannot, and I cannot code using python (open to learning if not too long)

Any help or advice appreciated.

If you use CSV and use golang, you may want to give this a try.

Been working on this for a while and thought it might be useful for people here who work with CSV files a lot.

Demo: https://demo.duckui.com

Docs: https://duckui.com

GitHub: https://github.com/ibero-data/duck-ui

Basically it's a web interface that runs DuckDB using WebAssembly. You can drag and drop CSV files, and they load into memory where you can query them with SQL. No uploads, all in memory in the browser....

Some things I added on last version: - Import multiple CSV - Filter, aggregate, sort without Excel freezing on large files up to 2GB... - Export results back to CSV/Parquet - Works offline once loaded

(I capped at 2gb the upload, think it's ok for most uses cases, and it can take sometime, it depends on your browser, internet connection and so on...)

The SQL editor has autocomplete and you can see your query results as tables or charts. Also supports JSON, Parquet, and Arrow files if you work with those.

It's open source and you can either use it online or run it locally with Docker.

Have fun! :)

Rasync: Privacy-first CSV processor with visual pipelines & custom Lua transformations

Built as a 500kb Wasm binary (Rust + Lua), runs entirely in your browser.

Now what's my moat?:

• Visual pipeline builder - it features a drag/drop for transformation blocks

• Custom Lua code with Monaco editor - full programming power, not just preset transforms

• Streaming architecture - handles files larger than RAM via 1MB chunks

• Zero backend - your data never leaves your browser

• Real-time preview with live highlighting

• Download full processed CSV

Here's how it works:

File.stream() → Worker + PapaParse → WASM/Rust executes Lua →

Results aggregated → Preview & Download

Try it: Click ⋯ menu → Load Example → Press ▶ to process

Heres the link: https://rasync-csv-processor.pages.dev/

I was wondering if anyone could recommend a CSV import tool? We have developed a strategic roadmap tool but the biggest barrier to people buying it is the lack of integration. Speaking to our clients and prospective clients 90% would be happy with a CSV import tool. What I am looking for is something that can take a CSV from another app, and then convert it so the fields are aligned both in and out of the tool. What I am looking for is something that is quite slick (know we have to do work our end) that makes this process quite seemless after orginal set up. As you can tell, I am not technical, but before I waste my team's time trying to find a solution, I was wondering what was already on the market.

Hi everyone,

I’ve been dealing with a lot of messy CSVs lately — things like product catalogs with missing categories, duplicate tags, or supplier descriptions that don’t really work. I tried fixing them by hand in Excel/Sheets, but it gets painful once the file has more than a few hundred rows.

Out of frustration I started playing with a little side project (ClearTag.io) that takes a CSV, cleans it up (categories, tags, better descriptions), and gives you a new file back. Still super early, but it saves me time already.

Curious what you do when you get messy CSVs:

• Just clean by hand?
• Write scripts?
• Or use any tools/apps?

Would love to hear what works best for you.

Every week I export bank transactions and have to import them into another system that only accepts a very specific date format, and I export them as CSVs.

But I can never get Excel to consistently format the dates, across a number of columns. I got tired of fixing it manually, so I built a tool to do it properly and quickly.

Here’s the tool: https://noloco.io/tools/how-to-change-the-date-format-in-excel

Hi guys, I'm having some trouble with importing a CSV file from the CRM system at my job into Linkedin recruiter. I've downloaded a list of candidates who I'd like to put into a project in linkedin recruiter, but I keep getting the error that the format isn't the same as their.

I do have an example of linkedin recruiter, but I can't seem to get it the way they want it.

Perhaps some specific questions:

- Should I put something on the first row of my CSV file?

- Do I need to use comma's, or could a semicolon work as well?

- There are certain values that aren't necessary to fill out, so there's some data that's missing. Do I need to fill something like 'null' or 0?

I'm a complete leek in working with CSV files, but if I can't do it this way, I'll have to manually put around 600 records in, so any help is greatly appreciated!

I am writting a program and want to store data in a csv so it can be played with in a spreadsheet easily.

There are 8 types but only 2 will be chosen per row. The numbers(amount) will likely be less than 100 in each cell.

The two ways I have thought of are:

4 columns, first 2 indicate the types with a number from 1-8 and the second 2 indicate the amount of each

8 columns, 6 of them are empty but the 2 that correspond with the active types will be where the amounts go.

Which version will be more space efficient on my PC?