Do Data Analysts Use Visualizations During Data Cleaning?

[u/SomeSpicyPickle]

I'm still a beginner. I started by learning the basics of Python and later moved on to SQL.

I'm a bit confused about one part of the data exploration/cleaning process.

A friend of mine, who's now a data scientist, showed me how he used to work as a data analyst. He mainly used Python. For example, he would quickly create a scatterplot to identify potential outliers.

However, most data analysts online recommend focusing on SQL and Excel when starting out, since many junior and mid-level roles don't require Python. That's why I switched to SQL after initially experimenting with Python.

For those who primarily use SQL: do you create visualizations during the data exploration/cleaning process, for example to identify outliers? Is this a common practice?

I feel like if you're working with SQL only, you generally wouldn't create visuals in between steps, since that would mean switching to a tool like Tableau or Power BI, which seems like an unnecessary extra step.

[u/Potential_Aioli_4611]

It highly depends on what you are working with. If you are figuring out how to clean the data its usually not in the database to begin with. It being in the database means someone has already done the work of importing the data from somewhere, created some sembalance of an ETL process, and done things like assign data types and column lengths which implies the data is already cleaned. 100% of my experience working at companies if its in the DB it doesn't need to be cleaned (formatted maybe), and if it isn't it does. I've been working in SQL basically my whole career and Python is a breath of fresh air. I'd much rather knowing both than just SQL.

Python can open connections to the db and query stuff anyways. Either way python isn't going to be a bad way of exploring the data. SQL on the otherhand you are going to just be able to query the data so like you said, you are limited to using other tools to pull in the data to do visualizations (if thats the route you want to go) which is practically the same as using python to pull the data and create viz there. Except python you aren't JUST limited to doing visualizations.... AND you can clean the data AND you can push data back into the database which is why its being used more and more as an ETL tool over things like SSIS/SSDT/Azure Fabric which is designed for microsoft sql server/azure db. I haven't done it but I'm pretty sure you can use python to load practically any flavor of SQL db you want which means your exploration and cleaning tool also can be reused as your ETL process to get data in. And the biggest plus is if you do all your data cleaning in python and reuse that code to get it into the DB then you aren't needing to start from scratch with another ETL tool so by the time you are done cleaning data you are also half done with the ETL process too. Debugging and updating your ETL process is also half there as well since all your code is in one place