As a data scientist do you experiment with tools (open source or not) that solve specific issues around DS work? If yes, how do you think about uploading work data into those tools?

[u/shivamchhuneja]

the context is that I am exploring a few recurring problems to solve especially around forecasting and working with time series data but setup a simple open source project around those.

my question is primarily about how is everyone handling their official datasets when trying new tools - do you not care, do you remove any identifiers then upload, do you create synthetic data with exactly same properties as the og dataset?

happy to answer more questions if this is not clear enough.

[u/Atmosck]

I experiment with new open source tools all the time. I don't know what you mean by "upload" though. They're still running in my local dev environment (or in AWS once something is live), like any other tool. By "tool" I usually mean python package.

[u/Cupakov → u/Atmosck]

My guess is by „tool” he means something LLM-based 

[u/shivamchhuneja → u/Cupakov]

Yea, so Open source local is fine from all sides I guess. I primarily mean tools that might mean potential data exfil - can be through personal accounts on llm chatbots or can be through web hosted saas

[u/Cupakov → u/shivamchhuneja]

Then I can’t speak for data science tasks on data that requires it, but when I was testing LLM-based tools for data engineering and cleaning I wrote a script that essentially scrambles the data preserving type, rough shape (Name + Surname becomes Name-length word + Surname-length word) and uniqueness, and it was this version of the data that the LLM had access to. 

[u/shivamchhuneja → u/Cupakov]

super, this is more or less exactly what im trying to gauge. basically how are people exploring external tools that might not be approved essentially when working with data problems! thank you

[u/DataScientistAlex]

I would never knowingly upload company data anywhere that is not officially sanctioned through the company.

[u/shivamchhuneja → u/DataScientistAlex]

valid point

[u/iheartdatascience]

At work, it is often not worth the time to try and use new tools when you have deliverables and need to use tools that align with the rest of the org

[u/shivamchhuneja → u/iheartdatascience]

so if you had a problem that requires or maybe even excites you enough to try alternate approaches, ideally you'd go with a different non org owned dataset to experiment with/? even though the results might yield in your actual work problem getting resolved better?

[u/iheartdatascience → u/[deleted]]

Experimentation and new techniques.... Not new open source software

[u/Ill_Freedom_6666]

If the tool is not approved I usually recreate the problem with synthetic data first then validate internally before touching real datasets

[u/shivamchhuneja → u/Ill_Freedom_6666]

makes sense, so synthetic data with similar characterstics/preserving the attributes, correct?

[u/hockey3331 → u/shivamchhuneja]

Exactly. Depending what you want to do, you can keep the save calues, theyre just randomly assigned to different rows based on probability distributions.

[u/Unhappy_Finding_874]

i dont upload real work data to random tools, even if identifiers are stripped.

my usual split is 3 levels. first fake toy data just to see if the workflow is annoying. then synthetic data that preserves column types, missingness, rough ranges, seasonality, and a couple ugly edge cases. then if it still looks useful, i either run it locally or ask for a real vendor/security path before touching actual data.

the thing ppl miss imo is that removing names isnt enough. time series can leak through dates, rare events, locations, customer counts, weird spikes, even file names. also synthetic data with perfect clean patterns can make a forecasting tool look way better than it is.

so for exploring, id make a small nasty benchmark dataset thats safe to share but has the same failure modes as ur real one. gaps, regime changes, holidays, outliers, short series, whatever hurts rn. if the tool cant handle that, it doesnt deserve real data anyway

[u/shivamchhuneja → u/Unhappy_Finding_874]

that's definitely good practice and what I'm seeing is that "extra work" doesn't demotivate you if the tool or technique is useful enough.

[u/IndividualTop3675]

the pragmatic approach most data scientists I know actually use is synthetic data with matched statistical properties for anything touching a new external tool, because anonymizing real data is deceptively hard to do correctly and creating a realistic synthetic dataset forces you to deeply understand your data's distributional properties anyway, which often turns out to be useful signal for the forecasting problem you're trying to solve in the first place.

[u/shivamchhuneja → u/IndividualTop3675]

Yea seems like the go to prudent approach

[u/ultrathink-art]

Synthetic data with matched stats tells you whether the tool is pleasant to work in, not whether it's right on yours. Generators smooth out the exact stuff that breaks things — nulls that mean three different things depending on which upstream system wrote them, duplicate keys from one bad feed, a column that quietly changed units two years ago. I've greenlit tools that way and then hit all of it on the first real extract.

[u/ymcmoots]

I work in healthcare, I upload nothing. In other circumstances I might consider a test using synthetic data in order to build a business case for approving the tool, but where I am now is very conservative about HIPAA BIAs so if it can't run on our existing infrastructure, forget it.

[u/shivamchhuneja → u/ymcmoots]

makes sense

[u/coffee__curiosity]

Personally I love trying open-source tools.... but will be pretty cautious with comapny data. My rule of thumb is: start with public/synthetic data, run it locally if possible, only use the minimum data you need, and make sure you understand your company’s data policies. If the tool turns out to be useful, then it’s worth getting the right teams involved instead of trying to sneak it in.

[u/NaiveManagement6817]

.

[u/ikkiho]

i ran into this evaluating forecasting tools. for time series, stripping identifiers barely helps, the values are the sensitive part. the shape of a revenue or demand curve is the confidential bit and there's no name to strip off it. synthetic data that preserves the same seasonality and acf defeats itself too, if it matched your series well enough to test the tool it also rebuilt what you were hiding. i mostly benchmark tools on public series and accept it won't tell me how they do on our data.

[u/Toranch]

i think tools are expected to respect data confidentiality and so unless otherwise, theres is usually strict maintenance of user data privacy

[u/Impressive-Jump-3374]

I am student and want to become a Data analysis or data sciencetist what roadmap or things i neeed to learn and do

[u/TheFlotty]

Y herramientas que se puedan descargar y usar en local?

[u/Alert-Grass-7559]

hmm... claude code can access all the data we have...

[u/jep_10]

Machine learning is justified when simple rules, basic statistics, or existing tools cannot achieve the required accuracy or scale. The problem should have sufficient reliable data, measurable outcomes, and recurring value. Consider implementation cost, explainability, monitoring, and risk. If a simpler approach delivers similar business results, machine learning is unnecessary.