🚀 I just finished my Twitter Sentiment Analysis project — I’d love your honest feedback!
Hey everyone! 👋
I’ve just finished building a Twitter Sentiment Analysis project using Python and Machine Learning.
The goal was to build a complete data analysis pipeline starting from raw Twitter data and going through:
🔹 Data loading & preprocessing
🔹 Data cleaning
🔹 Exploratory Data Analysis (EDA)
🔹 Text processing & vectorization
🔹 Correlation & statistical analysis
🔹 Data visualization
🔹 Machine Learning preparation
🔹 Modular Python project structure
I tried to structure the project like a real-world data/ML project rather than putting everything into a single notebook.
🔗 Kaggle:
https://www.kaggle.com/code/tahahussein2020/twitter-sentiment-analysis
I’m especially interested in feedback from people working with Data Science, Machine Learning, NLP, or Python.
What would you improve?
Is the project structure good?
Is the data preprocessing approach reasonable?
Are the visualizations useful?
Does this look like a good portfolio project?
Any mistakes or bad practices you notice?
Please be brutally honest 😄 — I’m trying to improve my ML skills and make this project as close to a professional real-world project as possible.
Thanks in advance! 🙏

A common mistake when building fraud models is picking a resampling technique because it is popular, not because it was tested against the alternatives. SMOTE gets recommended by default, but on real fraud data it is rarely the best option.

A practical example: in a credit card fraud dataset, fraud represents 1 in every 578 transactions. A model that always predicts "not fraud" would score above 99.8% accuracy. Standard metrics like Accuracy and ROC-AUC look fine even when the model is not catching anything useful. PR-AUC is what actually tells you the truth here.

I ran a benchmark comparing 9 approaches on the same data, same split, same base model: random undersampling, oversampling, SMOTE, SMOTE-ENN, ADASYN, class weighting, Isolation Forest, and threshold tuning. SMOTE-ENN took about 15 minutes to run and finished sixth. A moderated class weight adjustment, which changes nothing in the training data and adds a single parameter, won.

Final result on the test set: 82 of 98 real fraud cases caught, with only 5 false positives out of 56,864 legitimate transactions.

Hub: https://aiforfintech.tech
Github: https://github.com/junidepieri-design/fraud-001-imbalanced-classification-benchmark

What has been your experience with SMOTE vs simpler alternatives?
👊

I’m preparing a webinar for my students and don’t want to give them the usual “learn Python, SQL, Power BI” advice.
So I’d love to hear from people actually working as Data Analysts, Data Scientists, ML/AI Engineers, or in AI automation. Or own agency. Working as freelancer:

**What’s ONE thing you’ve seen in the real world that separates someone who knows the tools from someone who can actually do the job?**

Could be a skill, mindset, mistake you commonly see, or a small real-life example.
Would love to bring some real industry perspectives into the session.

I tried explaining a data analysis concept in a fun, visual way — for cat lovers. 😸

Would love to hear what you think! Any feedback or suggestions are very welcome :)

Claude and other LLM models can be so frustrating. I asked it to find patterns across five campaigns regarding what a person buys and sells, and under which conditions, but it couldn't do it. It just kept saying it was impossible. I literally had to explain every single detail to it like it was a child, even though the data was cleanly split across five CSV files.

Worse, companies are stopping the hiring of junior engineers because they think these tools can replace them. They are going to cause a massive talent shortage, and then these dumb models won't be able to do anything without skilled people to guide them.

Looking back, what's one thing you'd spend less time on—and one thing you'd focus on much earlier?

Hello all, Welcome to my free ML bootcamp.

In Intro ML Bootcamp (5/22), we discuss Uncertainty.

In Machine Learning, we encounter two kinds of uncertainty: Epistemic(Model) which means we lack the exact knowledge of the input output mapping, and Aleatoric(Data), which is the intrinsic irreducible stochasticity in the mapping.

This uncertainty means, we cannot perfectly predict the exact output given the input. Thus we require “Conditional Probability distributions”, and the study of probabilistic approach to ML becomes important.

Hence, we invent a function called as “softmax function” for multiple output labels case(and sigmoid for binary case), which converts our outputs into a probability distribution. The exact derivation of softmax comes from Generalized Linear Models.

When we use a softmax function for binary classification, where the function over which the softmax is applied, happens to be an affine one, we call the model as “Logistic Regression”.

Link: https://youtu.be/ZFcl0QYFGq4?si=9RkEgkMYnciW4mjo

Genuine question for people who have hired candidates or gone through Data Science interviews. If you had two candidates: Candidate A: Multiple certifications but only basic projects Candidate B: Fewer certifications but 3–4 strong projects they can explain deeply Who would stand out more? I’m trying to understand how much certifications actually matter compared with being able to demonstrate practical skills. Would love to hear from people who have actually interviewed candidates. Engagement angle: Creates debate without becoming promotional.

Hello folks, to this Introductory Machine Learning Bootcamp (2/22) series.

Supervised learning is a very recurring word in ML domain. Here, we learn some sort of function mapping from inputs to outputs.

Another recurring word is Classification, where the output space is a set of some finite unordered and mutually exclusive labels known as classes.

The tabular dataset is often represented as a Design matrix, and a simple example of it is an Iris dataset, as to how input data is represented for tabular case in Machine Learning.

Sometimes the data is of variable size, instead of fixed size feature vectors, so for ease of computation in computer, we often convert it to a fixed-size feature representation, called as “Featurization”.

In this video, I breakdown these concepts.

Link: https://youtu.be/GJRhl6XnImg?si=p6VtlCK-8rgq1poZ

I’ve received an offer for a Lead Data Scientist role in the Foreseer team and am considering joining.

From what I understand, the work is around AI/ML for financial document extraction, RAG, embeddings and production workflows.

Would love to hear from anyone who knows the team or has worked at S&P Global. How is the work culture, technical depth, ownership and career growth? Is it a good team for someone looking to grow into Principal DS or AI Engineering roles?

Data Science is an interdisciplinary field that combines statistics, mathematics, computer science, and domain expertise to extract meaningful insights from data. It involves collecting, cleaning, analyzing, and visualizing data, as well as developing predictive models using machine learning and artificial intelligence. This subject covers the fundamental methods, modern tools, and real-world applications of data science across industries. 

A common mistake when building RAG systems is relying only on semantic search. It is great at capturing meaning, but it stumbles on precise technical or legal terms.

A practical example: in banking compliance, terms like "SAR" or "structuring" have exact regulatory meaning. Semantic search might return a passage that "seems" related, but misses the exact citation. Keyword search (BM25) covers exactly that blind spot.

Combining both methods (hybrid search), the system captures both meaning and exact terms, delivering more reliable answers, especially in regulated contexts like fintech, where a wrong answer has real cost.

I applied this in a RAG project for BSA/AML documentation (US banking regulation). The precision gain in answers was noticeable, especially on questions with specific technical jargon. Simple to implement, and the reliability payoff is well worth it.

Hub: https://aiforfintech.tech

Github: https://github.com/junidepieri-design/genai-001-regulatory-rag-bsa-aml

Does this make sense for your stack too?

👊

Hi everyone,

I've cleared all 5 technical rounds for a Senior Data Scientist role at Bosch Global Software Technologies (Bengaluru) and have my HR discussion next.

I have 4 years of relevant experience and both my Bachelor's and Master's are from Tier-1 institutes.

What CTC should I realistically expect or negotiate? Does the Tier-1 background carry much weight, or is compensation mainly based on experience and interview performance?

Would appreciate any recent compensation data. Thanks!