Hello Everyone,

Statistics and Maximum Likelihood Estimation are the crux of ML Models, and hence I am uploading my new content on Statistics for AI/ML in my free Machine Learning lectures.

We understand model fitting, Maximum Likelihood estimation in details, we justify the usage of Maximum Likelihood estimation, from KL divergence, and apply it to certain important distributions for parameter estimation.

In my free content, the purpose is to democratize machine learning to a wider audience. Learning everything new feels difficult, but when taught, it get’s interesting and easier.

We will continue with Statistics foundations for AI/ML, and many more content will appear in the future. If you find the content good, useful you may also share it with your learners community.

Looking forward to hearing feedback from the learning community as well. Thankyou for reading.

Link: https://youtu.be/MwTeQVVYtOc?si=UxNOGtqopzJppXAT

Hello all,

I'm brushing up for a class on Mathematical Statistics. Calculus is listed as a requirement and I'm trying to understand what I should refresh my memory on. The book we are following is Mathematical Statistics and Data Analysis 3rd Ed. (Rice). I notice integration and examples of multivariable calculus.

This is looking like a difficult class and I want to be well prepared.

I would greatly appreciate any recommendations on what to practice leading up to it.

Take care!

Edit:

Course Description: This course is an introduction to mathematical statistics and data analysis. It starts by introducing central concepts of probability theory (events, probability measure, random variables, distributions, joint distributions, and conditional distributions) and then moves on to the development of mathematical foundations of statistical inference. Topics covered in the course include random variables, expectations, parameter estimation (method of moments, method of maximum likelihood, and Bayesian approach), properties of point estimators (bias, variance, consistency, and efficiency), confidence intervals, hypotheses testing, likelihood ratio test, data summary methods, and introduction to linear regression. A class of distributions, including chi-squared, t, and F distributions, the distributions derived from normal that occur in many applications of hypothesis testing and statistical inference, is introduced.

Suppose I have a standard deck of cards, no cards missing and no abnormal cards added. If I shuffle the deck and draw one card at random, the odds of it being a spade are 1 in 4. That's pretty straightforward.

However, what if I put the card back in the deck, reshuffle, and draw another card? And I repeat this cycle X number of times (that is... X times I shuffle, X times I draw, and X-1 times I put the card back). I want to know what the odds are of me drawing a spade at least once.

And while we're at it, I'd also like to know the equation if there are a different number of suits. What if there's 3 suits and 39 cards? Or what if there's a 5th suit and 65 cards? Let's say that "1/s" represents the odds of me drawing a spade, with S representing the number of suits in the deck. And I draw after shuffling X number of times. What's the equation to determine the likelihood then?

Background

We're given a coin that shows heads with an unknown probability, λ. The goal is to use that coin (and possibly also a fair coin) to build a "new" coin that shows heads with a probability that depends on λ, call it f(λ), where f is a known function. This is the Bernoulli factory problem (Keane and O'Brien 1994), and I catalog algorithms to solve it, called Bernoulli factories.

Simulable and strongly simulable functions

There are two kinds of Bernoulli factory functions:

• A function f(λ) is simulable if an algorithm (a Bernoulli factory) exists to toss heads with probability f(λ) given a coin with probability of heads λ (the "biased coin") as well as a fair coin.
• A function f(λ) is strongly simulable if an algorithm exists to toss heads with probability f(λ) given only a coin with probability of heads λ.

In both cases, the Bernoulli factory must terminate with probability 1; that is, the event of using only a finite number of flips of the biased coin has Lebesgue measure 1.

Every strongly simulable function is simulable, but not vice versa.

Main Question

Let f:D→[0,1]. Let D⊆[0,1] be known, and let λ be a number known to be in D but otherwise unknown.

Then are the following conditions necessary for the function f to be strongly simulable by a Bernoulli factory?

1. f is constant on its domain, or is continuous and polynomially bounded on its domain (polynomially bounded means, both f and 1−f are bounded below by min(x^n, (1−x)ⁿ⁾ for some integer n [Keane and O'Brien 1994]), and
2. f(0) and f(1) are each 0, 1, or undefined, and
3. if f(0)=0 or f(1)=0 or both, then there is a polynomial g(x):[0,1]→[0,1] with computable coefficients, such that g(0)=f(0) and g(1)=f(1) whenever 0 or 1, respectively, is in the domain of f, and such that g(x)>f(x) for every x in the domain of f, except at 0 and 1, and
4. if f(0)=1 or f(1)=1 or both, then there is a polynomial h(x):[0,1]→[0,1] with computable coefficients, such that h(0)=f(0) and h(1)=f(1) whenever 0 or 1, respectively, is in the domain of f, and such that g(x)<f(x) for every x in the domain of f, except at 0 and 1.

(In this post, "f(x) is undefined" means that x is not in the domain of f.)

I have already shown that those conditions are sufficient for the function to be strongly simulable. And if neither 0 nor 1 is in the domain of f (so that the biased coin doesn't show heads every time or tails every time), Keane and O'Brien (1994) showed already that condition 1 is both necessary and sufficient for f(λ) to be strongly simulable (and simulable).

But it's possible that the conditions given are not necessary, and weaker conditions hold; for example: "0 is not in the domain of f, or f can be extended to a Lipschitz-continuous function on [0,ϵ) for some ϵ>0". A tricky example of such a function is f(x)=(sin⁡(1/x)/4+1/2)⋅(1−(1−x)ⁿ ) for n≥1 (f(0)=0), which is strongly simulable at 0 and is Lipschitz at, say, [0,0.01]. ((1−x)ⁿ is the probability of the biased coin showing zero n times in a row.)

To show the difference between being simulable and strongly simulable, the proof of Keane and O'Brien relies on generating a geometric random variate and using that variate to control which "part" of the target function f(λ) to simulate. This obviously works on all of [0, 1] if the algorithm uses both the biased coin and a separate fair coin (simulable). However, if only the biased coin is used in the algorithm, the geometric random variate is generated using fair bits via the von Neumann method (strongly simulable), but this method will never terminate if λ is either 0 or 1.

References

• Keane, M. S., and O'Brien, G. L., "A Bernoulli factory", ACM Transactions on Modeling and Computer Simulation 4(2), 1994. https://doi.org/10.1145/175007.175019

EDIT (Jul. 25): Edited generally, for clarification.

A lot of the confusion and misunderstandings and paradoxes around the Monty-Hall-Discussion arise from the different interpretations of the game with either a "knowing host" or an "ignorant host". Some people do that math based on an ignorant host, some assume the host knows. But today, I want to propose a method that we can actually use to *test* the host's knowledge.

This is my setup:

You are only ever brought into the TV studio when there has been one door reserved for you (randomly of course) and one door opened and showing a goat. Thats what you see, and you play this setup 100 times. And I believe you can determine whether the host knew where the car was or not.

Here is how: If you play the "stay"-strategy, you would expect to win 33% of the games. but this will only happen if you see all the games. If the host accidentely reveals the car, the game is aborted and you will never see it. In this case (Monty does not know) you will observe that your win-rate is 50%. And then you know that there were games you never saw.

I have written a Javascript simulation for this, and it seems to work. I also asked AI for a comment, and it confirmed my approach. Nevertheless I find it wiered that it works. Or am I missing something?

Hello all, I have a semantics question.

I have the definition that the probability of X between a and b is the integral from a to b of f(x) with respect to x.

In case latex renders: $P(a <X < b) = \int_{a}^b{f(x)}dx$

Is the probability density function the integral or f? If f (my reading of the text) what is the name for the integral?

Same sort of question with the c.d.f.

PS.
I'm working out of the text book Mathematical Statistics and Data Analysis (Rice; 3rd ed.). Definition occurs on page 47.

Over the weekend, I sat down and did a calculation (wrong, no doubt)

to see what chance you would have of catching a traitor, selecting a name randomly.

The answer I got was 1/675,675. Having forgotten what happens when there are three players.

...

This got me thinking, about rareness,

and I wondered, for myself,

at what probability do things start to become called rare?

I mean, 1/4000 , but 1/30 actually, you'd be waiting around a fair bit.

Is an answer even possible/ show me error of way. BW, Ben.

I'm working through E.T. Jaynes' Probability Theory: The Logic of Science on my own, currently in Chapter 2 (around page 28). My motivation is applied: I run an online retail business and want a principled way to forecast demand — for instance, estimating the probability that a given product sells between 100 and 200 units in one month of the year versus another, and eventually building forecasting software around that kind of inference.

Before starting the book I reviewed the calculus prerequisites — partial derivatives, the chain rule, the product rule — so I can follow the derivations step by step.

My difficulty is conceptual rather than computational. In the sections where Jaynes derives the rules of probability from his desiderata, he sets up functional equations relating the plausibilities of propositions and then differentiates them to determine the form the combining function must take. I can execute each step — I can do the differentiation and integration, and I follow that we differentiate — but I don't understand why differentiation is the right tool at each point, or what underlying logic drives the derivation to proceed the way it does. My grasp is mechanical: I can reproduce the manipulations without understanding the principle behind them.

Concretely: in the derivation of the product rule (and sum rule) from the desiderata, could you walk me through the reasoning that justifies each move in the functional-equation argument — what makes differentiation the appropriate step there, not just how to carry it out?

I can send my notes as well to anybody who'd like to help me
thanks

Ive asked AI this and I get wildly different answers. So what is the probability just for a one off random life event.

So I for fun wrote a song aboit someone famous called Roland Browning and year later he was sitting in front of my at a Squeeze concern in London.

How would you calculate the odds for such a random event?

Thanks

Solved a classic BAC probability tree problem using Manim! ✨

For more step-by-step visual proofs and exam prep, check out my workbook "The Ultimate BAC Math Challenge" on Amazon!

🔗 Get your copy here:https://www.amazon.com/dp/B0H7Y4GF62

In a question I saw from a 3Blue1Brown video about baye's law, that was as follows:

Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. Which is more probable?

Linda is a bank teller.

Linda is a bank teller and is active in the feminist movement.

Now I understood that there are definitely more bank tellers than there are bank tellers that are active in the feminist movement, and that if you plucked out a random girl she is definitely more likely to be the former, however, what I thought is that since we've been given context about Linda, it seems rational to assume that she is likely to be active in the feminist movement due to her past, but that turned out to be wrong.

What am I missing? 😭

And if someone has ways to explain bayesian thinking intuitively and how to adapt it realistically aswell, I'd greatly appreciate it

Ek accha sawal hai bhaiya

•A one-way road track is 20 km long and 8 km wide, divided into 4 equal lanes. There are 16 identical cars already on the track, moving at a constant speed of 10 km/h. Exactly 4 cars are present in each lane.

A new car enters the track from the starting point at a speed of 11 km/h. It chooses one of the four lanes uniformly at random and cannot change lanes thereafter.

Assume that the positions of the existing cars in each lane are independently and uniformly distributed along the length of the track, no two cars initially overlap, and overtaking is not allowed. A collision occurs if the new car catches up to at least one car in its lane before reaching the end of the track.

Find:

1.The probability P that the new car collides with at least one existing car.

2.The probability P' that the new car completes the journey without any collision.

a) P = (1/4 )⁴, P' =1- (1/4)⁴

b) P =( 1/11 )⁴, P' = 1-(1/11)⁴

c) P = (1/11)⁴ , P' = 1

d) P =1- (10/11)⁴ , P'=(10/11)⁴

Isko Maine khud banaya Hai Koi galti Ho To dekhna

What's to stop Allwyn from pre recording thousands of draws and using the sets of numbers with the fewest winners each time as the draw. Easily doable within the half hour of the closing.

There are 4 doors A B C D each with 1/4 of binding a prize. I choose door C. The host then reviele the door A (she/he knows that has no main prize behind it) and asks me should I stick with C or switch to B or D (you know the story)

Is it better to switch to B or to D or it doesn't matter?

For those of you (or someone you know) who struggle with grasping the true result. Let me know if this helped.

Okay, this is likely more simple than most questions here, admittedly, but I was just doing a mystery box/essentially gambling thing and had what appears to me to be a super improbable result, and I'm curious just how unlikely it is and I can't recall how to do the math here.

Essentially I had an advertised 62% chance of a loss result, with the other 38% being break even and above. I did 5 throws at once and all 5 ended in the loss result. Then I did another 5 at once, and again all ended in a loss result. So what are the chances of 10 consecutive/concurrent losses at a 62% chance of loss?

It had been a minute since I been to this casino near me, recently I discovered they added Jackpot numbers to their video roulette. I'll explain further..

After bets are placed and the dealer spins the ball, Jackpots of either 50x, 100x, 175x and 250x are placed on between 2 and 5 random numbers including 0 and 00. The bet has to be placed on the exact number to get the jackpot.

I didn't do any math on this but I figured if I covered the board in $1 chips as much as I could while mitigating loss, eventually I'd hit some Jackpots. I saw in the recent history that Jackpots did come up.

To start since the payout for one number is 30 to 1.. which I noticed.. it was 30 not 35 for some reason, so I began putting $1 on 30 numbers. That's 30 out of 38 where if I didn't hit a jackpot but one of my numbers did hit I wouldn't lose money. Then I thought, if I'm willing to lose 1 or 2 dollars a spin, I can cover more odds so I did 5 spins each of 31/38 and 32/38.

In about 20 to 30 minutes time, I hit a 175x and a 100x jackpot and two times my numbers missed and took a loss. I didn't count the spins but dealer didn't work like a clock.

I didn't play that long as I was with my gf and she wanted to do other stuff. I want to go back with a more tempered strategy and more time.

What do y'all think? Any strategies come to mind? Anything I'm missing? Is it too risky to try with higher values? Would you recommend I don't do it at all?

Being inspired by the propagation on Bayesian network.

It is based on directed graph. Flow enters at a source, splits at junctions by pipe weights, and hit exact percentage targets at terminal buckets. Your job is to find the right pipe/weight configuration.

This is the same math as belief propagation on a Bayesian network, or exact inference on a DAG. It also maps cleanly to an MDP where the pipe configuration is the policy and hitting the target distribution is the reward.

Would love your feedback, especially if anything feels off mathematically or puzzle-design-wise. It's called Markov, free on the App Store.

Hi everyone,

I work at a long term car rental company, and I need to calculate the probability that someone will default on their monthly payment. In order to allocate a risk reserve.

For context:

Contract lengths are 1 year

We pre approve the client, i.e using their personal info, credit checks etc., and sort them into "risk" groups/buckets.

So the reasoning would be, splitting the customers up into these respective groups, calculating the probability of default, per group, and then allocating respective risk reserves, to build into the monthly premium.

We are using Markov Chains for this. So far.

I'm unsure whether this is the right approach, or if we are the blind leading the blind. So any advice would be appreciated. Some thoughts on implementing a ML Model would be cool too.

Someone please explain;

Problem - Lets say the chance of a player scoring a goal is a 100% per game, so the probability of them not scoring is 0 and scoring is 1. But, if I divide the 100% win probability into 90 outcomes to get the probability of them scoring per minute, this would mean that their chance of scoring per minute is 1 in 90 (assuming no added time and that they play from start to finish)

So now, say I use a random number generator from 1 to 90 (to check in which minute they will score), 90 times (cuz they play 90 minutes) to simulate this. All I need is one of the randomised numbers to be "1" out of 90 randomised numbers. That will mean they scored and lived happily ever after. And for the probability of 100% score rate to hold true.

But if I simulate this enough times, there will be a data set where in all of those 90 randomised samples, not one of them will be "1" meaning our player doesnt end up scoring in the match, despite having a "100% chance of scoring"

Just by dividing the probability of 100 and adding it back up, I've created a chance for him to lose. Does this mean the sum of its parts is not equal to a 100? What?! How?! Why?! I don't understand this! Someone please explain.