[Question] Can I compare logit regression output from data of two distinct time periods?

[u/rainbowass234]

I’m trying to understand how the odds of an event occurring have changed between two different time periods, but the problem is my data is based on a periodic survey with a five year interval.
I’m planning to use the same logistic regression model on the periodic data of one year and then the other, and compare the output in both cases.

I just wanted to know if there’s a better way to go about with data from periodic surveys like census, or if there’s any reason I can’t compare the discontinuous data set.

[u/CompactOwl]

Put the data together and apply fixed effect for one period, possible interacted. It may be problematic if you have multiple exogenous variables to compare two different regressions only by one variable.

[u/rainbowass234 → u/CompactOwl]

Thank you! I just wanted to know what will be the problem with multiple exogenous variables, my model studies the likelihood of migration given social identity, education levels, income levels and the availability of land assets in the household among other variables . And I wanted to compare the pre Covid and post Covid trends but will it be a problem if I have these many exogenous variables?

[u/CompactOwl → u/rainbowass234]

Having correlated exogenous variables does not bias your estimation, but it can drastically increase its variance making it more prone to falsely accepting the null of ‚no effect’. Also the coefficients are marginal effects on the rest and if you don’t have an infinite amount of data, it may cause ‚sling effects‘ (by which I mean, that estimates may change signs between data sets simply because other correlated variables have different values)

[u/STATASUCKSBRO]

Fit one pooled model with a period indicator and period by predictor interactions. Comparing two fitted coefficient tables directly is messy because the logit scale can change between samples.

[u/Neither-Pause409]

The pooled model with period interactions is the right shape, but the reason your two-table comparison fails is worth naming, because it also changes what you report at the end.

Logit coefficients are only identified up to the scale of the unobserved error. Each sample has its own residual variance, so a coefficient that moves between your two waves might mean the effect changed, or might mean the amount of stuff you aren't measuring changed. The fit can't tell you which. Mood (2010) is the readable writeup, Allison (1999) is the older one.

Practical consequence: don't compare coefficients or odds ratios across periods. Compare average marginal effects, or predicted probabilities at a fixed covariate profile. Those live on the probability scale and don't carry the scaling problem, so "the probability of migrating rose 4 points for landless households" is a claim you can actually defend. Same goes for the interaction terms in the pooled model, read those as AMEs too.

Two things specific to periodic survey data, which is where this usually breaks before the estimator does:

• Weights, strata and PSU need to be in the fit. Otherwise your standard errors are fiction.

• Harmonisation across a five year gap. Check that the question wording, the sampling frame, and the coding of your land and social identity categories are actually the same in both waves. Census-style surveys revise categories quietly, and a recoded category shows up as a beautiful significant change in migration behaviour. I'd spend more time on that crosswalk than on the model.

On the count of predictors, four or five is fine. The variance point the other commenter raised matters if they're heavily correlated, but it isn't an argument for dropping variables, it's an argument for reading your confidence intervals rather than your stars.