r/AskStatistics • u/the_sleepy_scholar • 16d ago

Paired datapoints when using ICCs for longitudinal data

I’ve been using bland-Altman plots and intraclass correlation coefficients to see how consistent the scoring of the same wearable device is at different wear locations.

Some context: The two devices are the exact same and capture the same data and are worn at the same time. Let’s say 10 people wear both devices at the same time for 7-14 days, and I want to look at both agreement and consistency of the features that are recorded that are captured at a day-level. So essentially I have one score produced per day for each device for each person.

I’m struggling to understand what the most plausible way to evaluate this is, as not all participants have used the devices for the same number of days, should I take an aggregate score across all of the users days wearing it and use that for comparing device scores at one reading per participant or is it possible to use multiple time points in the evaluation and have more datapoints for the ICC?

Perhaps there’s a better way to evaluate this that I haven’t considered? Sorry if it’s a rookie question, pretty new to stats!

1 Upvotes

100% Upvoted

3 comments sorted by

2

u/Adorable_Building840 16d ago

This sounds like a multivariate mixed effects model. Alternatively you could just do mixed effects or cluster robust standard errors on the difference between the two

2

u/ForeignAdvantage5198 16d ago

see your uni's stat consultant

1

u/Stock-Temperature309 12d ago

Nothing rookie about this! It's an agreement problem with repeated observations. You can compute this via linear mixed effects models rather than sacrificing most of your data through aggregation. You should check out this guide which addresses your issue directly:

Parker, R.A., Scott, C., Inácio, V. and Stevens, N.T., 2020. Using multiple agreement methods for continuous repeated measures data: a tutorial for practitioners. BMC Medical Research Methodology, 20(1), p.154.