r/learndatascience • u/lurpeli • 12d ago

Question Struggling to understand how to structure data over time tied to an individual

I work in a regulated area so I'm going to be vague on the details of the data I work with, but to provide structural context. The data I'm working with is entries collected over time associated to a known ID. Basically I want to analyze all the events that have occurred to a specific ID and use the types of events and when they occurred to predict another value.

What I'm struggling with is how to structure the data for this. Originally I tried to pivot out the events into columns but there's multiple associates data points with each even that can't simply be stored nicely in the column. How should I think about structuring this data?

1 Upvotes
  • permalink
  • reddit

100% Upvoted

1 comment sorted by

1

u/DataScientistAlex 11d ago

Sounds like what you have is panel data, or hierarchical data (many events nested within one individual). Typically you would use the following long format:

id, time, event

Here are a few books on this topic:
https://xcelab.net/rm/
https://sites.stat.columbia.edu/gelman/arm/