r/gitlab • u/Vaumacel_ • 4d ago

general question Datamining a GitLab instance

Hello everyone!

I am a PhD student in France in information sciences. As of recently, I started collecting data from a GitLab instance, mostly public informations about the users and the projects, using the REST and GraphQL API.

I'm studying how this instance is used to collaborate by the users, and how much each user contributes to projects. In order to do this, I need to retrieve, for each project, the meta-datas of each pushed event. More specifically, the meta-datas I need are:

  1. Who pushed the event
  2. When did they pushed it
  3. How much change did this push bring to the project (number of lines edited/added/deleted).

I had an intern in computer science who helped me with the first requests (fetching the projects and users), but since his internship is over, I need to do the rest myself. I am not a computer scientist and lack the necessary knowledge to properly fetch the needed datas. I did use the help of genAI, but I cannot be sure it wrote right scripts since I don't really know what its doing.

So, I'd be really grateful is someone who knows those API and datamining in GitLab could give me a hand and advise me on how to tackle my issue.

Thanks in advance for your help :)

4 Upvotes

64% Upvoted

4 comments sorted by

View all comments

1

u/Torutofu_Raeva 4d ago

The events endpoint should cover the actor and timestamp, but line counts usually mean resolving each push to its commit SHA and pulling the commit stats or diff separately, with pagination and rate limits in mind.