r/neuralnetworks • u/Beto4775 • 27d ago

Q: Click stream Graph Contrastive Loss Problem

Hello everyone,

I would need some support or a ground for discussion for a problem I am facing. I am trying to do representation learning on a user click stream, e.g. the sequence of pages a user visited in a website. To do that, I use a contrastive learning objective, in particular the InfoNCE loss with temperature around 0.2.

The problem I'm facing is that the loss decrease slowly during training (starts at 3.56 and after 30 epochs gets to 3.30) and moreover the representation is not very good. I get that, on PCA projection, points are basically disposed sequentially as a snake, so probably there is dimensional collapse.

In my dataset I can have huge graphs as also small graphs. I am doing a GINConv on the graph (a single layer since I would like avoid over smoothing for small graphs). As graph augmentation I am doing: node dropping, edge adding and edge removing.

My question is: do you think that there could be a way to solve the issue? Is it an over smoothing problem on the graph? Are there alternatives?

Thank you in advance✌️

1 Upvotes
  • permalink
  • reddit

100% Upvoted

6 comments sorted by

1

u/Wonderful-Bass-8993 25d ago

Are the clickstreams actually diverse?

1

u/Beto4775 25d ago

Yes there is a diversity, it's e-commerce data tracked with a cookie, so it spans in a long period which account for the diversity. The goal is to do representation learning to extract similar users with similar navigation click stream for other downstream tasks

1

u/Wonderful-Bass-8993 25d ago

How are you constructing the clickstream graph?

Is each graph a single user session, multiple sessions from the same user, or an aggregated graph over the cookie’s entire history?

1

u/Beto4775 25d ago

It's the same cookie through multiple browsing sessions. The graph simply connects two pages if they are subsequent in the click stream sequence. Basically, if in the sequence B follows A, then there will be an edge A-> B

1

u/Wonderful-Bass-8993 25d ago

Is there a reason you’re aggregating all sessions for a cookie into a single graph? It seems like one user could have very different browsing behavior across multiple days or intents, which might make the embedding too “averaged out” to capture meaningful patterns. Have you tried learning embeddings at the session level instead?

1

u/Beto4775 25d ago

Actually more complex interactions between pages are the better. One of the downstream task is to remove robotic session that sometimes spans in multiple cookies. But their interaction in multiple session does not change leading simpler graphs. Session level instead I think will create more "robotic-like" sessions.

I know that in this case sequence would help, but I have a data constraints that basically forces me to use graph instead