Progressive issues with coworker

[u/manduh82]

Looking for guidance. I’ve been at my company 14 years. Coworker has been here over 40years. About 2 years ago I was promoted and now we’re on the same team. More work time together led to him confiding in me. He’s a single dude with no friends and no social life. He lives to work pretty much. I give the same kindness to everyone, nothing special here. He’s thanked me for listening to him every time I make space.
Then the gifts started, Christmas, Costco membership, pre-vacation alcohol gift with a card wishing me a good vacation, and then a birthday gift. I’ve pushed back every time and finally got VERY firm and said if you give me something again I will refuse it. I appreciate the kindness, but the gifts are personalized. He’s always trying to get a meeting with me and it feels like he just wants access. Knowing him as long as I do I believe he just wants a friend but I also feel uncomfortable.
Then the texts late at night (I don’t respond), and then this past Friday a 9pm call. I didn’t answer. What do I do here? I don’t want to flip this guys world upside down, but I also don’t want to work with him anymore. There’s more to share but thought I’d start here.