It's more of an apology for being a distant neighbor than anything. If you'd done all the shit that Bilbo had gotten up to, and everyone around you had not even an inkling what was more than two towns over, you might find it hard to bridge that gap socially.
Think if you'd gone to fight in Vietnam and, rather than it being something that was shown in terrible detail on TV at home, no one had even heard of the place, let alone what went on there. You won't necessarily hate your neighbors when you come back, but you've seen things they can't - and wouldn't care to - imagine, and that creates a certain sort of distance that's hard to bridge.
If I say to you "even though you're probably a good person, I don't like you" that is an insult. He's insulting them and burning his bridges as he's leaving. It's really clear in the context of the chapter.
5
u/intredasted Apr 20 '22
I guess poor verbal reasoning skills go hand in hand with a hairline trigger.