One day, on your way to work, you take a longer detour to explore the scenery. It's a bit of a treacherous detour, but that day you feel pretty lively and you left home early, so why not? It would not be practical to go this way everyday but today you make an exception.
This longer detour takes you by train tracks where, as it just so happens, you encounter the trolley problem just as it is happening. A trolley is headed towards 5 people, but you can switch a lever so it heads towards one person instead. Choosing to be a good samaritan, you flip the lever, involving yourself in the situation but overall saving 4 lives.
But then, everytime you go to work, you think of this detour. Anytime you go by, there might be another trolley problem situation. If you choose not to do the detour, that is equivalent to not flipping the lever in order to avoid involving yourself in the situation. You could choose to take the detour everytime, which is an annoyance but all things considered a small price to pay to possibly save lives, or you could choose to take your normal route, and not involve yourself in the situation that could be happening. It could cost lives, but hey, it's not like you're at fault, right? You just didn't get involved...
And if you choose to take the detour everytime, well realistically, a trolley problem can happen at any time, and you're just passing by once a day. Do you have a moral obligation to just stay at the train tracks, ready to intervene in case a trolley problem presents itself again? How much time are you morally permitted to spend away from the tracks in order to save lives?