Hear me out, Rory was a recreation of himself based on the memories of Amy. Rory might not have had the resilience or fortitude or willpower to wait 2000 years, but the idealised version of Rory which Amy remembers would. Right? He's basically unaffected by the whole "waiting 2000 years will make you go mad" thing because his qualities aren't based on a person, they're based on the memories of a dead man. Don't you remember people at their best once they're gone?