What is your least favorite thing about traveling and why?

For me it's when you hear someone speaking in their obviously native language and you have to awkwardly decide whether to respond in English or their native language. It's extremely anxiety-inducing and I've had multiple instances where I did one and the other person responded in the other.