What age do we start wearing "old man" clothes?

Ya know, the button down, khakis type thing. I'm 30 and one of my friends is 35, he said he's starting to dress more like an old man. He's a software engineer btw so he can wear whatever he wants.