Have you been able to afford a home when switching jobs?

I currently work in a high paying job but the culture is toxic, and I wish I could leave. But I am currently paying off my house in a big city, and I don't know if I can afford to switch jobs seeing as the housing market is in crisis and the majority of jobs at my skill level pay less than my current position. Anyone else in this situation?