The worst thing about learning a language with a small number of native speakers, e.g. Swedish, Polish, Czech etc.

After years of learning Swedish, Danish, Dutch, and Italian, the worst thing about these kind of languages is that a huge amount of their speakers unfortunately expect you to speak English to them. I can easily talk to Swedish people, but after around a minute or two of conversation, they directly respond to me in English. I used to search for online language partners, but after 5-10 minutes of conversation with a Swede, they'd suddenly switch to English with me. You can't even practice Swedish with Swedes!