AITA for refusing to look after grandkids because I want to spend time with my other son.

I (50m) have two sons, Stefan (29) and Jax (25). Jax is severely autistic, and requires a lot of attention and care, he is non verbal and can’t do a lot of things on his own. I’m a widower, my wife died 6 years ago, and she was the best support for me and the boys. After she passed away I decided to move to a house which has special facilities for Jax so he could live as comfortable as possible. <br><br>Stefan was fairly annoyed about the whole thing, I could understand it because he was grieving and wanted to get away from all this to get some space for himself and move on. He did, he moved to another city and he has been living there for 3 years. He married Sarah, who is a really sweet lady and I love her like a daughter. <br><br>They have two kids, a 5 year old girl and a 2 year old boy. They also are very sweet. I love all four of them. But I’m a bit resentful towards Stefan because he moved away. <br><br>I understand that he needed to be somewhere else and have some space for himself. And maybe I shouldn’t be resentful over it, but I was used to having him around a lot because we were supporting each other about raising Jax. I take care of Jax now, and it’s not a problem for me, I have someone to help me sometimes. I’m not complaining but it wouldn’t be nice if I had support and someone to talk to, because I’m sure I’m not the only parent in this situation.<br><br>Now I have to get to the problem. Stefan and his wife came to visit me last week and told me that they were moving to another city again, about 4 hours from my town. They said they were going to stay a few more days with me and pass by some other relatives. <br><br>AITA for this; They asked me to look after the kids for a day, I refused. They were upset, and kind of offended. I said I wanted to spend time with Stefan as well because he’s gone for months and I barely see him and the family. They said I see them a lot more now than Jax, because Jax doesn’t understand I’m not there, and I only see Stefan a couple of times per year. They called me unfair for Stefan and called me an asshole.<br><br>I don’t think I’m an asshole, I have the right to spend time with my son and other grandkids. I’m actually a pretty fair dad, I make sure Stefan and Jax have equal amount of time with me when Stefan is around.<br><br>Edit: I didn’t help with the babies but I didn’t leave them with Stefan either. I spent the whole day with Stefan like I wanted to. I helped with the kids all the other times they’ve been with me. I only wanted to have the time with Stefan and the family to catch up. It’s definitely not about Jax, I love all my grandkids equally.