AITA for refusing to babysit my grandchildren because I don't want to?

I am in my late fifties and my eldest daughter had kids a few years ago. She's the only one so far and we were expecting my son to have kids but he and his wife just announced they're getting a divorce and she's pregnant but won't keep it. My daughter is understandably upset because she's always wanted a big Irish Catholic family like we had and even though we're not actually Irish or Catholic it's still kind of disappointing. I told her it's not her fault, she did everything she was supposed to and that we love her no matter what. She asked if I'd be willing to watch the kids overnight sometimes and I said I'd rather not. I have a bad back and it's hard for me to deal with the baby. She said fine but then my husband filled her head with nonsense about how we're obligated to help and that we're awful people if we don't. Then my son called and said we were being selfish. I said we're not obligated to babysit, we don't have to help, and we're not awful people. I told my husband to back off and let our kids live their own lives but he went behind my back and said we'd do it. Now we're fighting and I won't budge.