I hate incest but I wish I could still have a relationship with my brother.

We had sex when we were younger but we were together in a way that was more than just friends and more than just siblings. It feels like a part of me has been ripped out. I don't see him anymore and his life is so busy with work and other things. I didn't know what to do. I love him so much and I don't know what to do.