Is the human sense of morality a system of general agreement?

I'm not the first person to say this, but after watching a terrible situation where a person I know was treated poorly but nothing was done to stop it because it wasn't really illegal and virtually no one I know considers it morally wrong, just makes me feel the same way I do when I see humans do shitty things to animals. Is general agreement all that truly defines something as moral or immoral?