First it's not being trained from user input so the creators have total control over training data. *chan can't flood it with Hitler. Second ChatGPT was trained using a reward model generated from supervised learning in which human participants played both parts of the conversation. That is, they actively taught it to be informative and not horrible. There is also a safety layer on top of the user facing interface with it. However users have still been able to trick it into saying offensive things, despite all that!

But it is racist: https://twitter.com/spiantado/status/1599462375887114240