If it did it would be simple to train a healthy and truthful facts AI, you would simply train it off from the truthful and healthy facts.

Not how AI works. I mean, it’s technically possible to train an AI that only generates truth, but it would be so overfitted that it’s functionally no different from the search bar on Wikipedia.

Large language models need randomness to function. They cut up every true sentence in the training data into tiny tokens and reassemble them into… well, whatever arrangement of tokens satisfies the discriminator or the humans who grade the output. Discriminators can’t tell fact from fiction and the humans generally don’t care to.

Even if you valued truth above all else during the training and rejected every false statement you encounter, there’s no way you could judge the truth of every possible statement the system could ever produce. And with randomness most statements will be false, that’s simply the nature of truth.