• 1 Post
  • 51 Comments
Joined 1 year ago
cake
Cake day: February 5th, 2025

help-circle




  • This is the wrong approach to try and prove/disprove your hypothesis.

    Its a statistical model that by its nature will answer differently every time. The only deterministic parts are fundamental truths about how the system operates (r in strawberry) and guardrails that have been put in by devs (cannot talk about this topic).

    Therefore fundamentally this would require a statistical approach. A couple of those have already been done of course.

    If this is your experience you could try and actually fact check the output. I believe coding is good for this because issues / misunderstandings are pretty immediately obvious. But I use Kagi Assistant a lot instead of search and there are factual issues all the time. And that’s already just summarizing search results.

    Then also, as long as we are using LLMs for this, they are fundamentally still “find the next most likely word” machines. So they will be influenced by context a lot. The “truth” is not a concept that exists in LLMs.