

I was trying to have some insightful discussion on the actual capability of LLM which is difficult when the involvement of the human element is played down amd the role of the LLM is played up to feed the hype machine. It’s hard to acknowledge the real capabilities and weaknesses when the capabilities are always over reported and the weaknesses down played or denied.
It’s great that so many bugs are getting discovered but as I say there is no reporting on what effort was needed to sift and review the LLM output or how functional or understandable any PoC were… The article doesn’t directly even state the PoC were directly produced by the LLM and reads very ambigously.


It’s states they were produced which I’m taking to mean found and it’s ambigously worded so it’s unclear if the article is actually claiming it generated PoC for all of them. It doesn’t say how many if any hallucinated results were produced or how much effort was needed to fully confirm, basically down played the human involvement.
It’s great that so many bugs are being found and squashed but it’s implied LLMs are doing all the work when what they are actually doing is assisting as a tool.