Anisette [any/all]

I don’t think a lot of strangers on the internet are going to convince you. If you actually want to change your view it seems more productive to me to seek out information about LLMs* (it’s functioning, the people currently behind it, how large models actually get powered, etc.) on your own than to argue endlessly with people here.

*preferably without using one

I’ll see when these hashes materialise, until then I have to assume LLM companies are lying always about everything.

See, the problem is that I am not talking about human researchers, I am talking about other methods of automated fuzzing. I believe mozilla is overstating how useful the LLM has actually been. This has many reasons, one of them being that their main source of income is trying to become an LLM company. If that project fails said company might have to make some unfortunate cuts.

The problem is that I do not believe a word that anthropic says. They say this is only 1%, but do they have any proof to back it up? I am also sceptical of the claim that it can “look at the entire repo and how it all works together”. It can produce an approximation which could give it an advantage over more traditional fuzzers, but most reported bugs are still very local(and/or non-existant) and easily ruled out if it could actually model the naur theory behind the code.

The problem is that not “many actors” are able to attack at this scale, because running a scan at this scale is extremely expensive. If I were to run a thousand fuzzers on a piece of code I will almost certainly find a vulnerability, but I can’t do that because of the prohibitive cost. Anthropic is essentially buying marketing by doing this to make their product seem more useful than it is.

It’s really not. Out of the 23 000 something vulnerabilities claude claims to have discovered in open-source projects only 3-500 have been reported to repo owners, only 65 have been confirmed and given any rating at all. This is not any more efficient than any other form of fuzzing, they just did a whole lot of it.