When I listen to apocalyptic predictions as a result of AI (transformer based generative LLMs to be specific), they’re all based on assumption that it “adds value, but at a high energy cost”.
They don’t consider the destruction of human knowledge, where bullshit generators are “informing” decisions, and “curating” insights. Similar to how all steel made after the invention of nuclear weapons is useless for certain applications, so I find books written after the rise of LLMs.
If only it also didn’t come at the low cost of destroying the ability to reason (as numerous studies have shown). Silverlining is that it’s also absurdly energy demanding, and further pushing the climate past the point of no return. At the very least, we’re are in for a hefty and long recession when the bubble pops. What’s not to like?
My capacity for apocalyptic news has to be carefully metered to prevent burnout and stress. (It’s a balancing act.) Humanity seems to be hellbent on heading down a dark road with a bunch of ugly possible turns.
The potential to stifle FOSS is bad enough. What I find even more concerning is when the LLM gets good enough to reliably fool developers and can inject very complicated exploits from multiple sources.
I’m a programmer by trade, but it’s highly specialized. Otherwise, I’m just a power user/script kiddie. My crystal ball is pinging alarm bells about some of this. It’s hard for me to make an accurate assesment of the risk, I lack the knowledge. I expect unspecified bad things and am in a reactionary mode on this.
I hear you. I’m very much the same, both in trying to not pay too much attention for the same reasons, but also the trade, though perhaps not all that specialised.
Once the economic aspect of this comes to the conclusion we already know: it isn’t sustainable. I think we might start to see a more sensible approach to LLM usage.
The current status is as if people are asking LLMs if a mushroom they picked is safe to eat, and then serving the whole family. A more sensible approach would be to get a name suggestion from the LLM, then use that as an entry point to manually verify it.
The LLM user should always be the expert. I.e., don’t serve something potentially poisonous. Let it come with suggestions, by all means. But if you don’t know enough to verify the correctness of what it says, then you already lost. Unfortunately, this is how most people use it now. Followed by being shocked “it lied”.
Thanks for the additional information. However, where is the evidence that these are submissions by Anthropic’s employees, as this post claims? Those submissions all appear to be from random authors.
I would think this could be solved easily by requiring reputation to have a PR considered, otherwise with nothing preventing anyone from making an account and submitting anything at all, spam is inevitable, regardless of what tools were used to generate the spam.
The issue has more to do with the burden of reviewing code, vs the ease in which a poor contribution can be made that isn’t worth reviewing. The signal to noise becomes so bad, that maintainer are in many cases, out of necessity, rejecting contributions that are made with LLMs. By hiding LLM tell-tales, as the prompt in question here aims to do, it compounds an unethical and arrogant take that the contributions would somehow become more useful. As if commit message structure, comments or otherwise discussion, was the problem (when it suggests its by LLMs), and not the low quality of code changes (as is, by and large, the case).
As you point out, that is a more general discussion, and not specific to Anthropic employees.
Your suggested solution leaves me wanting to sigh. That’s what many open source projects have needed to do. Reject all external contributions. Modern software is extensively based on open source, and the work done by millions of developers, for free. There is a good will here, and hard work, that has been carried out under a sense of “furthering humanity”, where you just hope that you are able to contribute, in some way. Spam wasn’t a problem before LLMs. The goal of spam is to pass filters, in order to cause some kind of harm. This takes effort for humans, but trivial for a bullshit generator. Which, is even worse than my take, which was that these contributions were well intended, but just delusional as to its usefulness. Though, I’m sure that motivation to sabotage projects exists. Not sure how “active and deliberate sabotage” would paint a better picture of Anthropic employees, but it seems like you actually get why we might find it particularly repulsive?
In any case, if we assume best intentions, and that there can be value in contributions made by, or with the help of, LLMs. Then lying about this in PRs, is both unethical, and in contradiction with the altruistic mindset of open source development. Thinking your LLM based contribution is special, as opposed to all the other slop, and thus not deserve being put in some low priority review queue, so you lie about it, and instruct your LLM to lie about it, etc, is exactly the kind of skill-issue and arrogant delusion that pisses people off. And, what a monumental disaster for humanity it is, that what LLMs have managed to do, is force many open source maintainers to reject external contributions, not just those “by AI”, but all external contributions, since it is too costly to find valuable contributions in a sea of slop
I’ve read some of your comments. You don’t seem to understand the underlying issue. Have a read through some of these: https://gist.github.com/bagder/07f7581f6e3d78ef37dfbfc81fd1d1cd Maybe that’ll help
Woah. Not who you’re responding to. Had a read through GitHub and one of the reports.
Been seeing the rumblings about AI fuckery and open source but haven’t followed closely.
That’s some spooky shit.
Indeed.
Here is the article that lead me to it: https://acko.net/blog/the-l-in-llm-stands-for-lying/
When I listen to apocalyptic predictions as a result of AI (transformer based generative LLMs to be specific), they’re all based on assumption that it “adds value, but at a high energy cost”.
They don’t consider the destruction of human knowledge, where bullshit generators are “informing” decisions, and “curating” insights. Similar to how all steel made after the invention of nuclear weapons is useless for certain applications, so I find books written after the rise of LLMs.
If only it also didn’t come at the low cost of destroying the ability to reason (as numerous studies have shown). Silverlining is that it’s also absurdly energy demanding, and further pushing the climate past the point of no return. At the very least, we’re are in for a hefty and long recession when the bubble pops. What’s not to like?
My capacity for apocalyptic news has to be carefully metered to prevent burnout and stress. (It’s a balancing act.) Humanity seems to be hellbent on heading down a dark road with a bunch of ugly possible turns.
The potential to stifle FOSS is bad enough. What I find even more concerning is when the LLM gets good enough to reliably fool developers and can inject very complicated exploits from multiple sources.
I’m a programmer by trade, but it’s highly specialized. Otherwise, I’m just a power user/script kiddie. My crystal ball is pinging alarm bells about some of this. It’s hard for me to make an accurate assesment of the risk, I lack the knowledge. I expect unspecified bad things and am in a reactionary mode on this.
I hear you. I’m very much the same, both in trying to not pay too much attention for the same reasons, but also the trade, though perhaps not all that specialised.
Once the economic aspect of this comes to the conclusion we already know: it isn’t sustainable. I think we might start to see a more sensible approach to LLM usage.
The current status is as if people are asking LLMs if a mushroom they picked is safe to eat, and then serving the whole family. A more sensible approach would be to get a name suggestion from the LLM, then use that as an entry point to manually verify it.
The LLM user should always be the expert. I.e., don’t serve something potentially poisonous. Let it come with suggestions, by all means. But if you don’t know enough to verify the correctness of what it says, then you already lost. Unfortunately, this is how most people use it now. Followed by being shocked “it lied”.
Thanks for the additional information. However, where is the evidence that these are submissions by Anthropic’s employees, as this post claims? Those submissions all appear to be from random authors.
I would think this could be solved easily by requiring reputation to have a PR considered, otherwise with nothing preventing anyone from making an account and submitting anything at all, spam is inevitable, regardless of what tools were used to generate the spam.
The issue has more to do with the burden of reviewing code, vs the ease in which a poor contribution can be made that isn’t worth reviewing. The signal to noise becomes so bad, that maintainer are in many cases, out of necessity, rejecting contributions that are made with LLMs. By hiding LLM tell-tales, as the prompt in question here aims to do, it compounds an unethical and arrogant take that the contributions would somehow become more useful. As if commit message structure, comments or otherwise discussion, was the problem (when it suggests its by LLMs), and not the low quality of code changes (as is, by and large, the case).
As you point out, that is a more general discussion, and not specific to Anthropic employees.
Your suggested solution leaves me wanting to sigh. That’s what many open source projects have needed to do. Reject all external contributions. Modern software is extensively based on open source, and the work done by millions of developers, for free. There is a good will here, and hard work, that has been carried out under a sense of “furthering humanity”, where you just hope that you are able to contribute, in some way. Spam wasn’t a problem before LLMs. The goal of spam is to pass filters, in order to cause some kind of harm. This takes effort for humans, but trivial for a bullshit generator. Which, is even worse than my take, which was that these contributions were well intended, but just delusional as to its usefulness. Though, I’m sure that motivation to sabotage projects exists. Not sure how “active and deliberate sabotage” would paint a better picture of Anthropic employees, but it seems like you actually get why we might find it particularly repulsive?
In any case, if we assume best intentions, and that there can be value in contributions made by, or with the help of, LLMs. Then lying about this in PRs, is both unethical, and in contradiction with the altruistic mindset of open source development. Thinking your LLM based contribution is special, as opposed to all the other slop, and thus not deserve being put in some low priority review queue, so you lie about it, and instruct your LLM to lie about it, etc, is exactly the kind of skill-issue and arrogant delusion that pisses people off. And, what a monumental disaster for humanity it is, that what LLMs have managed to do, is force many open source maintainers to reject external contributions, not just those “by AI”, but all external contributions, since it is too costly to find valuable contributions in a sea of slop