Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy (short paper)
arxiv.org
The wording of natural language prompts has been shown to influence the performance of large language models (LLMs), yet the role of politeness and tone remains underexplored. In this study, we investigate how varying levels of prompt politeness affect model accuracy on multiple-choice questions. We created a dataset of 50 base questions spanning mathematics, science, and history, each rewritten into five tone variants: Very Polite, Polite, Neutral, Rude, and Very Rude, yielding 250 unique prompts. Using ChatGPT 4o, we evaluated responses across these conditions and applied paired sample t-tests to assess statistical significance. Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts. These findings differ from earlier studies that associated rudeness with poorer outcomes, suggesting that newer LLMs may respond differently to tonal variation. Our results highlight the importance of studying pragmatic aspects of prompting and raise broader questions about the social dimensions of human-AI interaction.
  • okwhateverdude@lemmy.worldEnglish
    42·
    13 days ago

    Not surprised. RLHF is going to steer the model to do better when confronted with frustration, anger, and negativity. In other words, yes, abusing the clankers does make them perform better and it feels gross to do it.

    • cardfire@sh.itjust.worksEnglish
      4·
      12 days ago

      Only if you’re a decent person. I recall articles earlier in the Siri and Alexa lifecycle about how men were telling on themselves in front of prospective dates by being assholes to their virtual assistants. Thinking about that for a decade is why I’m still trying to use whole sentences and communicate with the clangers in a way that the transcript could be a conversation with any of my friends.

      I try to talk to the Spicy Autocorrect as if it were a person deserving of respect, so that I don’t lose my humanity along the way.