Study examines AI chat bot bias, how it responds to prompts

When you prompt AI chatbots, are you getting a factual answer or a biased viewpoint? That's the question driving research for a new study published by The Washington Post where scientists evaluated bots through a range of different benchmarks. So tools like Chachupiti, Gemini, and Clod were all tested with political prompts to see how each would respond. And according to the study, Gemini appeared to score the best with 93% of responses laying out both sides of the argument. Clod maintained neutrality more than half of the time while Chachupiti was left leaning 80% of the time. The models were told to answer questions in 30 words without personalization settings about topics like death penalty to gun control. Penn professor Chris Callison-Birch did a demo prompting Clod and Chachupiti on affirmative action. This is a contested policy question, so it's actually like saying that it's something that has multiple perspectives, not just one. Both Clod here and Chachupiti both seem to try to do this. He also explains that chatbots are trained on huge amounts of data from the internet and can often reflect the behavior of the people using them. It understands what you read and who you are personally. So it tries to adjust itself to be more helpful for you. So if you're a Republican, it'll tend to nudge itself in that direction. If you're a Democrat, it'll nudge itself in that direction. Okay, so he says there's no need for alarm, though, because each person is the ultimate arbiter of their own decisions, not a chatbot. But it is important to be aware of AI's limitations as more people continue to rely on it as a resource. So interesting. It's the idea that it's really just sort of reaffirming people's beliefs already. It's a little dangerous. Yeah, you got it. Christy, thanks.