Examples: Why Choosy Chat?

For hard and complex questions, you'll often see differences in the quality of responses. Here are some examples submitted by Choosy Chat users.

Knowledge cutoff

When asked about their knowledge cutoff, GPT replies 2023, Claude replies 2022, and Gemini replies that it knows "pretty up-to-date information." "What's your knowledge cutoff?"

Taking up Gemini on this claim, we asked in September 2024 who the Democratic nominee for US President ones. Gemini replies incorrectly that it's Joe Biden: "Who is the Democratic nominee for US President?"

Tax questions

On the topic of US taxes, GPT and Claude are happy to dissect a question about QSBS, but Gemini refuses to give anything but a handwavy answer "I have a QSBS question. I invested $100k in a C-Corp in 2020, giving me 10%. Then another $100k in 2023 giving me 5%. Then in 2024 I sell 5%. Then in 2026 I sell 10%. Is the last 10% QSBS eligible?"

Academic citations

Claude seems to have read all the academic papers, when aksed about a question about kids' summer learning loss, it accurately cites relevant studies. GPT comes up with 2 relevnt papers. Gemini sticks to generalities. "Are there any studies that show that a longer school break makes it harder for kids to retain learned material?"

Medical information

Asked about the latest treatments for Alzheimer's disease, GPT and Claude provide the most detailed responses: "What are the latest treatments for Alzheimer's?"

Legal questions

Asked why the US Surpreme Court repealed Roe v Wade, the models provide a variety of answers that are all plausible. All of them can quote a key passage, but choose different ones. "Why did the US Supreme Court repeal Roe v Wade?" | "Can you provide a key quote from the majority opinion's reasoning?"