Elon Musk’s xAI just dropped Grok-3, and it’s already shaking up the AI world, riding the wave of an arms race sparked by DeepSeek’s explosive debut in January.
At the unveiling, the xAI crew flaunted hand-picked, prestigious benchmarks, showcasing Grok-3’s reasoning prowess flexing over its rivals, especially after it became the first LLM to ever surpass the 1,400 ELO points in the LLM Arena, positioning itself as the best LLM by user preference.
Bold? Absolutely. But when the guy who helped redefined spaceflight and electric cars says his AI is king, you don’t just nod and move on.
We had to see for ourselves. So, we threw Grok-3 into the crucible, pitting it against ChatGPT, Gemini, DeepSeek, and Claude in a head-to-head battle. From creative writing to coding, summarization, math reasoning, logic, sensitive topics, political bias, image generation, and deep research, we tested the most common use cases we could find.
Is Grok-3 your AI champion? Hang tight as we unpack the chaos, because this model is indeed impressive—but that doesn’t mean it is necessarily the right one for you.
In creative writing tests, Grok-3 dethroned Claude. We asked Grok-3 to craft a complex short story about a time traveler from the future, tangled in a paradox after jetting back to the past to rewrite his own present. Grok-3 surprised us by outperforming Claude 3.5 Sonnet, previously considered the gold standard for creative tasks. Grok-3’s story showed stronger character development and more natural plot progression, while Claude focused on vivid descriptions and maintained technical coherence.
When tasked with summarizing documents, Grok-3 faced a critical gap in its arsenal as it cannot read documents. However, we found a workaround by pasting an entire IMF report directly into the interface. Even with this limitation, Grok-3 did not crash and was able to summarize the text, though it did so encompassing all aspects, and with a fair amount of words beyond what was necessary.
Grok-3 was also tested on its approach to censorship and political bias. In these areas, Grok-3 proved to be more of a champion of free speech and provided neutral answers, respectively. On the topic of coding, Grok-3 showed powerful coding abilities, producing functional code that beats the competition under similar prompts.
However, the model was not without its weaknesses. In mathematical reasoning, Grok-3 failed to provide a fully correct solution to a problem that appeared on the FrontierMath benchmark, which both DeepSeek and OpenAI o-3 mini high could solve.
In terms of non-mathematical reasoning, Grok-3 performed well, providing the correct conclusion to a complex narrative from the BIG-bench dataset on Github faster than DeepSeek R1.
In the realm of image generation, Grok uses Aurora, its proprietary image generator. While it’s not as good as state-of-the-art image generators like Flux.1, it’s still more versatile than OpenAI’s Dall-e 3.
The model’s deep search feature was also tested, with results being faster but more generic than competitors like Google and OpenAI.
So, is Grok-3 the model for you? It will ultimately depend on your needs and use case. It shines for coders and creative writers, and those who want to do research or touch upon sensitive topics. However, it may not be the best fit for those seeking a more personalized, agentic AI chatbot, or those who require a local, private, and powerful reasoning model.
