- The more LLMs think, the worse they translateAcross several thinking strategies, models translated worse the more they reasoned. Ensembles were the exception.
- Claude 4 is good at translation, but nothing specialClaude 4 Sonnet and Opus on our open-source benchmark: solid, but not a step change.
- Evaluating the best language models at translation V2Concise, raw data on translation quality across the top models.
- The best AI models at translating German (+ local models)German translation on our new benchmark, with some unexpected results for coherence.
- Experimentation Matters: why Nuenki isn't using pairwise evaluationsPairwise evals need huge data and cost; here's why Nuenki skips them.
- Benchmarking LLMs is Rather Difficult, Actually: Designing Better ExperimentsHow bias and indecision creep in, and how to design better translation experiments.
- The best translator is a hybrid translatorCombining the best LLMs into one accurate, idiomatic translation, and it's open source.
- GPT-4.1 and Grok are excellent at translationGPT-4.1, 4.1 Mini and Nano, and Grok, benchmarked against Quasar, Sonnet, and Llama.
- Llama 4 performs worse than Llama 3 at translationLlama 4 is a disappointing release: it translates worse than Llama 3.2 70B.
- Whatever Quasar Alpha is, it's excellent at translationWhoever built it, Quasar Alpha beats every other model at translation, on every metric.
- Lingvanex translates quickly but literallyHow a dedicated machine-translation service compares to LLMs on quality and latency.
- Gemma 3's reasoning improvements don't translate to translationsGemma 3 translates well, but gains less than its reasoning advances suggest.
- Which LLMs are best at translation?Quantitatively comparing LLM translation quality, with a focus on low latency.
- Accelerating Bloom Filters by 310x using Rust and WASMHow we squeezed two orders of magnitude out of a Bloom filter.