Standard benchmarking techniques using LLM as a judge have strong limitations. First it creates a circular loop and reflects the flaws present in the AI judges. Then, the perceived quality depends on the end user: an enterprise LLM appeals to professionals and business people, while a generic one appeals to laymen. The two have almost opposite criteria to assess the value. Finally, benchmarking metrics currently in use fail to capture many of the unique features of specialized LLMs, such as exhaustivity, or the quality of the relevancy and trustworthiness scores attached to each element in the response. In fact, besides xLLM, very few if any LLMs display such scores to the user.
I now discuss these points, as well as the choice of test prompts, and preliminary results about xLLM, compared to others.
-- Structured output vs standard response --
A peculiarity of xLLM is that if offers two types of responses. The top layer is the classic response, though much less reworded than in other systems to keep it close to the original corpus, and well organized. The layer below — we call it the structured output — is accessible to authorized end users via the UI; it displays clickable summary boxes with raw extracts and contextual elements (title, category, tags, timestamp, contact person and so on). It also shows relevancy and trustworthiness scores:
➡️ Trustworthiness score: it tells you how trustworthy the input source is, for each summary box. In particular, if the same information is found in two different input sources but with a mismatch, the trustworthiness score tells you which one is most reliable.
➡️ Relevancy score: it tells you how relevant a summary box is to your prompt.
The structured output provides very precise links to where the information is coming from. Also, models based mostly on transformers are not able to generate meaningful [...]
Stay Ahead of AI Risks - Free Live Session for Tech Leaders
Exclusive working session about trustworthy AI, for senior tech leaders. Register at https://lu.ma/zrxsvy6c
AI isn’t slowing down, but poorly planned AI adoption will slow you down. Hallucinations, security risks, bloated compute costs, and “black box” outputs are already tripping up top teams, burning budgets, and eroding trust. That’s why this session blends three things you can’t get from a typical AI webinar:
- Practical expertise: GenAI pioneer Vincent Granville will share a real-world framework for deploying hallucination-free, secure, and lightweight AI, without endless vendor contracts or GPU farms.
- Candid Q&A: Get direct answers from Vincent and your peers in an open discussion, so you leave with clarity on the challenges that matter most to you.
➡️ What You’ll Get in 60 Minutes:
- 20-min Expert Briefing — actionable principles and architectures from Vincent Granville.
- 25-min Facilitated Working Session — collaborate with fellow tech leaders, guided by Sidebar facilitators, to share hard-won lessons and leave with peer-tested solutions.
- 15-min Q&A — bring your biggest questions and walk away with clear, practical guidance.
➡️ Why this matters for you:
- Protect your org from costly AI mistakes others are already making.
- Stay credible in the C-suite with clear, confident AI strategy.
- Move faster than competitors without sacrificing security, trust, or control.
➡️ About the Speaker
Vincent Granville — GenAI scientist and co-founder of BondingAI.io, building secure, hallucination-free LLMs for enterprise. Former senior leader at Visa, Microsoft, eBay, NBC, and Wells Fargo. Author with Elsevier and Wiley. Post-doc in computational statistics from the University of Cambridge. Successful startup exit.
