Stefan Krawczyk

This post is related to hallucinations but it delves into a couple of interesting points 1) Should we expect that LLM's will flag non-sense prompts, I would vote yes 2) Users need to stress test models by asking them non-sense questions

7

This looks useful for folks in research or scanning what's coming out on the frontier: Lumi from Google's PAIR research team, a tool for better understanding papers on arxiv. Reading smarter, not harder -- that tagline brings me back to Mortimer Adler's classic 'How to Read a Book'! From their GH repo: Lumi uses AI to help you quickly read and understand arXiv papers. Features include: ✏️ AI-augmented annotations - read summaries at multiple granularities 🔖 Smart highlights - highlight text + ask questions 🖼️ Figure explanations - ask Lumi about images in the paper Demo: https://lnkd.in/ex7erTBb GH repo: https://lnkd.in/emz_Ghcs Medium article: https://lnkd.in/enky8dKC GitHub discussions: https://lnkd.in/efAmiCcp

19

Excited to share a new blog post on our "Scalable Framework for Evaluating Health LLMs" 🚀 (https://lnkd.in/gzhuvTfN) Daniel McDuff and I summarize how this novel framework addresses key challenges in evaluating AI models to health queries, especially when user query includes data from wearables, blood biomarkers, and/or user-provided historical health data. This framework is designed to: (1) Significantly reduce evaluation time. (2) Improve inter-rater reliability. (3) Precisely identify quality gaps in LLM responses. A huge thanks to our incredible team, and a special shout-out to my former intern, Neil Mallinar, who was the main driving force behind this project. Great contributions from: Ali Heydari, PhD, Xin Liu, Tony Faranesh, Brent Winslow, Nova H., Benjamin Graef, Cathy Speed, Mark Malhotra, Shwetak Patel, and Xavi Prieto. #AI #LLM #evaluation #health #wearables #biomarkers

172

3 Comments

At this juncture, such research effort is probably much more important than squeezing out an extra percentage point of performance on some academic benchmarks, or arguing semantics on whether LLMs can "reason". https://lnkd.in/gKxEPDRF

9

"a consensus framework to guide the responsible use of generative artificial intelligence in mental healthcare, which, it is argued, has cognitive demands and features distinct from physical medicine. We propose such a framework, Clinical Reach into the Cognitive Space (CRITiCS), to support clinician involvement in the deployment of these technologies." https://lnkd.in/ePpQPBfX

3

𝗪𝗵𝗮𝘁 𝗱𝗼𝗲𝘀 𝗶𝘁 𝗺𝗲𝗮𝗻 𝘁𝗼 𝘀𝗰𝗮𝗹𝗲 𝗰𝗼𝗻𝘀𝘂𝗺𝗲𝗿 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝘀 𝗶𝗻 𝗮𝗻 𝗔𝗜-𝗳𝗶𝗿𝘀𝘁 𝘄𝗼𝗿𝗹𝗱? Dived deep into this at the roundtable at Bits n Atoms, hosted by The Product Folks last night — and the conversations stuck with me. Scaling consumer products in an AI world demands thinking across multiple dimensions, and these are just some of the lenses through which our group explored it.  • 𝗣𝗲𝗿𝘀𝗼𝗻𝗮𝗹𝗶𝘀𝗮𝘁𝗶𝗼𝗻 — not just smart recommendations, but designing for one and actually impacting behaviour change  • 𝗧𝗿𝘂𝘀𝘁 — especially for this new generation of AI-native products, where UX becomes critical to adoption  • 𝗖𝗼𝘀𝘁-𝗲𝗳𝗳𝗲𝗰𝘁𝗶𝘃𝗲𝗻𝗲𝘀𝘀 — using AI to boost efficiency and also designing AI systems that can scale without exploding infra or model costs  • 𝗠𝗼𝗻𝗲𝘁𝗶𝘇𝗮𝘁𝗶𝗼𝗻 — whether through subscriptions or new models that make sense for AI-first experiences  • 𝗗𝗶𝘀𝘁𝗿𝗶𝗯𝘂𝘁𝗶𝗼𝗻 & 𝗚𝗧𝗠 — this differentiation matters more than ever, in a world where building products is getting faster and cheaper everyday  • 𝗔𝗰𝗰𝗲𝘀𝘀𝗶𝗯𝗶𝗹𝗶𝘁𝘆 — across regions, languages, devices, and modalities. If it only works for the few, does it really scale? We’re in an interesting moment — where product thinking is expanding, not just because of what AI can do, but because of the questions it’s forcing us to ask. Loved the sharp, honest, and wide-ranging perspectives from everyone Shreyanshi C. Kushal Khandelwal Ashish Jain Madhav Bhartia Tanmay Saxena Pranav Agrawal Sachin Kamkar Shashank Raghavendra Nikhil Ankan Huge shout out to Abhay Jani Aditya Mohanty Suhas Motwani for driving these thoughtfully curated and highly impactful events!

92

6 Comments

Week 1 of #OlapOnTap covered: 1. Typing: how much it can matter in OLAP contexts: https://lnkd.in/gCQnNTNt 2. A deep dive into Cardinality, how it affects performance, and how using enums and LowCardinality() can help improve your performance: https://lnkd.in/g2d_pQJZ 3. Informed by this, how to efficiently design your table's ORDER BY to maximize your database's performance: https://lnkd.in/g2BnBCFF 4. How denormalization, and wider, flatter tables, are a better fit for OLAP database engines and query patterns https://lnkd.in/gyQQ62Gr The TL;DR for Week 1 is: 🤏 More restrictive types drastically improve performance, and low cardinality types can improve that even more 🔭 Structure your data based on your consumption patterns (both in terms of how you order your data, and how you structure your tables). Next topics coming up are: ➜ Modeling JSON & semi-structured data (noting how much heavy lifting the word "semi" is doing in this title) ➜ Nullability and default values ➜ Compression and encoding (and how you can take the reins more than you think here) ➜ Precomputing with materialized views The starting theme is modeling / setting up your OLAP database (if that wasn't obvious!). I am just kinda exploring topics I find interesting in the OLAP space, and I want to write LLM documents for my agents in respect of. But if any of you have particular topics you want me to cover, holler! Also, I'm focusing on ClickHouse implementations, but let me know if you want me to cover other databases. #DataEngineering #OLAP #MooseStack

11

Introducing Agentic Culture — an open-source experiment in collective memory and in-context cultural accumulation for multi-agent systems. A hot topic from the recent Andrej karpathy // dwarkesh podcast is that LLMs don't yet have culture. >>> So we built the scaffolding for them to develop one. Agno Agents can now tap into a shared cultural memory — knowledge that persists beyond individual sessions, users or memories. Culture is collective, evolving context that shapes agent behavior over time. Now they can create, read, explore and learn from their culture. You can use the `CultureManager` class to create and manage `CulturalKnowledge` objects. These are stored in your database of choice (SQLite for now) and automatically retrieved as in-context cultural grounding for your agents. To give your agents access to shared culture, just set `add_culture_to_context=True`. That's it. Agents now learn from collective experience. Explore the code and README: https://lnkd.in/e7E3AcJX Agno is open source — come build with us: https://agno.link/gh

98

4 Comments

A new transformer architecture called Co⁴ aims to emulate imagination and higher-level human mental states in artificial intelligence. Proposed by Ahsan Adeel and published at https://lnkd.in/grFd8zTb , Co⁴ introduces several key innovations: Key Highlights: Biological Inspiration: Co⁴ is inspired by recent neuroscience discoveries, specifically the function of layer 5 pyramidal two-point neurons (TPNs) in the human neocortex. These neurons process external sensory signals and internal contextual information in parallel—mechanisms thought to underlie different mental states such as wakefulness and various phases of sleep. Triadic Modulation Loops: The architecture features triadic loops connecting questions (Q), clues (keys, K), and hypotheses (values, V). This design enables multiple, parallel chains of reasoning, allowing the model to pre-select relevant information before attention is applied—similar to cognitive filtering in the human brain. Efficiency and Performance: By leveraging representation-level reasoning and eliminating traditional feedforward residual networks, Co⁴ achieves a computational complexity of approximately O(N), where N is the number of input tokens. This translates to improved learning speed and lower computational costs. Applications and Results: Co⁴ has shown strong performance in diverse areas, including reinforcement learning, computer vision, and natural language question answering. These results suggest its potential to advance AI reasoning and context-awareness. Significance: Co⁴ represents a step toward AI systems that not only process information more efficiently but also demonstrate context-aware, human-like reasoning abilities.

20

"Graphs make provenance first-class. Instead of burying "where did this come from?" inside a dense vector, a graph records sources, activities, and agents as nodes and edges you can query..." https://lnkd.in/efAdV9dx

26

This is a great experiment by the Anthropic team, showcasing the "true autonomous" capabilities of LLMs. By "true autonomous," I mean that Anthropic (I believe) purposefully did not bombard the LLM with numerous tools in areas where LLMs are weak. For example, LLMs are not good in math, and performance would have been much better if they hd provided a tool like "calculate_profitability." This might have prevented the $100 opportunity miss. In AI agent development, tools bridge the gap where LLMs are not "deterministic." This is currently the MOAT in the GenAI app business: the better your tools, the more deterministic results you can provide. Only till LLMs can write their own tools on the fly and execute them where they know they are weak and using deterministic softwares (tools) to fill these gaps. https://lnkd.in/gMk3VRJv

10

1 Comment

Avoiding Claude Code because the terminal feels intimidating? Cowork changes that. Same power, zero command line. Anthropic released Cowork today and I've been testing it. Here's what makes it different: Cowork lets Claude interact with your computer through a visual interface. No typing commands. No scary black screens. What you can actually do with it: - Organize messy file folders automatically - Turn a pile of receipts into a clean spreadsheet - Draft reports by pulling data from multiple documents - Rename hundreds of files based on patterns - Create presentations from raw research notes This is Claude Code's capability through a visual interface. The same parallel processing that developers use, but accessible to non-technical users through Claude desktop. Worth exploring if you spend hours on repetitive computer work.

2

(Note: I'm on a wireless network that blocks most video at the moment, so I haven't seen the specific rant. May add more comments later when I do.) SQL's strength back in the 80s and 90s had to do in part with the core idea that you could effectively join two rows from different tables through the use of primary key/foreign key pairs, though at the cost of creating a combinatoric explosion of result sets. The syntax was fairly arcane, but as programmers in general became skilled with it, they were able to do a great number of things if they had a good understanding of the core data model. It was not seen in these terms at the time, but SQL was an early graph traversal system. As many people in the ontology space know, SQL graphs are subsets of RDF-based graphs - you don't have the metadata connections with SQL, nor can you easily do multiple forward links, but creating a SQL-like query system is a common problem (and usually easily solved) in ontology circles. Having said that, I don't think that's really what Bob Martin is ranting about. The bigger issue may very well be, does it make sense to expose a specialized query interface to a database (such a SQL, SPARQL, GraphQL, or similar tool), or should such services be wrapped in API calls? This is a harder question to answer, especially if you can take advantage of data shapes and the ability to construct very generalized queries that can be resolved as graphs in multiple dimensions. In essence, these are shape graphs that use the fact that the schema is itself represented as graphs that can be traversed as part of the same query. I love SPARQL for its flexibility in creating synthetic graphs, especially compared to SQL, but I also recognize that it is easier to create multiple generalized solutions that can then be constrained to generate such graphs with proper parameterization. This is obviously more secure - as you can wrap such queries in basic API calls that let you catch most of the more egregious hacks (and you can shape the information after the fact), but it still requires consistency within the graph to pull off well as well as considerable knowledge about SPARQL and SHACL that is much less heavily represented in the programmer space than it probably should be. Are there better encodings for graph traversal, query, and manipulation? I rather doubt it. We could readily have gone the XQuery route of building a more "traditional" route of graph path traversal, and reifiers are still akin to dark magic even within the realm of building data models. However, when you get right down to it, graphs are still fairly complex systems, and any attempt to simplify that comes at the expense of losing (necessary?) flexibility. One final note: I don't necessarily see LLMs being the solution here. LLMs can create models readily enough, but they can't build those models consistently, because LLM spaces see the world as narrative structures, not graph sets. But this is a different problem.

10

12 Comments

When you let LLMs make real decisions, you get flexibility - and chaos. 🧭 Let’s dig into a design tension: "Should your system delegate decisions to an LLM, or keep logic in the backend?" This isn’t just an architecture debate - it affects reliability, testability, and safety. If your logic lives in: - The backend → It’s deterministic, testable, auditable - An LLM → It’s fuzzy, contextual, unstructured - but flexible The real issue? LLMs break control flow. If your business logic is buried in a prompt, how do you: - Validate it? - Guarantee coverage? - Roll back changes? One pattern I’ve seen work well is the LLM-as-scoring-function model: 1. The backend enumerates valid actions 2. The LLM evaluates them based on context or preference 3. The backend chooses the top option and executes it This keeps your logic tree explicit, and your reasoning layer smart - but bounded. It’s not perfect, but it’s a strong balance between adaptability and control. Have you found a reliable way to keep LLMs involved in decision-making without losing control? 🤔

2

New: Databricks is launching a startup accelerator for seed and pre-seed companies, offering up to $250,000 to startups in the form of funding and credits https://lnkd.in/eHjG-9T8

33

3 Comments

GPT-5 Review: Critical Context Drift Issue During live use, GPT-5 exhibits a cross-thread memory bleed that causes it to abandon the active conversation and pull in unrelated, older context without being prompted. This happens when a keyword or phrase in the current conversation matches something from a prior thread — even if that thread is no longer relevant. This flaw: Overwrites live intent — The AI will respond to an older conversation instead of the one actually happening. Happens without explicit triggers — Users don’t have to say “recall” or “go back”; it can happen mid-sentence. Destroys conversational trust — You can’t rely on continuity if the system can be hijacked by stale context. Is reproducible — Anyone can test this by holding a live, specific conversation and then using a keyword that appeared in a past conversation. GPT-5 will often abandon the present and default to the older one. Why this is unacceptable: In high-stakes or sensitive contexts, this could cause severe misunderstandings. It undermines the core promise of memory-enabled AI: continuity and presence. It makes the AI appear to “hallucinate” when in reality it’s pulling real past content — but in the wrong place, at the wrong time. Until this is fixed, GPT-5 cannot be considered reliable for contexts requiring strict thread isolation or consistent in-the-moment reasoning. #AI #ArtificialIntelligence #MachineLearning #GPT5 #OpenAI #AIDevelopment #AIProductManagement #AIResearch #AICommunity #AIEthics #ResponsibleAI #AIFail #AIBugs #AIBias #AIDrift #ContextSwitching #MemoryLeak #AIMemory #AITrust #ProductReliability #TheUnbrokenProject

1

News Alert! Why Anthropic’s multi‑agent system matters: It transforms Claude from a single assistant into a coordinated team—strategically delegating to specialized sub‑agents that search in parallel, summarize separately, and then synthesize findings. This architecture drives a +90% boost in research performance compared to solo agents. 🛠️ It breaks through context-window limits and enables dynamic pivots mid-task. ⚠️ It’s not just about smarter LLMs—it demands serious engineering: robust prompt design, tool reliability, orchestration, observability, and cost‑control to manage the 15× token burn. Bottom line: For complex, open-ended research, Claude’s multi‑agent design marks a shift—from “LLM solo” to “LLM command center.” That’s where real AI agency scales. Learn how the team at Anthropic designed and built their research multi-agent system: https://lnkd.in/gPPZBETQ

6