R1 Deep Researcher
Fully local research assistant w @deepseek_ai R1 + @ollama. Give R1 a topic and watch it search web, learn, reflect, search more, repeat as long as you want. Gives you a report w/ sources at end. All open source ..
00:00
Lance Martin
1,294 posts
Lance Martin
@RLanceMartin
MTS @anthropicai
- I built an app that uses ChatGPT for question-answering over all 365 episodes of the @lexfridman podcast. Uses @OpenAI Whisper model for audio-to-text and @langchain. All code is open source (linked below). App: lex-gpt.fly.dev
GIF - o3-mini researcher Give it a topic, use o3-mini for report planning w/ human feedback, then parallelize all research/writing when plan is accepted. All open source (code below)
00:00 - RAG From Scratch Here's a set of short (5-10 min videos) and notebooks explaining > a dozen of my favorite RAG papers. Took a stab at implementing each idea myself (all code open source) and grouped according to the diagram. Repo: github.com/langchain-ai/r… Video playlist:
- I'm open-sourcing a tool I use to auto-evaluate LLM Q+A chains: given inputs docs, app will use an LLM to auto-generate a Q+A eval set, run on a user-selected chain (model, retriever, etc) built w/ @langchain, use an LLM to grade, and store each expt. github.com/PineappleExpre…
GIF - Finally got GPT4 API access, so built an app to test it: here's Q+A assistant for all 121 episodes of the @theallinpod. You can ask any question abt the shows. It uses @OpenAI whisper model for audio -> text, @pinecone, @langchain. App is here: besties-gpt.fly.dev
GIF - Fully local / open source deep researcher Works w/ any local model hosted via @ollama + @lmstudio. Uses diff tools (@perplexity_ai, @tavilyai, SearXNG, DDG). MCP for local files coming soon. Code: github.com/langchain-ai/l…
00:00 - Agents from scratch This repo covers the basics of building agents: + Fundamentals + Build an agent + Agent eval + Agent w/ human-in-the-loop + Agent w/ long-term memory Builds to a deployable agent to run your email Code (all open source): github.com/langchain-ai/a…
- Context Engineering @dbreunig and I did a meetup on context engineering last night. Wanted to share slides (below) + a recap of some themes / discussion points. 1/ Context grows w/ agents. @manusai mentions typical task requires ~50 tool calls. manus.im/blog/Context-E… 2/
Replying to @_mohansoloBut embedding search becomes unreliable as a retrieval heuristic as the size of the codebase grows. Instead, we must rely on a combination of techniques like grep/file search, knowledge graph based retrieval, and more. With all these heuristics, a re-ranking step also becomes - Deconstructing RAG It can be hard to follow all of the RAG strategies that have come out over the past months. I created a few guides to organize them into major themes and show how to build multi-modal / semi-structured RAG on complex docs (w/ images, tables). Here's a few
- Here's a simple (< 100 lines of code) app to run #ChatGPT question-answering on any uploaded document (using @langchain DBQA w/ ChatGPT API): …-doc-gpt-doc-gpt-q0823l.streamlit.app
- Document splitting is common for vector storage / retrieval, but useful context can be lost. @langchain has 3 new "context-aware" text splitters that keep metadata about where each split came from. Works for code (py, js) c/o @cristobal_dev, PDFs c/o @CorranMac, and Markdown ..
- open-deep-research is the best performing fully open source deep research agent on DeepResearchBench (100 PhD-level research tasks across 22 distinct fields). leaderboard: huggingface.co/spaces/Ayanami… code: github.com/langchain-ai/o…
- Building Agents: Free Course We just released a course with > 20 videos & notebooks focused on building agents. All code is open-source and the course is free! Context Back in June, I gave at talk at @aiDotEngineer on building agents with LangGraph. I got ~2 hrs of questions.
