ragas
186 posts
ragas
@ragas_io
Supercharge Your LLM Application Evaluations 🚀
Github: github.com/vibrantlabsai/…
Discord: discord.gg/5djav8GGNZ
Joined March 2024
- We are preparing the roadmap for ragas v0.2 and would love to get all your feedback. We invite the community to discuss these plans, share your thoughts, and help shape the future of ragas. ▸ Poll here app.rallly.co/invite/WOSpLw4… to let us know your preferred timings for the
- Weekly release update : ragas v0.1.14 is out 🚀 Major new features 👉🏽 ▸ New Metric based on @vectara 's new HHEM model which is a better Hallucination Detection Model. ▸ Integration with @helicone_ai (YC W23) which is an open-source platform for logging, monitoring, and
- This would not have been possible without the love of our oss community ❤️Ragas is now a first class eval suite — mentioned directly in the Intel's latest RAG Foundry project and earlier OpenAI dev day mention Built from BLR & Kochi via YC 🇮🇳 x AI 🙌
- Reach out to us on [email protected] if you're interested in being part of this 🔥Been cooking something 🍳 for synthetic test data generation from documents to evaluate RAG systems. ▸ 10x improvement over the last version in terms of quality and scalability. DM me if you're interested in participating in private beta testing of this feature.
- This is exactly what we are solving for & we do it as open-source.A barrier to faster progress in generative AI is evaluations (evals), particularly of custom AI applications that generate free-form text. Let’s say you have a multi-agent research system that includes a researcher agent and a writer agent. Would adding a fact-checking agent
- Join us on May 30th to learn about some of new things we are working on from @Shahules786Join us for @Haystack_AI's May 30th Webinar: Evaluating RAG Pipelines. 🚀 Featuring talks by our @atitaarora and @Shahules786 from @ragas_io. We'll explore a critical step in bringing your RAG application to production: effective RAG evaluation. lu.ma/7vc43x9q
- When building your #GenAI application with @vectara, you can now evaluate your pipeline using @ragas. Thanks @Shahules786 and Jithin James. Learn how in this new blog post: vectara.com/blog/evaluatin…
- We are welcoming feature requests to add new metrics to Ragas. 🚀 ⭐️Fill out this simple 3-question form to have your desired metrics added to ragas forms.gle/pKvxPM4AVAn7R7… You're free to propose any metrics or tasks that can help with evaluating LLM applications. We will also
- Releasing our first custom model as part of the ragas framework. Many more to come as we tackle the hard problem of LLM application evaluation and testing. Do check it out.As our journey with YC W24 wraps up, we're thrilled to share a special gift with you all! 🎁 We are releasing one of first model as part of @Ragas_io for synthetic test data generation - ragas critic LLM to replace GPT-4 as critic. ⭐️ ▸ Finetuned + GPTQ quantised Qwen 1.8B
GIF 
00:00- Replying to @cmgriffingthanks for the feedback. We will take a look at this.
- Loved it ❤️A critical skill for AI engineers → evaluation ← RAGAS is a library built for evaluating RAG pipelines, here we dive into metrics-driven development for AI agents (built with @langchain @pinecone @cohere and @AnthropicAI) #GenAI #AIEng
00:00
