Lumin

Bright and Intelligent: Connecting as many LLM Agents as you want to solve your problems and answer your question. Teamwork equals Dreamwork.

Comment

Inspiration

The inspiration for Lumin came from the challenges faced by current AI systems, particularly in maintaining accuracy and reliability across varied and complex tasks. Our aim was to enhance the integrity and efficiency of AI-driven solutions, making them more trustworthy and versatile across different domains.

What we built

  • Lumin: A system that allows easy integration of LLMs into existing systems.
    • Lumin uses a multi-LLM approach, letting multiple LLM agents with different roles work together in teams to accomplish a task. Multiple different configurations of Teams can be created which then can be interacted with through sessions (API, or through the Lumin frontend app).
    • Lumin has an integrated vector database which can be populated by uploading files, e.g. pdfs, which allows for integration of organization data which can then be further used by the agents
  • Lumin frontend app: An application that allows to use the features of the Lumin API in a simple to use application:
    • uploading files, that will then be processed and stored in the vector database
    • managing different team configurations
    • creating and interacting (chatting) with sessions (in OOP terms: sessions essentially instances of a configuration)
    • prompt the Teams through the created sessions (possible with or without giving the agents access to the vector database)

Tech Stack:

  • Lumin B

  • Lumin Example (a solution to the Continental challenge):

    • backend:
      • logging of errors through an api endpoint /log/add and retrieval through /log
      • getting a systematic description of an error through the api endpoint at /wtf
    • frontend
      • view logs (/logs api endpoint)
      • check what an error means (/wtf api endpoint)
      • (the /log/add api endpoint is not integrated in the frontend, since errors should be sent to the server by the external automation system)

Tech Stack

  • Lumin backend

    • python
    • fast-api
    • weaviate
    • sqlite (any other database could be used as well)

  • Lumin frontend app

    • python
    • streamlit

  • Lumin Example (Continental Challenge):

    • backend
      • python
      • fast-api
    • frontend
      • react

What It Does

Lumin utilizes a novel architecture where multiple chat agents collaborate and submit their outputs as drafts to multiple supervisory agents. These supervisory agents are specialized to handle multiple different aspects of verification, such as factual accuracy, context relevance, or language fluency. This ensures a comprehensive review process where each response is meticulously vetted before being chosen or returned for more work to be done automatically.

How We Built It

To implement this complex system, we used:

  • Docker: To manage our application in a contained environment, ensuring consistency regardless of the deployment platform.
  • FastAPI: For crafting efficient and scalable APIs that facilitate robust communication between agents and supervisors.
  • Weaviate: To provide semantic search capabilities that allow our agents to pull contextually appropriate data.
  • MetaGPT and other LLMs: Employed as supervisory agents, each fine-tuned for specific oversight tasks to ensure the highest quality of output.

The system is designed to be flexible, allowing for easy addition of more agents or supervisors based on the task complexity and requirements.

Challenges We Ran Into

  • Coordination Among Agents: Ensuring smooth and efficient communication between multiple agents and supervisors presented logistical challenges.
  • Balancing Load: Optimally distributing tasks among various agents and supervisors to prevent bottlenecks.
  • Quality Control: Developing a sophisticated feedback loop where supervisors effectively vet and improve agent drafts without excessive delays.

Accomplishments That We're Proud Of

  • Innovative Multi-Agent Framework: Successfully developed a multi-layered AI system where outputs are more accurate and context-aware than those produced by traditional single-agent systems.
  • Dynamic Scalability: Demonstrated the system's ability to scale dynamically by adding more agents and supervisors based on the complexity of the task and throughput requirements.
  • Enhanced Reliability: Significantly reduced the rate of erroneous information (hallucinations) typically seen in AI-generated content.

What We Learned

  • Advanced AI Coordination: Learned to coordinate multiple AI models effectively, enhancing overall system performance.
  • Feedback Efficiency: Gained insights into creating efficient feedback mechanisms that enhance learning and accuracy without compromising response times.
  • Scalability Challenges: Understood the complexities involved in scaling multi-agent systems, particularly in maintaining performance and accuracy.

What’s Next for Lumin

Moving forward, we plan to:

  • Expand to More Domains: Explore applications in more specialized fields, adapting our agent-supervisor model to meet specific industry needs.
  • Improve Inter-Agent Communication: Enhance the AI’s ability to communicate and collaborate more effectively.
  • Explore Real-World Applications: Test and refine Lumin in real-world scenarios, potentially partnering with organizations for pilot programs to further validate and improve our system.

Something like Lumin is poised to set a new standard in AI-driven solutions, offering unprecedented reliability and adaptability across a wide range of applications, by influencing the probability of hallucinations influencing the quality of generative work and information.

The Lumin Example (Continental Challenge)

Since it hasn't been mentioned earlier i'll talk a little bit about the example. The frontend is build using react (please appreciate my hard work in making it look halfway decent :) The backend utilizes the Lumin API to check for errors and what they could mean. To make this reliable, the vector database is populated with data provided for the challenge. This data has been uploaded using the Lumin frontend app (which has a nice button for it :), very easy). The example frontend calls the example backend, which (with the use some prompt engineering) then sends requests to the Lumin API to process the request.

Built With

Share this project:

Updates