AI Music Video Show

Nexartis: Where Human Creativity Meets AI Technology

Comment

Inspiration

Our journey began with The AI Music Video Show in October 2023, showcasing where Human Creativity meets AI Technology in the digital media world. We quickly identified a gap in the industry: the lack of viable revenue streams and transparent compensation models for creators. This inspired us to create the Open Commercial Media Ecosystem (OCME) in June 2024. As our community grew, we realized the need for a tool to streamline the curation process for the increasing volume of digital media content, leading us to develop the OCME Curator Manager.

What it does

The OCME Curator Manager is a powerful tool designed to streamline the curation process for The AI Music and Video Channel and future channels in the Lotus Network. It enables curators to efficiently manage and organize the vast array of creative works within our Open Commercial Media Ecosystem. The tool facilitates the creation of themed compilations and special programming, as demonstrated by our ability to craft a Halloween special for our AI Music Video Live Shows.

How we built it

We developed the OCME Curator Manager using a comprehensive and modern tech stack:

  • Started with a Vercel Turbo Repo Mono Repo, setting up three React apps for our web pages.
  • Implemented Infrastructure as Code (IaC) using SST (Serverless Stack) and AWS to deploy our apps to production environments (www, ocme, curator).ragathonlink.
  • Built a llama-create app incorporating Pinecone for vector storage and a multi-agent workflow.
  • Deployed the front-end of llama-create to production using IaC/SST.
  • Developed and deployed the llama-create Python back-end to production using Render.
  • Enhanced our Python back-end with a Toolbox for extended functionality.
  • Integrated Arize into our Python back-end for enhanced monitoring and observability.

The core of our system is a multi-agent RAG (Retrieval-Augmented Generation) system, leveraging LlamaIndex, PineCone and ToolBox for efficient information retrieval and processing of digital media content.

Challenges we ran into

Our main technical challenge was deploying the llama-create python backend as a serverless lambda function. Due to significant technical challenges and the time constraints of the Rag-A-Thon, we were ultimately prevented from continuing to work on this project.

We also faced challenges in integrating multiple technologies (React, Python, AWS, Pinecone, Arize) into a cohesive system.

  • Integrating multiple technologies (React, Python, AWS, Pinecone, Arize) into a cohesive system.
  • Ensuring efficient retrieval and processing of large volumes of digital media content.
  • Implementing a multi-agent workflow that could handle complex curation tasks.

Accomplishments that we're proud of

Successfully deploying a multi-agent RAG system into production - a significant technical achievement!

Creating a fully functional Curator Manager during the Rag-A-Thon, addressing a critical need in our ecosystem.

What we learned

Through this project, we gained valuable insights and skills:

  • How to build and deploy a multi-agent RAG system with "all the bells and whistles."
  • Effective use of LlamaIndex for information retrieval and processing in AI-driven applications.
  • Techniques for optimizing Python backends for serverless environments.
  • Integration of various cloud services (AWS, Render) and tools (Pinecone, Arize) for a robust, scalable application.
  • The importance of Infrastructure as Code for managing complex, multi-component systems.

What's next for AI Music Video Show and OCME

Nexartis will be developing the following tools to enhance our ecosystem:

  1. Identity Media Suite
  2. Media Registry
  3. Player
  4. Streamer
  5. Payment Engine

These tools will integrate with our OCME Curator Manager, expanding its capabilities and efficiency in managing digital media content.

We've learned so much from this Rag-A-Thon and all the products showcased here. We're excited to apply this knowledge as we continue building and improving our system, leveraging the latest advancements in AI and digital media content management technologies.

Our ecosystem includes various projects such as:

All of these are part of our broader vision to shape the future of content commercialization and distribution within the Open Commercial Media Ecosystem (OCME).

Built With

  • amazon-web-services
  • infrastructure-as-code
  • llamacreate
  • llamaindex
  • monorepo
  • pinecone
  • python
  • reactapp
  • render
  • serverless
  • svelte
  • toolhouse
  • turborepo
  • twelvelabs
  • vercel
Share this project:

Updates