Magic Events

Organizer Dashboard
Company Dashboard
Floor Plan

Inspiration

Two of us have organized over 15 events in the past three years. But we stopped organizing them because it was too operationally challenging. So we wondered, can we take most of the work out of organizing events and focus on creating the best in-person experience?

Problem

Event organizers face numerous complexities, such as managing venues, logistics, catering, and attendee information. Think about organizing hackathons such as MHacks, for example. These tasks are time-consuming and often require meticulous attention to detail. Moreover, organizations sending groups to these events must manually handle each participant's details, including names, emails, and travel arrangements, adding to the logistical challenges.

What it does

Magic Events is an all-encompassing solution designed to simplify and enhance the event management process. Key features include:

Automated Venue Selection and Catering Arrangements: Simplifies the process of choosing venues and organizing catering, including AI-assisted negotiations with food vendors. Personalized Floor Plan Creation: Offers a visual representation tailored to specific event types, aiding in spatial planning. Streamlined Event Scheduling: Facilitates the construction of well-organized event itineraries. AI Negotiator: Employs speech-to-text and text-to-speech technologies for efficient food vendor negotiations, adhering to budget constraints. Group Travel Arrangements for Companies: Enables easy flight and hotel bookings for groups via CSV spreadsheet inputs, automating what was previously a manual process.

How we built it

The technical stack includes:

Frontend: Developed using Next.js and JavaScript. Backend: Built with FastAPI and Python. Hosting: Utilized Google Cloud, Cloud Run, and Docker for deployment. Floor Plan Visualizer: Integrated GPT-4V with OCR and OpenCV for automatic image labelling. Scheduler: GPT-4 with data from floor plan. AI Negotiator: Combines voice conversion, GPT-based text generation, and ElevenLabs' voice output. Flight Details: Implemented through the reverse engineering of Great Escape and automated with Selenium for flight bookings.

Challenges we ran into

Image Labeling Challenge

Our initial approach was to use GPT-4V for direct image labelling, but we soon discovered that it couldn't accurately create bounding boxes independently. This was a significant hurdle because we needed precise segmentation of each floor plan area for event management.

To overcome this, we developed a novel solution: combining TensorFlow with GPT. We created a model in TensorFlow to label each segment of the floor plan with a unique number. Then, we used GPT to assign an activity component to each numbered segment. This way, we could search for these numbers using OCR and utilize the bounding boxes generated by it. This hybrid approach, blending advanced machine learning models, was innovative and proved more effective than our initial attempts.

AI Conversation System Challenge

Another complex task was creating a real-time AI conversation system for our AI negotiator feature. The existing models required cumbersome processes involving recording audio, saving files, converting them to text, generating responses, and then converting these back to voice. This process was far too slow for our needs.

We devised a streaming approach to create a more seamless and real-time experience. We implemented a system that took 3-second samples of spoken audio, transcribing it directly into an audio file. This file was then streamed to GPT, which quickly generated a response. The response was immediately streamed into the voice model, which streamed the audio output. This method drastically reduced the delay, making the conversation feel real-time and much more natural.

Accomplishments that we're proud of

Our most significant accomplishment with Magic Events is creating a unified, comprehensive tool that significantly eases the event planning process, particularly for non-profits with limited resources. We're proud of how we've managed to integrate complex systems like TensorFlow, GPT-4V, and real-time AI negotiations into a seamless, user-friendly platform. This integration showcases technological innovation and directly addresses the practical challenges event organizers face. By reducing logistical burdens, Magic Events allows these organizations, especially non-profits, to focus more on their core mission and less on operational complexities, making event planning more accessible and efficient.

What we learned

During this project, we learned that sometimes it is not good to implement all of the features we think of. We thought that the project would not show its value if we did not include features such as automated systems to find food vendors, venues, and more. However, we fell short of finishing our front end, though all of our backend services worked.

What we finished:

ML segmentation of venue maps, as well as the labelling
Automated scheduling based on the type of event and event details
Finding appropriate food vendors based on the schedule
AI agent negotiator to find the cheapest food vendors to fit into the event budget
Finding the cheapest flights and hotels for companies travelling to events

What is left to finish: