ARgentum

Inspiration

Our inspiration for ARgentum stemmed from the desire to revolutionize education through augmented reality (AR) technology. We were intrigued by the potential of AR to enhance learning experiences and make education more engaging and immersive.

What it does

ARGentum is an AR-based educational platform that creates a virtual teaching environment where students can interact with virtual avatars of their teachers or create their own avatars. The platform utilizes AR technology to overlay virtual avatars of teachers, allowing students to feel like they are learning from their teachers.

How we built it

We built ARgentum using a combination of AR development tools, machine learning algorithms, and various text to speech API's. We started by working on the avatar creation and modelling and then progressed to work on the backend of how the avatar takes a voice input and responds. Currently we are using GPT-3.5 turbo LLM model to generate our responses. Lastly, we used a feature to convert the text generated to speech.

Challenges we ran into

The main challenge we faced was working around the multitude of API's as well as finding free API's. If not for the money constraint, we would have had a plethora of options to choose from and various API's we could use. Another challenge was working with the new NEXT.js template since many of the documentations present online have example code only for the previous. A few other problems were our time constraint due to personal reasons.

Accomplishments that we're proud of

We're proud to have developed a cutting-edge educational platform that leverages AR technology to enhance the learning experience. Our platform has the potential to revolutionize education by making it more interactive, engaging, and accessible to students around the world.

What we learned

Through the development of ARgentum, we learned a great deal about AR technology, Large language models, Readyplayerme api and many more. We gained valuable insights into the challenges and opportunities of integrating emerging technologies into the education sector.

What's next for ARgentum

We plan to make the following changes to our project: -> Use locally hosted large language models like llama2 or gemma and train them based on the teacher's temperament, style, etc. -> Make use of paid APIs like eleven labs to reproduce the teacher's voice for a more immersive experience. -> Integrate image and video generation while teaching using tools like DALL-E, Sora, etc. -> Make custom animations for our avatars using high level tools like Blender and Unity. ...and much more.

ARgentum isn't merely a project. It is a phenom in the virtual world that will change the way we live.

Built With

• chakraui
• javascript
• keras
• next
• openai
• python
• react
• readyplayerme
• tensorflow
• webrtc