Recipeasy

Harnesses AR and Snap Spectacles to teach you to make excellent food. Easy-peasy, Recipeasy!

Comment 1

Inspiration

As college students, cooking on our own for the first time was a challenge, and we wish there was a way to easily learn essential tricks and techniques to prepare delicious meals. Recipeasy is what we came up with!

What it does

Recipeasy is a Snap AR lens built specifically for the Snap Spectacles that guides you through a selection of hand-picked recipes. Recipeasy works like this:

  1. Open the lens and use your voice to search for a recipe from our extensive selection. Try, for example, “Search Chicken Casserole.” A list of our top recipes matching the search will be presented. Don’t know what to make? Scan the ingredients you have on hand, and Recipeasy will find the best recipes to make the most out of your ingredients!
  2. Recipeasy will then walk you through step-by-step instructions on how to prepare the specified recipe, featuring a fully interactive virtual AR tutorial so you can get your hands dirty before actually getting your hands dirty.
  3. Ready to cook? Recipeasy on the Snap Spectacles will serve as a heads-up display, showing you the recipe while you cook. No need to unlock your phone with your dirty hands to read the next set of instructions!
  4. Enjoy a delicious, home-cooked meal!

How we built it

We used the Snap Lens Studio, JavaScript, and a variety of different libraries for ML/AI/AR.

We split up tasks based on the user flow and then combined all our separate projects in the end by exporting objects and importing them into the main project on one computer.

Challenges we ran into

None of us had the experience of working with Lens Studio, and at first it was difficult to establish a starting point and direction. When we first imported the first template, there were so many parts of the Lens Studio that we were a bit overwhelmed. What are objects and how do we create them? How do we call scripts on certain inputs from the user? With the help of the documentation and mentors, we gradually started to navigate our way around the tool. We split off tasks and started working towards a solution by learning how each part of the tutorial code worked and then creating our own based on it.

Another notable challenge that we ran into was collaboration. Comfortable with using GitHub for collaboration, we were met with a roadblock when we wanted to put all of our different parts together. After consulting with a mentor, we decided the best path of action was to export in Objects and move all the code as imports to one main computer.

Accomplishments that we're proud of

In the final product we created, we are proud of the egg-grabbing physics-based animation, as well as the speech recognition and parsing we implemented, which made the whole AR experience as seamless as possible.

We’re also proud of the fact that we were able to take a problem that we all encounter on a daily basis and solve it with a piece of technology that we were all very new to yet interested in. We all have the shared sentiment that this is the hackathon where the project that we wanted to create required the most knowledge that we didn’t have in the beginning. So, we’re proud of what came out of the hours and hours of somewhat frustrating and extremely rewarding time we spent building Recipeasy.

What we learned

Throughout the course of this hackathon, we learned a lot about the intricacies of Lens Studio, a technology we were wholly unfamiliar with at the onset of the hackathon. We also learned the importance of ideation: we went through many different ideas and iterations before finally settling with Recipeasy as it is currently, which helped us to direct our development efforts and make sure no time was wasted developing a feature that wouldn't make it into the final pitch.

What's next for Recipeasy

In the future, we hope to build Recipeasy into a full-fledged application. We hope to expand our collection of recipes to be more robust and useful. We would also like to implement real-time object detection and tracking, so that Recipeasy can highlight the ingredients and utensils you need while you are in the act of cooking.

Built With

Share this project:

Updates