SketchBook.AI

Relive your best memories through a sketchbook generated with the power of Artificial Intelligence.

Comment

Inspiration

The inspiration for SketchBook.AI stemmed from our desire to create a meaningful application that combines the power of artificial intelligence with personal memories. One of our major goals for this hackathon was to create an application that reflected the theme of HackGTX and that we also felt was meaningful. We were motivated by the idea of helping users relive their cherished memories through a digital sketchbook. With a strong belief in the potential of AI, we set out to develop an application that would provide a unique and creative way for people to revisit their past.

What it does

SketchBook.AI is an innovative application that leverages Flutter (Dart) and Firebase for backend operations. The app incorporates two essential APIs: OpenAI's DALL·E for text-to-image generation and a Flutter plugin for text-to-speech. The core functionality of SketchBook.AI revolves around the user's ability to relive their memories through AI-generated sketches.

Here's how it works:

  1. User Login: The user begins by logging into the application.

  2. Recording Memories: Once logged in, the user can record themselves describing a specific memory. This step is crucial as it forms the basis for generating sketches.

  3. Text-to-Speech Generation: The recorded speech is then converted to text using the text-to-speech plugin. The generated text appears in a text box, allowing the user to review it.

  4. Editing and Submission: Users have the option to either submit the generated text as is or make edits if the text-to-speech plugin didn't accurately transcribe their speech.

  5. AI-Powered Sketch Generation: After finalizing the text description, it's sent to OpenAI's DALL·E model using a POST request. The resulting image is obtained through a GET request.

  6. Digital Sketchbook: The generated image is displayed on a digital sketchbook page along with the user's description. Users can create up to six pages within their sketchbook and create multiple sketchbooks to revisit various memories.

Firebase is used to store user login credentials, as well as to save the images generated from the user's prompts and their sketchbooks.

How we built it

The foundation of SketchBook.AI is built using Flutter and Dart for the frontend, providing a visually appealing and user-friendly interface. Firebase serves as the backend for user authentication and image storage, ensuring secure data handling.

The integration with external AI services was a crucial part of our project:

  • For text-to-speech, we utilized the TextToSpeech plugin for Android.
  • For text-to-image generation, we connected to OpenAI's DALL·E model via API calls. A POST request was used to send the text description, and a GET request was employed to retrieve the generated image.

The entire development process was collaborative, with each team member contributing their skills in design, programming, and AI integration.

Challenges we ran into

Creating SketchBook.AI came with its fair share of challenges:

  • AI Integration: Integrating external AI services like DALL·E and ensuring a seamless flow of data between the app and the AI models was a complex task.

  • Accuracy: Ensuring the accuracy of the text-to-speech plugin was crucial, as it impacts the quality of the generated text and subsequent sketches.

  • User Experience: Designing an intuitive and engaging user interface to provide a smooth experience for users was a challenging aspect of our project.

Accomplishments that we're proud of

We're proud of several key accomplishments with SketchBook.AI:

  • Successfully integrating external AI services into the application to create a unique and engaging user experience.

  • Building a user-friendly and visually appealing interface to encourage users to relive their cherished memories.

  • Enabling users to create and revisit multiple sketchbooks, each filled with AI-generated sketches of their memories.

What we learned

Through the development of SketchBook.AI, we gained valuable insights and knowledge. We learned:

  • The practical application of AI in creative and personal contexts.

  • The challenges and complexities of integrating external AI models into a mobile application.

  • The importance of user experience design and the impact it has on the app's success.

Almost all of our team had little to no experience with Dart, Flutter, or Firebase, so many of us learned a lot about these technologies. We all feel like we have a better understanding of how we leverage Flutter, Dart, and Firebase to make exciting new applications.

What's next for SketchBook.AI

In the future, we have several exciting plans for SketchBook.AI:

  • Enhancing AI Integration: We aim to further improve the accuracy and capabilities of text-to-image generation by exploring advanced AI models and techniques.

  • User Feedback: Gathering user feedback and insights to continuously enhance the application and tailor it to the needs and preferences of our users.

  • Expanding Features: Expanding the app's features to provide users with more options for customization and personalization.

  • Wider Platform Support: Expanding the app to support a wider range of platforms and devices to reach a broader audience.

  • Community Building: Building a community around SketchBook.AI, where users can share their AI-generated sketches and memories with others, fostering a sense of connection and nostalgia.

SketchBook.AI is a testament to the creative potential of AI, and we're excited to continue evolving and improving the application for the benefit of our users.

Share this project:

Updates