Storyboarding

Inspiration

As writers, we were inspired to streamline the writing process by creating a storyboarding app. We wanted to help people write freely without having to draw their own storyboards by using AI to generate images instead.

What it does

Our application allows users to create a text description by writing or using speech-to-text. The user can then upload an image from their own file system or choose to generate an image based on the text. The scene will then be added to the main screen for the user to peruse while writing their story. Users can also rearrange panels to flexibly adjust the order of their story and plan their plot progression easier.

How we built it

We used React to create the web application. Using Chakra, we were able to render components on the browser. Deepgram's API was used for the text-to-speech. Stable Diffusion was used to generate images based on the text. By utilizing AI, we were able to streamline the process of creating the storyboard.

Challenges we ran into

The most significant challenge we ran into was figuring out the drag and drop component of the application. The panels displayed on the screen should have been able to be moved around by the user. However, the components kept glitching, and it was difficult to debug, as the dnd module's documentation was not complete enough to understand.

Accomplishments that we're proud of

We are proud of successfully using Deepgram's API to generate text based on speech and using Stable Diffusion to successfully generate images based on text entered by the user.