Vis10nary

• 

  Project Banner

• 

  Hand drawn system design of Online Book Store

• 

  The final generated code comprising all the components

• 

  The image is stored in the specific folder named "files"

• 

  Flask REST API

• 

  Books.js page in React

• 

  Orders.js page in React

• 

  package.json file for the React project

• 

  docker-compose.yml file for running the project as dockerized

Inspiration

"Hey let's build a software and make a business" every programmer with a creative mindset has faced this statement from friends/colleagues. This statement has changed history of innovation several times, in fact, many tools that we are using today is a result of this conversation. Although we hear this statement few times a year, many of the great ideas fade in time while doing all the preparatory works like planning, drawing a data flow, user identification and the most important part - coding. Completing all these stages to create a MVP(Minimum Viable Product) to launch or to showcase to investors is a tedious job as competitions are tough and launching to the market as fast as possible is a priority.

What it does

To cut down this timeframe from days to few seconds, project Vis10nary takes a hand drawn image of page flow design from the brainstorming session and creates the equivalent MVP in no time. With the help of Google's Gemini 1.5 Pro and Gemini 1.0 Pro Vision this tool creates real life projects in no time.

How we built it

Vis10nary first takes the snapshot of the page flow hand drawing from a page which is then passed to Gemini 1.0 Pro Vision model to extract different components like Frontend, Backend and Database and pages/modules present in each component. It then generates code files for each components and is saved in a variable. Then generated content is then fed to the Gemini 1.5 Pro model and a JSON formatted output is returned where each key denotes the file name (e.g. app.py, Home.js) and the value denotes the content. Each key in JSON is iterated and written into respective files. In addition a docker-compose.yml file is also generated which eases the process of running the application buy just entering the command docker-compose up provided Docker is already installed.

Challenges we ran into

In some runs, the output produced were not properly indented, rather the whole code came on a single line. In some runs, the output was not generated completely i.e. output ended in the middle of statements.

Accomplishments that we're proud of

The project could successfully generate ready-to-run code files which could easily be executed by using docker-compose up command. Most of the times, the outputs were easy to read and following coding standards.

What we learned

There is a huge potential with the Gemini API to translate real life drawings to codes and there are numerous use cases for this project. I learned how to train and fine tune Gemini in the Google AI Studio and use this as an API in any application, which provides great power to the projects.

What's next for Vis10nary

Next plan is to implement real time transformation of drawing to code, as time progresses and the plan drawing changes over time, the codes change accordingly. Also keep the project in running mode as when the drawing changes, the user can see the changes in the computer screen. Second, the AI sometimes cannot generate fully runnable code project, the uses had to do some tweaks in the codes in order to run the project successfully.

Built With

• gemini
• python
• react