-
-
Login page. Domain main will take the user to this page.
-
Registration page. New user needs to register, and then login.
-
User can assign a project name after clicking "Create New Project"
-
User can logging to account and see the list of projects, or create a new one.
-
After uploading images in project view, the user can click "Begin Modeling" to run the panoptic segmentation modeling on each image.
-
The PyTorch model will generate the labels for each image and display them below each image. User can click "Home" to create a new project.
-
Full application concept with both use cases released. The user will be able to select the generated labels to control creating a new image.
Inspiration
Art is always present as we experience the world. We find it everywhere through life; nature, sports and sculptures. It is part of what defines the human race. As we are overwhelmed with information, sometimes we do not have time to appreciate the art in the experiences we have. We believe our experiences should be art. This why Pyvinci was created, to make life trips and experiences a beautiful canvas.
What it does
Pyvinci is a hashtag generator and an image art painter that allows the user to upload images for a given experience or trip and it generates art according to the objects observed in the image. Besides creating beautiful art, Pyvinci provides the right hashtags for the user to use in their social media platforms.
How we built it
The main areas of the application; client, server, and the models were deployed in separate docker containers to allow the app to have easier and faster deployment. The machine learning model of Pyvinci was deployed as a worker so it could be a more scalable solution. The model picks new jobs for every new projects that runs the model. We used a panoptic segmentation model to catch every possible background and foreground object image. The main server API receives a new job from the client and adds it to the database for the workers to pick up and perform modeling on.
Challenges we ran into
The biggest challenge we encountered was when developing the algorithm that creates the art. We wanted to use a layer base sequential framework for scene generator with GANs as the second worker model. However, the strategy that we use did not allow us to have the required time to train the model to the level of accuracy that we wanted.
Accomplishments that we're proud of
We are very proud to have complete a deployed app that uses a machine learning model.
What we learned
How to work with PyTorch and its all many of its components. How to create panoptic segmentation models and many other types with Detectron2. Also, how to work with docker containers to have a better structure to scale the application. How to best deploy machine learning models with an application.
What's next for PyVinci
We are going to better train the GAN model to be able to generate art from the multiple objects we extract from the images to give users the ability to control how the image is generated. Also we would like to create a queue that the workers pick up jobs from rather than pick them up from the database.
Log in or sign up for Devpost to join the conversation.