Loving Vincent

Converting movie frames to paintings using Deep Learning.

Comment

Inspiration

Loving Vincent is a 2017 experimental animated biographical drama film about the life of the painter Vincent van Gogh. It is the first fully painted animated feature film. The film, written and directed by Dorota Kobielaand Hugh Welchman, is a Polish-UK co-production, funded by the Polish Film Institute, and partially through a Kickstarter campaign.

Each of the film's 65,000 frames is an oil painting on canvas, created using the same techniques as Van Gogh by a team of 125 artists drawn from around the globe. It won Best Animated Feature Film Award at the 30th European Film Awards in Berlin and was nominated for Best Animated Feature at the 90th Academy Awards.

A movie production of this kind demands a huge capital and a huge time frame. For an arts student trying to re-create this movie under a student budget, the entire process is an impossible task.

From a financial perspective: • Hiring a team of 2000 painters over a 6-year course period accounts to a production cost in millions. • A selection round of 5000 painters is again a hefty task

What it does

We are planning to automate the process of converting real-life movie frames to paintings using Deep Learning algorithms.

How we built it

We plan to use Neural Style Transfer on movie frames and then concatenate them to form a smooth video file. We target movie makers and art students, as we provide them with a tool to automate conversions of movie scenes to paintings.

Challenges we ran into

  1. Huge Processing time
  2. Choosing the perfect fps to make the output non-choppy

Accomplishments that we're proud of

A traditional approach to this problem would take up to 6 years of work and a million- dollar budget. We aim at cutting down these factors as we believe time is a major resource in the art world. We enable users to let out their creativity instantly without any barrier of time and capital. We reduce the unnecessary time delay and capital depreciation, so the user only has to focus on the story line rather than hiring painters or bulking up on bank loans.

What's next for Loving Vincent

  1. A Graphical User Interface for easy file selection.
  2. Smooth controls to alter the features of the paintings selected.
  3. Custom GPU/TPU selection
  4. Neural Network Optimization
  5. Distributing workflow over multiple GPUs
  6. Constructing a cloud-based GPU workforce for around the clock conversions
  7. Credit approvals for Cloud access
  8. Pipeline transfer to cloud
  9. Encrypted data-transfer throughout the workflow to avoid data penetration.

Built With

Share this project:

Updates