Last Minute PPTs

Project which automatically generates crucial business and leisure presentation slides based on what you say in real time.

Comment

Inspiration

Too many projects to present? Too many presentations to make? Ever hoped making presentations could be easier and less tedious? Our project helps solve just this.

What it does

Project which automatically generates crucial business and leisure presentation slides based on what you say in real time. One just has to speak and everything will generated for you, be it title, summarization, images or charts, it covers everything.

How we built it

We made a website for easy interaction and accessibility, user visits the website and chooses a template which suits their requirements, and he/she clicks on the audio button and starts speaking, slides get inserted automatically and in no time you will have the presentation ready.

Speech to text

Behind the scenes, audio gets converted to base64 string and is sent to google speech-to-text API which is in turn is sent to our backend. Firstly it detects what is the context of the message, whether it is a title, summarization, image scraping, chart or tabulation task.

Backend

After detecting our NLP and DL models extract the required information, and run any required sub-process like image-scraping, chart/table generation and send the components to the frontend, where they are dynamically arranged to get slides!

Classifying the text

For text classification, all the current models are very heavy, we had to find a lite alternative and hence we used Spacy CNN model to get a feature representation of text, which we used to train our own neural network to classify messages

Extracting key sub phrases

We explored several models for summarization from seq2seq to TextRank but none of them fit our needs as we wanted a very concise summary, essentially a phrase so we wrote our own algorithm to parse a dependency tree and identify important sub phrases

Identifying entities in text

Our application has the feature to allow users to add images, plots and tables. It was important to detect the subject of images and other such entities to get the correct image or plot Hence we again turned to the trusty dependency tree and parsed it to extract entities

Challenges we ran into

First challenge we ran into was how to get the context of the text, i.e. whether this part of text is for important for title, a part corresponds to chart, table, image or summarization.

Secondly, all the current summarizers are very heavy and it was necessary for our project to make slides in realtime.

Making charts in realtime from the extracted features was another setback.

Identifying the image subject was also not an easy task

Accomplishments that we're proud of

Making a real-time model to parse and decode the features, which is still a hard task to achieve with higher accuracy, from the given text.

Providing a variety of features only based on the text.

What we learned

How to collaborate under pressure

What's next for Last Minute PPTs

Google Slides integration.

Share this project:

Updates