Inspiration
Being frustrated at looking through tables of contents, indexes, when technology provides easier, more functional searching.
What it does
An iOS app that enables you to search real life documents by simply taking a picture of them. If the picture is of a page of a book, we also tell you other places within that book where your query matches! Whether you're a recruiter looking for candidates with specific skills, or simply a student trying to find which page talks about method overriding, this app is for you.
How I built it
Using Swift 2, iOS 8.4, we made an iOS app (the first app for most of us on the team). We used tesseract, an open-source library for OCR, to extract text from pictures. We also use google books API to find book matches and to search within a book.
Challenges I ran into
Making api calls in Swift (and using the JSON it returns) is not exactly the most intuitive task, but thankfully we found a library (SwiftJSON) that helps a lot. We also found that tesseract does not expose each character's enclosing rectangles in easy ways, so the task of searching within the recognized string, and only highlighting the matched characters is still a bit buggy. We are often able to highlight characters close to the match, but a few characters offset. Given more time, (and we will definitely continue developing this), we could probably improve the accuracy of the highlighting.
Accomplishments that I'm proud of
Creating and running my first iOS app. Being able to click a button and debug on my own phone was quite exciting. I also finally got to learn Swift 2 and put it to practice.
What I learned
Basic (if messy) iOS app organization. How to use the iOS frameworks to build dynamic apps, etc.
What's next for Highlight
Highlight still needs a better connection to google books. Right now, the api doesn't expose the actual page numbers where our query matches, so we linked to a UIWebView of the online google book viewer. Not only that, but enabling the google books api section of the app makes it a little unstable, so that is something we still need to work on.
Log in or sign up for Devpost to join the conversation.