Spotlight

Inspiration

The project started with a very simple goal: how can we be less forgetful of things we care about, in this age of information fragmentization.

With the current technologies delivering breaking news around the globe 24/7 to our screens, we often mistakenly believe that we're more connected with people we care about than ever before. Yet most of the times, it's quite the opposite: this exact surplus of information has shortened our attention span so much that we could read about missing children on the borders, decide to follow the story till the end, and still get distracted by another story a week after -- completely forget about our 'promises' to ourselves, which could further lead to people in need getting less help without proper public attention, since we certainly aren't alone with this problem.

Thus, Spotlight came about (honoring the inspiring Oscar winner with the name), aiming exactly at such pain point -- give back to people not only a longer attention span on news stories they care about, but also more compassion and hopefully more social impact by advocating for their beliefs over a longer period of time.

What it does

Spotlight has two core functions: 'Spotlight' a news thread with keyword or URL to get real-time updates on them & See popular 'Spotlights' from similar people in terms of interest with the option to 'Spotlight' or mark as 'Not interested' (Yes it evolves to be more relevant over time)

How we built it:

We divided our tasks into three parts: user interface design, ios app development and web scraping & server. For UI, we used the Marvel App to map out all the design features and interfaces we want. For ios app development, we coded in Swift on the XCode IDE. To generate the data from the keywords input, we wrote a python program to scrape relevant information (headline, summary, time, topics) from the Guardian news website and set up a Paperspace virtual server to respond the requests sent by the clients.

Challenges we ran into:

We spent a great deal of time trying to fix server/client communication. We chose HTTP protocol for its simplicity, but there's very little documentation we can find. To make things worse, the server we rented didn't allow administrator access. We struggled a lot before successfully sending the requests and receiving the responses. The second challenge is actually about web-scraping. We used the Google API to search for keywords the user inputted and return the URLs found so that we could use regular expression to find useful information in the html code. However, the search engine often recognizes us as a robot and asks for verification. We spent lots of time trying to pinpoint the actual command that looked suspicious to the search engine and find a way around it.

Accomplishments that we're proud of:

With scrapping program and a headline matching algorithm, we can outline a news event by showing the timeline of critical reports and their summaries. We successfully built an app that not only allows people to follow certain news topic, but also see how the events develop historically by looking at the story lines of social movements like MeToo.

What we learned:

We experienced firsthand the gap between "theoretically feasible" and "actually working". We always run into errors that we didn't expect. We gradually learned the underlying mechanisms behind the APIs and tools that we are using and honed our problem solving skills in the mean time. We also learned several amazing coding languages and techniques. Finding and following tutorials and reading documentations will also become great assets for us.

What's next for Spotlight

We want to use not only the Guardian, but multiple news sources including CNN, BBC, WSJ to ensure the authenticity and impartiality of the news alerts and story lines we provide. Many of these news sources do not come with a summary at the beginning of their posts, which means we'll need to use NLP APIs to automatically generate a summary. We've decided to use the Attention-Based Summarization method from a Google Brain paper in 2016 and train it with the summary and Guardian news story data set. Moreover, we'll keep working on the user interface design to make our app look more attractive and appealing. We'll make the interface more colorful and add interesting icons and features.