Social Sense

Sentiment break down

Inspiration

In the age of social media dominance where the internet dominates every facet of our lives, we thought it was important to tackle an issue related to our online exposure. In particular, mental health as a result of the algorithms we subliminally receive online was of great importance. After watching items like the Social Dilemma and seeing how social media is designed to maintain your attention, we wanted to see to what lengths these algorithms were willing to go. We also wanted to display to students like ourselves who often get caught up in these algorithms the kind of content they are watching. Thus, we decided to use Sentiment Analysis on different posts to analyze the concentration of our time spent going down these rabbit holes. This project is very prevalent in our lives and many others as well.

What it does

Social Sense is a tool that lets you monitor the content you are being exposed to on social media. Using natural language processing and sentiment analysis, Social Sense is able to categories the posts you see based on the perceived sentiment. This data is presented to users through our Social Sense dashboard which provides a visual representation and breakdown of what social media sources their content is coming from. In a digital world where we are constantly bombarded with negative content, Social Sense gives us the insight we need into where and what see online.

How we built it

The social media data was retrieved through the Facebook Graph API and Facebook Groups API. this allowed us to access the posts users were viewing but also the specific online groups and communities they came from. From the retrieved data, the text portions of recent posts were separated and ran through Google Cloud's Natural Language API which would return a sentiment score ranging from -1.0 to 1.0 (Negative sentiment to positive). These two APIs were encompassed in a Python Flask app which was capable of communicating with our dashboard backend through HTTP requests.

The database was built using CockroachCloud which has a similar syntax to PostgreSQL. The schema was a simple one data table to identify the sentiments listed above, Facebook user ids, & Facebook Group ids. With limited tests per hour allowed by the Facebook API, it was important to have a shared cloud database so that we did not exhaust our usage of the API. Using Cockroach also helped us save time by having to repeat queries into the DB and performing similar tests from different users. On the backend, Express and Sequelize were used to respectively host and easily connect with the database.

Challenges we ran into

Learning CockroachCloud over the weekend was challenging, especially attempting to integrate it into the backend and locating some of the files like the certificate. In addition, during the day the servers had high latency so to access the schemas and validate our work we had to transition to Cockroach's command line. Though the learning curve seemed steep, once Cockroach was fully integrated it acted similarly to other DB's used in the past and had an excellent query process.

The second challenge faced was figuring out why sending a get request from the frontend with parameters did not display as part of the request body when receiving the data. After trying over and over to manipulate the data to display as part of the body, we realized that HTTP protocol does not allow data to be sent as part of the body for get requests. This is because a get request is not meant to have transformative data that is meant to affect the backend, and as such, it is only stored in the less solidified query parameter.

The third challenge we encountered was the actual retrieval of data from social media sources. Originally we planned you using a web scraper to gather data from sources but, this proved to be a complicated task when requiring a constant update on the user's feed, and when gathering data from multiple social media platforms. To resolve this challenge we resorted to using APIs provided by Facebook that allows the retrieval of post data, with this new tool we focused our efforts on Facebook content and provided a breakdown of sources within the platform.

Accomplishments that we're proud of

First and foremost we are proud that we create a project and develop a prototype that focuses on the issue of mental health and the toll that social media plays on our well-being. We also take pride in the fact we tackled this issue while using new and fascinating tools in the development process.

Another major accomplishment for us was creating a full-stack project that covers a broad range of development tools and stages. Some of these include integration with CockroachCloud for our database, the interactions between our front and backend, and the AI implementation through our Python Flask app. The interaction between different aspects of this project allowed our team to strike a balance between gaining knowledge through our work while playing off of each other's strengths. The team built upon a core set of technical competencies and developed them with new situations. Building a full-stack application pushed us to integrate all the components of the application while ensuring the flow of data stayed intact.

What we learned

One of the main takeaways from this experience was how we learned to take advantage of our team dynamic and each other's skillsets. When tackling a comprehensive project such as this one many times groups tend to lean on a single person for the brunt of the workload, this is a situation all of our team members have experienced in the past. As we worked we quickly became aware of what each other were comfortable with and took the lead on our own respective tasks. This allowed us to gain valuable insight on leading portions of a project and learn technical skills by playing a supportive role in portions of the project we were less familiar with. Some of the skills we learned from each other include, utilizing Figma for frontend visuals, the integration of databases into a full-stack project, and machine learning techniques focused on natural language processing and sentiment analysis.

What's next for Social Sense

We believe Social Sense only touched the tip of the iceberg. There are hundreds if not thousands of untapped verticals relating to understanding the psychology and human patterns behind the algorithms dominating social media. The most important piece of the future is collecting more data. Our hypothesis is that if people were able to accurately see the categories they watch and a potent visualization of the path they are led down, they will become more conscious about their choices of watch they watch. We are also hoping to generate a general level of understanding of the dangers of not diversifying the sources we receive information. Therefore some specific objectives that align with these initiatives include:

Display the information in a roadmap or tree to see how the algorithms choose what you watch next
Use more accurate machine learning models to better classify the data (i.e. more sentiment categories outside of just positive vs negative)
Display the data by category and demonstrate how concentrated our time becomes on certain sectors
Cross Analyze different platforms to see which companies focus the most on maintaining customer focus and at what price (i.e will they maintain your attention for negative content)