D.I.E.

Inspiration:

Policies pertaining to diversity, equity, and inclusion (DEI) are essential for creating inclusive and equitable institutions, governance, and workplaces. It is difficult to monitor policy changes over time and comprehend their effects because of:

1. The large number of policy changes.
2. Absence of sentiment analysis: are laws getting more lenient or more inclusive?
3. Finding themes might be challenging; what subjects are becoming more or less important?
4. In order to examine, contrast, and illustrate how DEI policies change and whether they have a beneficial or detrimental effect on inclusion, we set out to develop an AI-powered solution.

We had the opportunity to meet with Eric Gordon, a Communications Professor at Boston University, and Harlen Weber from Code for Boston. Their insights and expertise inspired us to explore diversity and inclusion policies as a key area of analysis. Their guidance helped us recognize the importance of leveraging AI to track and understand policy sentiment trends, ultimately shaping the direction of our project.

What it does:

1. Monitoring Policy Changes: Examines policy language changes before and after. 2.Sentiment Analysis: This method classifies policies as either positive, neutral, or negative using BERT-based NLP models.
2. Thematic Extraction: This method uses TF-IDF + Named Entity Recognition (NER) to identify important themes, such as Diversity, Equity, Inclusion, and Healthcare.
3. Policy Network Graph: This tool uses PyVis and NetworkX to map relationships between policies according to common DEI themes.
4. Heatmaps and Tables: These tools offer lucid insights into thematic movements and trends in policy attitude.
5. Streamlit's Interactive Dashboard enables users to dynamically examine and visualise policy trends.

How we built it:

1.Data collection: 18F Handbook and other sources of DEI policy documents were used.

1. Text Preprocessing: Policy text was cleaned and prepared for analysis (removal of stopwords, tokenisation).
2. Sentiment Analysis: Accurate sentiment classification with refined BERT-based NLP models. Policy Similarity Measurement: To compare policy texts, TF-IDF + Cosine Similarity was applied.
3. Policy Network Graph: Using NetworkX and PyVis, an interactive graph was created to show the connections between policies according to recurring themes. 5 Visualisation & Dashboard: Using Seaborn, Matplotlib, and Pandas, interactive charts, heatmaps, and tables were created.
4. Streamlit Integration: Developed a user-friendly web application for exploring policy sentiment and network connectivity in real time.

Challenges we ran into:

1.Managing Complex Policy Texts Sentiment detection is difficult in policies because of their legal and bureaucratic nature. 2.Conflicting Feelings in a Single Policy: Some policies use terminology that is both inclusive and restricted.

1. Thematic Classification Issues: Using dynamic definitions of popular DEI themes instead of pre-made keyword lists.
2. Graph Structure Optimisation: Determining the most effective method for meaningfully representing policy relationships.
3. Computational Limitations: In order to optimise BERT models when running on huge texts, GPU support was necessary. 6.Optimising real-time processing and visualisations for seamless engagement is the goal of Streamlit Deployment.

Accomplishments that we're proud of:

1.The first-of-its-kind DEI Policy Tracker is an AI-powered system that monitors sentiment and policy changes. 2.Accurate Sentiment Analysis: A refined BERT model has increased the accuracy of sentiment identification. 3.An interactive network of DEI policies and their thematic links was created using the Policy Network Graph.

1. Real-Time Interactive Dashboard: An easy-to-use Streamlit software for policy monitoring has been successfully launched.
2. Better Policy Insights: Presented proof of DEI trends and sentiment changes over time based on data.
3. Scalable and Expandable: The framework can be used for healthcare, climate, and financial policies in addition to DEI policies.

What we learned:

1. AI in Policy Analysis: Sentiment categorisation in policy texts is greatly enhanced by BERT-based NLP models.
2. NetworkX + PyVis facilitates the development of significant connections between policy documents through graph theory and visualisation.
3. The Value of Data Cleaning: For NLP models to function effectively, legal and bureaucratic language must be preprocessed.
4. Interactive Dashboards Are Important: Streamlit's real-time visualisation enhances accessibility and user experience.
5. A more comprehensive perspective of policy changes was obtained by combining many techniques, including TF-IDF, NER, and graph embeddings.

What's next for D.I.E.:

1.Increasing the Dataset: Add more policy documents from many fields, such as healthcare, finance, and climate. 2.Better Sentiment & Bias Detection: Use more sophisticated AI models (like Llama-2 and RoBERTa) to identify implicit biases in policies.

1. Geospatial Policy Analysis: Use Geopandas + Folium to map the differences in DEI policies by nation or region. 4.Connecting Policy Changes to Practical Effects: Examine if DEI policy changes result in real benefits at work or in institutions.
2. Real-time policy modifications can be obtained by integrating web scraping tools such as BeautifulSoup and Scratchy.
3. Make sentiment classification easier for policymakers to understand and comprehend with Explainable AI (XAI).

Built With

• and
• and-natural-language-processing.->-hugging-face-transformers:-classifying-sentiment-(bert
• clustering
• dashboard
• data
• developed
• distilbert).
• graph
• heatmaps
• interactive
• k-means
• matplotlib
• network
• networkx
• policy
• pyvis
• scikit-learn:
• seaborn
• sentiment-analysis
• streamlit:
• tech-stack:->-python:-policy-comparison
• tf-idf
• vectorisation.
• visualisation.
• visualisations