Second Insight

Inspiration

Recently at a Morgan Stanley event, Homer learned that average investors frequently overlook risks. This insight motivated him to explore LLMs for risk evaluation, intrigued by AutoGPT's capabilities despite some limitations. He aims to leverage these technologies to help investors make informed, risk-aware decisions.

What it does

This innovative platform undertakes essential tasks such as scrutinizing annual reports and stock prices, and utilizing collected data to provide insightful risk assessments.

How we built it

Our platform is developed with OpenAI's LLM technology, akin to the RAG framework. It performs risk analysis by dynamically generating tasks tailored to each investment decision type. For our demo, we leverage public financial APIs to access stock price history and annual financial reports, ensuring a comprehensive and data-driven analysis.

Challenges we ran into

One significant challenge we encountered is OpenAI's tendency for "hallucination"—producing coherent but inaccurate risk analyses. Acknowledging that solving this well-known issue entirely during this hackathon is unrealistic, our approach instead demonstrates how the model can learn and improve over time. By analyzing both successful and unsuccessful runs, we refine the prompts used to guide the model's thought process. This iterative learning strategy shows promise for enhancing the accuracy and reliability of the analyses.

Accomplishments that we're proud of

Our team's collaborative effort led to the development of functional code (Demo: https://youtu.be/i5kxJhkhHWE), marking a significant achievement. We conducted evaluations using a combination of simulated data and human moderation to establish a reliable baseline. Beyond presenting our current progress, we take pride in having outlined a forward-looking plan and strategy for continuous improvement.

What we learned

1. GPT-4's limitations in comprehending financial facts became apparent, particularly with terminology not fully supported. Incorporating a dedicated financial knowledge base or employing a fine-tuned approach may be necessary.
2. We discovered that evaluating a combination of investment products, rather than focusing on a single stock as demonstrated, could offer more utility.
3. The significance of employing a knowledge graph (or Neuro-symbolic AI) for financial analysis was underscored. While GPT-4 showcased its capabilities, it highlighted the need for further advancements in this area.
4. The process of training—acquiring new knowledge and improving AI—emerged as a critical, long-term priority over the current system's capacity at its inception.

What's next for Second Insight

1. Expand the data sources to include a broader spectrum of information, such as competitors' performance, management team profiles, and industry sector analyses, to enrich our analysis.
2. Develop an internal knowledge base using a vector database, aiming to deepen our understanding and improve the model's accuracy.
3. Enhance the explainability of our risk analysis feature, focusing on providing users with clear insights into the reasoning behind risk assessments.
4. Implement a training procedure inspired by robotic task planning, combining fine-tuning of LLM with Reinforcement Learning (RL) techniques, to continuously improve the model's performance and decision-making capabilities.

Built With

• langchaine
• node.js
• openai
• python