AIMPATHY

Inspiration

Humanity has been building product that tracks us our health, such as heart rate, respiratory rate, blood oxygen level. However, there is lack of mechanisms to utilize speech as part of our health. Speech, however, is a great way to understand our health and condition. GPT is great at understanding the context of our speech in plain text and it even has great speech to text capability. However, in the process of processing our speech, it strips away the most important part of the speech which are the tone, pitch, and pattern. With digital signal processing that can quantify the tone, pitch, and pattern, along with GPT's text interpretability, now we can build out a way to measure health and condition from the entire aspects of speech.

What it does

1. Speech from the phone to Text - (still under development)
2. Spectral Analysis of the speech to quantify pitch, tone and speech pattern in categorical values
3. Plain Text to gpt-4 along with the pitch, tone and speech pattern information
4. Get information about the individual's condition and GPT's suggestions of music, food and ambient light setting

How we built it

1. Speech from the phone to Text - (still under development)
2. Spectral Analysis of the speech to quantify pitch, tone and speech pattern in categorical values
3. Plain Text to gpt-4 along with the pitch, tone and speech pattern information
4. Get information about the individual's condition and GPT's suggestions of music, food and ambient light setting

Challenges we ran into

Time Limit

Accomplishments that we're proud of

Unique idea and creativity.

What we learned

GPT is powerful and now most important aspect of project is creativity, impact and implementation

What's next for AIMPATHY

1. Have [pitch, tone, speech pattern, and other health information] as features and gpt generated condition as the label
2. Build individualized health monitoring model (since individual will have varying baseline tone, pitch and other biological information)

Built With

• gpt-4
• python