Linky

Linky pipeline

Inspiration

Do you love listening to podcast, watching videos, influencers but find yourself drowning in the sea of content? Are you frustrated by missing out on crucial information or struggling to distill key insights from lengthy episodes or videos?

What it does

Enter Linky, your personalized content companion. Linky revolutionizes your podcast experience by curating interactive episodes and videos from your subscribed channels. Let's say you're into health-related podcasts like Huberman and Paul Walker. You're keen on enhancing your sleep quality and learn actionable tips, but the combined runtime of these podcasts exceeds 15 hours. With Linky, you can extract the essence of these podcasts and even pose personalized queries, empowering you to implement the best practices seamlessly.

For instance: "Hey Linky, how can I optimize my sleep based on Paul Walker's research?" "Hi there! To enhance your sleep quality..."

Additionally, Linky keeps you updated on recent news events to ensure you stay informed. Whether you're craving the latest releases from your favorite channels or seeking a quick summary of trending topics, Linky has you covered.

For example: "Hey Linky, any recent updates from my subscribed channels?" "Sure thing! Also, would you like a brief summary of XXX?" "Hmm, I'm in the mood for news. Save this podcast for my commute home." "Thanks, Linky. Could you provide more details on the recent developments in XXX?" "Absolutely! Fetching additional information for you."

You can either talk to Linky, or write your questions directly, whatever works best for you at the moment you are seeking the information!

Linky isn't just a tool; it's your content partner, streamlining your listening experience and keeping you informed, engaged, and empowered. Ready to revolutionize your content journey? Let's dive in together.

How we built it

Our backend service can run on a virtual instance (e.g. AWS EC2) or a local laptop. It uses cloud-hosted LLM services (e.g. Groq) or local LLM solutions (e.g. ollama) with open-source model weights (e.g. Meta Llama). We rely on iPhone Siri for seamless front-end experience. After downloading our Apple Shortcut, Linky can be started by saying "Hey Linky". Linky can be connected to various social media platforms (e.g. YouTube, Spotify) based on existing subscriptions. It can also fetch content directly from the internet (e.g. Brave browser).

Process and steps

Data Gathering: We grab data from the YouTube Data API and the YouTube Transcript API. This allows us to access detailed information about podcast episodes, including titles, descriptions, and the actual spoken content, which is crucial for content summarization and query answering.

Web Searching Capability: We use Brave as the web browser to search the internet. This choice was made due to Brave's emphasis on privacy and security, ensuring that users' data and search history are protected.

User Subscription Integration: We connect our agent directly to the user's subscribed channels. This integration ensures that Linky only interacts with trusted sources and provides personalized content based on the user’s specific interests.

Utility Functions Development: We developed a series of utility functions to retrieve specific information from the APIs. These functions handle tasks like listing channels, fetching video descriptions, and extracting crucial data points necessary for our service.

Integration with AI and NLP Services: We integrated Llama 3 with Grok to leverage advanced natural language processing capabilities. This integration allows Linky to understand complex user queries, provide accurate responses, and summarize content effectively.

Interactive Interface: Users can interact with Linky either through voice commands or text inputs, making it versatile and user-friendly. This dual-mode interaction is supported by robust backend logic that interprets and processes user requests promptly.

Challenges we ran into

Building LLM based applications can be challenging. We have been using Weights and Biases to log and trace our application. Connecting to many APIs.

Accomplishments that we're proud of

Seamless experience by talking directly to Siri. We built an awesome service that can aggregate content you subcribe to and uses state of the art language models. More engaging experience with content you consume on social platforms.