Trajectify

Accelerate your career trajectory with the power of deep learning.

Comment

✨ Inspiration

Finding and planning tangible ways to grow into the person you want to be is difficult. For aspiring students, even developing a career trajectory and breaking it into meaningful steps is difficult. In the world of tech especially, there are more paths than ever, with the number of technologies reaching the thousands.

Trajectify, our project for Jamhacks 7, simplifies this process by utilizing a state-of-the-art deep neural network we trained ourselves to allow anyone to break into their career trajectory.


🚀 What it does

Trajectify is a web app that helps you build and achieve your future career goals based on what other people in your same shoes have achieved! You input your LinkedIn profile, after which Trajectify will generate a customized career path for you, complete with tangible steps you can take to reach that goal. As opposed to pre-existing language models, Trajectify uses a fine-tuned neural network trained directly on students from institutions across Canada, allowing for a much higher degree of accuracy.


⚙️ How we built it

Architecture Overview

After entering your LinkedIn username into the application, the frontend—built with React, TypeScript, Next.JS, and hosted on Vercel—sends the username to the backend. The backend—written in Python and FastAPI—scrapes your profile from LinkedIn and sends it back to the frontend for display. Meanwhile, the backend passes the content in your profile (tagline, summary, experiences, volunteering, education) through a language model we trained.

We trained LLaMA-7B [1]—Meta’s 7 billion parameter foundational language model—on 3,710 LinkedIn profiles we scraped to predict realistic future careers for a user. The task during training is what we call “career modeling”; during training, we hide a user’s most recent experience, and the language model must predict it. The model took around 15 hours to train on two Nvidia Titan RTXs, even when using parameter-efficient fine tuning (in particular, LoRA [2]) due to GPU memory limitations. This gave us a model to predict a user’s potential future career, using a training script implemented in PyTorch and HuggingFace. We chose to train our own model instead of using an off-the-shelf language model (like GPT-4) because we can be more confident in the accuracy of a model we train ourselves. That is, our model is trained specifically on other students and alumni from Canadian institutions to predict future careers, whereas off-the-shelf models like GPT-4 are trained on general data.

Finally, the backend performs in-context learning with OpenAI’s GPT-4 to provide attainable goals to prepare the user for their potential future career. After passing the profile through the machine learning model, the backend passes this information back to the frontend for display.


🚧 Challenges we ran into

  • Initially, models trained with LoRA did not converge properly, which was stressful under time pressure
  • Extreme rate limiting by LinkedIn while scraping data to train our models, delaying our training


😁 Accomplishments that we’re proud of

We’re proud of being able to make a functioning web application that ties in concepts new to us in deep learning and data scraping in just 36 hours. Overall, we are proud of building an app that many people can use to advance their careers (including people at Jamhacks)!


📚 What we learned

  • How to train large (>1B parameters) language models in practice
  • Inner workings of the LinkedIn APIs
  • How to effectively scrape data under a strict rate limit


⏭ What’s next for Trajectify

Overall we’re really happy with what we’ve made! In the future, we plan to…

  • Provide online resources directly with next steps (e.g. if a frontend job is generated, link a React resource as a next step)
  • Generate entries and next steps for other LinkedIn sections (e.g. volunteering and educational opportunities)
  • Connect Trajectify users with a similar trajectory via LinkedIn (e.g. a UW student interested in AI is prompted to connect with a senior UW student who has interned at OpenAI)

References

  1. Touvron, Hugo, et al. “LLaMa: Open and efficient foundation language models.” arXiv preprint arXiv:2302.13971 (2023).
  2. Hu, Edward J., et al. “LoRA: Low-rank adaptation of large language models.” ICLR (2022).

Built With

Share this project:

Updates