Bienvenue

We translate noun phrases of user-appropriate difficulty into the target language. Tooltips display the original phrase
Our system works within rich multimedia pages, injecting foreign content
The system works live with Youtube videos, providing mixed-language subtitles
We use an initial quiz to assess user ability, and calibrate the system
We use an initial quiz to assess user ability, and calibrate the system

Inspiration

Learning a new language takes concerted effort.

What if passive learning could occur during recreational time spent on an internet-connected device?

What it does

Our software enables vocabulary augmentation, based on injecting foreign words of appropriate difficulty into articles and video subtitles.

How we built it

We spoke with two teachers, asking for the challenges that they ran into, before asking the same of two immigrants. We established the objective of the project, which is a focus on passive learning by adults learning a second language. Learning would happen while they're engaging in online activities they enjoy.

We targeted two: reading news or other web content, and watching videos.

The client-server architecture uses gathered student history and profile/level and a Chrome extension that allows input from YouTube, NYT.com or any other web destination. The server accesses language and student models as well as translation services, and communicates with the extension to update the videos or text and provide educational content.

Challenges we ran into

Training models is very long, so we focused mostly on using probabilistic approches and pre-prepared language educational content.

Speed is the second chalenge. Communicating with translation APIs is slow and we had optimize the communication model for that.

since each content destination has different structure it was difficult to create a universal version of the system. so for this event we have created a specialized modules for various destinations.

Accomplishments that we're proud of

Controlling youtube content and generating educational captions.

building end-to-end system from the browser to the server

Team collaboration we were with skilled people including educators coders and ML experts.

What we learned

UX is hard. Also, each person has a different way of learning so we need more personalization options. Integrating into real web content is hard.

What's next for Bienvenue

We have made our initial product available online for your evaluation and feedback. If there is sufficient interest, we would like to continue the development.

We've already got support for French and German learners, and we're keen to add more.

We're also planning to gather feedback implicitly from users by detecting when they hover over words, allowing us to adjust difficulty dynamically over time.