-
CheckMate's logo
-
-
Example of 'brilliant' rizz
-
-
-
-
Inspiration
Do you have terrible game? Spent months trying to find that special someone - but keep getting sent back to the dog house? Does the cold, unending passage of time keep you up at night?
For years romance gurus have attempted to guide desperate sods like yourself, yet to no avail. Countless man hours deposited into the ever increasing sands and time - and for what, to continue to fumble 10/10s?
What we needed was an accurate, trustworthy and concise feedback system.
Enter chess.com analysis.
We rate your DM game and give a ranking of your moves in the form of “Brilliant, great, book, mistake and blunder”.
Peer reviewed studies suggest a 73% improvement in “pulling ability” when utilising the “CheckMate” system.
What it does
Launch an overlay you can have over your messaging app / Instagram / dating app, that when clicked will read the messages on the screen and send them to our server. The server then returns a rating in the style of the Chess.com analysis system (e.g. Brilliant, Good, Blunder). The overlay then shows the icon for the returned rating.
How we built it
We made a Flask backend for sending the message data as a prompt. We used the Ollama Python library to access the LLM. We chose the model ‘Dolphin-Mistral’ as this was more open to flirting messages than other LLMs. We have spent most of the time building the prompt to be able to explain the different Chess.com ratings and to help the LLM understand how to interpret the json format. After the LLM has decided on the rating, it returns this And some comments for how the user can improve in the json format.
For the app, we largely abused of the Android platform API. In order to show the score in the apps, we made an overlay view, which allows you to place views in a floating “window” across the whole phone. This functionality is almost exclusively used by malware, and by looking it up you’ll find a load of resources from Google begging you to use a less malicious API.
To get messages from the app (in our case, Instagram, because my teammates weren’t dedicated enough to sign up for Grindr), we used an AccessibilityService. Accessibility services allow apps to see some or all accessibility data that the phone is displaying, such as labels on text views. This API is also heavily used by malware. Annoyingly, accessibility nodes don’t seem to give the accessibility hints, so we just hardcoded messages such that:
- Messages sent from the user are text views with their right hand side at 1256 pixels on the screen.
- For messages sent to the user, we use text views that are 168 pixels from the left.
- The “Tap and hold to react” text caused false positives, so we just ignore any nodes with that label.
This approach had the side effect that the app only works on a Pixel 9 Pro emulator.
Challenges we ran into
The Android ecosystem is made to cause harm. APIs are poorly documented, usually only accompanied by sample code from 6 years ago, as was the case with the screen capture API (which we were planning on using before switching to an accessibility service).
In order to use the aforementioned overlay view, we had to pass the Android WindowManager a defined View. Since we wanted to use Jetpack Compose, the way to pass Composable UI functions was through a ComposeView. The problem with this was this content then never refreshed itself, defeating the whole point of using compose in the first place. In order to get around this, we had to manually call setContent on the View to update it every time a request was made. It is awful.
Natural language was never meant to be processed.
Prompt engineering really sucks, a lot of the time was spent trying to explain the concept of Chess.com’s ratings to the LLM and to make sure it could distinguish between each user’s messages. The alternative would be to produce a “rizz database” and make an embedding model - however these datasets don't exist and I don't want to go around a hackathon asking for everyone's best LLVM pickup lines.
Coming with flirtatious conversation for testing was hard because none of us have game.
Accomplishments that we're proud of
It actually worked and was finished before the time limit.
What we learned
We already knew Android development was awful but we learnt it is even more awful than we could have imagined. We also learnt how to do development with self-hosted LLMs, which is apparently a valuable skill nowadays for some reason.
What's next for CheckMate
Elo system.
Working on more than just the pixel 9 pro.
Support for any dating / messaging app.
Log in or sign up for Devpost to join the conversation.