Control your Mac or PC with AI — from your phone.
iMessage · Discord · Slack · WhatsApp · FaceTime voice
Send a message from your phone. Your computer does the rest.
| You send | Your Mac/PC does |
|---|---|
| "FaceTime me" | Starts FaceTime, shares screen — then use voice to control the computer |
| "Open Chrome and search X on Google" | Opens Chrome, runs the search, opens the first result |
| "Open Notepad" | Opens Notepad and types a test message |
| "Open Calculator" | Opens the Calculator app |
| "Go to YouTube Music and play some music" | Opens Chrome, navigates to YouTube Music |
Only your messages are processed; everyone else is ignored.
Message from your phone → task runs on your computer. Architecture image generated by AI and made with love by Aditya Punjani and Sakshi Tripathy.
flowchart LR
subgraph Phone
A[iMessage / Discord / Slack / WhatsApp]
end
subgraph Your Computer
B[Message bridge]
C[Backend]
D[FaceTime voice]
E[Stagehand · Browser]
F[Agent-S · Desktop]
end
A --> B --> C
C --> D
C --> E
C --> F
D --> G[Your Mac/PC]
E --> G
F --> G
| Layer | Role |
|---|---|
| Message bridge | Receives your messages (iMessage, Discord, Slack, WhatsApp). |
| Backend | Routes each message to the right service. |
| FaceTime | Text "FaceTime me" → call starts, screen shares; then use voice to control the Mac. |
| Stagehand | Browser tasks (Chrome, Google, YouTube, Gmail, etc.). |
| Agent-S | Desktop tasks (Notepad, Calculator, files, apps). |
git clone https://github.com/AdityaYC/Cortex-Treehacks.git
cd Cortex-Treehacks
cp .env.example .env # Add API keys, FACETIME_APPLE_ID, etc.
./run_order.shThen text your Mac from iMessage. Say "FaceTime me" for a call + screen share and voice control, or send any other task (browser or desktop).
git clone https://github.com/AdityaYC/Cortex-Treehacks.git
cd Cortex-Treehacks
copy .env.example .env
.\run_windows.ps1This starts the message bridge, backend, Agent-S, Stagehand, and frontend.
| Platform | ngrok? | Setup |
|---|---|---|
| iMessage (Mac) | No | Run with run_order.sh; text from your iPhone to trigger tasks and "FaceTime me" for voice |
| Discord | No | Discord & Slack setup → |
| Slack | No | Discord & Slack setup → |
| Yes | WhatsApp setup → |
Add to .env:
# Agent-S (required)
ANTHROPIC_API_KEY=sk-ant-your-key
# Stagehand (browser tasks)
OPENAI_API_KEY=sk-your-openai-key
# Discord / Slack / WhatsApp — see setup docs| Folder | Purpose |
|---|---|
Agent-S |
Desktop automation (vision + pyautogui) |
backend |
Flask server, message bridges, Stagehand service |
frontend |
Electron UI |
docs |
Setup guides (Discord, Slack, WhatsApp, testing) |
scripts |
Utilities and run scripts |
| Doc | Description |
|---|---|
| Video calls & FaceTime | "FaceTime me", Zoom, Meet, WhatsApp |
| Windows setup | Full Windows-only setup |
| Discord & Slack | Bot tokens and allowed user IDs |
| WhatsApp (Twilio) | Twilio and ngrok |
| Testing | How to test messages and flows |
- Cortex-AI — Logo and architecture imagery generated by AI and made with love by Aditya Punjani and Sakshi Tripathy.
- Agent-S — computer-use agent (simular-ai)
- Stagehand — browser automation (browserbase)
Last updated: February 2026