Skip to content

fix: #199#238

Merged
louis030195 merged 2 commits into
mainfrom
fix-#199
Sep 11, 2024
Merged

fix: #199#238
louis030195 merged 2 commits into
mainfrom
fix-#199

Conversation

@louis030195

Copy link
Copy Markdown
Collaborator

looking for someone on macos beta (>=15.0) to test this PR (i'm on 14.6)

how to test:

cargo build --release --features metal
./target/release/screenpipe

this should use your default audio output

play some youtube video or something like audiobook idk someone speaking

after 1 min you should see some transcriptions in the terminal

if this is the case can merge

@Lunchb0ne

Copy link
Copy Markdown

I'm on 15.1 beta does that work for you?

@louis030195

Copy link
Copy Markdown
Collaborator Author

@Lunchb0ne yes, although if you come for the bounty just testing the pr is $50 (since i fixed the issue myself)

@sayalovesu

Copy link
Copy Markdown
Screenshot 2024-08-29 at 14 22 03 Seems to work.

@sayalovesu

sayalovesu commented Aug 29, 2024

Copy link
Copy Markdown

Not sure how should I claim this. Do I create a PR without changes or what?

@louis030195

Copy link
Copy Markdown
Collaborator Author

/tip $50 @kunokareal

@algora-pbc

algora-pbc Bot commented Aug 29, 2024

Copy link
Copy Markdown

🎉🎈 @kunokareal has been awarded $50! 🎈🎊

@louis030195

Copy link
Copy Markdown
Collaborator Author

ETA: will merge tomorrow

@Lunchb0ne

Copy link
Copy Markdown
[2024-08-30T08:12:34Z INFO  screenpipe]   MacBook Pro Microphone (input)
[2024-08-30T08:12:34Z INFO  screenpipe]   Display 1 (output)
[2024-08-30T08:12:34Z INFO  screenpipe_server::db] Migrations executed successfully.
[2024-08-30T08:12:34Z INFO  screenpipe] Database initialized, will store files in /Users/lunchb0ne/.screenpipe
[2024-08-30T08:12:34Z INFO  screenpipe] Server started on http://localhost:3030



                                            _
   __________________  ___  ____     ____  (_____  ___
  / ___/ ___/ ___/ _ \/ _ \/ __ \   / __ \/ / __ \/ _ \
 (__  / /__/ /  /  __/  __/ / / /  / /_/ / / /_/ /  __/
/____/\___/_/   \___/\___/_/ /_/  / .___/_/ .___/\___/
                                 /_/     /_/



Build AI apps that have the full context
Open source | Runs locally | Developer friendly


┌─────────────────────┬────────────────────────────────────┐
│ Setting             │ Value                              │
├─────────────────────┼────────────────────────────────────┤
│ FPS                 │ 0.2                                │
│ Audio Chunk Duration│ 30 seconds                         │
│ Port                │ 3030                               │
│ Audio Disabled      │ false                              │
│ Self Healing        │ false                              │
│ Save Text Files     │ false                              │
│ Audio Engine        │ WhisperDistilLargeV3               │
│ OCR Engine          │ AppleNative                        │
│ Monitor ID          │ 1                                  │
│ Data Directory      │ /Users/lunchb0ne/.screenpipe       │
│ Debug Mode          │ false                              │
│ Restart Interval    │ Disabled                           │
├─────────────────────┼────────────────────────────────────┤
│ Audio Devices       │                                    │
│                     │ MacBook Pro Microphone (input)     │
│                     │ Display 1 (output)                 │
└─────────────────────┴────────────────────────────────────┘
You are using local processing. All your data stays on your computer.

[2024-08-30T08:12:34Z INFO  screenpipe_server::server] Starting server on 0.0.0.0:3030
[2024-08-30T08:12:34Z INFO  screenpipe_audio::stt] device = Metal(MetalDevice(DeviceId(1)))
[2024-08-30T08:12:34Z INFO  hf_hub] Token file not found "/Users/lunchb0ne/.cache/huggingface/token"
[2024-08-30T08:12:37Z INFO  screenpipe_server::video] Starting new video capture
[2024-08-30T08:12:37Z INFO  screenpipe_server::video] Started capture thread
[2024-08-30T08:12:37Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:12:38Z INFO  screenpipe_server::video] Starting FFmpeg process for file: /Users/lunchb0ne/.screenpipe/data/2024-08-30_08-12-38.mp4
[2024-08-30T08:12:43Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:12:45Z INFO  screenpipe_server::resource_monitor] Runtime: 10s, Total Memory: 4% (1 GB / 16 GB), Total CPU: 0%, NPU: N/A
[2024-08-30T08:12:49Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:12:50Z INFO  screenpipe_audio::core] device: "Display 1 (output)"
[2024-08-30T08:12:51Z INFO  screenpipe_audio::core] Recording Display 1 (output) for 30 seconds
[2024-08-30T08:12:51Z INFO  screenpipe_audio::core] device: "MacBook Pro Microphone (input)"
[2024-08-30T08:12:51Z INFO  screenpipe_audio::core] Recording MacBook Pro Microphone (input) for 30 seconds
[2024-08-30T08:12:55Z INFO  screenpipe_server::resource_monitor] Runtime: 20s, Total Memory: 5% (1 GB / 16 GB), Total CPU: 12%, NPU: N/A
[2024-08-30T08:12:56Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:13:02Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:13:05Z INFO  screenpipe_server::resource_monitor] Runtime: 30s, Total Memory: 6% (1 GB / 16 GB), Total CPU: 27%, NPU: N/A
[2024-08-30T08:13:08Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:13:14Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:13:15Z INFO  screenpipe_server::resource_monitor] Runtime: 40s, Total Memory: 7% (1 GB / 16 GB), Total CPU: 28%, NPU: N/A
[2024-08-30T08:13:20Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:13:21Z INFO  screenpipe_audio::core] Recording stopped, wrote to /Users/lunchb0ne/.screenpipe/data/Display 1 (output)_2024-08-30_08-12-50.mp4. Now triggering transcription
[2024-08-30T08:13:21Z INFO  screenpipe_server::core] Finished record_and_transcribe for device Display 1 (output) (iteration 1)
[2024-08-30T08:13:21Z INFO  screenpipe_server::core] Recording complete for device Display 1 (output) (iteration 1): "/Users/lunchb0ne/.screenpipe/data/Display 1 (output)_2024-08-30_08-12-50.mp4"
[2024-08-30T08:13:21Z INFO  screenpipe_server::core] Finished iteration 1 for device Display 1 (output)
[2024-08-30T08:13:21Z INFO  screenpipe_audio::core] device: "Display 1 (output)"
[2024-08-30T08:13:21Z INFO  screenpipe_audio::core] Recording Display 1 (output) for 30 seconds
[2024-08-30T08:13:21Z INFO  screenpipe_audio::stt] Resampling from 48000 Hz to 16000 Hz
[2024-08-30T08:13:21Z INFO  screenpipe_audio::core] Recording stopped, wrote to /Users/lunchb0ne/.screenpipe/data/MacBook Pro Microphone (input)_2024-08-30_08-12-51.mp4. Now triggering transcription
[2024-08-30T08:13:21Z INFO  screenpipe_server::core] Finished record_and_transcribe for device MacBook Pro Microphone (input) (iteration 1)
[2024-08-30T08:13:21Z INFO  screenpipe_server::core] Recording complete for device MacBook Pro Microphone (input) (iteration 1): "/Users/lunchb0ne/.screenpipe/data/MacBook Pro Microphone (input)_2024-08-30_08-12-51.mp4"
[2024-08-30T08:13:21Z INFO  screenpipe_server::core] Finished iteration 1 for device MacBook Pro Microphone (input)
[2024-08-30T08:13:21Z INFO  screenpipe_audio::core] device: "MacBook Pro Microphone (input)"
[2024-08-30T08:13:21Z INFO  screenpipe_audio::stt] Total audio_frames processed: 2990, frames that include speech: 2770
[2024-08-30T08:13:21Z INFO  screenpipe_audio::core] Recording MacBook Pro Microphone (input) for 30 seconds
[2024-08-30T08:13:23Z INFO  screenpipe_audio::multilingual] detected language: ("en", "english")
[2024-08-30T08:13:25Z INFO  screenpipe_server::resource_monitor] Runtime: 50s, Total Memory: 3% (1 GB / 16 GB), Total CPU: 48%, NPU: N/A
[2024-08-30T08:13:27Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:13:33Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:13:39Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:13:45Z INFO  screenpipe_server::resource_monitor] Runtime: 70s, Total Memory: 8% (1 GB / 16 GB), Total CPU: 26%, NPU: N/A
[2024-08-30T08:13:46Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:13:49Z INFO  screenpipe_audio::stt] 0.0s -- 30.0s
[2024-08-30T08:13:49Z INFO  screenpipe_audio::stt]   0.0s-0.0s:
[2024-08-30T08:13:49Z INFO  screenpipe_audio::stt]   0.0s-3.2s:  competition we can't still be like that's nice color i also have an race
[2024-08-30T08:13:49Z INFO  screenpipe_audio::stt]   3.2s-7.6s:  oh my chicken fat is building you're gonna no no no you're gonna look you're gonna see what i bring out
[2024-08-30T08:13:49Z INFO  screenpipe_audio::stt]   7.6s-15.0s:  oh my mushroom is better than it was my mushroom loves me and the awesome then food arrived oh so good
[2024-08-30T08:13:49Z INFO  screenpipe_audio::stt]   15.0s-27.7s:  that looks like baby food somebody says no it's so good oh my heartroats a hundred even this is that good no it's shocking what's my i don't want to know it oh come here
[2024-08-30T08:13:51Z INFO  screenpipe_audio::stt] 30.0s -- 45.0s
[2024-08-30T08:13:51Z INFO  screenpipe_audio::stt]   0.0s-0.0s:
[2024-08-30T08:13:51Z INFO  screenpipe_audio::stt]   0.0s-1.0s:  You know,
[2024-08-30T08:13:51Z INFO  screenpipe_audio::stt] Resampling from 0 Hz to 16000 Hz
thread 'tokio-runtime-worker' panicked at library/alloc/src/raw_vec.rs:25:5:
capacity overflow
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[2024-08-30T08:13:51Z INFO  screenpipe_server::core] Received transcription
[2024-08-30T08:13:51Z INFO  screenpipe_server::core] Inserting audio chunk: "/Users/lunchb0ne/.screenpipe/data/Display 1 (output)_2024-08-30_08-12-50.mp4"
[2024-08-30T08:13:51Z INFO  screenpipe_audio::core] Recording stopped, wrote to /Users/lunchb0ne/.screenpipe/data/Display 1 (output)_2024-08-30_08-13-21.mp4. Now triggering transcription
[2024-08-30T08:13:51Z ERROR screenpipe_audio::core] Failed to send audio to audio model: channel closed
[2024-08-30T08:13:51Z INFO  screenpipe_server::core] Finished record_and_transcribe for device Display 1 (output) (iteration 2)
[2024-08-30T08:13:51Z INFO  screenpipe_server::core] Recording complete for device Display 1 (output) (iteration 2): "/Users/lunchb0ne/.screenpipe/data/Display 1 (output)_2024-08-30_08-13-21.mp4"
[2024-08-30T08:13:51Z INFO  screenpipe_server::core] Finished iteration 2 for device Display 1 (output)
[2024-08-30T08:13:51Z INFO  screenpipe_audio::core] device: "Display 1 (output)"
[2024-08-30T08:13:51Z INFO  screenpipe_audio::core] Recording Display 1 (output) for 30 seconds
[2024-08-30T08:13:51Z INFO  screenpipe_audio::core] Recording stopped, wrote to /Users/lunchb0ne/.screenpipe/data/MacBook Pro Microphone (input)_2024-08-30_08-13-21.mp4. Now triggering transcription
[2024-08-30T08:13:51Z ERROR screenpipe_audio::core] Failed to send audio to audio model: channel closed
[2024-08-30T08:13:51Z INFO  screenpipe_server::core] Finished record_and_transcribe for device MacBook Pro Microphone (input) (iteration 2)
[2024-08-30T08:13:51Z INFO  screenpipe_server::core] Recording complete for device MacBook Pro Microphone (input) (iteration 2): "/Users/lunchb0ne/.screenpipe/data/MacBook Pro Microphone (input)_2024-08-30_08-13-21.mp4"
[2024-08-30T08:13:51Z INFO  screenpipe_server::core] Finished iteration 2 for device MacBook Pro Microphone (input)
[2024-08-30T08:13:51Z INFO  screenpipe_audio::core] device: "MacBook Pro Microphone (input)"
[2024-08-30T08:13:51Z INFO  screenpipe_audio::core] Recording MacBook Pro Microphone (input) for 30 seconds
[2024-08-30T08:13:52Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:13:55Z INFO  screenpipe_server::resource_monitor] Runtime: 80s, Total Memory: 10% (2 GB / 16 GB), Total CPU: 33%, NPU: N/A
[2024-08-30T08:13:58Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:14:05Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:14:05Z INFO  screenpipe_server::resource_monitor] Runtime: 90s, Total Memory: 11% (2 GB / 16 GB), Total CPU: 18%, NPU: N/A
[2024-08-30T08:14:11Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:14:15Z INFO  screenpipe_server::resource_monitor] Runtime: 100s, Total Memory: 12% (2 GB / 16 GB), Total CPU: 36%, NPU: N/A
[2024-08-30T08:14:17Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:14:21Z INFO  screenpipe_audio::core] Recording stopped, wrote to /Users/lunchb0ne/.screenpipe/data/Display 1 (output)_2024-08-30_08-13-51.mp4. Now triggering transcription
[2024-08-30T08:14:21Z ERROR screenpipe_audio::core] Failed to send audio to audio model: channel closed
[2024-08-30T08:14:21Z INFO  screenpipe_server::core] Finished record_and_transcribe for device Display 1 (output) (iteration 3)
[2024-08-30T08:14:21Z INFO  screenpipe_server::core] Recording complete for device Display 1 (output) (iteration 3): "/Users/lunchb0ne/.screenpipe/data/Display 1 (output)_2024-08-30_08-13-51.mp4"
[2024-08-30T08:14:21Z INFO  screenpipe_server::core] Finished iteration 3 for device Display 1 (output)
[2024-08-30T08:14:21Z INFO  screenpipe_audio::core] device: "Display 1 (output)"
[2024-08-30T08:14:21Z INFO  screenpipe_audio::core] Recording Display 1 (output) for 30 seconds
[2024-08-30T08:14:21Z INFO  screenpipe_audio::core] Recording stopped, wrote to /Users/lunchb0ne/.screenpipe/data/MacBook Pro Microphone (input)_2024-08-30_08-13-51.mp4. Now triggering transcription
[2024-08-30T08:14:21Z ERROR screenpipe_audio::core] Failed to send audio to audio model: channel closed
[2024-08-30T08:14:21Z INFO  screenpipe_server::core] Finished record_and_transcribe for device MacBook Pro Microphone (input) (iteration 3)
[2024-08-30T08:14:21Z INFO  screenpipe_server::core] Recording complete for device MacBook Pro Microphone (input) (iteration 3): "/Users/lunchb0ne/.screenpipe/data/MacBook Pro Microphone (input)_2024-08-30_08-13-51.mp4"
[2024-08-30T08:14:21Z INFO  screenpipe_server::core] Finished iteration 3 for device MacBook Pro Microphone (input)
[2024-08-30T08:14:21Z INFO  screenpipe_audio::core] device: "MacBook Pro Microphone (input)"
[2024-08-30T08:14:21Z INFO  screenpipe_audio::core] Recording MacBook Pro Microphone (input) for 30 seconds
[2024-08-30T08:14:23Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:14:25Z INFO  screenpipe_server::resource_monitor] Runtime: 110s, Total Memory: 13% (2 GB / 16 GB), Total CPU: 30%, NPU: N/A
[2024-08-30T08:14:29Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:14:36Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:14:42Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T08:14:45Z INFO  screenpipe_server::resource_monitor] Runtime: 130s, Total Memory: 15% (2 GB / 16 GB), Total CPU: 25%, NPU: N/A

This is my run on 15.1 Beta (24B5035e)

@louis030195

louis030195 commented Aug 30, 2024

Copy link
Copy Markdown
Collaborator Author

@Lunchb0ne thanks

does this vision error happen on main branch? usually vision can show a few errors on some windows but should stop quickly or be infrequent

also do you have idea what is this untitled window? cursor tab maybe?

regarding the audio error i will not merge then - will do more testing with @m13v next week

@Lunchb0ne

Lunchb0ne commented Aug 30, 2024

Copy link
Copy Markdown

Here's from the latest on main. My guess for the Unknown window is Arc Browser, that is the only other window I have opened.

CleanShot 2024-08-30 at 12  09 35@2x

[2024-08-30T10:59:29Z INFO  screenpipe]   MacBook Pro Microphone (input)
[2024-08-30T10:59:29Z INFO  screenpipe]   MacBook Pro Speakers (output)
[2024-08-30T10:59:29Z INFO  screenpipe_server::db] Migrations executed successfully.
[2024-08-30T10:59:29Z INFO  screenpipe] Database initialized, will store files in /Users/lunchb0ne/.screenpipe
[2024-08-30T10:59:29Z INFO  screenpipe] Server started on http://localhost:3030



                                            _
   __________________  ___  ____     ____  (_____  ___
  / ___/ ___/ ___/ _ \/ _ \/ __ \   / __ \/ / __ \/ _ \
 (__  / /__/ /  /  __/  __/ / / /  / /_/ / / /_/ /  __/
/____/\___/_/   \___/\___/_/ /_/  / .___/_/ .___/\___/
                                 /_/     /_/



Build AI apps that have the full context
Open source | Runs locally | Developer friendly


┌─────────────────────┬────────────────────────────────────┐
│ Setting             │ Value                              │
├─────────────────────┼────────────────────────────────────┤
│ FPS                 │ 0.2                                │
│ Audio Chunk Duration│ 30 seconds                         │
│ Port                │ 3030                               │
│ Audio Disabled      │ false                              │
│ Self Healing        │ false                              │
│ Save Text Files     │ false                              │
│ Audio Engine        │ WhisperDistilLargeV3               │
│ OCR Engine          │ AppleNative                        │
│ Monitor ID          │ 1                                  │
│ Data Directory      │ /Users/lunchb0ne/.screenpipe       │
│ Debug Mode          │ false                              │
│ Restart Interval    │ Disabled                           │
├─────────────────────┼────────────────────────────────────┤
│ Audio Devices       │                                    │
│                     │ MacBook Pro Microphone (input)     │
│                     │ MacBook Pro Speakers (output)      │
└─────────────────────┴────────────────────────────────────┘
You are using local processing. All your data stays on your computer.

[2024-08-30T10:59:29Z INFO  screenpipe_server::server] Starting server on 0.0.0.0:3030
[2024-08-30T10:59:29Z INFO  screenpipe_audio::stt] device = Metal(MetalDevice(DeviceId(1)))
[2024-08-30T10:59:29Z INFO  hf_hub] Token file not found "/Users/lunchb0ne/.cache/huggingface/token"
[2024-08-30T10:59:32Z INFO  screenpipe_server::video] Starting new video capture
[2024-08-30T10:59:32Z INFO  screenpipe_server::video] Started capture thread
[2024-08-30T10:59:32Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T10:59:34Z INFO  screenpipe_server::video] Starting FFmpeg process for file: /Users/lunchb0ne/.screenpipe/data/2024-08-30_10-59-34.mp4
[2024-08-30T10:59:39Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T10:59:40Z INFO  screenpipe_server::resource_monitor] Runtime: 11s, Total Memory: 4% (1 GB / 16 GB), Total CPU: 0%, NPU: N/A
[2024-08-30T10:59:44Z INFO  screenpipe_audio::core] device: "MacBook Pro Speakers (output)"
[2024-08-30T10:59:44Z INFO  screenpipe_audio::core] device: "MacBook Pro Microphone (input)"
[2024-08-30T10:59:44Z WARN  screenpipe_audio::core] Audio output not supported on macOS 15.0 and above
[2024-08-30T10:59:44Z INFO  screenpipe_server::core] Finished record_and_transcribe for device MacBook Pro Speakers (output) (iteration 1)
[2024-08-30T10:59:44Z ERROR screenpipe_server::core] Error in record_and_transcribe for device MacBook Pro Speakers (output) (iteration 1): Audio output not supported on this macOS version, stopping thread
[2024-08-30T10:59:44Z INFO  screenpipe_server::core] Exiting audio capture thread for device: MacBook Pro Speakers (output)
[2024-08-30T10:59:44Z INFO  screenpipe_audio::core] Recording MacBook Pro Microphone (input) for 30 seconds
[2024-08-30T10:59:44Z INFO  screenpipe_server::core] Handle for device MacBook Pro Speakers (output) has finished
[2024-08-30T10:59:45Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T10:59:50Z INFO  screenpipe_server::resource_monitor] Runtime: 21s, Total Memory: 5% (1 GB / 16 GB), Total CPU: 15%, NPU: N/A
[2024-08-30T10:59:52Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T10:59:58Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:00:00Z INFO  screenpipe_server::resource_monitor] Runtime: 31s, Total Memory: 7% (1 GB / 16 GB), Total CPU: 30%, NPU: N/A
[2024-08-30T11:00:04Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:00:10Z INFO  screenpipe_server::resource_monitor] Runtime: 41s, Total Memory: 7% (1 GB / 16 GB), Total CPU: 22%, NPU: N/A
[2024-08-30T11:00:11Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:00:14Z INFO  screenpipe_audio::core] Recording stopped, wrote to /Users/lunchb0ne/.screenpipe/data/MacBook Pro Microphone (input)_2024-08-30_10-59-44.mp4. Now triggering transcription
[2024-08-30T11:00:14Z INFO  screenpipe_server::core] Finished record_and_transcribe for device MacBook Pro Microphone (input) (iteration 1)
[2024-08-30T11:00:14Z INFO  screenpipe_server::core] Recording complete for device MacBook Pro Microphone (input) (iteration 1): "/Users/lunchb0ne/.screenpipe/data/MacBook Pro Microphone (input)_2024-08-30_10-59-44.mp4"
[2024-08-30T11:00:14Z INFO  screenpipe_server::core] Finished iteration 1 for device MacBook Pro Microphone (input)
[2024-08-30T11:00:14Z INFO  screenpipe_audio::core] device: "MacBook Pro Microphone (input)"
[2024-08-30T11:00:14Z INFO  screenpipe_audio::core] Recording MacBook Pro Microphone (input) for 30 seconds
[2024-08-30T11:00:14Z INFO  screenpipe_audio::stt] Resampling from 0 Hz to 16000 Hz
thread 'tokio-runtime-worker' panicked at library/alloc/src/raw_vec.rs:25:5:
capacity overflow
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[2024-08-30T11:00:17Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:00:21Z INFO  screenpipe_server::resource_monitor] Runtime: 51s, Total Memory: 9% (1 GB / 16 GB), Total CPU: 34%, NPU: N/A
[2024-08-30T11:00:24Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:00:30Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:00:37Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:00:39Z INFO  screenpipe_server::resource_monitor] Runtime: 70s, Total Memory: 11% (2 GB / 16 GB), Total CPU: 27%, NPU: N/A
[2024-08-30T11:00:43Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:00:44Z INFO  screenpipe_audio::core] Recording stopped, wrote to /Users/lunchb0ne/.screenpipe/data/MacBook Pro Microphone (input)_2024-08-30_11-00-14.mp4. Now triggering transcription
[2024-08-30T11:00:44Z ERROR screenpipe_audio::core] Failed to send audio to audio model: channel closed
[2024-08-30T11:00:44Z INFO  screenpipe_server::core] Finished record_and_transcribe for device MacBook Pro Microphone (input) (iteration 2)
[2024-08-30T11:00:44Z INFO  screenpipe_server::core] Recording complete for device MacBook Pro Microphone (input) (iteration 2): "/Users/lunchb0ne/.screenpipe/data/MacBook Pro Microphone (input)_2024-08-30_11-00-14.mp4"
[2024-08-30T11:00:44Z INFO  screenpipe_server::core] Finished iteration 2 for device MacBook Pro Microphone (input)
[2024-08-30T11:00:44Z INFO  screenpipe_audio::core] device: "MacBook Pro Microphone (input)"
[2024-08-30T11:00:44Z INFO  screenpipe_audio::core] Recording MacBook Pro Microphone (input) for 30 seconds
[2024-08-30T11:00:49Z INFO  screenpipe_server::resource_monitor] Runtime: 80s, Total Memory: 12% (2 GB / 16 GB), Total CPU: 18%, NPU: N/A
[2024-08-30T11:00:49Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:00:56Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:00:59Z INFO  screenpipe_server::resource_monitor] Runtime: 90s, Total Memory: 13% (2 GB / 16 GB), Total CPU: 33%, NPU: N/A
[2024-08-30T11:01:02Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:01:09Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:01:09Z INFO  screenpipe_server::resource_monitor] Runtime: 100s, Total Memory: 14% (2 GB / 16 GB), Total CPU: 27%, NPU: N/A
[2024-08-30T11:01:14Z INFO  screenpipe_audio::core] Recording stopped, wrote to /Users/lunchb0ne/.screenpipe/data/MacBook Pro Microphone (input)_2024-08-30_11-00-44.mp4. Now triggering transcription
[2024-08-30T11:01:14Z ERROR screenpipe_audio::core] Failed to send audio to audio model: channel closed
[2024-08-30T11:01:14Z INFO  screenpipe_server::core] Finished record_and_transcribe for device MacBook Pro Microphone (input) (iteration 3)
[2024-08-30T11:01:14Z INFO  screenpipe_server::core] Recording complete for device MacBook Pro Microphone (input) (iteration 3): "/Users/lunchb0ne/.screenpipe/data/MacBook Pro Microphone (input)_2024-08-30_11-00-44.mp4"
[2024-08-30T11:01:14Z INFO  screenpipe_server::core] Finished iteration 3 for device MacBook Pro Microphone (input)
[2024-08-30T11:01:14Z INFO  screenpipe_audio::core] device: "MacBook Pro Microphone (input)"
[2024-08-30T11:01:14Z INFO  screenpipe_audio::core] Recording MacBook Pro Microphone (input) for 30 seconds
[2024-08-30T11:01:15Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:01:20Z INFO  screenpipe_server::resource_monitor] Runtime: 110s, Total Memory: 15% (2 GB / 16 GB), Total CPU: 28%, NPU: N/A
[2024-08-30T11:01:22Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:01:28Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:01:35Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:01:39Z INFO  screenpipe_server::resource_monitor] Runtime: 130s, Total Memory: 17% (3 GB / 16 GB), Total CPU: 26%, NPU: N/A
[2024-08-30T11:01:41Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:01:44Z INFO  screenpipe_audio::core] Recording stopped, wrote to /Users/lunchb0ne/.screenpipe/data/MacBook Pro Microphone (input)_2024-08-30_11-01-14.mp4. Now triggering transcription
[2024-08-30T11:01:44Z ERROR screenpipe_audio::core] Failed to send audio to audio model: channel closed
[2024-08-30T11:01:44Z INFO  screenpipe_server::core] Finished record_and_transcribe for device MacBook Pro Microphone (input) (iteration 4)
[2024-08-30T11:01:44Z INFO  screenpipe_server::core] Recording complete for device MacBook Pro Microphone (input) (iteration 4): "/Users/lunchb0ne/.screenpipe/data/MacBook Pro Microphone (input)_2024-08-30_11-01-14.mp4"
[2024-08-30T11:01:44Z INFO  screenpipe_server::core] Finished iteration 4 for device MacBook Pro Microphone (input)
[2024-08-30T11:01:44Z INFO  screenpipe_audio::core] device: "MacBook Pro Microphone (input)"
[2024-08-30T11:01:45Z INFO  screenpipe_audio::core] Recording MacBook Pro Microphone (input) for 30 seconds
[2024-08-30T11:01:47Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:01:49Z INFO  screenpipe_server::resource_monitor] Runtime: 140s, Total Memory: 18% (3 GB / 16 GB), Total CPU: 33%, NPU: N/A
[2024-08-30T11:01:54Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }
[2024-08-30T11:01:59Z INFO  screenpipe_server::resource_monitor] Runtime: 150s, Total Memory: 19% (3 GB / 16 GB), Total CPU: 18%, NPU: N/A
[2024-08-30T11:02:00Z ERROR screenpipe_vision::capture_screenshot_by_window] Failed to capture image for window Untitled on monitor Monitor #41039: Capture failed 100 CGRect { origin: CGPoint { x: 1742.0, y: 1169.0 }, size: CGSize { width: 0.0, height: 0.0 } }

@louis030195

Copy link
Copy Markdown
Collaborator Author

Oh
Weird, nobody had this issue on Mac before and so it's not this branch the problem

How do you build?

Wonder what's special in your environment that cause this

@Lunchb0ne

Copy link
Copy Markdown

I don't think I'm doing anything differently. Using the latest stable rust toolchain on macos 15.1

Do you reckon I can get any other logs to help?

@louis030195

Copy link
Copy Markdown
Collaborator Author

@m13v can you test this? if it work for you lets resolve conflicts & merge

also now anyone can record audio output using virtual device instead of display by creating aggregate audio device

Screenshot 2024-09-02 at 13 26 13

typically need thing like https://github.com/ExistentialAudio/BlackHole though

@louis030195 louis030195 merged commit 55e9518 into main Sep 11, 2024
@screenpipe screenpipe deleted a comment from algora-pbc Bot Dec 6, 2024
@screenpipe screenpipe deleted a comment from algora-pbc Bot Dec 6, 2024
louis030195 pushed a commit that referenced this pull request Jun 5, 2026
scoped to the LLM-reference block only. cross-checked against the codebase,
the brain, and the in-flight pricing launch (website #238 / app #3846):

- pricing: drop removed lifetime/$400/$600/$39; use launch tiers
  Standard $25/mo, Pro $50/seat/mo, Enterprise $150/seat/mo. lifetime
  grandfathered, no new lifetime sales. core engine + CLI stay free/OSS.
- ai search: SQLite FTS5 keyword search, not "semantic embeddings"
  (embeddings exist only for speaker-id; content embeddings are a future TODO)
- built-in pipes: replace nonexistent obsidian-sync/reminders/idea-tracker
  with the actually-shipped pipes (meeting-summary, day-recap, etc.)
- audio: Whisper Large-V3-Turbo locally / Deepgram cloud
- mcp: pin @latest to match the install snippet
- faq + comparison table: one-time/lifetime -> subscription reality

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
louis030195 added a commit that referenced this pull request Jun 5, 2026
scoped to the LLM-reference block only. cross-checked against the codebase,
the brain, and the in-flight pricing launch (website #238 / app #3846):

- pricing: drop removed lifetime/$400/$600/$39; use launch tiers
  Standard $25/mo, Pro $50/seat/mo, Enterprise $150/seat/mo. lifetime
  grandfathered, no new lifetime sales. core engine + CLI stay free/OSS.
- ai search: SQLite FTS5 keyword search, not "semantic embeddings"
  (embeddings exist only for speaker-id; content embeddings are a future TODO)
- built-in pipes: replace nonexistent obsidian-sync/reminders/idea-tracker
  with the actually-shipped pipes (meeting-summary, day-recap, etc.)
- audio: Whisper Large-V3-Turbo locally / Deepgram cloud
- mcp: pin @latest to match the install snippet
- faq + comparison table: one-time/lifetime -> subscription reality

Co-authored-by: Louis Beaumont <louis@screenpi.pe>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants