AI Agent Detection and Management for Ruby web applications. Spyglasses provides comprehensive AI agent detection and management capabilities for Ruby web applications, including Rails, Sinatra, and other Rack-based frameworks.
- 🤖 AI Bot Detection: Automatically detect AI agents like GPTBot, ClaudeBot, ChatGPT-User, and more
- 🧠 AI Referrer Detection: Track traffic from AI platforms like ChatGPT, Claude, Perplexity
- 🚫 Flexible Blocking: Configure blocking rules via the Spyglasses platform
- 📊 Request Logging: Non-blocking request logging to Spyglasses collector
- ⚡ High Performance: Minimal overhead with pattern caching and background processing
- 🔧 Easy Integration: Drop-in Rack middleware for any Ruby web framework
- 🛡️ Thread Safe: Built for concurrent Ruby applications
Add this line to your application's Gemfile:
gem 'spyglasses'And then execute:
$ bundle installOr install it yourself as:
$ gem install spyglassesSign up at spyglasses.io to get your API key.
export SPYGLASSES_API_KEY=your_api_key_hereAdd to your config/application.rb:
# config/application.rb
class Application < Rails::Application
config.middleware.use Spyglasses::Middleware
endOr with options:
# config/application.rb
class Application < Rails::Application
config.middleware.use Spyglasses::Middleware, {
api_key: ENV['SPYGLASSES_API_KEY'],
debug: Rails.env.development?,
exclude_paths: ['/admin', '/internal']
}
endrequire 'sinatra'
require 'spyglasses'
use Spyglasses::Middleware, api_key: ENV['SPYGLASSES_API_KEY']
get '/' do
'Hello World!'
end# config.ru
require 'spyglasses'
use Spyglasses::Middleware, api_key: ENV['SPYGLASSES_API_KEY']
app = lambda do |env|
[200, {'Content-Type' => 'text/plain'}, ['Hello World!']]
end
run app| Variable | Description | Default |
|---|---|---|
SPYGLASSES_API_KEY |
Your Spyglasses API key | Required |
SPYGLASSES_DEBUG |
Enable debug logging | false |
SPYGLASSES_AUTO_SYNC |
Auto-sync patterns on startup | true |
SPYGLASSES_CACHE_TTL |
Pattern cache TTL in seconds | 86400 (24 hours) |
SPYGLASSES_COLLECT_ENDPOINT |
Custom collector endpoint | https://www.spyglasses.io/api/collect |
SPYGLASSES_PATTERNS_ENDPOINT |
Custom patterns endpoint | https://www.spyglasses.io/api/patterns |
use Spyglasses::Middleware, {
api_key: 'your-api-key', # API key (overrides env var)
debug: true, # Enable debug logging
auto_sync: true, # Auto-sync patterns
platform_type: 'rails', # Platform identifier
exclude_paths: [ # Paths to exclude from monitoring
'/admin',
'/api/internal',
/\.json$/ # Regex patterns supported
],
collect_endpoint: 'https://...', # Custom collector endpoint
patterns_endpoint: 'https://...' # Custom patterns endpoint
}# config/initializers/spyglasses.rb (Rails)
Spyglasses.configure do |config|
config.api_key = ENV['SPYGLASSES_API_KEY']
config.debug = Rails.env.development?
config.platform_type = 'rails'
config.exclude_paths = ['/admin', '/internal']
endrequire 'spyglasses'
# Configure the client
client = Spyglasses::Client.new(
Spyglasses::Configuration.new.tap do |config|
config.api_key = 'your-api-key'
end
)
# Detect bots
result = client.detect_bot('GPTBot/1.0')
puts "Bot detected: #{result.is_bot}" # => true
puts "Should block: #{result.should_block}" # => depends on your settings
# Detect AI referrers
result = client.detect_ai_referrer('https://chat.openai.com/')
puts "AI referrer: #{result.source_type}" # => 'ai_referrer'
# Combined detection
result = client.detect('Mozilla/5.0', 'https://chat.openai.com/')
puts "Source type: #{result.source_type}" # => 'ai_referrer'class MyCustomMiddleware
def initialize(app)
@app = app
@spyglasses = Spyglasses::Client.new
end
def call(env)
request = Rack::Request.new(env)
# Detect using Spyglasses
result = @spyglasses.detect(request.user_agent, request.referrer)
if result.is_bot && result.should_block
return [403, {}, ['Forbidden']]
end
# Log the request if something was detected
if result.source_type != 'none'
@spyglasses.log_request(result, {
url: request.url,
user_agent: request.user_agent,
# ... other request info
})
end
@app.call(env)
end
endBlocking rules are configured through the Spyglasses platform dashboard, not in code:
- Global AI Model Trainer Blocking: Block all AI training bots (GPTBot, ClaudeBot, etc.)
- Custom Block Rules: Block specific categories, subcategories, or individual patterns
- Custom Allow Rules: Create exceptions for specific bots
The middleware automatically loads and applies these settings.
The gem includes default patterns for common AI agents:
- ChatGPT-User/* - OpenAI ChatGPT user requests
- Claude-User/* - Anthropic Claude user requests
- Perplexity-User/* - Perplexity AI user requests
- Gemini-User/* - Google Gemini user requests
- GPTBot/* - OpenAI training crawler
- ClaudeBot/* - Anthropic training crawler
- CCBot/* - Common Crawl bot
- meta-externalagent/* - Meta training crawler
- Applebot-Extended/* - Apple training crawler
- chat.openai.com, chatgpt.com - ChatGPT
- claude.ai - Claude
- perplexity.ai - Perplexity
- gemini.google.com - Gemini
- copilot.microsoft.com - Microsoft Copilot
For Rails applications, you can also create an initializer:
# config/initializers/spyglasses.rb
Spyglasses.configure do |config|
config.api_key = Rails.application.credentials.spyglasses_api_key
config.debug = Rails.env.development?
config.platform_type = 'rails'
config.exclude_paths = [
'/rails/active_storage',
'/admin',
/^\/api\/internal/
]
end
# Add middleware
Rails.application.config.middleware.use Spyglasses::Middleware# app.rb
require 'sinatra'
require 'spyglasses'
configure do
Spyglasses.configure do |config|
config.api_key = ENV['SPYGLASSES_API_KEY']
config.platform_type = 'sinatra'
end
use Spyglasses::Middleware
endAdd to your test helper:
# spec/spec_helper.rb or test/test_helper.rb
require 'spyglasses'
# Disable API calls in tests
Spyglasses.configure do |config|
config.api_key = nil
config.auto_sync = false
endRun the test suite:
$ bundle exec rspecAfter checking out the repo, run:
$ bin/setup # Install dependencies
$ rake spec # Run tests
$ rake console # Interactive console
$ rake check # Run all checks (tests + linting)The Spyglasses middleware is designed for high-performance applications:
- Pattern Caching: Regex patterns are compiled and cached
- Background Logging: API calls are made in background threads
- Minimal Overhead: Typical overhead is <1ms per request
- Smart Exclusions: Static assets and health checks are automatically excluded
Bug reports and pull requests are welcome on GitHub at https://github.com/spyglasses/spyglasses-ruby.
The gem is available as open source under the terms of the MIT License.
- 📧 Email: support@spyglasses.io
- 📖 Documentation: https://www.spyglasses.io/docs/platforms/ruby
- 🐛 Issues: https://github.com/spyglasses/spyglasses-ruby/issues