Skip to content

unioney/FETCHUB

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Fetchub

Fetch, translate, and publish RSS feeds automatically using AI-powered processing with a robust per-feed cron scheduling system.


Overview

Fetchub is an advanced WordPress plugin that automates the aggregation, translation, and publication of RSS feeds. It fetches feed items, detects their language, and translates them into a target language using AI providers such as OpenAI, Gemini, Claude, and Grok. With features like AI-driven content ranking, customizable per-feed cron intervals, and an intuitive admin UI, Fetchub streamlines content management.

A valid token is required for full functionality. New users get a 3-day trial, then $1/month thereafter.


Key Features

  • AI-Powered Translation: Translate feed content using OpenAI, Gemini, Claude, or Grok with support for 90+ languages.
  • Content Ranking: AI evaluates and ranks feed items by importance using ai_rank_importance.
  • Custom Cron Scheduling: Set unique fetching intervals per feed via WP-Cron. 1.2 dynamically registers minute schedules like every_{N}_minutes and ensures a weekly schedule exists.
  • Cron Jobs Panel (NEW): From the admin page, list only Fetchub cron events and Run now, Delete, or Reschedule per feed (nonce-protected).
  • Manual Fetching: Trigger instant updates via AJAX for one or all feeds. 1.2 hardens “Instant fetch” to prevent double-click races, queue a one-shot run, and ensure the recurring event exists.
  • Stop All Feeds (Improved): Halt all active requests. 1.2 adds a global pause flag, doesn’t flip row enabled/disabled states, and sends a short-lived kill-switch to any running job.
  • Feed Configuration: Per-feed intervals, categories, max posts, AI models, temperature, custom prompts, target languages, and block words.
  • Smart Logic / Duplicate Control: AI-driven ranking plus semantic duplicate detection and guard rails to avoid re-publishing similar stories.
  • Featured Image Support: Automatically fetches and sets featured images from feeds or linked pages.
  • Export/Import Settings: Backup/restore settings as JSON. 1.2 automatically re-schedules all feeds after import.
  • Token System: Activate with a token; admin header shows Token active and precise Expires date/time in your site timezone.
  • Revoke Token: Deactivate the token for use on another site.
  • Secure by Design: Nonces for AJAX; WordPress coding standards. 1.2 adds a license gate that blocks all Fetchub AJAX/cron when the token is invalid/expired.
  • Logs UI (Improved): Cleaner, collapsible, proper timezone handling; supports legacy time strings and new GMT timestamps.
  • Post Status Options: Publish as publish or draft by default.
  • Feed URL & Fetched URL Columns: See both the source feed and the original fetched link in the Posts list.

Supported AI Models

OpenAI (newest → older)

  • GPT-5 (gpt-5)
  • GPT-5 mini (gpt-5-mini)
  • GPT-5 nano (gpt-5-nano)
  • GPT-4.1 (gpt-4.1)
  • GPT-4.1 mini (gpt-4.1-mini)
  • GPT-4.1 nano (gpt-4.1-nano)
  • GPT-4o (gpt-4o)
  • GPT-4o mini (gpt-4o-mini)
  • o3 mini (o3-mini)
  • o1 (o1)
  • o1 mini (o1-mini)
  • GPT-3.5 turbo (gpt-3.5-turbo)

Gemini (newest → older)

  • Gemini 2.5 Pro (gemini-2.5-pro)
  • Gemini 2.5 Flash (gemini-2.5-flash)
  • Gemini 2.5 Flash-Lite (gemini-2.5-flash-lite)
  • Gemini 2.0 Flash (gemini-2.0-flash)
  • Gemini 2.0 Flash Lite (gemini-2.0-flash-lite)

Claude (newest → older)

  • Claude Opus 4.1 (claude-opus-4-1-20250805)
  • Claude Opus 4 (claude-opus-4-20250514)
  • Claude Sonnet 4 (claude-sonnet-4-20250514)
  • Claude 3.7 Sonnet (claude-3-7-sonnet-20250219)
  • Claude 3.5 Haiku (claude-3-5-haiku-20241022)

Grok (newest → older)

  • Grok 4 (grok-4)
  • Grok 3 (grok-3)
  • Grok 3 mini (grok-3-mini)

Deprecated / Unsupported models (removed from default picker)

  • Claude 3.5 Sonnet (claude-3-5-sonnet-20241022) — Unsupported
  • Claude 3 Opus (claude-3-opus-20240229) — Unsupported
  • Claude 3 Sonnet (claude-3-sonnet-20240229) — Unsupported
  • Claude 3 Haiku (claude-3-haiku-20240307) — Unsupported
  • Gemini 1.5 Flash-8B (gemini-1.5-flash-8b) — Deprecated
  • Gemini 1.5 Flash (gemini-1.5-flash) — Deprecated
  • Gemini 1.5 Pro (gemini-1.5-pro) — Deprecated

Requirements

  • WordPress 6.0+ — provides core polyfills like str_starts_with() used by Fetchub.
  • PHP 7.4+ — arrow functions (fn()) and modern features. Fully tested on PHP 8.0 – 8.4.
  • PHP extensions: cURL, OpenSSL, DOM/XML (enabled by default on mainstream hosts).
  • WP-Cron active (or real system cron) for per-feed schedules.
  • Memory: 512 MB memory_limit recommended when using multiple AI models.

Best practice for performance
For smooth AI processing and reliable cron execution, we recommend ihost WordPress Cloud Hosting. See FAQ 12 for an ihost-optimized php.ini template.


Installation

  1. Upload the fetchub folder to /wp-content/plugins/.
  2. Activate the plugin in Plugins → Installed Plugins.
  3. Go to Fetchub in your WordPress dashboard.
  4. Activate with a valid token. New users can start a 3-day trial at https://fetchub.com and enter the trial email + token.
  5. Enter API key(s) for your AI provider(s) and configure feed settings (URLs, intervals, categories, etc.).
  6. Save settings once to schedule per-feed cron events.

External Services

This plugin contacts five external APIs. What is sent, why, and where:


Frequently Asked Questions

1. How does the custom cron scheduling work?

Each feed can have its own interval (in minutes). Fetchub dynamically registers those intervals (e.g., every_7_minutes) and schedules fetchub_feed_cron per feed. Saving settings regenerates the schedules. On deactivation, related events are cleared to prevent duplication.

2. Which AI providers are supported?

OpenAI, Gemini, Claude, and Grok. Enable your preferred provider(s) and add API keys in settings.

3. How does content ranking work?

ai_rank_importance uses AI to score items 0–10 by interest/importance, allowing prioritization when Smart Logic is enabled.

4. What if I don’t have an AI provider API key?

You can still fetch and publish feeds. AI features (translation, ranking) require at least one active provider.

5. What is best practice for AI model usage and approximate cost?

In our experience, using Gemini 2.0 Flash Lite across ~49 feeds ran about $20/month without “Couple Title”. With “Couple Title” enabled, about $8/month. Actual costs vary by provider usage and pricing.

6. How do I activate the 3-day trial?

Visit https://fetchub.com to sign up. You’ll receive an email and token; enter them in the Fetchub settings page.

7. How much does the plugin cost monthly?

We built Fetchub to be accessible for journalists, editors, freelancers, and starters. The plugin is $1/month. With the usage described in #5, total monthly spend is approximately $11/month including provider costs (varies by usage).

8. What if a feed fetch fails?

Verify feed URLs and server resources. Enable WP_DEBUG in wp-config.php to log errors.

9. How do I migrate settings to another site?

Use Export Settings to download JSON, then Import Settings on the new site to merge configurations. After import, Fetchub automatically re-schedules all feeds.

10. What Title and Content prompts were used in testing?

We used translation into YOUR_LOCAL_LANGUAGE and fetched news.

Title prompt (summary):

  • Analyze the provided news content (any language). Create a concise (under 10 words), highly engaging title in fluent, natural YOUR_LOCAL_LANGUAGE. Use correct terminology and professional headline style. Avoid literal translations. Only capitalize the first word and proper names. Use correct brand names (e.g., iPhone, xAI, iOS). Avoid duplicates; if repeated, skip and create a new post. No formatting in the title.

Content prompt (summary):

  • Produce a professional news article in YOUR_LOCAL_LANGUAGE (not a short bulletin).
    • Unique Introduction: Start with original sentences; do not reuse or rephrase the title/excerpt.
    • Second Paragraph: Must start exactly with one of:
      • [NAME_OF_YOUR_MEDIA|https://name_of_your_media.com] informing that,
        (Use precisely as written.)
    • Length: Max 390 words; condense while keeping key details; never cut off abruptly.
    • Conclusion: Always end with a clear, concise conclusion.
    • Editor Attribution (exactly one name below the conclusion):
      • Politics/World/Other: Asad Freeman
      • Technology/Sports/Finance/Economics/Business: Fred Alison
      • Health: Sunny Springfield
      • Science/Vehicles: Aileen August
    • No extraneous metadata, dates, headings, or social text. Plain article text only.
    • Language/Style: Fluent, professional YOUR_LOCAL_LANGUAGE with correct grammar and spelling.
    • Uniqueness: Clean, complete, ready for publication, natural to native readers.
    • Brand names: Use correct forms (iPhone, xAI, iOS).
    • Name localization: Localize personal names appropriately in YOUR_LOCAL_LANGUAGE. Company names remain untranslated; apply suffixes as usual.
    • Output plain text only.

Remember to replace YOUR_LOCAL_LANGUAGE with your language/country and NAME_OF_YOUR_MEDIA|https://name_of_your_media.com with your site. On the frontend it will render as: [https://name_of_your_media.com](NAME_OF_YOUR_MEDIA).

11. How do I contact support?

Use https://fetchub.com/contact and include details, screenshots, or videos.

12. What PHP settings are recommended for best performance on ihost?

We run well on modern hosts, but for “zero-tuning” speed/reliability we recommend
ihost WordPress Cloud Hosting.

Why ihost works well with Fetchub

  • Web optimization suite (minifies/compresses assets).
  • Free, unlimited ihostCDN (full global CDN).
  • Server-side image resizing/compression/lazy-loading; code minification.
  • Edge caching for static assets worldwide.
  • Optimized PHP (custom PHP-FPM/OPcache); multiple PHP versions (5.6 → 8.4).

ihost PHP directives (copy into “PHP Options → Additional directives”)

asp_tags = 0
auto_append_file =
auto_prepend_file =
display_errors = 0
error_reporting = 22527
file_uploads = 1
max_execution_time = 300
max_file_uploads = 50
max_input_time = 300
max_input_vars = 10000
memory_limit = 1024M
output_buffering = 0
post_max_size = 256M
short_open_tag = 1
upload_max_filesize = 256M
zlib.output_compression = 0

13. Can I change cache limits?

add_filter( 'fetchub_guid_cache_size', function () { return 2000; } );
add_filter( 'fetchub_summary_keep',    function () { return 800;  } );

14. What is the Cron Jobs Panel?

It filters to Fetchub-only cron events and lets you Run now, Delete, or Reschedule per feed. Actions are nonce-protected; rescheduling sanitises input before use.

15. What does “Stop All” do now?

It unschedules all Fetchub jobs without changing row enabled/disabled states, sets a global pause flag, and triggers a short-lived kill-switch for jobs already running.


Screenshots

  1. Admin Settings Page – Configure AI providers, API keys, and general settings.
  2. Feed Configuration Table – Manage feeds with options for intervals, AI models, and advanced settings.
  3. Export / Import & Revoke Token Panel – Backup settings, import JSON, or revoke your token in one place.
  4. Feed & Fetched URL Column – See the original feed URL and fetched URL directly in the WordPress Posts list.
  5. Logs & Cron Panel (NEW) – Filtered Fetchub cron jobs list (run now, delete, reschedule) plus improved logs view.

Functions

– Singleton instance of the main plugin class.
– Handles plugin activation, including initial setup and scheduling.
– Handles plugin deactivation, including clearing all scheduled cron jobs.
– Registers plugin settings.
– Enqueues admin scripts and styles.
– Adds the main admin menu page.
– Displays the plugin settings page HTML.
– AJAX handler for manual instant feed fetching (now with double-click lock and immediate one-shot queue).
– AJAX handler to fetch and process all feeds instantly.
– AJAX handler to stop all feeds and clear running jobs (global pause + kill-switch).
AJAX handler to update AI model options based on active providers.
NEW: AJAX handlers for Cron Jobs Panel: list jobs, run now, delete schedules, and reschedule minutes (sanitised input).
– WP-Cron hook for per-feed processing (supports a forced one-time run even for disabled rows).
– Processes a feed URL, parses items, and publishes posts.
– Generates post title using AI from content.
– Ranks feed items using AI.
– Identifies content language for translation.
– Translates text using the selected AI provider.
– Processes hyperlink markers in translated text.
– Unified API call method for all AI providers.
– Checks if a post with the given GUID already exists.
– Checks if an attachment with a matching file exists.
– Sets the featured image for a post from a URL.
– Scrapes the first image URL from a given link.
– Checks if a given text contains any of the configured block words.
– Removes trailing opinion phrases from content.
– Removes leading duplicate title from content.
– Simplifies text for slug-like comparison.
– Cleans raw HTML/text content.
– Checks if a GUID has been globally seen (cached).
– Marks a GUID as globally seen and caches it.
– Handles the import of settings from a JSON file (auto re-schedule on success).
– Handles the export of settings to a JSON file.
– Handles token revocation submission.
– Saves a one-sentence summary of a post for duplicate checks.
– WP-Cron task to clean up old post summaries.
– Adds the Feed URL column to the posts list.
– Displays the Feed URL in the posts list column.
– Adds the Fetched URL column to the posts list.
– Displays the Fetched URL in the posts list column.
– Adds Settings/Activate links on the Plugins page.
Security: Blocks all Fetchub AJAX/cron when licence is invalid/expired.
UI: Token header shows Token active and Expires: {date time} in site timezone; Logs UI has proper timezone handling and supports legacy/new timestamps.


Changelog

1.2

  • New – Cron Jobs Panel: Manage only Fetchub’s cron events from the admin: list jobs, Run now, Delete, and Reschedule per feed (nonce-protected).
  • New – Dynamic minute schedules: Automatically registers every_{N}_minutes schedules based on the intervals used by your feeds; also ensures a weekly schedule exists.
  • Improved – Instant Fetch: Prevents double-clicks, queues an immediate single run, and guarantees the recurring event exists for that feed.
  • Improved – Stop All: Unschedules everything without changing row states, sets a global pause flag, and signals a kill-switch for jobs already running.
  • Improved – Logs UI & timekeeping: Logs render with proper timezone handling and accept both legacy time strings and new GMT timestamps.
  • Improved – Import: After importing settings JSON, Fetchub automatically re-schedules all feeds.
  • UI – Token header: When activated, the admin header shows Token active and Expires: {date time} in site timezone.
  • Security: Hardened admin AJAX for rescheduling and license gate prevents any Fetchub AJAX/cron if the token is invalid/expired.
  • Fix: These models are no longer available from supported providers and are removed from the default picker:
  • Claude 3.5 Sonnet (claude-3-5-sonnet-20241022) — Unsupported
  • Claude 3 Opus (claude-3-opus-20240229) — Unsupported
  • Claude 3 Sonnet (claude-3-sonnet-20240229) — Unsupported
  • Claude 3 Haiku (claude-3-haiku-20240307) — Unsupported
  • Gemini 1.5 Flash-8B (gemini-1.5-flash-8b) — Deprecated
  • Gemini 1.5 Flash (gemini-1.5-flash) — Deprecated
  • Gemini 1.5 Pro (gemini-1.5-pro) — Deprecated

1.1

  • Fix: Resolved issue causing cron jobs to duplicate on plugin deactivation and reactivation by implementing aggressive cleanup of all related scheduled events.
  • New: Weekly cron task (fetchub_cleanup_summaries) that trims _fetchub_summary post-meta to the 500 most-recent rows, preventing long-term table growth.
  • New: Duplicate-guard settings — threshold, look-back depth and a fast slug-overlap check before the LLM call.
  • New: Added "Fetched URL" column to the WordPress Posts list table, showing the original source URL of fetched items.
  • Dev: All summary-cleanup SQL now uses prepare() and inline PHPCS whitelisting; PHPCS reports come back clean.
  • UI: Added tooltips and inline help for duplicate guard and cleanup options.
  • Fix: AI-model dropdown instantly refreshes when providers are toggled.
  • Fix: Minor UI/UX tweaks and translation string improvements.
  • Perf: Leaner database queries and transient handling across the board.
  • Fetchub Logs: We log up to 1,000 feeds here. Please scroll down within the below of Fetchub page, Logs section to view all available entries.

1.0

  • Initial release with full AI integration, cron scheduling, and token system.

Upgrade Notice

1.2

Adds Cron Jobs panel, dynamic minute schedules, safer instant-fetch, improved Stop All, better logs timekeeping, and auto-reschedule after import. After updating, open Fetchub → Settings, review feed intervals, and (optionally) reschedule from the new Cron panel.

1.1

Fixes cron duplication on deactivation/reactivation. Adds “Duplicate guard”, “Cleanup” settings, and “Fetched URL” column in Posts list. Visit Fetchub settings to configure new features after updating.

1.0

Initial release with full AI integration, cron scheduling, and token system.


License

This plugin is released under the GNU General Public License v2 or later. See license.txt for full details.

About

Fetch, translate, and publish RSS feeds automatically using AI-powered processing.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors