Skip to content

fix: prevent RecursionError when caching link previews#2117

Merged
prathameshkurunkar7 merged 1 commit into
The-Commit-Company:developfrom
barredterra:line-preview-caching
Apr 10, 2026
Merged

fix: prevent RecursionError when caching link previews#2117
prathameshkurunkar7 merged 1 commit into
The-Commit-Company:developfrom
barredterra:line-preview-caching

Conversation

@barredterra

@barredterra barredterra commented Apr 9, 2026

Copy link
Copy Markdown
Contributor

Coerce link preview values to plain str before caching, preventing RecursionError during pickle.dumps.

Problem

The linkpreview library can return BeautifulSoup NavigableString objects instead of plain Python strings (notably from Generic.description via Tag.string). NavigableString is a str subclass that carries parent/sibling references into the entire HTML parse tree. When frappe.cache().set_value() pickles a dict containing such objects, pickle follows the tree references recursively and exceeds Python's recursion limit.

Traceback with variables (most recent call last):
  File "apps/frappe/frappe/app.py", line 120, in application
    response = frappe.api.handle(request)
      request = <Request '/api/method/raven.api.preview_links.get_preview_link?urls=["https://docs.frappe.io/school/framework-assignments/day-1"]' [GET]>
      response = None
      rollback = True
      e = RecursionError('maximum recursion depth exceeded')
  File "apps/frappe/frappe/api/__init__.py", line 52, in handle
    data = endpoint(**arguments)
      request = <Request '/api/method/raven.api.preview_links.get_preview_link?urls=["https://docs.frappe.io/school/framework-assignments/day-1"]' [GET]>
      endpoint = <function handle_rpc_call at 0x7f4ed41f2fc0>
      arguments = {'method': 'raven.api.preview_links.get_preview_link'}
  File "apps/frappe/frappe/api/v1.py", line 40, in handle_rpc_call
    return frappe.handler.handle()
      method = 'raven.api.preview_links.get_preview_link'
      frappe = <module 'frappe' from 'apps/frappe/frappe/__init__.py'>
  File "apps/frappe/frappe/handler.py", line 53, in handle
    data = execute_cmd(cmd)
      cmd = 'raven.api.preview_links.get_preview_link'
      data = None
  File "apps/frappe/frappe/handler.py", line 86, in execute_cmd
    return frappe.call(method, **frappe.form_dict)
      cmd = 'raven.api.preview_links.get_preview_link'
      from_async = False
      server_script = None
      method = <function get_preview_link at 0x7f4e8a742160>
  File "apps/frappe/frappe/__init__.py", line 1764, in call
    return fn(*args, **newargs)
      fn = <function get_preview_link at 0x7f4e8a742160>
      args = ()
      kwargs = {'urls': '["https://docs.frappe.io/school/framework-assignments/day-1"]', 'cmd': 'raven.api.preview_links.get_preview_link'}
      newargs = {'urls': '["https://docs.frappe.io/school/framework-assignments/day-1"]'}
  File "apps/frappe/frappe/utils/typing_validations.py", line 32, in wrapper
    return func(*args, **kwargs)
      args = []
      kwargs = {'urls': '["https://docs.frappe.io/school/framework-assignments/day-1"]'}
      apply_condition = <function whitelist.<locals>.innerfn.<locals>.<lambda> at 0x7f4e8a741940>
      func = <function get_preview_link at 0x7f4e8a7419e0>
  File "apps/raven/raven/api/preview_links.py", line 70, in get_preview_link
    frappe.cache().set_value(url, data)
      urls = ['https://docs.frappe.io/school/framework-assignments/day-1']
      data = {'title': '\n        \n    Day 1\n\n    ', 'description': 'Type to search documentation', 'image': '/files/airplane-ticker-form-day-1.webp', 'force_title': '\n        \n    Day 1\n\n    ', 'absolute_image': 'https://docs.frappe.io/files/airplane-ticker-form-day-1.webp', 'site_name': 'docs.frappe.io'}
      empty_data = {'title': '', 'description': '', 'image': '', 'force_title': '', 'absolute_image': '', 'site_name': ''}
      message_links = []
      url = 'https://docs.frappe.io/school/framework-assignments/day-1'
      preview = <linkpreview.linkpreview.LinkPreview object at 0x7f4e81f4bf10>
  File "apps/frappe/frappe/utils/redis_wrapper.py", line 69, in set_value
    self.set(key, pickle.dumps(val))
      self = RedisWrapper<ConnectionPool<Connection<host=localhost,port=13000,db=0>>>
      key = ********
      val = {'title': '\n        \n    Day 1\n\n    ', 'description': 'Type to search documentation', 'image': '/files/airplane-ticker-form-day-1.webp', 'force_title': '\n        \n    Day 1\n\n    ', 'absolute_image': 'https://docs.frappe.io/files/airplane-ticker-form-day-1.webp', 'site_name': 'docs.frappe.io'}
      user = None
      expires_in_sec = None
      shared = False
builtins.RecursionError: maximum recursion depth exceeded

Fix

Wrap every value extracted from LinkPreview with str(... or ""). This strips the BeautifulSoup tree references and also normalises None to "", consistent with the empty_data fallback already used in the same function.

@prathameshkurunkar7 prathameshkurunkar7 merged commit 56e1eba into The-Commit-Company:develop Apr 10, 2026
2 checks passed
@barredterra barredterra deleted the line-preview-caching branch April 10, 2026 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants