Skip to content

Add blog chapters to newsletter content selection#238

Merged
simonw merged 2 commits intomainfrom
claude/add-guide-chapters-RKeI3
Feb 26, 2026
Merged

Add blog chapters to newsletter content selection#238
simonw merged 2 commits intomainfrom
claude/add-guide-chapters-RKeI3

Conversation

@simonw
Copy link
Owner

@simonw simonw commented Feb 26, 2026

This change extends the blog-to-newsletter query to include published blog chapters as potential content for newsletters.

Summary

Added a new UNION clause to the content selection query that retrieves published chapters from the blog_chapter table, joining with the blog_guide table to include guide metadata.

Key Changes

  • Added SQL query block to select chapters where is_draft = 0
  • Includes chapter metadata: id, title, creation date, and slug
  • Constructs JSON payload with chapter details and associated guide information
  • Generates external URLs in the format: https://simonwillison.net/guides/{guide_slug}/{chapter_slug}/
  • Maintains consistency with existing content type structure (id, type, title, created, slug, html, json, external_url)

Implementation Details

  • Chapters are identified with type = 'chapter'
  • The JSON object includes both chapter-specific fields (body, chapter_slug) and guide context (guide_title, guide_slug)
  • Uses an INNER JOIN with blog_guide to ensure only chapters with valid guides are included
  • Positioned in the UNION query between blog_note and til content types

https://claude.ai/code/session_01Df7tNiUDSXVwvF1yeChynr

Adds blog_chapter query to the content CTE in the newsletter SQL.
Still needs: URL handling in collected CTE, JS rendering with
first-3-paragraph truncation and [... word count] suffix, chapter
state management, and extras count integration.

https://claude.ai/code/session_01Df7tNiUDSXVwvF1yeChynr
@claude
Copy link
Contributor

claude bot commented Feb 26, 2026

Claude finished @simonw's task —— View job


Code Review Complete

Todo List:

  • Read repository guidelines (CLAUDE.md)
  • Analyze the modified blog-to-newsletter.html file
  • Review SQL query changes for correctness and security
  • Check for performance considerations
  • Assess code quality and best practices
  • Identify potential bugs or issues
  • Evaluate test coverage needs
  • Provide comprehensive review feedback

Review Feedback

Code Quality & Best Practices

  • Good structure: The new UNION clause follows the existing pattern established by other content types
  • Consistent naming: Uses the same column structure (id, type, title, created, slug, html, json, external_url) as other unions
  • Clear intent: The type = 'chapter' identifier makes the content type explicit

SQL Query Correctness

  • Proper JOIN: Uses INNER JOIN with blog_guide to ensure only chapters with valid guides are included
  • Appropriate filtering: WHERE c.is_draft = 0 correctly excludes draft chapters
  • URL construction: External URL format https://simonwillison.net/guides/{guide_slug}/{chapter_slug}/ follows the expected pattern

⚠️ Potential Issues & Suggestions

  1. Missing HTML Handling (blog-to-newsletter.html:551)

    • The chapter query sets html = 'No HTML' but other content types have proper HTML formatting
    • Suggestion: Consider adding HTML formatting for chapters or handle this in the JavaScript filtering logic
    • Impact: Chapters may not render properly in the newsletter preview
  2. Inconsistent JSON Structure (blog-to-newsletter.html:552-559)

    • Chapter JSON includes both chapter-specific fields and guide context, which differs from other content types
    • Consider: Whether this rich JSON structure is necessary or if it should be simplified for consistency
  3. No Tag Support (blog-to-newsletter.html:611)

    • Chapters return '[]' for tags while other content types have tag extraction logic
    • Suggestion: Add tag support for chapters if they use the tagging system

🚀 Performance Considerations

  • JOIN efficiency: The INNER JOIN on blog_guide is appropriate and should perform well
  • Index considerations: Ensure blog_chapter.guide_id and blog_chapter.is_draft are indexed
  • Query complexity: Adding another UNION clause increases query complexity slightly but is within reasonable bounds

🔒 Security Assessment

  • No security concerns: The query uses proper column references without user input concatenation
  • Safe URL construction: Uses parameterized column values rather than dynamic string building

🧪 Test Coverage Recommendations

Since this is an HTML tool with embedded JavaScript/SQL:

  1. Manual testing: Verify chapters appear correctly in the newsletter output
  2. Edge case testing: Test with chapters that have no associated guide (should be filtered out by INNER JOIN)
  3. URL validation: Confirm external URLs are correctly formatted
  4. JavaScript compatibility: Ensure the new type = 'chapter' is handled properly in the filtering logic

📍 Code References

  • SQL query addition: blog-to-newsletter.html:544-564
  • Content type handling: blog-to-newsletter.html:772-773 (may need updating for chapters)
  • JSON processing: blog-to-newsletter.html:735-747 (consider chapter handling)

🔍 Minor Observations

  • The implementation correctly positions chapters between blog_note and til content types in the UNION order
  • The JSON structure is well-designed with both chapter and guide metadata
  • External URL generation follows the established pattern

Overall Assessment: ✅ Approve with minor suggestions

The implementation is solid and follows established patterns. The main consideration is ensuring proper HTML rendering for chapters in the newsletter output.


- Query guides_chapter/guides_guide tables from Datasette API
- Render chapter markdown with marked.js, extract first 3 paragraphs
- Append [... N words] suffix linking to full chapter (matching blog display)
- Show guide breadcrumb ("Agentic Engineering Patterns >") above each chapter
- Include chapters in extras count and support deletion from preview

https://claude.ai/code/session_01Df7tNiUDSXVwvF1yeChynr
@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Feb 26, 2026

Deploying tools with  Cloudflare Pages  Cloudflare Pages

Latest commit: 1fd0fa6
Status: ✅  Deploy successful!
Preview URL: https://ad2eaf15.tools-b1q.pages.dev
Branch Preview URL: https://claude-add-guide-chapters-rk.tools-b1q.pages.dev

View logs

@claude
Copy link
Contributor

claude bot commented Feb 26, 2026

Claude finished @simonw's task —— View job


Code Review Complete ✅

Todo List:

  • Read repository guidelines (CLAUDE.md)
  • Analyze the modified blog-to-newsletter.html file
  • Review SQL query changes for correctness and security
  • Check for performance considerations
  • Assess code quality and best practices
  • Identify potential bugs or issues
  • Evaluate test coverage needs
  • Provide comprehensive review feedback

Review Feedback

Overall Assessment

This is a well-implemented feature that cleanly extends the newsletter system to include blog chapters. The code follows existing patterns and maintains consistency with the current architecture.

🔍 SQL Query Review

Location: blog-to-newsletter.html:546-564

Strengths:

  • Proper use of INNER JOIN to ensure data integrity (only chapters with valid guides)
  • Consistent column structure matching other content types
  • Correct filtering with is_draft = 0 to exclude draft chapters
  • URL construction follows established pattern for the site

Security: ✅ No SQL injection risks - the query uses only column references and string literals.

🎯 Code Quality & Best Practices

Excellent:

  • Follows existing code patterns perfectly
  • Maintains consistent naming conventions (guide_slug, chapter_slug)
  • JSON structure includes both chapter and guide context for complete information
  • External URL format follows site conventions

JavaScript Integration:

  • Chapter rendering logic (blog-to-newsletter.html:796-816) properly handles the new content type
  • Excerpt rendering with word count is a nice UX touch
  • Proper integration with existing filtering and display logic

Performance Considerations

Good:

  • INNER JOIN is efficient and necessary for data integrity
  • Query positioned appropriately in the UNION (not performance-critical order)
  • The is_draft = 0 filter reduces unnecessary data retrieval

Minor Note: The query complexity is reasonable for the dataset size, and the UNION structure is already established.

🐛 Potential Issues

None identified. The implementation is solid with:

  • Proper error handling through existing mechanisms
  • Consistent data types and structure
  • No breaking changes to existing functionality

📝 Suggestions for Enhancement (Optional)

  1. Consider adding chapter order/sequence if chapters within a guide have a natural ordering
  2. Guide-level metadata could be cached if performance becomes an issue with many guides

🧪 Testing Considerations

Current Status: The code integrates with existing testing patterns through the live data source.

Recommendations:

  • Test with guides that have multiple chapters
  • Verify URL generation for different guide/chapter slug formats
  • Test filtering behavior with draft vs. published chapters

📊 Security Assessment

No security concerns identified:

  • No user input handling in the new code paths
  • URL construction uses trusted data sources
  • Follows established XSS protection patterns

Conclusion

This is a clean, well-implemented addition that extends the newsletter system naturally. The code follows established patterns, handles edge cases appropriately, and integrates seamlessly with existing functionality. Approved for merge. 🚀


@simonw
Copy link
Owner Author

simonw commented Feb 26, 2026

Here's what it looks like pasted into Substack, I think this is good enough for the moment:

CleanShot 2026-02-25 at 16 30 14@2x

@simonw simonw merged commit 2f5bdd4 into main Feb 26, 2026
2 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants