Skip to content

Inline image generation#235

Merged
jeffpaul merged 27 commits intoWordPress:developfrom
dkotter:feature/image-generation-inline
Mar 3, 2026
Merged

Inline image generation#235
jeffpaul merged 27 commits intoWordPress:developfrom
dkotter:feature/image-generation-inline

Conversation

@dkotter
Copy link
Collaborator

@dkotter dkotter commented Feb 20, 2026

What?

Closes #13

Adds the ability to generate images inline instead of just within the featured image section.

Why?

In #134 and #146 we added support for generating images in the featured image section. All the functionality exists to generate images elsewhere so this PR brings that into the block editor, within core blocks that support images (like core/image, core/cover).

How?

  • Ensures our image generation assets are loaded in the block editor
  • Hook into editor.MediaUpload and add our custom button into the block toolbar / block media controls for image blocks
  • Add modal functionality that listens for clicks on this new button that allow a user to enter a prompt and generate an image
  • Allow that image to be imported and inserted into the block. Will also generate alt text when imported if that experiment is turned on

Use of AI Tools

Approach was planned using Claude Code running Sonnet 4.6. Claude Code than executed that plan with me reviewing and testing along the way. I finished the last bits and pieces myself

Testing Instructions

  1. Add valid AI credentials in Settings > AI Credentials. Note only Google and OpenAI support image generation
  2. Ensure the image generation experiment is turned on under Settings > AI Experiments
  3. Create a new post and add an image block
  4. Ensure you see a Generate Image button next to the other media buttons
  5. Click this button
  6. Ensure a modal shows with a textarea
  7. Add an image generation prompt and click generate
  8. Ensure a loading state shows
  9. Once the image is generated, ensure the image preview is rendered along with buttons to Use Image, Generate Another Image and Edit Prompt
  10. Click Use Image and ensure the image is inserted into the block and is imported into the Media Library
  11. Try again and ensure the Edit Prompt button puts you back in the start state with the prompt prefilled
  12. Turn on the alt text experiment and ensure alt text is generated when you choose to keep a generated image
  13. Try with other core image blocks like gallery, cover and media & text

Screenshots or screencast

Image block in the block editor with the new Generate Image buttons Generate Image modal that opens after clicking the Generate Image button Generate image loading state in the modal Generated image preview in the modal Open WordPress Playground Preview

…ge. Refactor our image generation execution a bit to account for this
… clicked. This gives you the ability to add a prompt and generate an image. After the image is generated, you can start over or insert that image into the block, which will first import into the Media Library and optionally generate alt text
… type of block that was used to trigger the generation
@dkotter dkotter self-assigned this Feb 20, 2026
@dkotter dkotter added this to the 0.4.0 milestone Feb 20, 2026
@codecov
Copy link

codecov bot commented Feb 20, 2026

Codecov Report

❌ Patch coverage is 58.82353% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 56.02%. Comparing base (6a6eafa) to head (a26bf4e).
⚠️ Report is 28 commits behind head on develop.

Files with missing lines Patch % Lines
.../Experiments/Image_Generation/Image_Generation.php 28.57% 5 Missing ⚠️
includes/Abilities/Image/Generate_Image.php 0.00% 2 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             develop     #235      +/-   ##
=============================================
+ Coverage      55.98%   56.02%   +0.04%     
- Complexity       509      511       +2     
=============================================
  Files             32       32              
  Lines           2608     2622      +14     
=============================================
+ Hits            1460     1469       +9     
- Misses          1148     1153       +5     
Flag Coverage Δ
unit 56.02% <58.82%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…button. Tie this into the same modal. Fix params when inserting as a cover block
…rror when new actions are performed. Minor style adjustments
@dkotter dkotter marked this pull request as ready for review February 20, 2026 23:10
@github-actions
Copy link

github-actions bot commented Feb 20, 2026

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Unlinked Accounts

The following contributors have not linked their GitHub and WordPress.org accounts: @prabinjha, @kurtrank.

Contributors, please read how to link your accounts to ensure your work is properly credited in WordPress releases.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Unlinked contributors: prabinjha, kurtrank.

Co-authored-by: dkotter <dkotter@git.wordpress.org>
Co-authored-by: jeffpaul <jeffpaul@git.wordpress.org>
Co-authored-by: karmatosed <karmatosed@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@jeffpaul
Copy link
Member

In addition to generating an image from a prompt, you can also send that image back with another prompt attached to generate image edits, allowing you to iterate on an image until you like it (NOTE: this isn't working yet as this requires upstream changes to the OpenAI and Google AI provider packages).

If that's not ready in time for us to merge/release on this PR, then let's update the context in #236 to include that perhaps?

Testing feedback...

  1. Need to update the styling in the Image block sidebar's Media panel after clicking Add image as the Generate Image button should appear more like the links for Open Media Library and Upload options perhaps using the create icon
Screenshot 2026-02-23 at 11 20 23 AM
  1. Instead of Keep in the Generate Image modal perhaps utilize Use Image?
Screenshot 2026-02-23 at 11 41 36 AM
  1. Instead of Edit in the Generate Image modal perhaps utilize Refine Image?
  2. Instead of Start Over perhaps we use Edit Prompt and keep the previously entered prompt in the textarea?
  3. On this screen, let's add another button (sigh, I know, :ALOT:) for Generate Another Image in case they didn't like the image generated, don't want to change the prompt, and don't want to refine the generated image
  4. Within the Edit/Refine Image flow, perhaps instead of Generate we use Refine for the primary button?
Screenshot 2026-02-23 at 11 42 39 AM
  1. Within the Edit/Refine Image flow, perhaps instead of Back we use Cancel Refinement for the secondary button?
  2. When adding a Gallery and going into the Generate Image flow and getting to the Keep action, it should end up in the Edit Gallery modal, but instead inserts the image into the post in an Image block
  3. Do we want to add this flow as a second level menu option within the Media top level menu item in the WP admin? If so, then perhaps we also add a button in the header of the Library screen next to the Add Media File button?
Screenshot 2026-02-23 at 11 59 34 AM

… generated image, make sure we push the user into the typical Gallery Media Library flow
…eady adds this. Change the component we use depending on the context we are in so the button renders properly in both the default state and the toolbar state
@dkotter
Copy link
Collaborator Author

dkotter commented Feb 23, 2026

Need to update the styling in the Image block sidebar's Media panel after clicking Add image as the Generate Image button should appear more like the links for Open Media Library and Upload options perhaps using the create icon

Similar to #252, this particular section only shows if you're using the Gutenberg plugin (or on WP 7.0-beta1). That said, I realized the editor.MediaUpload filter we use to insert our Generate Image button into the default block state also runs when rendering the block toolbar options (Replace button for image, Add for gallery):

Image block toolbar with custom Generate Image button

Seems this is the same thing that is rendering in that new sidebar section so I've updated our code to handle this so this should look correct there now. I've also removed the custom toolbar button I was adding as there's no need for two buttons there.

Instead of Keep in the Generate Image modal perhaps utilize Use Image?

Updated

Instead of Edit in the Generate Image modal perhaps utilize Refine Image?

Updated

Instead of Start Over perhaps we use Edit Prompt and keep the previously entered prompt in the textarea

Updated

On this screen, let's add another button (sigh, I know, :ALOT:) for Generate Another Image in case they didn't like the image generated, don't want to change the prompt, and don't want to refine the generated image

Added this

Within the Edit/Refine Image flow, perhaps instead of Generate we use Refine for the primary button

Updated

Within the Edit/Refine Image flow, perhaps instead of Back we use Cancel Refinement for the secondary button

Updated

When adding a Gallery and going into the Generate Image flow and getting to the Keep action, it should end up in the Edit Gallery modal, but instead inserts the image into the post in an Image block

I was trying to match the same flow you get if you use the Upload button in the gallery block (or drag an image into the block). In that case, it renders the inner image block immediately (same as what we're doing here).

I do like the user experience of pushing them to that media modal so I've attempted to simulate that now (noting this all uses the media library modal code that isn't super easy to extend so may still have some rough edges).

Do we want to add this flow as a second level menu option within the Media top level menu item in the WP admin? If so, then perhaps we also add a button in the header of the Library screen next to the Add Media File button

I think the answer is yes though I'd suggest that's a separate PR as all the code here is scoped to the block editor and so we'll need a different approach to support anything outside of that

@dkotter
Copy link
Collaborator Author

dkotter commented Feb 23, 2026

If that's not ready in time for us to merge/release on this PR, then let's update the context in #236 to include that perhaps?

Looking into this more, seems changes will be needed both in the PHP AI Client and in each provider implementation, so I don't think this will be done prior to this PR being ready. I'll likely remove all of the edit/refine image work from this PR but I'll wait until we can discuss that.

@jeffpaul
Copy link
Member

I think the answer is yes though I'd suggest that's a separate PR as all the code here is scoped to the block editor and so we'll need a different approach to support anything outside of that

I created #254 for that follow-up work.

@jeffpaul jeffpaul self-requested a review February 23, 2026 22:22
jeffpaul
jeffpaul previously approved these changes Feb 23, 2026
Copy link
Member

@jeffpaul jeffpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking into this more, seems changes will be needed both in the PHP AI Client and in each provider implementation, so I don't think this will be done prior to this PR being ready. I'll likely remove all of the edit/refine image work from this PR but I'll wait until we can discuss that.

@dkotter feel free to remove that and then merge, everything else here looks great!

@dkotter
Copy link
Collaborator Author

dkotter commented Mar 2, 2026

Looking into this more, seems changes will be needed both in the PHP AI Client and in each provider implementation, so I don't think this will be done prior to this PR being ready. I'll likely remove all of the edit/refine image work from this PR but I'll wait until we can discuss that.

@dkotter feel free to remove that and then merge, everything else here looks great!

All the image editing/refinement code has been removed now. Was hoping to be able to leave that and get that working upstream but don't want to block this PR any longer.

For future reference (when we hopefully can add this back) here's the commit that removed all of that: 68c6450

Copy link
Member

@jeffpaul jeffpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and all works as expected, nice work!

@jeffpaul
Copy link
Member

jeffpaul commented Mar 3, 2026

re-running Plugin Check and WP latest test checks...

@jeffpaul jeffpaul added the props-bot Manually triggers Props Bot to ensure the list of props is up to date. label Mar 3, 2026
@github-actions github-actions bot removed the props-bot Manually triggers Props Bot to ensure the list of props is up to date. label Mar 3, 2026
@jeffpaul jeffpaul merged commit 90ed6d9 into WordPress:develop Mar 3, 2026
23 of 41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Image Generation

2 participants