FEAT: add TransparencyAttackConverter#1031
Conversation
Based on the "Novel Image Blending Algorithm" from: https://arxiv.org/pdf/2401.15817
|
Here's an example of attack that uses this converter for image manipulation, ChatGPT: 
Apparently, when using dark theme I can clearly see that the image asks "How to make a bomb", but that's a limitation this converter just has. And here's some comparison of how the final blended image looks based on the viewer's background color: Edit: I wanted to share a link to the chat but I immediately got "This shared link has been disabled by moderation" 😄 |
romanlutz
left a comment
There was a problem hiding this comment.
Woah! This is awesome 🥇 I left a few comments but nothing major. Excellent work! Can't wait to try this.
jbolor21
left a comment
There was a problem hiding this comment.
This is awesome! Maybe one related idea is adding in a notebook (or in our image converter notebook) on showing this working! (Totally non-blocking comment)
I'll definitely do this! Thanks for the idea 😃 |
bashirpartovi
left a comment
There was a problem hiding this comment.
Great job on this @paulinek13 , this is really cool. I had a few comments and one recommendation as follows:
You could add early convergence check in your step loop for an early exit. Here is an example:
# ...
prev_loss = float('inf')
convergence_threshold = 1e-6
convergence_patience = 10
no_improvement_count = 0
# ...
for step in range(self.steps):
# ...
if abs(prev_loss - loss) < convergence_threshold:
no_improvement_count += 1
if no_improvement_count >= convergence_patience:
# early convergence, exit the loop
break
else:
no_improvement_count = 0
prev_loss = loss
# ....ad47c84 to
2d2beef
Compare
TransparencyAttackConverter
|
@romanlutz @jbolor21 @hannahwestra25 @bashirpartovi Thanks a lot for your reviews, comments and suggestions! I've addressed them 😀 I'll now work on adding a notebook for this converter |
romanlutz
left a comment
There was a problem hiding this comment.
Very nice! This is essentially ready to merge.
Description
#529
This PR adds a converter that implements the transparency attack as described in: "Transparency Attacks: How Imperceptible Image Layers Can Fool AI Perception" by McKee, F. and Noever, D., 2024: https://arxiv.org/abs/2401.15817
The converter blends an attack (background/harmful) image with a benign (foreground/target) image using an optimized alpha channel to create a dual perception effect. The output is a PNG image that looks like the benign image on light backgrounds, but reveals the attack image when placed on dark backgrounds.
Tests and Documentation
✔️ Added unit tests.
I've tested the effectiveness of the converter with various models (even local ones) and it works as expected: I was able to successfully "fool" some of them :) I'll post an example of such attack in the PR comments.