I tried to unblock a blocked IP which gave the 500 error.
Same happened when trying to delete an IP from the Exception list.

Adding to the exception or the blocked list does work.