We should add the advanced options field to the block settings:

Since every block has this option we'll always be showing the BlockSettings button. So we can also simplify the code that decides to render the BlockSettings button according to InspectorControls having fills.