Version Used: VS 2017 15.3.5

Steps to Reproduce:

1. Create new Class Library (.NET Framework) project.
2. Edit Active Rule Set and make CS8019 (Unnecessary using directive) an 'Error'.
3. Compile project.

Expected Behavior:
Given that the project starts with a Class1.cs file containing unused using directives, compilation should fail.

Actual Behavior:
Compilation succeeds. The VS UI does show the IntelliSense error, however.

Attached repro project.
AnalyzerTest.zip