Version Used:
D15PreRel 26120.4

Steps to Reproduce:

1. Enable full solution analysis for C#
2. Create a C# console app with the following source code:

namespace ConsoleApp1
{
    class Program
    {
        void F(object o)
        {
            var x = o as Program;
        }

        static void Main(string[] args)
        {
        }
    }
}

3. Verify IDE0008 is shown in the error list.
4. In solution explorer, right click on Analyzers node under Project -> References -> Analyzers and click "Open active ruleset".
5. Search "IDE0008" in the ruleset editor, and change the severity of both the listed diagnostics to "Error".
6. Verify IDE0008 is now shown as "Error" in the error list
7. Close the source file

Expected Behavior:
Error IDE0008 is still shown in the error list. Any diagnostics marked as error in the project's ruleset should always be executed for closed files.

Actual Behavior:
IDE0008 will disappear, and only show for open files. Even closing and re-opening solution/VS doesn't seem to help.
The problem seems to be that many IDE analyzers compute OpenFileOnly value using just the Workspace level NotificationOption, see here. They should instead be using the effective severity of the diagnostic ID for the current compilation to decide this value.