Fixes WB-7468
Fixes WB-7692
...and a ton of other stuff

I apologize for the size of this PR and the length of the description below :)

Description

Context: currently, settings are handled in a very complex manner, making it somewhat challenging to add new features and to maintain the existing ones.

The main purpose of this PR (which is likely only the first one in a series of PRs) is to refactor the wandb.sdk.wandb_settings.Settings class to provide:

• Consistency, tight control, and encapsulation: implement general interfaces for settings.
  • Enable preprocessing, validation, and runtime hooks (e.g. a setting can depend on another setting) for individual settings.
  • Implement and enforce source priority logic in a consistent matter. Brake settings into policies and non-policy ones.
• Compactness

Having addressed the above, I focused on making sure that nothing is broken across the client.

It's been a lot of fun working on this, and I expect even more fun from the review process :)

Implementation details

This PR adds the Property class that is utilized by the Settings.

Property: a class to represent attributes (individual settings) of the Settings object.

• Encapsulates the logic of how to preprocess and validate values of settings throughout the lifetime of a class instance.
• Allows for runtime modification of settings with hooks, e.g. in the case when a setting depends on another setting.
• The update() method is used to update the value of a setting. Enforces source priority logic when updating values.
• The is_policy attribute determines the source priority when updating the property value. E.g. if is_policy is True, the smallest Source value takes precedence.
• Ability to freeze/unfreeze.

Here's a basic example (for more examples, see tests/wandb_settings_test.py)

from wandb.sdk.wandb_settings import Property, Source


def uses_https(x):
    if not x.startswith("https"):
        raise ValueError("Must use https")
    return True

base_url = Property(
    name="base_url",
    value="https://wandb.com/",
    preprocessor=lambda x: x.rstrip("/"),
    validator=[lambda x: isinstance(x, str), uses_https],
    source=Source.BASE,
)

endpoint = Property(
    name="endpoint",
    value="site",
    validator=lambda x: isinstance(x, str),
    hook=lambda x: "/".join([base_url.value, x]),
    source=Source.BASE,
)

>>> print(base_url)  # note the stripped "/"
'https://wandb.com'
>>> print(endpoint)  # note the runtime hook
'https://wandb.com/site'
>>> print(endpoint._value)  # raw value
'site'
>>> base_url.update(value="https://wandb.ai/", source=Source.INIT)
>>> print(endpoint)  # valid update with a higher priority source
'https://wandb.ai/site'
>>> base_url.update(value="http://wandb.ai/")  # invalid value - second validator will raise exception
ValueError: Must use https
>>> base_url.update(value="https://wandb.dev", source=Source.USER)
>>> print(endpoint)  # valid value from a lower priority source has no effect
'https://wandb.ai/site'

Settings: a class that uses Property objects to represent configurable settings.

• The code is supposed to be self-documented -- see wandb/sdk/wandb_settings.py :)
• Clearly and compactly defines all individual settings, their default values, preprocessors, validators, and runtime hooks as well as whether or not they are treated as policies.
• Provides a mechanism to update settings specifying source (which abides the corresponding Property source logic) via Settings.update(). Direct attribute assignment is not allowed.
• Careful Settings object copying.
• Mapping interface.
• Exposes attribute.value if attribute is a Property.
• Ability to freeze/unfreeze.
• Settings.make_static() method that we can use to replace StaticSettings.
• Adapted/reworked convenience methods to apply settings originating from different source.

Other stuff in this PR

• Fix WB-7468 (code saving and wandb.login()).
• Fix WB-7692 (drop some of the deps back-ported to py2).
• A note on using a debugger in CONTRIBUTING.md.
• Configurable '{"code_saving_enabled": true}' in mock_server ctx.
• Improved test suite robustness (see below).
• Expose run settings via a public attribute Run.settings. We should also start saving them by default with each run as it would provide a lot of useful information that is not otherwise captured.

Testing

Numerous (>100) tests, both settings-specific and not, were added or fixed.

• No more sloppiness in the way the settings are treated in the tests :)
• Improved isolation for numerous tests that were leaking state through filesystem and env variables and potentially affecting other tests (in a flaky way) executed by the same pytest worker.

Some notes

When designing this, I considered external libraries attrs and pydantic as potentially useful. Neither seemed like a good option for us:

• pydantic generates scary code behind the scenes.
• attrs only ensures conversion and validation at init.
• Neither have runtime hooks/converters/validation.
• No easy way to implement tracking and updating values based on source priorities.

Status as of 1/11/2022

• A single functional test is failing: disabled code saving. This happens bc the mock server sets {“code_saving_enabled”: True} in the user settings by default, which, since now code saving setting is a policy, has higher priority than the ENV variable that the test sets. In unit tests, I made that configurable and so I do live_mock_server.set_ctx({"code_saving_enabled": None}) before running the corresponding tests — now I am working on making this possible with yea. Shouldn't take too long, but I think the review can proceed in the meantime.
• Several tests on Windows are failing -- need to fix them, but I also think it's not a road block for the review.