Hi @sigJoe thanks for contributing, would like to hear more opinion on the change of behavior here

If someone is relying on the previous behaviour where samconfig parameter_overrides get completely discarded when specifying CLI parameter-overrides, this PR could be an issue.

Hi @sigJoe thanks for contributing, would like to hear more opinion on the change of behavior here

If someone is relying on the previous behaviour where samconfig parameter_overrides get completely discarded when specifying CLI parameter-overrides, this PR could be an issue.

Prior to this PR, specifying --parameter-overrides on the CLI will cause SAM to completely ignore the parameter_overrides section of samconfig.tom or samcomfig.yaml.

In this PR, SAM will merge both sources of overrides and build/deploy/etc with the resulting set.

I consider the new behaviour to be objectively better. It was also the basis for #2380.

However, I admit it is conceivable that somebody might rely on the way it is currently working, even though I personally don't have any use cases for it.

Edit: I didn't really think about it when I started, but I suppose it could be a good idea to make it retain the current default behaviour and only merge samconfig when someone specifies samconfig as an override (e.g. sam deploy --parameter-overrides samconfig foo=bar file://otherparams.yaml)

After thinking about it further, I removed the merging of samconfig with CLI parameter overrides as probably deserves further discussion

To recap: I changed the syntax that failed the Python3.9 tests and removed the merging of parameters that caused that integration test to fail (can re-add if necessary but should probably be its own thing).

and here's some documentation:

Parameter Overrides Syntax

samconfig.toml

Three supported parameter_overrides formats:

version = 0.1
[default.deploy.parameters]
stack_name = "sam-app"
region = "us-east-1"
parameter_overrides = [
  "file://params.toml",                  # NEW! Amazing file syntax
  "ParameterKey=foo,ParameterValue=bar", # Long form key/value pair. Unnecessarily verbose
  "a=1"                                  # Short form key/value pair
]

You can now also use parameter_overrides as a TOML table/section

version = 0.1
[default.deploy.parameters]
stack_name = "sam-app"
region = "us-east-1"

[default.deploy.parameters.parameter_overrides]
a = "1"

Multiple parameters per string are not best practice:

version = 0.1
[default.deploy.parameters]
stack_name = "sam-app"
region = "us-east-1"
parameter_overrides = [
  "a=1 b=2",                                    # Valid to bundle multiple parameters, although not as clean as putting them on separate lines
  "c=3 ParameterKey=d,ParameterValue=4",        # Invalid: Cannot mix syntax
  "d=4 file://params.toml",                     # Invalid: Still not allowed to mix syntax
  "file://params.toml file://otherparams.toml", # Invalid: Despite being the same syntax, it didn't feel necessary to support this
]

samconfig.yaml

Old format

version: 0.1
default:
  deploy:
    parameters:
      stack_name: sam-app
      region: us-east-1
      parameter_overrides:
        - A=1
        - B=2
        - Spaces='String with spaces'
        - ListParam='a,b,c'

NEW! Dictionary format support

Looks cleaner and supports IDE syntax highlighting

version: 0.1
default:
  deploy:
    parameters:
      stack_name: sam-app
      region: us-east-1
      parameter_overrides:
        - A: 1
        - B: 2
        - Spaces: String with spaces
        - ListParam: # Converts to "a,b,c"
          - a
          - b
          - c
        - file://params.yaml

NEW! Anchors and aliases format support

See https://www.educative.io/blog/advanced-yaml-syntax-cheatsheet#YAML-Anchors-and-Alias for more info.
The following example shows how you can re-use configuration blocks for parameter_overrides. While anchors and aliases technically did work previously, the problem was you would end up with nested lists that would not get parsed correctly. This PR flattens nested lists such that you can now do things like this:

version: 0.1
prod-a:
  deploy:
    parameters:
      stack_name: sam-app
      region: us-east-1
      parameter_overrides: &prod_a_overrides
        - Name: prod-a
        - ParamA: 1
        - ParamB: 2

prod-b:
  deploy:
    parameters:
      stack_name: sam-app-2
      region: us-east-1
      parameter_overrides:
        - *prod_a_overrides # Pulls in above params (Name, ParamA, and ParamB)
        - Name: prod-b      # But we want this one named differently

External TOML params files

When specifying a toml file in parameter-overrides, the file accepts basic key = value syntax. See https://toml.io/en/ for more info.

StringParam = "Pam Param"
NumberParam = 42
ListParam = "a,b,c"
ListParam2 = ["a", "b", "c"] # Converts to "a,b,c"

Note there are limitations to the format such that you cannot specify nested TOML files

StringParam = "Pam Param"
NumberParam = 42
file://another_toml_file.toml # This will fail because it is not valid TOML

External YAML params files

When specifying a yaml file in parameter-overrides, you get all the same config options
Root list:

- A=1 # OK if you really must
- ParameterKey=B,ParameterValue=2 # It works, but now you're just being silly
- StringParam: String with Spaces
- NumberParam: 42
- ListParam: a,b,c
- ListParam2: # Converts to "a,b,c"
  - a
  - b
  - c
- file://other_yaml_params.yaml
- file://toml_params_because_I_can.toml

Ideal Scenario

These features unlock a lot of different use cases, but personally I would be using it something like this:

version: 0.1
prod-a:
  deploy:
    parameters:
      stack_name: sam-app
      region: us-east-1
      parameter_overrides: file://prod-a.yaml # includes prod.yaml, which includes common.yaml

prod-b:
  deploy:
    parameters:
      stack_name: sam-app
      region: us-west-2
      parameter_overrides: file://prod-b.yaml # includes prod.yaml, which includes common.yaml

dev:
  deploy:
    parameters:
      stack_name: sam-app
      region: us-east-1
      parameter_overrides: file://dev.yaml # includes common.yaml

Thank you for your update. Pending review from team.

Sorry about that - I'm not sure how I missed those other failing tests.

Hi @roger-zhangg. With apologies, I accidentally reset your approval when I clicked the Commit suggestion button for your suggested change. Can you kindly re-approve? Thanks

Resolve the conflict which trigger the approval to reset. Pending workflow result and approval from maintainers.