feat: add support multiple dotenv files (#1497) by rekyungmin · Pull Request #3222 · pydantic/pydantic

rekyungmin · 2021-09-16T13:38:29Z

Change Summary

Allow BaseSettings to accept multiple .env files in Config as a list or tuple.
~~The first value takes priority over the second value.~~, this i changed below, the last file now takes priority.

Example:

.env file:

debug_mode=true
host=localhost
port=8000

.env.prod file:

debug_mode=false
host=https://example.com/services

test code:

from pydantic import BaseSettings


class Settings(BaseSettings):
    debug_mode: bool
    host: str
    port: int

    class Config:
        # `.env.prod` takes priority over `.env`
        env_file = ['.env', '.env.prod']  # (undated to reflect newer behaviour)
        env_file_encoding = 'utf-8'

s = Settings()
print(s.dict())
# {'debug_mode': False, 'host': 'https://example.com/services', 'port': 8000}

Related issue number

#1497

Checklist

Unit tests for the changes exist
Tests pass on CI and coverage remains at 100%
Documentation reflects the changes where applicable
changes/<pull request or issue id>-<github username>.md file added describing change
(see changes/README.md for details)
My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Allow BaseSettings to accept multiple dotenv files in Config as a list or tuple.

rekyungmin · 2021-09-16T13:51:44Z

please review

samuelcolvin

otherwise LGTM.

samuelcolvin · 2021-12-08T22:05:50Z

@@ -134,30 +136,47 @@ def __repr__(self) -> str:
 class EnvSettingsSource:


I wonder if it would be smarter to split dotenv processing into a separate source?

I suggested the same thing here but this case is more clear cut.

Could you elaborate on it?

See how sources work in settings. I propose moving dotenv logic into a whole new class.

Good idea, but it will break backwards compatibility.
This is because customise_sources needs to be changed to take 4 callables as arguments.

let's leave to v2.

samuelcolvin · 2021-12-08T22:11:56Z

Please update.

rekyungmin · 2021-12-21T00:43:42Z

please review

rekyungmin · 2021-12-26T06:09:14Z

Thanks for the review. Please review.

Resolve conflicts

phillipuniverse · 2022-02-11T14:37:26Z

@rekyungmin @samuelcolvin IMO this is counterintuitive, I would expect the files to be loaded in the order in which they are defined which would mean the last one in the list gets the final say. So instead of this:

for f in reverse(env_file):
    props = load_dotenv(f)

I would expect this:

for f in env_file:
    props = load_dotenv(f)

This also matches the semi-documented behavior for multiple sources in python-dotenv.

As long as it's consistent I suppose not a huge deal either way, I'm just excited to see this feature!! What's left here to merge?

rekyungmin · 2022-03-09T06:16:39Z

@rekyungmin @samuelcolvin IMO this is counterintuitive, I would expect the files to be loaded in the order in which they are defined which would mean the last one in the list gets the final say. So instead of this:
for f in reverse(env_file):
    props = load_dotenv(f)
I would expect this:
for f in env_file:
    props = load_dotenv(f)
This also matches the semi-documented behavior for multiple sources in python-dotenv.

As long as it's consistent I suppose not a huge deal either way, I'm just excited to see this feature!! What's left here to merge?

In Python, it is common for the rightmost value to win (e.g. unpacking, dict union).
However, in customize_sources, the leftmost value wins.
I think the left win strategy is better for consistency in the settings module.

@samuelcolvin, what do you think?

lucafaggianelli · 2022-07-12T12:59:55Z

Hey, any news on this PR? would love to see it merged! :)

samuelcolvin

Very nearly there, one tiny comment, but more generally I agree with @phillipuniverse - the right most entry in the list/tuple should "win".

Otherwise this is ready to go.

Please update.

samuelcolvin · 2022-08-04T14:37:27Z

        __pydantic_self__._init_private_attributes()

-    @no_type_check
+    @no_type_check  # noqa: C901 (ignore complexity)


Suggested change

@no_type_check # noqa: C901 (ignore complexity)

@no_type_check

samuelcolvin · 2022-08-04T14:55:11Z

please update.

samuelcolvin · 2022-08-12T12:04:15Z

Thanks @rekyungmin for this, and everyone for your feedback.

rekyungmin added 2 commits September 16, 2021 21:35

feat: support multiple dotenv files

3ec734f

Allow BaseSettings to accept multiple dotenv files in Config as a list or tuple.

docs: add multiple dotenv support

e59c86e

rekyungmin changed the title ~~feat: support multiple dotenv files~~ feat: add support multiple dotenv files (#1497) Sep 16, 2021

docs: add a change summary

eefaec5

github-actions Bot added the ready for review label Sep 16, 2021

github-actions Bot assigned PrettyWood and samuelcolvin Sep 16, 2021

rekyungmin force-pushed the feature/multiple-dotenv branch from 9b63964 to 5970635 Compare September 16, 2021 14:37

test: add missing tests

5e18e79

rekyungmin force-pushed the feature/multiple-dotenv branch from 5970635 to aabd806 Compare September 16, 2021 14:46

fix: test coverage

12f4cc3

rekyungmin force-pushed the feature/multiple-dotenv branch from aabd806 to 12f4cc3 Compare September 16, 2021 15:15

rekyungmin added 2 commits September 17, 2021 08:17

refactor: raise a TypeError when an object of an invalid type is passed

0adc1dc

fix: a lint error

7a050bc

rekyungmin force-pushed the feature/multiple-dotenv branch from a0dc6fa to 7a050bc Compare September 16, 2021 23:44

samuelcolvin reviewed Dec 8, 2021

View reviewed changes

github-actions Bot added awaiting author revision awaiting changes from the PR author and removed ready for review labels Dec 8, 2021

github-actions Bot assigned rekyungmin and unassigned samuelcolvin and PrettyWood Dec 8, 2021

rekyungmin added 2 commits December 11, 2021 15:02

refactor: remove _read_process_env_vars method

4939fea

docs: update priority

3375359

samuelcolvin mentioned this pull request Dec 12, 2021

Nested env #3159

Merged

5 tasks

rekyungmin added 2 commits December 12, 2021 22:31

refactor: apply a review

04a39cc

test: simplify testing

2735054

github-actions Bot added the ready for review label Dec 21, 2021

samuelcolvin assigned rekyungmin and unassigned PrettyWood Dec 24, 2021

github-actions Bot added ready for review and removed awaiting author revision awaiting changes from the PR author labels Dec 26, 2021

github-actions Bot assigned PrettyWood and samuelcolvin and unassigned rekyungmin Dec 26, 2021

refactor: apply reviews

b36360c

rekyungmin force-pushed the feature/multiple-dotenv branch from b9d1517 to b36360c Compare December 26, 2021 07:35

Merge branch 'master' into feature/multiple-dotenv

a8cf714

Resolve conflicts

samuelcolvin reviewed Aug 4, 2022

View reviewed changes

github-actions Bot added awaiting author revision awaiting changes from the PR author and removed ready for review labels Aug 4, 2022

github-actions Bot assigned rekyungmin and unassigned samuelcolvin and PrettyWood Aug 4, 2022

samuelcolvin added 5 commits August 12, 2022 12:50

reverse piority order, reduce diff size

24f498e

Merge branch 'master' into rekyungmin-feature/multiple-dotenv

e3a088d

fix change file grammar

f293c33

correct docs, avoid unnecessary copy of os.environ

c09f58a

correct wording in docs

79ded71

samuelcolvin enabled auto-merge (squash) August 12, 2022 12:03

samuelcolvin merged commit 0320a99 into pydantic:master Aug 12, 2022

bkis mentioned this pull request Jan 12, 2023

Allow BaseSettings to accept multiple .env files in Config as a list #1497

Closed

		@@ -134,30 +136,47 @@ def __repr__(self) -> str:
		class EnvSettingsSource:

	@no_type_check # noqa: C901 (ignore complexity)
	@no_type_check

Uh oh!

Conversation

rekyungmin commented Sep 16, 2021 • edited by samuelcolvin Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Change Summary

Related issue number

Checklist

Uh oh!

rekyungmin commented Sep 16, 2021

Uh oh!

samuelcolvin left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

samuelcolvin Dec 8, 2021

Choose a reason for hiding this comment

Uh oh!

rekyungmin Dec 11, 2021

Choose a reason for hiding this comment

Uh oh!

samuelcolvin Dec 11, 2021

Choose a reason for hiding this comment

Uh oh!

rekyungmin Dec 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

samuelcolvin Dec 24, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

samuelcolvin commented Dec 8, 2021

Uh oh!

rekyungmin commented Dec 21, 2021

Uh oh!

rekyungmin commented Dec 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

phillipuniverse commented Feb 11, 2022

Uh oh!

rekyungmin commented Mar 9, 2022

Uh oh!

lucafaggianelli commented Jul 12, 2022

Uh oh!

samuelcolvin left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

samuelcolvin Aug 4, 2022

Choose a reason for hiding this comment

Uh oh!

samuelcolvin commented Aug 4, 2022

Uh oh!

samuelcolvin commented Aug 12, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

rekyungmin commented Sep 16, 2021 •

edited by samuelcolvin

Loading

samuelcolvin left a comment •

edited

Loading

rekyungmin Dec 12, 2021 •

edited

Loading

rekyungmin commented Dec 26, 2021 •

edited

Loading

samuelcolvin left a comment •

edited

Loading