Skip to content

feat-51094: add OAuth 2 / XOAUTH2 support via auth_type & token/credential extras#53554

Merged
eladkal merged 15 commits intoapache:mainfrom
nailo2c:feat-51094_support_oauth2
Aug 5, 2025
Merged

feat-51094: add OAuth 2 / XOAUTH2 support via auth_type & token/credential extras#53554
eladkal merged 15 commits intoapache:mainfrom
nailo2c:feat-51094_support_oauth2

Conversation

@nailo2c
Copy link
Contributor

@nailo2c nailo2c commented Jul 19, 2025

Closes: #51094

Why

Airflow does not currently support OAuth for SMTP. Microsoft is removing BasicAuth as capability from Exchange Online globally this September 2025 (Source: Microsoft announcement).

How

Area Change
Core send_email_smtp / send_mime_email now accept auth_type="oauth2" and an optional access_token.
Provider SmtpHook, SmtpNotifier, and EmailOperator accept auth_type; the hook reads either access_token or client_id + client_secret from Connection extra.
UI Connection form exposes the new fields (auth_type, access_token, client_id, …).
Docs Updated connections/smtp.rst.
Tests New unit tests cover both Basic Auth and OAuth 2 paths.

What

Step 1

Use the Goole Oauth Playground to obtain an access_token.

Step 2

Start Airflow locally with breeze start-airflow and add a connection:

airflow connections add 'smtp_gmail_oauth2' \
  --conn-type smtp \
  --conn-host smtp.gmail.com \
  --conn-port 587 \
  --conn-login 'nailo2c@gmail.com' \
  --conn-extra '{
    "from_email": "nailo2c@gmail.com",
    "auth_type": "oauth2",
    "access_token": "ya29.*****",
    "disable_ssl": "true"
  }'

Step 3

Tested by run this DAG

from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.providers.smtp.hooks.smtp import SmtpHook
from datetime import datetime

def gmail_oauth2_test():
    with SmtpHook(smtp_conn_id="smtp_gmail_oauth2", auth_type="oauth2") as hook:
        hook.send_email_smtp(
            to="nailo2c@gmail.com",
            subject="[Airflow→Gmail] OAuth2 OK",
            html_content="<h3>Gmail XOAUTH2 works 🎉</h3>",
        )

with DAG(
    dag_id="test_gmail_oauth2",
    start_date=datetime(2025,7,1),
    schedule=None,
    catchup=False,
) as dag:
    PythonOperator(task_id="send_mail", python_callable=gmail_oauth2_test)

Or add in airflow.cfg

[smtp]
smtp_host = smtp.gmail.com
smtp_port = 587
smtp_starttls = True
smtp_ssl = False

Then in a Python shell:

from airflow.utils.email import send_email_smtp
send_email_smtp(
    to="nailo2c@gmail.com",
    subject="[Airflow utils] XOAUTH2 test",
    html_content="<h3>🎉 utils.send_email_smtp works</h3>",
    conn_id="smtp_gmail_oauth2",
    auth_type="oauth2",
    access_token="ya29.*****"
)

Step 4

Confirm the message arrived in mailbox.
截圖 2025-07-19 下午2 53 06

@nailo2c nailo2c requested a review from hussein-awala as a code owner July 19, 2025 22:02
@nailo2c nailo2c closed this Jul 20, 2025
@nailo2c nailo2c reopened this Jul 20, 2025
eladkal
eladkal reviewed Jul 20, 2025
View reviewed changes
@nailo2c
Copy link
Contributor Author

nailo2c commented Jul 28, 2025

Hi @eladkal, I’ve moved all the changes from email.py into SmtpHook.

With that done, I think this PR is ready to merge for the new feature, and we can handle deprecating email.py in a follow-up PR. Let me know what you think. Thanks for your guidance!

@eladkal
Copy link
Contributor

eladkal commented Jul 28, 2025

Nice!
I think what we are missing is some doc explnations.
Right now there are many settings to configure but it's not easy to know which setting relates to what.
I think we need to have something like the AWS docs:
https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/connections/aws.html

where we explain the different options and how to set them.

we can handle deprecating email.py in a follow-up PR.

That would be great!

@potiuk
Copy link
Member

potiuk commented Jul 29, 2025

Right now there are many settings to configure but it's not easy to know which setting relates to what.
I think we need to have something like the AWS docs:
https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/connections/aws.html

Agree. More user docs is better.

@nailo2c
Copy link
Contributor Author

nailo2c commented Jul 29, 2025

Sure, let me work on that.

@nailo2c
Copy link
Contributor Author

nailo2c commented Aug 4, 2025

Done. I referred to aws.rst to make the document more organized and to add clearer explanations. I also provided additional runnable examples wherever possible.

截圖 2025-08-03 下午6 16 48

@eladkal eladkal merged commit 42f581f into apache:main Aug 5, 2025
71 checks passed
Nataneljpwd pushed a commit to Asquator/airflow that referenced this pull request Aug 5, 2025
@nailo2c
Add OAuth 2 / XOAUTH2 support via auth_type & token/credential extr…
6e7a415 
…as (apache#53554)

* support auth_type=oauth2 for smtp

* add auth_type to notifications

* add ut for build_xoauth2_string & send_mime_email

* add test for smtp

* add test for notifications

* add ui widgets & property for oauth2

* add extra parameters to the docs

* fix CI test

* fix CI test apache#2

* fix CI test apache#3

* expose `build_xoauth2_string` & add fallback for old cores

* rollback email.py and test_email.py & refactor smtp.py

* refactor the doc to enrich examples and explnations
HsiuChuanHsu pushed a commit to HsiuChuanHsu/airflow that referenced this pull request Aug 5, 2025
@nailo2c @HsiuChuanHsu
Add OAuth 2 / XOAUTH2 support via auth_type & token/credential extr…
d2bc7d9 
…as (apache#53554)

* support auth_type=oauth2 for smtp

* add auth_type to notifications

* add ut for build_xoauth2_string & send_mime_email

* add test for smtp

* add test for notifications

* add ui widgets & property for oauth2

* add extra parameters to the docs

* fix CI test

* fix CI test apache#2

* fix CI test apache#3

* expose `build_xoauth2_string` & add fallback for old cores

* rollback email.py and test_email.py & refactor smtp.py

* refactor the doc to enrich examples and explnations
@eladkal eladkal mentioned this pull request Aug 7, 2025
Closed
40 tasks
ferruzzi pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Aug 7, 2025
@nailo2c @ferruzzi
Add OAuth 2 / XOAUTH2 support via auth_type & token/credential extr…
bea2088 
…as (apache#53554)

* support auth_type=oauth2 for smtp

* add auth_type to notifications

* add ut for build_xoauth2_string & send_mime_email

* add test for smtp

* add test for notifications

* add ui widgets & property for oauth2

* add extra parameters to the docs

* fix CI test

* fix CI test #2

* fix CI test #3

* expose `build_xoauth2_string` & add fallback for old cores

* rollback email.py and test_email.py & refactor smtp.py

* refactor the doc to enrich examples and explnations
fweilun pushed a commit to fweilun/airflow that referenced this pull request Aug 11, 2025
@nailo2c @fweilun
Add OAuth 2 / XOAUTH2 support via auth_type & token/credential extr…
8cc09e3 
…as (apache#53554)

* support auth_type=oauth2 for smtp

* add auth_type to notifications

* add ut for build_xoauth2_string & send_mime_email

* add test for smtp

* add test for notifications

* add ui widgets & property for oauth2

* add extra parameters to the docs

* fix CI test

* fix CI test apache#2

* fix CI test apache#3

* expose `build_xoauth2_string` & add fallback for old cores

* rollback email.py and test_email.py & refactor smtp.py

* refactor the doc to enrich examples and explnations
@boelaars
Copy link

boelaars commented Aug 25, 2025

@eladkal Do you know if this is intended to make its way into the 2.x Airflow release at some point, or will it only be 3.x ? (Hope it's alright to ask here.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@eladkal eladkal eladkal approved these changes

@hussein-awala hussein-awala Awaiting requested review from hussein-awala hussein-awala is a code owner

Assignees

No one assigned

Labels

area:providers kind:documentation provider:smtp

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

OAuth for SMTP notifications

4 participants

@nailo2c @eladkal @potiuk @boelaars