Describe the enhancement requested

Add python bindings so that the new C++ AzureFilesystem can be used from Python code. abfs:// and abfss:// URIs should automatically select the AzureFilesystem.

This should enable pyarrow users to get much improved performance and reliability compared to https://github.com/fsspec/adlfs.

Work on the C++ AzureFilesystem is tracked by #18014

Sub issues:

• [Python][FS][Azure] Enable Azure in pyarrow wheel builds #39352
• [CI][Packaging] Enable Azure in more tests and builds #40079
• [Python][FS][Azure] Minimal python bindings for AzureFilesystem #39968
• [Python][FS][Azure] Expose all the configuration options to python #40572
• [Python][Azure][Docs] Add documentation about AzureFilesystem #41496

Component(s)

Python