What's __init__.py? PREMIUM

How can you make a Python module out of a directory?

What are Python packages?

If you see a directory that has a __init__.py file within it that directory is meant to be a Python package.

For example, this food_utils directory is meant to be a Python package:

$ tree food_utils
food_utils
├── fruits.py
├── __init__.py
└── non_fruits.py
0 directories, 3 files

A package is a directory that Python treats like it's a module.

We can import this directory, and the Python module we get back is actually the __init__.py file:

>>> import food_utils
>>> food_utils
<module 'food_utils' from '/home/trey/food_utils/__init__.py'>

If we look at the variables within this module object, we'll see that there's not much there right now.

>>> dir(food_utils)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__p
ackage__', '__path__', '__spec__']

There's not much there because our __init__.py file is actually empty. That's not uncommon for Python packages.

All of the code in this package right now lives in its submodules, which are the other .py files that exist in the food_utils directory. Submodules are essentially modules within a module.

We can import a submodule by using the dot notation ({package_name}.{submodule_name}):

>>> import food_utils.fruits

Once we have a submodule imported, we can use it just like we would use any other module: by accessing its attributes.

>>> food_utils.fruits.watermelon
'WATERMELON'

Many modules are packages in disguise

You may not think that you've seen Python packages very often before, but you might be surprised.

For example, the json module in the Python standard library is actually a package. It's a json directory with a __init__.py file within it:

>>> import json
>>> json.__file__
'/usr/lib/python3.10/json/__init__.py'

Though, not all of the code within this json package is defined in its __init__.py file. For example, the JSONEncoder class is actually defined in an encoder submodule:

>>> json.JSONEncoder
<class 'json.encoder.JSONEncoder'>

We don't import it from that encoder submodule, but that's where it was actually defined.

So from our perspective as users of the json module, we can think of it as just like any other module. We don't need to know that json is technically a package.

Why not?

Well, the developers of the json package decided to split up their code into submodules and then they imported all of the objects relevant to json module users into the __init__.py file.

from .decoder import JSONDecoder, JSONDecodeError
from .encoder import JSONEncoder
import codecs

# ...

We can treat json just like any other module because of this.

This is a little bit uncommon for folks to do when you're making a package for your own Python code (and in fact I don't usually recommend it as it involves more code for little benefit). But it's pretty common to see in third-party packages that are meant to be used by many different developers.

Making a Python package without a __init__.py file

One last thing: it is possible to make a Python package that doesn't have a __init__.py file.

This food_utils directory doesn't have a __init__.py file anymore, but it is a valid Python package:

$ rm food_utils/__init__.py
$ tree food_utils
food_utils
├── fruits.py
└── non_fruits.py
0 directories, 2 files

This is called an implicit namespace package, and you probably don't want to make these.

Implicit namespace packages act a little different from other packages and their slightly different behaviors aren't usually desirable.

For example implicit namespace packages have no __file__ attribute:

>>> import food_utils
>>> food_utils.__file__
>>> food_utils
<module 'food_utils' (<_frozen_importlib_external._NamespaceLoader object at 0x7f16e
9e94b80>)>

Many tools that look for Python packages (pytest for example) won't find packages that don't have __init__.py file within them. So I highly recommend always including a __init__.py file in your Python packages.

Summary

A __init__.py file labels a directory as a Python package. A Python package is a module made from a directory: they can be imported, just as any other Python module can be imported. The .py files within a package directory act as submodules of that package.