Reading and Writing TOML Configuration Files in Python

TOML (Tom‘s Obvious, Minimal Language) is a configuration file format that emphasizes simplicity and readability. With a minimal syntax intended to be easy for humans to read and write, TOML is a popular alternative to formats like JSON or YAML for storing program settings and preferences. This guide will introduce you to working with TOML data in Python using the third-party toml module.

What is TOML and When Should You Use It?

TOML stands for "Tom‘s Obvious, Minimal Language" and was created by Tom Preston-Werner, co-founder of GitHub. It focuses on being a minimal configuration file format that‘s easy for humans to read and write.

Some key characteristics of TOML files:

• Uses a simple key-value syntax (similar to INI files)
• Supports nested tables (sections) for organization
• Contains data types like strings, ints, floats, booleans, datetimes
• Includes arrays and inline table definitions
• Comments using # symbol
• Focused on configuration files rather than complex data structures

TOML excels as a format for configuration files and storing program preferences. The syntax aims to be easy to parse and generate. It‘s a simpler alternative to JSON and YAML if you don‘t need those formats‘ more complex data structuring capabilities.

Some good examples of using TOML configurations:

• Application defaults and settings
• Development/testing preferences
• Deployment environments and server configs
• Storing key-value options by plugin or module

So in summary, TOML shines for writing human-readable configuration files that don‘t require elaborate data modeling.

Reading and Writing TOML Files with Python‘s toml Module

Since TOML support does not currently exist in Python‘s standard libraries, you need to install the toml module from PyPI to work with TOML data:

pip install toml

This toml module contains several functions for parsing TOML strings into Python dicts and vice versa:

Loading/Reading

• toml.load() – Parse a TOML file and return a dict
• toml.loads() – Parse a TOML string and return a dict

Dumping/Writing

• toml.dump() – Write a dict to a TOML file
• toml.dumps() – Convert a dict to a TOML formatted string

Let‘s look at some examples using this toml module for reading and writing TOML configurations with Python.

Loading a TOML File into a Python Dictionary

Using toml.load(), you can easily parse a TOML file from disk and access the configuration data just like a normal Python dictionary.

For example, say we have a TOML file settings.toml with some application configuration:

# settings.toml

[application]
name = "My App"
version = "1.0"

[logging]
level = "info"
output = "logfile.log"

[server]
host = "127.0.0.1"
port = 8000

We can load this into a Python dictionary with:

import toml

config = toml.load("settings.toml")

print(config)

This would output:

{‘application‘: {‘name‘: ‘My App‘, ‘version‘: ‘1.0‘}, 
‘logging‘: {‘level‘: ‘info‘, ‘output‘: ‘logfile.log‘},
‘server‘: {‘host‘: ‘127.0.0.1‘, ‘port‘: 8000}}

We now have a normal Python dictionary that we can query just like any other using:

print(config["logging"]["level"]) # info
print(config["server"]["port"]) # 8000 

So toml.load() provides an easy way to parse and load any TOML file into memory.

Parsing Only Parts of a TOML Document

If you only wanted to parse a specific part of a TOML document, you could use toml.loads() instead to parse just a TOML text snippet.

For example:

import toml

toml_str = """
title = "Habits"

[habits]
  [habits.morning]
  wake_time = "7:00am"
  walk_dog = false

  [habits.evening]
  bedtime = "11:00pm" 
"""

data = toml.loads(toml_str)
print(data)

This parses just this TOML string and returns:

{‘title‘: ‘Habits‘, ‘habits‘: {‘morning‘: {‘wake_time‘: ‘7:00am‘, ‘walk_dog‘: False}, ‘evening‘: {‘bedtime‘: ‘11:00pm‘}}}

So toml.loads() lets you parse TOML snippets on demand instead of having to load a full file.

Writing a Python Dict as TOML

We can also go the other way – converting Python dictionary data into TOML text. The toml module contains toml.dump() for writing TOML files and toml.dumps() for writing TOML strings.

For example:

import toml

data = {
  "product": {
    "name": "Super Product",
    "price": 9.99,
    "tags": ["awesome", "fantastic"]
  },
  "company": {
    "name": "Awesome Company",  
    "email": "info@awesomecompany.com",
  }
}

toml_str = toml.dumps(data)
print(toml_str)

This would convert the dictionary into:

product.name = "Super Product"
product.price = 9.99
product.tags = ["awesome", "fantastic"] 

[company]  
name = "Awesome Company"
email = "info@awesomecompany.com"

We can also write the output to a new TOML file instead with toml.dump():

with open("data.toml", "w") as outfile:
    toml.dump(data, outfile)

So whether you need to serialize a Python dictionary to a tempfile or save out application settings, toml.dump() handles writing TOML output files.

TOML Tables, Arrays, and Other Data Structures

So far we‘ve looked at simple key-value pairs, but TOML also supports more advanced data structures like nested tables, datetime objects, arrays, inline tables, etc.

Here is an example with some different TOML syntax and types:

title = "My Schedule"

[days]

  [[days.monday]]
  name = "Morning Jog"
  start_time = 2022-03-14T07:00:00

  [[days.monday]]
  name = "Work Meeting" 
  start_time = 2022-03-14T09:00:00
  end_time = 2022-03-14T10:00:00

  [[days.tuesday]]
  name = "Grocery Shopping"
  priority = "high"

  [days.wednesday]
  events = ["Haircut", "Car Wash"]

This TOML file utilizes:

• Nested tables (days.monday, days.tuesday)
• Array of tables syntax ([[days.monday]])
• Datetime values
• Inline table arrays

The toml module will handle parsing all of these more complex data structures into the equivalent Python types like dicts, lists, datetimes, etc.

So you can build fairly elaborate configurations and toml.load() will take care of converting things properly into Python types.

Converting Config Files to TOML Format

If you have existing configuration files or preferences files saved in JSON, YAML, INI, or other formats, you may want to convert those over to TOML format to take advantage of the readability benefits.

The toml module doesn‘t directly include utilities for converting from other formats, but you can load JSON/YAML using the built-in Python libraries and then dump those dicts out as TOML.

For example:

JSON > TOML

import json
import toml

with open("config.json") as jsonfile:
  data = json.load(jsonfile)

with open("config.toml", "w") as tomlfile:
   toml.dump(data, tomlfile)  

YAML > TOML

import yaml 
import toml

with open("settings.yml") as yamlfile:
  data = yaml.safe_load(yamlfile)

with open("settings.toml", "w") as tomlfile:
  toml.dump(data, tomlfile)

So while not direct conversion functions, you can easily translate JSON or YAML over to TOML yourself.

When to Use JSON, YAML or TOML?

JSON, YAML, and TOML all occupy a similar niche for data interchange and storing structured configurations. So which one should you use?

Some key differences and use cases:

JSON

• More strict specification
• Verbose syntax but very clear structure
• Excellent for web APIs and services
• Wide programming language support

YAML

• Compact syntax with significant whitespace
• Supports complex data modeling
• Great for bulky configuration files
• Language library support more limited

TOML

• Focused on simplicity and minimalism
• Aims for extreme human readability
• Excellent for basic config files
• Very limited data structure capabilities

So in summary:

• JSON – Strict spec and portable – good for web services
• YAML – Feature-rich data modeling – good for complex configs
• TOML – Readable and minimalist – best for basic configs

Consider your use case and audience when deciding between these lightweight data formats.

Wrap Up

TOML offers a simplified syntax focused around readability that‘s perfect for storing program configurations and preferences. By using Python‘s toml module, reading and writing TOML data is easy.

Key takeaways:

• TOML excels at human-friendly configuration formats
• toml.load() parses TOML files into Python dicts
• toml.dump() serializes Python values out to the TOML format
• Supports tables, datetimes, arrays, inline tables and other data types
• Converts reasonably cleanly to/from JSON or YAML formats

Hopefully this overview has shown how easy it can be to integrate TOML configuration files into your Python applications and scripts!