Ansible Nested Variables: A Comprehensive Guide

Ansible is a powerful open-source automation tool used by developers and ops teams to automate provisioning, configuration management, and application deployments.

One of Ansible‘s most useful features is its flexible variable system. Variables allow you to store reusable data and reference it throughout your playbooks, roles, and templates.

Nested variables take Ansible‘s variables to the next level by allowing hierarchical variable definitions. If you‘re not leveraging nesting, you‘re missing out on a key capability for managing complex configs.

In this comprehensive 3200+ word guide, you’ll learn:

• What are Nested Variables?
• Why Use Nested Variables?
• Defining Nested Variables
• Accessing Nested Variable Values
• Setting Nested Variable Definitions
• Looping Over Nested Variable Subsets
• Chaining Nested Variable Dependencies
• Nested Variables Best Practices
• Nested Variables vs Other Tools
• The Importance of Mastering Nested Variables

What are Nested Variables?

Ansible nested variables refer to variables defined within other variables allowing complex, multi-layer data structures.

For example, you can define:

user_accounts:
  john:
    uid: 1001
    home: "/home/john"
  jane:  
     uid: 1002
     home: "/home/jane"

Here the parent user_accounts variable contains two nested variables john and jane. And each nested variable further contains subkeys uid and home with user-specific values.

This demonstrates a simple case, but nesting can involve unlimited layers depending on the use case. Nested variables don‘t have to just be dictionaries either – you can nest arrays and other data types.

Why Use Nested Variables?

Nested variables offer several key advantages:

Organized Data

Nesting allows you to logically organize related configuration data under a hierarchy of parent-child variables. This massively improves readability of complex data for both humans and Ansible.

For example, instead of a flat dictionary:

john_uid: 1001
john_home: /home/john
jane_uid: 1002 
jane_home: /home/jane

You can semantically group the related user data:

users:
  john: 
    uid: 1001
    home: /home/john

  jane:
    uid: 1002
    home: /home/jane

This avoids clutter and makes the purpose of each value explicit through hierarchy.

Simplify Complex Configs

Ansible automation often involves managing hundreds or thousands of parameters across layers of infrastructure.

Nested variables allow you to encapsulate related low level configuration under parent variables to simplify management:

apache:
  max_clients: 100
  user: www-data
  enable_modules:
    - headers
    - rewrite

php:
  memory_limit: 128M  
  opcache_enabled: true
  extensions: 
    - mysqli
    - curl

Grouping these configurations simplifies reasoning about the collective purpose of each nested variable.

Reuse and Extend Data

Common data can be defined once at a parent variable level, but specialized for each child variable through extension:

server_defaults:
  num_cpus: 2
  memory: 4G
  os: ubuntu

servers:
  web:
    <<: *server_defaults
    cores: 4 
    memory: 8G

  db:
    <<: *server_defaults
    memory: 16G  

Here common defaults are set once, then extended only where needed. This YAML anchor technique avoids repetition.

Dynamic References

Nested variables allow child values to be defined in terms of parent values or even other runtime variables.

For example:

username: john

users:
  {{ username }}:
     uid: 1001
     home: /home/{{ username }}

Here uid and home values are dynamically constructed for flexible reuse.

This technique is employed heavily by Ansible roles to configure components in relation to other factors.

In summary, nested variables organize data hierarchically for simplified management while avoiding redundancy and allowing flexible reuse.

Defining Nested Variables

Ansible leverages YAML syntax to allow flexible definition of nested variable structures within playbooks and included files.

Basic YAML Nesting

Here is a YAML snippet demonstrating variable nesting:

parent_var:
  child_var1: value1
  child_var2: value2

The indentation of child_var1 and child_var2 nests them under parent_var.

You can nest arbitarily deep, indenting each level, like so:

grandparent_var:
  parent_var:  
    child_var: value

Where child_var is nested twice underneath.

Data Structure Nesting

In addition to nesting simple key-value pairs, Ansible readily allows nesting complex structures like lists and dictionaries:

users:
  - name: john
    groups: 
      - wheel 
      - dev

  - name: jane 
    groups:
      - wheel

Here a list contains two nested user dictionaries with further nested groups arrays.

You can also nest dictionaries in dictionaries:

app_config:
  frontend: 
    port: 80
    server: nginx

  backend:
    port: 8080  
    server: apache

This allows you to model arbitrarily complex, multi-layer data hierarchies.

Accessing Nested Variable Values

Ansible offers simple syntax to traverse nested variable hierarchies for accessing sub-values.

Dot Notation

Dot notation walks down the variable hierarchy from parent to child using literal names.

For example:

{{ users.john.uid }}

Accesses the uid subkey nested under john.

You can chain dot notation to access variables nested at any depth:

{{ companies.acme.apps.frontend.port }}

Dot notation is ideal for direct literal references.

Bracket Notation

Bracket notation is more flexible in allowing dynamic nesting traversal using either variables or literal strings:

{{ users[‘john‘][‘uid‘] }}

Accesses the same uid subkey.

And bracket notation can leverage variables:

{{ users[username].uid }}

Here the username would be substituted dynamically.

Mixing dot and bracket notation is common for readability.

Setting Nested Variable Definitions

Nested variables can be defined and initialized across multiple Ansible file types and objects.

Inventory Files

You can directly set nested variable values within inventory INI or YAML files:

inventory.yml

[webservers]
www1.example.com
www2.example.com

[webservers:vars]
nginx:
  max_clients: 200 
  fastcgi: "fastcgi_pass unix:/var/run/php/php7.2-fpm.sock;"

apache:
  max_clients: 100

Here multiple nested variable definitions belonging to the webservers group are defined.

Playbooks

In playbook YAML files, set variables at either the play or task level:

playbook.yml

---
- hosts: webservers

  vars:
    http:  
      nginx: 
        port: 80
      apache: 
        port: 8080

  tasks:
    - name: Print ports  
      debug: 
        msg: "{{ http[item].port }}"
      loop: 
        - nginx
        - apache

This playbook initializes two nested variables and prints the port values.

Included Files

Store nested variable definitions in external YAML files and import them:

roles/common/vars/users.yml

john:
  uid: 1001
  group: users 

jane:
  uid: 1002  
  group: admins

playbook.yml

---

- hosts: all  

  vars_files:
    - "roles/common/vars/users.yml"

  tasks:
    - name: Print john uid
      debug:  
        msg: "{{ users.john.uid }}"

Including variable files avoids playbook clutter.

Registered Variables

Even registered task results can be leveraged as nested variables:

- name: Get user records
  uri:
    url: "https://api.example.com/v1/users"
  register: result

- name: Show jane uid 
  debug:
    msg: "{{ result.json.jane.uid }}"

Here an API call is registered, then parsed as nested JSON.

Looping Over Nested Variable Subsets

A common use case is needing to iterate over a subset of items in a nested data structure.

Ansible offers several useful plugins for looping over portions of nested variables without having to flatten them beforehand:

with_dict

The with_dict lookup plugin iterates over the key-value pairs of a nested dictionary:

- name: Start services
  ansible.builtin.service:
    name: "{{ item.key }}" 
    state: started  
  loop: "{{ services|dict2items }}"

vars:
  services: 
    nginx: 
      enabled: true
    mysql:
      enabled: false 

This allows acting on each service without requiring a separate services list.

with_subelements

with_subelements iterates over sub-portions of arrays and dictionaries:

- name: Add users 
  user:
    name: "{{ item.0.name }}"
    uid: "{{ item.0.uid }}"
    groups: "{{ item.1 }}"  
  loop: "{{ users|subelements(‘groups‘) }}"  

vars:
  users:    
    - name: john
      uid: 1001  
      groups: 
        - wheel
        - dev
    - name: jane    
      uid: 1002
      groups: 
        - wheel

Here the user subarrays are parsed to pass groups cleanly.

There are also with_list and with_together plugins for more complex looping.

Chaining Nested Variable Dependencies

A unique capability unlocked by nesting is dynamically varying playbook behavior based on dependencies between variables.

Some examples:

Variable App Roles

vars:
  app_list: [app1, app2, app3]

roles:
  - role: "{{ item }}"
    loop: "{{ app_list }}"

Here the app_list var controls what roles get applied.

User-Specific App Installs

users:
  john: 
    groups:  
      - dev
      - docker

    apps:
      - git  
      - vim
      - tmux

  jane:
    groups:
      - docker

Now app installs depend on user groups.

By chaining nested variable dependencies in this way, you gain extreme flexibility to adapt playbook functionality based on runtime data as opposed to hard-coded logic.

Nested Variables Best Practices

To effectively leverage nested variables:

Start Simple

Avoid overly complex nesting up front. Begin simple and refactor hierarchies once functionality is working.

Comment Extensively

Comment all nested variable blocks to explicitly capture intent for future readers.

Test Rigorously

Thoroughly test variable references to detect issues early, otherwise debugging problems is challenging once playbooks grow large.

Abstract Common Data

Identify common data subsets to abstract into shareable nested variables.

Modularize Large Hierarchies

Break deeply complex nested structures into multiple easier to manage files.

Mastering these nested variable best practices takes time but pays off tremendously in more maintainable automation as complexity scales up.

Nested Variables vs Other Tools

It‘s worth comparing Ansible‘s capabilities for nested variables and data modeling to other widespread DevOps tools:

• Terraform has variables but no native support for nested structures. Requires workaround HCL/JSON hacks.
• CloudFormation similarly lacks built-in nesting requiring messy JSON/YAML escapes.
• Kubernetes only supports rudimentary YAML config nesting – no variable references.
• Docker flattens all compose file vars into top level keys.
• Jenkins nests pipeline var syntax but doesn‘t allow variable reuse or references.

Among major competitors, Ansible unmatched in deeply integrating recursive data modeling into variable system DNA. This focus manifests in more elegantly written automation.

The Importance of Mastering Nested Variables

Why bother learning the intricacies of Ansible‘s nested variables as opposed to just using simple flat variables?

Flat is Fragile

Lacking structured hierarchies, flat variables eventually grow disorganized. Dependencies get scattered across disparate tasks rather than modularized.

Flat Doesn‘t Scale

With flat variables, duplicate data has to be copied everywhere it‘s referenced leading to consistency issues. Files grow huge.

Flat is Chaotic

Undifferentiated flat variables turn into a soup difficult to parse for humans or Ansible. Technical debt balloons out of control.

Complexity Never Diminishes

Infrastructure complexity doesn‘t decrease over time – it expands. Technical debt compounds rapidly without thoughtful abstraction.

You can only sweep complexity under the rug with flat variables for so long before it catches up as disorder.

Mission Critical Requires Nesting

If your Ansible usage is small scale or non-critical, flat may suffice in the short term. But mission critical automation at production scale inevitably demands nested variables to control chaos through data architecture.

That‘s because complexity is inherent with large scale systems – it can‘t be removed, only organized. Ansible nesting allows imposing order rather than being dominated by entropy.

Ironically then, while more complex to initially comprehend, nested variables ultimately simplify usage of Ansible at scale by preventing variable sprawl.

This order from chaos approach is a hallmark of mature engineering teams who vanguard large operational systems.

So while requiring an upfront learning curve investment, fluency with Ansible nested variables pays perpetual maintenance dividends in proportion to infrastructure scale and criticality.