[{"content":"In this post I will show you how to develop an Ansible custom filter plugin that transforms live data obtained from a network device into the correct format for further use in subsequent Ansible playbook tasks.\nWhat are Ansible filters? Ansible filters are powerful Python functions for manipulating data within Ansible. Filters receive input data via a variable, then perform a fixed operation and returns the result. Use cases are combining lists, transforming JSON data, manipulating strings, parsing data, …\nWithin Ansible there are three types of filters:\nAnsible-specific filters (present in core.py) Official built-in Jinja2 filters Custom filter plugins An example of the built-in \u0026ldquo;dict2items\u0026rdquo; filter in ansible playbook task:\n- name: VLAN \u0026gt;\u0026gt; parsed with custom filter set_fact: vlans_live: \u0026#34;{{ vlans | dict2items }}\u0026#34; Develop a custom Ansible filter for JSON manipulation In this example, we receive an overview of the configured switch vlans via REST API. The JSON response is a nested dictionary structure that we want to transform through our custom filter plugin \u0026ldquo;nesteddict2items\u0026rdquo; into a simple list of dictionaries containing the desired key: value pairs. This to structure and prepare the data for further Ansible Playbook tasks that go through the vlan list via the loop method, to perform for example configuration validation or generation.\nvlan output requested from the device:\n\u0026#34;vlans\u0026#34;: { \u0026#34;1\u0026#34;: { \u0026#34;vlanDescription\u0026#34;: \u0026#34;vlan-1\u0026#34;, \u0026#34;vlanNumber\u0026#34;: \u0026#34;1\u0026#34; }, \u0026#34;10\u0026#34;: { \u0026#34;vlanDescription\u0026#34;: \u0026#34;vlan-10\u0026#34;, \u0026#34;vlanNumber\u0026#34;: \u0026#34;10\u0026#34; } } vlan output after parsing with the \u0026ldquo;nesteddict2items\u0026rdquo; filter:\n\u0026#34;vlans\u0026#34;: [ { \u0026#34;vlanDescription\u0026#34;: \u0026#34;vlan-1\u0026#34;, \u0026#34;vlanNumber\u0026#34;: \u0026#34;1\u0026#34; }, { \u0026#34;vlanDescription\u0026#34;: \u0026#34;vlan-10\u0026#34;, \u0026#34;vlanNumber\u0026#34;: \u0026#34;10\u0026#34; } ] 1. Create a Python function Using the built-in Ansible filter \u0026ldquo;dict2items\u0026rdquo; it is possible to transform a dictionary into a list, but this filter does not give us the desired result in combination with a nested dictionary. We need a simple Python function that takes the value of each dictionary item and adds it to a new list.\nCreate the file \u0026rsquo;nesteddict2items.py\u0026rsquo; with the following code and place it in the folder \u0026ldquo;filter_plugins\u0026rdquo; within your project.\n#!/usr/bin/python class FilterModule(object): \u0026#39;\u0026#39;\u0026#39; Nested dict filter \u0026#39;\u0026#39;\u0026#39; def filters(self): return { \u0026#39;nesteddict2items\u0026#39;: self.nesteddict2items } def nesteddict2items(self, vlans_live): vlans = [] for v_key, v_value in vlans_live.items(): vlans.append(v_value) return vlans 2. Add filter_plugins path folder to ansible.cfg To make effective use of the plugin, the only thing left to do is to refer to our new plugin folder via ansible.cfg. Remove the comment from the \u0026ldquo;filter_plugins\u0026rdquo; line and add the path \u0026ldquo;filter_plugins\u0026rdquo; so that Ansible\u0026rsquo;s filter search path can find our custom plugin file.\n# set plugin path directories here, separate with colons #action_plugins = /usr/share/ansible/plugins/action #become_plugins = /usr/share/ansible/plugins/become #cache_plugins = /usr/share/ansible/plugins/cache #callback_plugins = /usr/share/ansible/plugins/callback #connection_plugins = /usr/share/ansible/plugins/connection #lookup_plugins = /usr/share/ansible/plugins/lookup #inventory_plugins = /usr/share/ansible/plugins/inventory #vars_plugins = /usr/share/ansible/plugins/vars filter_plugins = filter_plugins #test_plugins = /usr/share/ansible/plugins/test #terminal_plugins = /usr/share/ansible/plugins/terminal #strategy_plugins = /usr/share/ansible/plugins/strategy 3. Custom filter in Ansible Playbooks Now you can use the filter in your Ansible Playbook. Filters can be used consecutively, so the possibilities are endless.\nPython type testing via type_debug filter he built-in filter \u0026ldquo;type_debug\u0026rdquo; stores the underlying Python data type for our variable. We add these to check and illustrate whether the Python dictionary has been effectively converted to a list.\nPython type without filter transformation:\nAnsible Playbook task: - name: DEBUG \u0026gt;\u0026gt; vlan python type debug: var: vlans_type: \u0026#34;{{ vlans | type_debug }}\u0026#34; Terminal output: TASK [DEBUG \u0026gt;\u0026gt; vlan parsed] ok: [NET-SWI-001] =\u0026gt; { \u0026#34;vlans_type\u0026#34;: \u0026#34;dict\u0026#34; } Python type with filter transformation:\nAnsible Playbook task: - name: DEBUG \u0026gt;\u0026gt; vlan python type debug: var: vlans_type: \u0026#34;{{ vlans | nesteddict2items | type_debug }}\u0026#34; Terminal output: TASK [DEBUG \u0026gt;\u0026gt; vlan parsed] ok: [NET-SWI-001] =\u0026gt; { \u0026#34;vlans_type\u0026#34;: \u0026#34;list\u0026#34; } ","permalink":"https://netdevops.be/developing-a-custom-ansible-filter-plugin/","summary":"\u003cp\u003eIn this post I will show you how to develop an Ansible custom filter plugin that transforms live data obtained from a network device into the correct format for further use in subsequent Ansible playbook tasks.\u003c/p\u003e\n\u003ch2 id=\"what-are-ansible-filters\"\u003eWhat are Ansible filters?\u003c/h2\u003e\n\u003cp\u003e\u003ca href=\"https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html\"\u003eAnsible filters\u003c/a\u003e are powerful Python functions for manipulating data within Ansible. Filters receive input data via a variable, then perform a fixed operation and returns the result. Use cases are combining lists, transforming JSON data, manipulating strings, parsing data, …\u003c/p\u003e","title":"Developing a custom Ansible filter plugin"},{"content":"This demo shows how you can easily backup your entire Alcatel-Lucent Enterprise Omniswitch environment using Ansible. You can store the backup files locally or even better, push them to a remote GIT repository. The result is a CMDB (configuration management database) in which the different backup versions are stored and where you can easily compare them to track changes.\nSwitch configuration stored into a GIT repository For AOS6 devices we use Gilbert Moisio\u0026rsquo;s \u0026ldquo;ale_aos\u0026rdquo; Ansible collection to save the CLI output captured from the \u0026ldquo;show configuration snapshot\u0026rdquo; command through SSH. For AOS8 devices we can utilize the REST API to send the necessary GET requests via the Ansible built-in \u0026ldquo;uri\u0026rdquo; module.\nWhat is Ansible: Ansible is an open-source orchestration tool for automating application deployment and configuration management, enabling infrastructure as a code.\nYou can find all the demo files in the ansible-aos-demo GitHub repository. Clone the remote repository to a local folder to get started:\ngit clone https://github.com/jefvantongerloo/ansible-aos-demo.git Installation: Python modules, Ansible collections Before we get started we need to install some Python modules and the desired Ansible collections. For example, the ale_aos Ansible collection makes underlying use of the Python package Netmiko. We install from two files in which we track dependencies: requirements.txt and requirements.yml.\nansible-base netmiko --- collections: - name: gmoisio.ale - name: lvrfrc87.git_acp version: \u0026#34;1.1.3\u0026#34; - name: community.general Install Python requirements pip3 install -r requirements.txt 2.Install Ansible-galaxy collections\nansible-galaxy collection install -r requirements.yml Test the Ansible and the Ansible-galaxy collections installation by running the following terminal commands:\nansible --version ansible-galaxy collection list Example output \u0026quot;ansible --version\u0026quot;:\njefvantongerloo:$ ansible --version ansible 2.10.16 config file = None configured module search path = [\u0026#39;/home/p064033/.ansible/plugins/modules\u0026#39;, \u0026#39;/usr/share/ansible/plugins/modules\u0026#39;] ansible python module location = /home/p064033/.local/lib/python3.8/site-packages/ansible executable location = /home/p064033/.local/bin/ansible python Host and variable configuration Add all desired hosts to the inventory.yml file and specify the correct device_os of the device, respectively aos6 and aos8. Through this parameter, we will later determine whether to use SSH (slow) or REST API (fast) to interact with the device.\n--- all: hosts: NET-SWI-001: ansible_host: 10.10.0.1 device_vendor: alcatel device_os: aos8 In our Ansible playbook we use the variable \u0026ldquo;ale_username\u0026rdquo; and \u0026ldquo;ale_password\u0026rdquo; to login to the switches. By adding this as a group variable in the file \u0026ldquo;group_vars / all / all.yml\u0026rdquo; they are available for the group \u0026ldquo;all\u0026rdquo;, i.e. all hosts.\n--- ale_username: admin ale_password: switch The Ansible backup playbook can store the backup configuration locally or on a remote git repository. By altering the \u0026ldquo;backup_git\u0026rdquo; variable with \u0026ldquo;true\u0026rdquo; or \u0026ldquo;false\u0026rdquo; you can activate this functionality. If you are pushing backups to git, also complete the set of git parameters to allow Ansible accessing your GIT repository.\nbackup_local: true backup_git: false git_token: \u0026#34;\u0026#34; git_username: \u0026#34;\u0026#34; git_url: \u0026#34;\u0026#34; git_branch: \u0026#34;master\u0026#34; Execute Ansible playbook backup.yml Final step is to run the backup-all.yml ansible playbook:\nansible-playbook backup-all.yml All playbook tasks from the backup-all.yml file are now executed sequentially. Taking into account our \u0026ldquo;device_os\u0026rdquo; parameter, only relevant tasks for the host are performed. If everything goes well, the counter of failed tasks will remain at zero.\nIf the local backup option is activated, a folder structure has been created in the \u0026ldquo;backups\u0026rdquo; folder containing the configuration files.\n","permalink":"https://netdevops.be/alcatel-lucent-enterprise-omniswitch-aos6-aos8-ansible-backup/","summary":"\u003cp\u003eThis demo shows how you can easily backup your entire \u003cstrong\u003eAlcatel-Lucent Enterprise Omniswitch\u003c/strong\u003e environment using Ansible. You can store the backup files locally or even better, push them to a remote GIT repository. The result is a \u003cstrong\u003eCMDB (configuration management database)\u003c/strong\u003e in which the different backup versions are stored and where you can easily compare them to track changes.\u003c/p\u003e\n\u003cfigure\u003e\n    \u003cimg loading=\"lazy\" src=\"/img/20210527-git-compare.png\"\n         alt=\"Switch configuration stored into a GIT repository\"/\u003e \u003cfigcaption\u003e\n            Switch configuration stored into a GIT repository\n        \u003c/figcaption\u003e\n\u003c/figure\u003e\n\n\u003cp\u003eFor AOS6 devices we use \u003ca href=\"https://galaxy.ansible.com/gmoisio/ale\"\u003eGilbert Moisio\u0026rsquo;s \u0026ldquo;ale_aos\u0026rdquo; Ansible collection\u003c/a\u003e to save the CLI output captured from the \u0026ldquo;show configuration snapshot\u0026rdquo; command through SSH. For AOS8 devices we can utilize the REST API to send the necessary GET requests via the Ansible built-in \u0026ldquo;uri\u0026rdquo; module.\u003c/p\u003e","title":"Backup Alcatel-Lucent Enterprise Omniswitch AOS6 \u0026 AOS8 using Ansible"}]