01. January 2019 · Comments Off on Ansible playbook to manage objects on a Cisco Firepower Management Center (FMC) · Categories: Ansible, Cisco, Firewall, Networking · Tags: , , , , , , , , , , , ,

I really wish Cisco would support the DevOps community and release Ansible modules for their products like most other vendors. That being said, since there are no modules for the Cisco Firepower you have to manage the device through the APIs directly. Managing anything using raw API requests in Ansible can be a little tricky but not impossible. When creating playbooks like this you will typically spend most time figuring out the structure of responses and how best to iterate through them.

The following Ansible playbook is a refactor of a previous script I wrote last year to post/delete objects up to a firepower in bulk. I have spent a lot of time with Ansible playbooks and I recommend grouping and modularizing related tasks into separate importable YAML files. This not only makes reusing common groups of tasks much easier but also means later those logical task groupings can simply be copied up into a role with little to no effort.

main.yaml: a /usr/bin/ansible-playbook -f 10 script text executable, ASCII text

#!/usr/bin/ansible-playbook -f 10
## Ansible playbook to manage objects on a FMC
# 2019 (v.01) - Playbook from www.davideaves.com
---
- name: manage firepower objects
  hosts: fmc
  connection: local
  gather_facts: no

  vars:

  - ansible_connection: "local"
  - ansible_python_interpreter: "/usr/bin/env python"

  - fmc_provider:
      username: "{{ username | default('apiuser') }}"
      password: "{{ password | default('api1234') }}"

  - fmc_objects:
    - name: server1
      value: 192.0.2.1
      description: Test Server

  tasks:

  ## Note ##
  # Firepower Management Center REST API authentication tokens are valid for 30 minutes, and can be refreshed up to three times
  # Ref: https://www.cisco.com/c/en/us/td/docs/security/firepower/623/api/REST/Firepower_Management_Center_REST_API_Quick_Start_Guide_623/Connecting_with_a_Client.html

  - name: "fmc_platform: generatetoken"
    local_action:
      module: uri
      url: "https://{{ inventory_hostname }}/api/fmc_platform/v1/auth/generatetoken"
      method: POST
      user: "{{ fmc_provider.username }}"
      password: "{{ fmc_provider.password }}"
      validate_certs: no
      return_content: no
      force_basic_auth: yes
      status_code: 204
    register: auth

  - include: fmc_objects.yaml
    when: auth.x_auth_access_token is defined

The following is the task grouping that will make object changes to the FMC using Ansibles built in URI module. I have tried to make this playbook as idempotent as possible so I first register an array with all of the objects that exist on the FMC. I then iterate through that array in subsequent tasks so I only change what does not match. If it sees a fmc_object name key with no value set, the delete task will remove the object from the FMC.

fmc_objects.yaml: ASCII text

## Cisco FMC object management tasks for Ansible
## Requires: VAR:auth.x_auth_access_token
## 2019 (v.01) - Playbook from www.davideaves.com
#
## VARIABLE EXAMPLE ##
#
#  - fmc_objects:
#    - name: server1
#      value: 192.0.2.1
#
## USAGE EXAMPLE ##
#  - include: fmc_objects.yaml
#    when: auth.x_auth_access_token is defined
#
---
 
## NOTE ##
# Currently only handling host and network objects!
# Other object types will likely require a j2 template to construct the body submission.

- name: "fmc_config: get all objects"
  local_action:
    module: uri
    url: "https://{{ inventory_hostname }}/api/fmc_config/v1/domain/{{ auth.domain_uuid }}/object/{{ item }}?limit=10000&expanded=true"
    method: GET
    validate_certs: no
    status_code: 200
    headers:
      Content-Type: application/json
      X-auth-access-token: "{{ auth.x_auth_access_token }}"
  with_items:
    - hosts
    - networks
  register: "all_objects_raw"
 
# Unable to figure out how to do this without a j2 template.
# FMC returns too many subelements to easily filter.

- name: "fmc_config: post new objects"
  local_action:
    module: uri
    url: "https://{{ inventory_hostname }}/api/fmc_config/v1/domain/{{ auth.domain_uuid }}/object/{{ fmc_objects | selectattr('name', 'equalto', item) | map(attribute='type') | list | last | default('hosts') | lower }}"
    method: POST
    validate_certs: no
    status_code: 201
    headers:
      Content-Type: application/json
      X-auth-access-token: "{{ auth.x_auth_access_token }}"
    body_format: json
    body:
      name: "{{ item }}"
      value: "{{ fmc_objects | selectattr('name', 'equalto', item) | map(attribute='value') | list | last }}"
      description: "{{ fmc_objects | selectattr('name', 'equalto', item) | map(attribute='description') | list | last | default('Ansible Created') }}"
      overridable: "{{ fmc_objects | selectattr('name', 'equalto', item) | map(attribute='overridable') | list | last | default('False') | bool }}"
  with_items: "{{ lookup('template', 'fmc_objects-missing.j2').split('\n') }}"
  when: (item != "") and (fmc_objects | selectattr('name', 'equalto', item) | map(attribute='value') | list | last is defined)
  changed_when: True
 
## NOTE ##
# The conditions below will not catch the sudden removal of the description or overridable key

- name: "fmc_config: modify existing objects"
  local_action:
    module: uri
    url: "{{ item.1.links.self }}"
    method: PUT
    validate_certs: no
    status_code: 200
    headers:
      Content-Type: application/json
      X-auth-access-token: "{{ auth.x_auth_access_token }}"
    body_format: json
    body:
      name: "{{ item.1.name }}"
      id: "{{ item.1.id }}"
      type: "{{ item.1.type }}"
      value: "{{ fmc_objects | selectattr('name', 'equalto', item.1.name) | map(attribute='value') | list | last }}"
      description: "{{ fmc_objects | selectattr('name', 'equalto', item.1.name) | map(attribute='description') | list | last | default('Ansible Created') }}"
      overridable: "{{ fmc_objects | selectattr('name', 'equalto', item.1.name) | map(attribute='overridable') | list | last | default('False') | bool }}"
  with_subelements:
    - "{{ all_objects_raw['results'] }}"
    - json.items
  when: (fmc_objects | selectattr('name', 'equalto', item.1.name) | list | count > 0) and
        (((fmc_objects | selectattr('name', 'equalto', item.1.name) | map(attribute='value') | list | last is defined) and (fmc_objects | selectattr('name', 'equalto', item.1.name) | map(attribute='value') | list | last != item.1.value)) or
         ((fmc_objects | selectattr('name', 'equalto', item.1.name) | map(attribute='description') | list | last is defined) and (fmc_objects | selectattr('name', 'equalto', item.1.name) | map(attribute='description') | list | last | default('Ansible Created') != item.1.description)) or
         ((fmc_objects | selectattr('name', 'equalto', item.1.name) | map(attribute='overridable') | list | last is defined) and (fmc_objects | selectattr('name', 'equalto', item.1.name) | map(attribute='overridable') | list | last | default('False') | bool != item.1.overridable)))
  changed_when: True

- name: "fmc_config: delete objects"
  local_action:
    module: uri
    url: "{{ item.1.links.self }}"
    method: DELETE
    validate_certs: no
    status_code: 200
    headers:
      X-auth-access-token: "{{ auth.x_auth_access_token }}"
  with_subelements:
    - "{{ all_objects_raw['results'] }}"
    - json.items
  when: (fmc_objects | selectattr('name', 'equalto', item.1.name) | list | count > 0)
        and(fmc_objects | selectattr('name', 'equalto', item.1.name) | map(attribute='name') | list | last is defined)
        and(fmc_objects | selectattr('name', 'equalto', item.1.name) | map(attribute='value') | list | last is undefined)
  changed_when: True

Sometimes when trying to munge an array and perform comparisons you have to do it in a Jinja2 Template. The following template creates a list of existing object names then will check to see if that object needs to be created. This is what my POST task uses to determine what new objects will be created.

templates/fmc_objects-missing.j2: ASCII text

{#- Build a list of the existing objects -#}
{% set EXISTING = [] %}
{% for object_result in all_objects_raw['results'] %}
{% for object_line in object_result['json']['items'] %}
{{- EXISTING.append( object_line['name'] ) -}}
{% endfor %}
{% endfor %}
 
{#- Check fmc_objects to see if missing -#}
{% for fmc_object in fmc_objects %}
{% if fmc_object['name'] not in EXISTING %}
{{ fmc_object['name'] }}
{% endif %}
{% endfor %}
19. December 2018 · Comments Off on Collect all sensor information from the FMC. · Categories: Cisco, Firewall, Linux Scripts, Networking, Uncategorized · Tags: , , , , , ,

Eventually I plan on refactoring all my firepower scripts into Ansible Playbooks. But in the meanwhile the following is a quick script that will collect all sensor information from a Firepower Management Center and save that information to a CSV file. The output is pretty handy for migrations and general data collection.

#!/bin/bash
## Collect all sensor devicerecords from a FMC.
## Requires: python:PyYAML,shyaml
## 2018 (v.01) - Script from www.davideaves.com
 
username="fmcusername"
password="fmcpassword"
 
FMC="192.0.2.13 192.0.2.14 192.0.2.15 192.0.2.16 192.0.2.17 192.0.2.18 192.0.2.21 192.0.2.22 192.0.2.23"
 
### Convert JSON to YAML.
j2y() {
 python -c 'import sys, yaml, json; yaml.safe_dump(json.load(sys.stdin), sys.stdout, default_flow_style=False)' 2> /dev/null
}
 
### Convert YAML to JSON.
y2j() {
 python -c 'import sys, yaml, json; y=yaml.load(sys.stdin.read()); print json.dumps(y)' 2> /dev/null
}
 
echo "FMC,healthStatus,hostName,model,name," > "$(basename ${0%.*}).csv"
 
# Itterate through all FMC devices
for firepower in ${FMC}
 do eval "$(curl -skX POST https://${firepower}/api/fmc_platform/v1/auth/generatetoken \
        -H "Authorization: Basic $(printf "${username}:${password}" | base64)" -D - |\
        awk '/(auth|DOMAIN|global)/{gsub(/[\r|:]/,""); gsub(/-/,"_",$1); print $1"=\""$2"\""}')"
 
    ### Get expanded of list devices
    curl -skX GET "https://${firepower}/api/fmc_config/v1/domain/${DOMAIN_UUID}/devices/devicerecords?offset=0&limit=1000&expanded=true" -H "X-auth-access-token: ${X_auth_access_token}" |\
     j2y | awk 'BEGIN{ X=0; }/^(-|  [a-z])/{if($1 == "-") {X+=1; printf "'''${firepower}''',"} else if($1 == "healthStatus:" || $1 == "hostName:" || $1 == "model:" || $1 == "name:") {printf $NF","} else if($1 == "type:") {printf "\n"}}'
 
done >> "$(basename ${0%.*}).csv"
04. December 2018 · Comments Off on Using Ansible to perform a Netscaler backup · Categories: Ansible, Load Balancing, NetScaler · Tags: , , , ,

The following Ansible playbook is a rewrite of a script from a long time ago to perform backups of a Netscaler. As far as I know, there are no native Ansible or Vendor modules to perform a system backup. Within the playbook I am simply performing a raw call using the URI module against the Nitro API and fetching the backup file.

The following Vendor links contain good/related reference information:

netscaler_systembackup.yaml: a /usr/bin/ansible-playbook -f 10 script text executable, ASCII text

#!/usr/bin/ansible-playbook -f 10
## Ansible playbook to perform a full backup of Netscaler systems
## 2018 (v.01) - Playbook from www.davideaves.com
---
- name: Netscaler full backup
  hosts: netscalers
  connection: local
  gather_facts: False

  vars:

    ansible_connection: "local"
    ansible_python_interpreter: "/usr/bin/env python"

    backup_location: "/srv/nsbackup"

    ns_sys_backup: "/var/ns_sys_backup"

  tasks:

    - name: Check backup file status
      local_action:
        module: stat
        path: "{{ backup_location }}/{{ inventory_hostname }}_{{ lookup('pipe', 'date +%Y%m%d') }}_nsbackup.tgz"
      register: stat_result

    - name: Check backup directory location
      local_action:
        module: file
        path: "{{ backup_location }}"
        state: directory
        mode: 0775
        recurse: yes
      run_once: True
      when: stat_result.stat.exists == False

    - name: Full backup of Netscaler configuration.
      block:

      - name: Create Netscaler system backup
        local_action:
          module: uri
          url: "https://{{ inventory_hostname }}/nitro/v1/config/systembackup?action=create"
          method: POST
          validate_certs: no
          return_content: yes
          headers:
            X-NITRO-USER: "{{ nitro_user | default('nsroot') }}"
            X-NITRO-PASS: "{{ nitro_pass | default('nsroot') }}"
          body_format: json
          body: 
            systembackup:
              filename: "{{ inventory_hostname | hash('md5') }}"
              level: full
              comment: Ansible Generated Backup

      - name: Fetch Netscaler system backup
        local_action:
          module: uri
          url: "https://{{ inventory_hostname }}/nitro/v1/config/systemfile?args=filename:{{ inventory_hostname | hash('md5') }}.tgz,filelocation:{{ ns_sys_backup | replace('/','%2F') }}"
          method: GET
          status_code: 200
          validate_certs: no
          return_content: yes
          headers:
            X-NITRO-USER: "{{ nitro_user | default('nsroot') }}"
            X-NITRO-PASS: "{{ nitro_pass | default('nsroot') }}"
        register: result

      - name: Save Netscaler system backup to backup directory
        local_action: "shell echo '{{ result.json.systemfile[0].filecontent }}' | base64 -d > '{{ backup_location }}/{{ inventory_hostname }}_{{ lookup('pipe', 'date +%Y%m%d') }}_nsbackup.tgz'"

      - name: Chmod saved backup file permissions
        local_action:
          module: file
          path: "{{ backup_location }}/{{ inventory_hostname }}_{{ lookup('pipe', 'date +%Y%m%d') }}_nsbackup.tgz"
          mode: 0644

      always:

      - name: Delete system backup from Netscaler
        local_action:
          module: uri
          url: "https://{{ inventory_hostname }}/nitro/v1/config/systembackup/{{ inventory_hostname | hash('md5') }}.tgz"
          method: DELETE
          validate_certs: no
          return_content: yes
          headers:
            X-NITRO-USER: "{{ nitro_user | default('nsroot') }}"
            X-NITRO-PASS: "{{ nitro_pass | default('nsroot') }}"

      - name: Locate backup files older than 90 days
        local_action:
          module: find
          paths: "{{ backup_location }}"
          age: "1d"
        run_once: true
        register: files_matched

      - name: Purge old backup files
        local_action:
          module: file
          path: "{{ item.path }}"
          state: absent
        run_once: true
        with_items: "{{ files_matched.files }}"

      when: stat_result.stat.exists == False
10. August 2018 · Comments Off on Cisco Firepower Management Center (FMC) bulk modifications of policy rules. · Categories: Cisco, Firewall, Linux, Linux Scripts, Networking · Tags: , , , , , , ,

As stated in my previous post, the Cisco Migration tools are very limited and Cisco has not shared any tools publicly that can perform bulk modifications. When migrating to a firepower unless all the previous access rules on the ASA had logging enabled any rule converted will continue to not have logging enabled. This is a major bummer for auditors, security teams or for anyone looking to eventually migrate L4 rules to L7 rules.

Just because I can, the following example script will bulk enable logBegin & sendEventsToFMC on all policy rules on a Firepower (FMC) appliance. This script is intended to be a reference, however modifying policy rules can broadly be broken down into the following steps:

  1. GET the accessrule
  2. Purge the metadata & link value trees from the json
  3. Change, add or remove whatever values you want
  4. PUT the accessrule
#!/bin/bash
## Example script that can bulk modify policy rules on a Cisco FMC.
## Requires: python:PyYAML,shyaml
## 2018 (v.01) - Script from www.davideaves.com
 
firepower="192.0.2.10"
username="apiuser"
password="api1234"
 
# Convert JSON to YAML
j2y() {
 python -c 'import sys, yaml, json; yaml.safe_dump(json.load(sys.stdin), sys.stdout, default_flow_style=False)' 2> /dev/null
}
 
# Convert YAML to JSON
y2j() {
 python -c 'import sys, yaml, json; y=yaml.load(sys.stdin.read()); print json.dumps(y)' 2> /dev/null
}
 
# Pop forbidden put values (metadata, links)
jpop() {
 python -c 'import sys, json; j=json.load(sys.stdin); j.pop("metadata"); j.pop("links"); print json.dumps(j)' 2> /dev/null
}
 
fmcauth() {
 ### Post credentials and eval header return data.
 
 if [[ -z "${auth_epoch}" || "${auth_epoch}" -lt "$(($(date +%s) - 1500))" ]]
  then
 
   if [ -z "${X_auth_access_toke}" ]
    then eval "auth_epoch=$(date +%s)"
        eval "$(curl -skX POST https://${firepower}/api/fmc_platform/v1/auth/generatetoken \
          -H "Authorization: Basic $(printf "${username}:${password}" | base64)" -D - |\
          awk '/(auth|DOMAIN|global)/{gsub(/[\r|:]/,""); gsub(/-/,"_",$1); print $1"=\""$2"\""}')"
    else eval "auth_epoch=$(date +%s)"
         eval "$(curl -skX POST https://${firepower}/api/fmc_platform/v1/auth/refreshtoken \
          -H "X-auth-access-token: ${X_auth_access_token}" \
          -H "X-auth-refresh-token: ${X_auth_refresh_token}" -D - |\
          awk '/(auth|DOMAIN|global)/{gsub(/[\r|:]/,""); gsub(/-/,"_",$1); print $1"=\""$2"\""}')"
   fi
 
 fi
}
 
# Get a list of all access control policies.
fmcauth && curl -skX GET https://${firepower}/api/fmc_config/v1/domain/${DOMAIN_UUID}/policy/accesspolicies \
 -H "X-auth-access-token: ${X_auth_access_token}" | j2y | shyaml get-value items | awk '/self:/{print $NF}' | while read POLICY
do fmcauth
 
   # Get a list of all access rules in each policy.
   curl -skX GET ${POLICY}/accessrules?limit=10000 -H "X-auth-access-token: ${X_auth_access_token}" |\
   j2y | shyaml get-value items 2> /dev/null | awk '/self:/{print $NF}' | while read RULE
   do
 
      # ID of the access rule.
      UUID="$(basename ${RULE})"
 
      # Collect the access rule resource and pop forbidden values.
      RESOURCE=`curl -skX GET ${RULE} -H "X-auth-access-token: ${X_auth_access_token}" | jpop`
 
      # Modify only if resource exists.
      if [[ ${RESOURCE} != *"Resource not found."* ]]
       then echo -ne "$(date) - PUT:${UUID} - MSG: "
 
       # Use sed to modify values in payload before putting back to FMC.
       curl -skX PUT ${RULE} \
        -H "Content-Type: application/json" \
        -H "X-auth-access-token: ${X_auth_access_token}" \
        -d "$(echo ${RESOURCE} | j2y | sed 's/logBegin: false/logBegin: true/;s/sendEventsToFMC: false/sendEventsToFMC: true/' | y2j)" && echo
      fi
 
   done
 
done