17. November 2018 · Comments Off on Convert ASA access-list rules to a parseable YAML format. · Categories: AWK, Cisco, Firewall, Linux Scripts, Networking · Tags: , , , , ,

This script spun out of a string of firewall migrations off the legacy ASA platform, I need the ability to convert access-lists to a parseable format. There are multiple reasons for needing this script. First is for human readability and auditing purposes. Second is to have a parseable rule base for duplication or migration to other firewall types.

ASA_acls.sh: Bourne-Again shell script text executable, ASCII text

#!/bin/bash
## Convert ASA access-list rules to a parseable YAML format.
## 2018 (v.01) - Script from www.davideaves.com
 
### VARIABLES ###
 
asa_config_file="${1}"
search_string="${2}"
 
### MAIN SCRIPT ###
 
[ -z "${asa_config_file}" ] && { echo -e "${0} - ERROR: missing ASA config"; exit 0; }
 
for ACCESSGROUP in `awk '/^access-group /{print $2}' "${asa_config_file}" | sort --ignore-case`
 do
 
  echo "${ACCESSGROUP}:"
  awk 'BEGIN{ REMARK=""; ACTION=""; SERVICE=""; SOURCE=""; DESTINATION=""; PORT=""; LOG=""; DISABLED=""; previous="" }
 
        # convert number to bits
        function bits(N){
          c = 0
          for(i=0; i<8; ++i) if(and(2**i, N)) ++c
          return c
        }
 
        # convert ipv4 to prefix
        function to_prefix(mask) {
          split(mask, octet, ".")
          return bits(octet[1]) + bits(octet[2]) + bits(octet[3]) + bits(octet[4])
        }
 
        # test if a string is an ipv4 address
        function is_v4(address) {
          split(address, octet, ".")
          if ( octet[1] <= 255 && octet[2] <= 255 && octet[3] <= 255 && octet[4] <= 255 )
          return address
        }
 
        # Only look at access-lists lines
        /^access-list '''${ACCESSGROUP}''' .*'''${search_string}'''/{
 
        # If line is a remark store it else continue
        if ( $3 == "remark" ) { $1=$2=$3=""; REMARK=substr($0,4) }
        else { $1=$2=$3=""; gsub("^   ", "")
 
          # Itterate through columns
          for(col = 1; col <= NF; col++) {
 
           # Append prefix to SOURCE & DESTINATION
           if ( is_v4(previous) && is_v4($col) ) {
            if ( DESTINATION != "" ) { DESTINATION=DESTINATION"/"to_prefix($col); previous="" }
            else if ( SOURCE != "" ) { SOURCE=SOURCE"/"to_prefix($col); previous="" }
          } else {
 
            # Determine col variable
            if ( col == "1" ) { ACTION=$col; SERVICE=""; SOURCE=""; DESTINATION=""; PORT=""; LOG=""; DISABLED=""; previous="" }
            else if ( $col ~ /^(eq|interface|object|object-group)$/ ) { previous=$col }
            else if ( SERVICE == "" && $col !~ /^(host|object|object-group)$/ ) { SERVICE=$col; PORT=""; previous="" }
            else if ( SOURCE == "" && $col !~ /^(host|object|object-group)$/ ) {
              if ( previous == "interface" ) { SOURCE=previous"/"$col }
              else { SOURCE=$col }; PORT=""; previous=to_prefix($col) }
            else if ( DESTINATION == "" && $col !~ /^(host|object|object-group)$/ ) {
              if ( previous == "interface" ) { DESTINATION=previous"/"$col }
              else { DESTINATION=$col }; PORT=""; previous=to_prefix($col) }
            else if ( previous ~ /^(eq|object-group)$/ ) { PORT=$col; previous="" }
            else if ( $col == "log" ) { LOG=$col; previous="" }
            else if ( $col == "inactive" ) { DISABLED=$col; previous="" }
            else { LAST=$col; previous="" }
 
          }
 
        }}
 
        # Display the output
        if ( DESTINATION != "" ) { count++
          print "  - name: '''${ACCESSGROUP}''' rule",count,"line",NR
          print "    debug:",$0
          if ( REMARK != "" ) { print "    description:",REMARK }
          print "    action:",ACTION
          print "    source:",SOURCE
          print "    destination:",DESTINATION
          if ( PORT == "" ) { print "    service:",SERVICE }
          else { print "    service:",SERVICE"/"PORT }
          if ( LOG != "" ) { print "    log: true" }
          if ( DISABLED != "" ) { print "    disabled: true" }
          REMARK=""; ACTION=""; SERVICE=""; SOURCE=""; DESTINATION=""; PORT=""; LOG=""; DISABLED=""; previous=""
        }
 
  }' "${asa_config_file}"
 
done
08. August 2018 · Comments Off on Cisco Firepower Management Center (FMC) bulk import & delete objects · Categories: Cisco, Firewall, Linux, Linux Scripts, Networking · Tags: , , , , , ,

I’ve been doing a lot of migration work with the Cisco Firepower. I quickly noticed a general lack of tools to assist with migrations short of a few scattered and limited (hopefully incomplete and will continue to be developed) migration tools provided by Cisco:

If you are looking for tools to perform bulk rule changes or help convert from Layer4 rules to Layer7, like the PaloAlto Migration tool, you are out of luck. That being said, as an engineer trying to use the FMC, I quickly found the experience of working within the firepower interface slow, tedious and generally painful. This has forced me to try to interface with the API’s as much as possible to save time and to avoid using the interface.

First thing; API access needs to be enabled on the FMC; by default they are, but if disabled you can enable them by going to System > REST API Preferences and enabling them. If API’s are enabled the documentation will be accessible via: https://<FMCHOST>/api/api-explorer

API’s on the FMC take an imperative approach and are simple to trigger and depending on the HTTP method you can command the FMC to make changes. There are 4 basic methods the FMC will accept:

  • GET – Retrieves data from the specified object. GET is a read-only operation.
  • PUT – Adds supplied information to the specified object; returns a 404 Resource Not Found error if the object does not exist.
  • POST – Creates the object with the supplied information. POST operations are be followed with a payload consisting of JSON.
  • DELETE – Delete and object.

Concerning the types of objects that can be modified; the following are currently supported:

  • icmpv4objects
  • icmpv6objects
  • interfacegroups
  • networkgroups
  • networks
  • portobjectgroups
  • protocolportobjects
  • ranges
  • securityzones
  • slamonitors
  • urlgroups
  • urls
  • vlangrouptags
  • vlantags

As far as I can tell vpn policies, flexconfig and other objects of the sort must be created by hand. The following json & script are very rudimentary, however it is a working example that uses cURL to perform a bulk import of objects into a firepower. At the very least the curl commands can be used as a reference in your own projects.

fmc-objects.json: ASCII text, with very long lines

[{"value":"10.0.0.0","overridable":false,"name":"10.0.0.0_32","type":"Hosts","description":"Test REST API Object"},{"value":"10.0.0.1","overridable":false,"name":"10.0.0.1_32","type":"Hosts","description":"Test REST API Object"},{"value":"10.0.0.2","overridable":false,"name":"10.0.0.2_32","type":"Hosts","description":"Test REST API Object"},{"value":"10.0.0.3","overridable":false,"name":"10.0.0.3_32","type":"Hosts","description":"Test REST API Object"},{"value":"10.0.0.4","overridable":false,"name":"10.0.0.4_32","type":"Hosts","description":"Test REST API Object"},{"value":"10.0.0.5","overridable":false,"name":"10.0.0.5_32","type":"Hosts","description":"Test REST API Object"},{"value":"10.0.0.6","overridable":false,"name":"10.0.0.6_32","type":"Hosts","description":"Test REST API Object"},{"value":"10.0.0.7","overridable":false,"name":"10.0.0.7_32","type":"Hosts","description":"Test REST API Object"},{"value":"10.0.0.8/29","overridable":false,"name":"10.0.0.8_29","type":"Networks","description":"Test REST API Object"}]

Because I do a lot with Ansible I have a strong preference to work with the YAML data serialization format. The script includes 2 unused functions that converts JSON to YAML and vice versa. I left those functions in this example script because they are handy and provide me flexibility of what and how I import. When posting to the FMC, its important to make sure the data is in JSON format and any metadata & link value trees have been popped. One thing I am lacking is a function to verify the object has been created, its easy to do by simply performing a GET after the POST, it just makes things slower and is simply not in this example.

fmc-objects.sh: Bourne-Again shell script, ASCII text executable

#!/bin/bash
## Example script that can bulk import and delete objects from a Cisco FMC.
## Requires: python:PyYAML,shyaml
## 2018 (v.01) - Script from www.davideaves.com
 
firepower="192.0.2.10"
username="apiuser"
password="api1234"
 
# Convert JSON to YAML
j2y() {
 python -c 'import sys, yaml, json; yaml.safe_dump(json.load(sys.stdin), sys.stdout, default_flow_style=False)' 2> /dev/null
}
 
# Convert YAML to JSON
y2j() {
 python -c 'import sys, yaml, json; y=yaml.load(sys.stdin.read()); print json.dumps(y)' 2> /dev/null
}
 
fmcauth() {
 ### Post credentials and eval header return data.
 
 if [[ -z "${auth_epoch}" || "${auth_epoch}" -lt "$(($(date +%s) - 1500))" ]]
  then
 
   if [ -z "${X_auth_access_toke}" ]
    then eval "auth_epoch=$(date +%s)"
        eval "$(curl -skX POST https://${firepower}/api/fmc_platform/v1/auth/generatetoken \
          -H "Authorization: Basic $(printf "${username}:${password}" | base64)" -D - |\
          awk '/(auth|DOMAIN|global)/{gsub(/[\r|:]/,""); gsub(/-/,"_",$1); print $1"=\""$2"\""}')"
    else eval "auth_epoch=$(date +%s)"
         eval "$(curl -skX POST https://${firepower}/api/fmc_platform/v1/auth/refreshtoken \
          -H "X-auth-access-token: ${X_auth_access_token}" \
          -H "X-auth-refresh-token: ${X_auth_refresh_token}" -D - |\
          awk '/(auth|DOMAIN|global)/{gsub(/[\r|:]/,""); gsub(/-/,"_",$1); print $1"=\""$2"\""}')"
   fi
 
 fi
}
 
 
### Bulk add diffrent type objects into FMC
 
# Cat single line example containing diffrent objects & use awk to extract json stanza.
cat "fmc-objects.json" | awk -F'[}{]' '{ for (i=1; i< =NF; i++) if(length($i) > 5) print "{"$i"}" }' |\
while read LINE
 do fmcauth
 
   # Determine Type & Value.
   TYPE="$(echo ${LINE} | shyaml get-value type | awk '{print tolower($0)}')"
   VALUE="$(echo ${LINE} | shyaml get-value value | awk '{print tolower($0)}')"
 
   echo -ne "$(date) - POST:${TYPE}>${VALUE} - MSG: "
 
   # Post new object to FMC.
   curl -skX POST https://${firepower}/api/fmc_config/v1/domain/${DOMAIN_UUID}/object/${TYPE} \
     -H "Content-Type: application/json" \
     -H "X-auth-access-token: ${X_auth_access_token}" \
     -d "${LINE}" && echo
done
 
### Bulk delete all unused objects from FMC...
 
# Valid objects types are: hosts icmpv4objects icmpv6objects interfacegroups networkgroups networks portobjectgroups protocolportobjects ranges securityzones slamonitors urlgroups urls vlangrouptags vlantags
for TYPE in hosts networks
 do fmcauth
    echo "### DELETEING ${TYPE} OBJECTS ###"
 
    # Get a list of objects.
    curl -skX GET https://${firepower}/api/fmc_config/v1/domain/${DOMAIN_UUID}/object/${TYPE}?limit=10000 \
     -H "X-auth-access-token: ${X_auth_access_token}" | j2y | shyaml get-value items 2> /dev/null | awk '/self:/{print $NF}' | while read SELF
    do  fmcauth
        echo -ne "$(date) - DELETE:${TYPE} - MSG: "
 
        # Delete the object.
        curl -skX DELETE ${SELF} -H "X-auth-access-token: ${X_auth_access_token}" && echo
    done
done
01. March 2018 · Comments Off on PHP YAML data collector to build Vultr servers · Categories: Ansible, Cloud, Linux, PHP, Vultr · Tags: , , , , , ,

A few days ago I posted an Ansible playbook to provision & deprovision virtual servers on VULTR. One of the key features of that playbook would read a separate YAML file for its list of servers to build. The following is a simple yaml data collector written in PHP to generate the YAML for an Ansible playbook to consume.

VR Server Request

Long story short the heart of the script is the yaml_emit_file function that I use to dump POST data directly into a YAML file. As stated in my previous posting, anything could generate that file, be it the user themselves or a ticketing system.

yaml-data-collector.php: PHP script, ASCII text

< ?php
/*
   # yaml-data-collector
   # Simple php form to collect information from a user on what servers to build.
   2017 (v1.0) - Script from www.davideaves.com
*/
 
if (basename($_SERVER["PHP_SELF"]) == basename(__FILE__) && isset($_GET["source"])) {
  header("Content-type: text/plain");
  exit(file_get_contents(basename($_SERVER["PHP_SELF"])));
}
 
// Uncomment to view debug points in the code.
//$debug = "yes";
 
if ($_SERVER['REQUEST_METHOD'] == 'POST') {
  // Update $yamlfile
  yaml_emit_file("$_POST[tag].yaml", $_POST);
}
 
if(isset($_REQUEST['DATA'])):
  if (file_exists($_REQUEST['DATA'])) {
    $yamldata = yaml_parse_file($_REQUEST['DATA']);
    // Reindex the SERVERS array numerically incase broken.
    $yamldata['vultr']['servers'] = array_values($yamldata['vultr']['servers']);
  }
endif
?>
< !DOCTYPE html>
<html>
  <head>
     <title>< ?php echo gethostname(); ?>: VULTR Server Req.</title>
     <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
     <meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate"/>
     <meta http-equiv="Pragma" content="no-cache"/>
     <script>
     /**
      * Projet: Dynamic dataTable.
      * Add/Delet rows from an html table.
      */
 
     var maxRows = 5;
 
     function addRow(tableID) {
          var table = document.getElementById(tableID);
          var rowCount = table.rows.length;
          if(rowCount < maxRows) {   // limit the user from creating fields more than your limits
               var row = table.insertRow(rowCount);
               var colCount = table.rows[0].cells.length;
               for(var i=0; i<colCount; i++) {
                    var newcell = row.insertCell(i);
                    newcell.innerHTML = table.rows[0].cells[i].innerHTML.replace(RegExp('\[[0-9]+\]'), '\[' + rowCount + '\]');
               }
          } else {
               alert("Maximum rows reached.");
          }
     }
     function deleteRow(tableID) {
          var table = document.getElementById(tableID);
          var rowCount = table.rows.length;
          for(var i=0; i<rowCount; i++) {
               var row = table.rows[i];
               var chkbox = row.cells[0].childNodes[0];
               if(null != chkbox && true == chkbox.checked) {
                    if(rowCount <= 1) {   // limit the user from removing all the fields
                         alert("Cannot remove all rows.");
                         break;
                    }
                    table.deleteRow(i);
                    rowCount--;
                    i--;
               }
          }
     }
     </script>
     <style>
     body {background-color: grey;}
     form {margin-top: 15px; margin-left: 15px; margin-right: 15px;}
     form a {color: blue; text-decoration: none; margin: 0px;}
     form label {display:inline-block; width:48%;}
     form input[type="text"] {width:48%;}
     form input[type="submit"] {width:150px;}
     form select {width:48%;}
     form .line {clear:both;}
     #DATANAV {background-color: #4ff; opacity: 0.5; border: solid 1px #999; border-radius: 5px; font-family: "Trebuchet MS", Arial, Helvetica, sans-serif; text-decoration: none; text-align: center; position: absolute; top: 15px; right: 15px; padding: 0 15px 0 15px; align: center;}
     #DATANAV a {text-decoration: none; font-size: 15px; color: red;}
     #DATANAV a:hover {font-weight: bold; color: maroon;}
     #DATANAV label {text-decoration: underline; color: black;}
     #TEMPLATE {width:1000px; margin:20px auto; background: #EEE; border: solid 1px #999; position: float; vertical-align: middle; border-radius: 5px;}
     #TEMPLATE fieldset {border-radius: 5px; border-width:2px; border-style:dotted; margin:5px;}
     #TEMPLATE h2 {margin:0; font-family:Arial, Sans-Serif; background:#5f9be3; color:#fff; white-space: nowrap; font-weight:bold; padding:0px; text-align: center;}
     #TEMPLATE span.config {background-color:black; color:white; font-family:monospace; white-space:pre; display:block;}
     #TEMPLATE mark {background-color:black; color:yellow; font-style:italic;}
     #TEMPLATE form {margin-top: 15px; margin-left: 15px; margin-right: 15px;}
     #TEMPLATE form label {display:inline-block; width:48%;}
     #TEMPLATE form a {color: blue; text-decoration: none; margin: 0px;}
     #TEMPLATE form input[type="text"] {width:50%;}
     #TEMPLATE form input[type="submit"] {width:100%;}
     #TEMPLATE form select {width:50.6%;}
     #TEMPLATE form .line {clear:both;}
     </style>
  </script></head>
<body>< ?php if (isset($debug)) {
  echo "<span class=\"debug\">";
  print_r(yaml_emit($_POST));
  echo "<hr /><br />\n";
} ?>
 
<div id="DATANAV">
<p><a class="button" href="?">New Request</a></p>
<p><label>Previous Tags</label>
<br />< ?php // SELECT THE TEMPLATE
  $templates = glob("*.yaml");
  foreach ($templates as $file) {
    echo "<a href='" . $_SERVER['PHP_SELF'] . "?DATA=" . $file . "'>" . preg_replace('/\\.[^.\\s]{3,4}$/', '', $file). "<br />\n";
  }
?></p></div>
 
 
<div id="TEMPLATE">
<form action="<?php echo $_SERVER['PHP_SELF']; ?>" method="POST" onsubmit="return confirm('Are you sure you want to submit this request?');">
<h2>Vultr Server Request</h2>
<fieldset class="row1">
 <legend>Request Details</legend>
 
 <p>
  <div class="line"><label>Tag</label>
   <input name="tag" type="text" required="required" value="<?php if(isset($yamldata['tag'])) { echo $yamldata['tag']; } ?/>" />
  </div>
  <div class="line"><label>Firewall Group</label>
   <select name="firewall_group">
    < ?php if(!isset($yamldata['firewall_group'])) { echo "<option value=\"\" disabled selected>Select your option"; } ?>
    <option value="shellserver" <?php if(isset($yamldata['firewall_group']) && $yamldata['firewall_group'] == "shellserver") { echo selected; } ?>>shellserver</option>
    <option value="webserver" <?php if(isset($yamldata['firewall_group']) && $yamldata['firewall_group'] == "webserver") { echo selected; } ?>>webserver</option>
   </select>
  </div>
 </p>
 
</fieldset>
 
<input type="hidden" name="modified" value="<?php echo date(DATE_RFC2822);?/>">
 
<fieldset class="row2">
 <legend>Server Details</legend>
 
  <p>
   <input type="button" value="Add Server" onClick="addRow('dataTable')" />
   <input type="button" value="Remove Server" onClick="deleteRow('dataTable')" />
  </p>
 
  <table id="dataTable" class="form">
   <tbody>< ?php if(isset($yamldata['vultr']['servers'])) { foreach($yamldata['vultr']['servers'] as $COUNT=>$SERVER){ ?><tr>
    <td><input type="checkbox" /></td>
    <td>
     <label>Domain Name</label>
     <input type="text" required="required" name="vultr[servers][<?php echo $COUNT; ?/>][name]" value="< ?php echo $SERVER['name']; ?>" />
    </td>
    <td>
     <label>Server OS</label>
     <select name="vultr[servers][<?php echo $COUNT; ?>][os]" required="required">
      < ?php if(!isset($yamldata['vultr']['servers'])) { echo "<option value=\"\" disabled selected>Select your option"; } ?>
      <option value="CentOS 7 x64" <?php if(isset($yamldata[vultr][servers][$COUNT][os]) && $yamldata[vultr][servers][$COUNT][os] == "CentOS 7 x64") { echo selected; } ?>>CentOS 7</option>
      <option value="Debian 9 x64 (stretch)" <?php if(isset($yamldata[vultr][servers][$COUNT][os]) && $yamldata[vultr][servers][$COUNT][os] == "Debian 9 x64 (stretch)") { echo selected; } ?>>Debian 9</option>
      <option value="Fedora 27 x64" <?php if(isset($yamldata[vultr][servers][$COUNT][os]) && $yamldata[vultr][servers][$COUNT][os] == "Fedora 27 x64") { echo selected; } ?>>Fedora 27</option>
      <option value="Ubuntu 17.10 x64" <?php if(isset($yamldata[vultr][servers][$COUNT][os]) && $yamldata[vultr][servers][$COUNT][os] == "Ubuntu 17.10 x64") { echo selected; } ?>>Ubuntu 17.10</option>
     </select>
    </td>
    <td>
     <label>Plan</label>
     <select name="vultr[servers][<?php echo $COUNT; ?>][plan]" required="required">
      < ?php if(!isset($yamldata['vultr']['servers'])) { echo "<option value=\"\" disabled selected>Select your option"; } ?>
      <option value="1024 MB RAM,25 GB SSD,1.00 TB BW" <?php if(isset($yamldata['vultr']['servers'][$COUNT]['plan']) && $yamldata['vultr']['servers'][$COUNT]['plan'] == "1024 MB RAM,25 GB SSD,1.00 TB BW") { echo selected; } ?>>$5/month</option>
      <option value="2048 MB RAM,40 GB SSD,2.00 TB BW" <?php if(isset($yamldata['vultr']['servers'][$COUNT]['plan']) && $yamldata['vultr']['servers'][$COUNT]['plan'] == "2048 MB RAM,40 GB SSD,2.00 TB BW") { echo selected; } ?>>$10/month</option>
      <option value="4096 MB RAM,60 GB SSD,3.00 TB BW" <?php if(isset($yamldata['vultr']['servers'][$COUNT]['plan']) && $yamldata['vultr']['servers'][$COUNT]['plan'] == "4096 MB RAM,60 GB SSD,3.00 TB BW") { echo selected; } ?>>$20/month</option>
      <option value="8192 MB RAM,100 GB SSD,4.00 TB BW" <?php if(isset($yamldata['vultr']['servers'][$COUNT]['plan']) && $yamldata['vultr']['servers'][$COUNT]['plan'] == "8192 MB RAM,100 GB SSD,4.00 TB BW") { echo selected; } ?>>$40/month</option>
      <option value="16384 MB RAM,200 GB SSD,5.00 TB BW" <?php if(isset($yamldata['vultr']['servers'][$COUNT]['plan']) && $yamldata['vultr']['servers'][$COUNT]['plan'] == "16384 MB RAM,200 GB SSD,5.00 TB BW") { echo selected; } ?>>$80/month</option>
      <option value="32768 MB RAM,300 GB SSD,6.00 TB BW" <?php if(isset($yamldata['vultr']['servers'][$COUNT]['plan']) && $yamldata['vultr']['servers'][$COUNT]['plan'] == "32768 MB RAM,300 GB SSD,6.00 TB BW") { echo selected; } ?>>$160/month</option>
     </select>
    </td>
    <td>
     <label>Region</label>
     <select name="vultr[servers][<?php echo $COUNT; ?>][region]" required="required">
      < ?php if(!isset($yamldata['vultr']['servers'])) { echo "<option value=\"\" disabled selected>Select your option"; } ?>
      <option value="Atlanta" <?php if(isset($yamldata['vultr']['servers'][$COUNT]['region']) && $yamldata['vultr']['servers'][$COUNT]['region'] == "Atlanta") { echo selected; } ?>>Atlanta</option>
      <option value="Chicago" <?php if(isset($yamldata['vultr']['servers'][$COUNT]['region']) && $yamldata['vultr']['servers'][$COUNT]['region'] == "Chicago") { echo selected; } ?>>Chicago</option>
      <option value="Dallas" <?php if(isset($yamldata['vultr']['servers'][$COUNT]['region']) && $yamldata['vultr']['servers'][$COUNT]['region'] == "Dallas") { echo selected; } ?>>Dallas</option>
      <option value="Los Angeles" <?php if(isset($yamldata['vultr']['servers'][$COUNT]['region']) && $yamldata['vultr']['servers'][$COUNT]['region'] == "Los Angeles") { echo selected; } ?>>Los Angeles</option>
      <option value="Miami" <?php if(isset($yamldata['vultr']['servers'][$COUNT]['region']) && $yamldata['vultr']['servers'][$COUNT]['region'] == "Miami") { echo selected; } ?>>Miami</option>
      <option value="New Jersey" <?php if(isset($yamldata['vultr']['servers'][$COUNT]['region']) && $yamldata['vultr']['servers'][$COUNT]['region'] == "New Jersey") { echo selected; } ?>>New Jersey</option>
      <option value="Seattle" <?php if(isset($yamldata['vultr']['servers'][$COUNT]['region']) && $yamldata['vultr']['servers'][$COUNT]['region'] == "Seattle") { echo selected; } ?>>Seattle</option>
      <option value="Silicon Valley" <?php if(isset($yamldata['vultr']['servers'][$COUNT]['region']) && $yamldata['vultr']['servers'][$COUNT]['region'] == "Silicon Valley") { echo selected; } ?>>Silicon Valley</option>
     </select>
    </td>
   </tr>< ?php } } else { ?><tr>
    <td><input type="checkbox" /></td>
    <td>
     <label>Domain Name</label>
     <input type="text" required="required" name="vultr[servers][0][name]" />
    </td>
    <td>
     <label>Server OS</label>
     <select name="vultr[servers][0][os]" required="required">
      < ?php if(!isset($yamldata['vultr[servers][0][os]'])) { echo "<option value=\"\" disabled selected>Select your option"; } ?>
      <option value="CentOS 7 x64">CentOS 7</option>
      <option value="Debian 9 x64 (stretch)">Debian 9</option>
      <option value="Fedora 27 x64">Fedora 27</option>
      <option value="Ubuntu 17.10 x64">Ubuntu 17.10</option>
     </select>
    </td>
    <td>
     <label>Plan</label>
     <select name="vultr[servers][0][plan]" required="required">
      <option value="" disabled selected>Select your option</option>
      <option value="1024 MB RAM,25 GB SSD,1.00 TB BW">$5/Month</option>
      <option value="2048 MB RAM,40 GB SSD,2.00 TB BW">$10/Month</option>
      <option value="4096 MB RAM,60 GB SSD,3.00 TB BW">$20/Month</option>
      <option value="8192 MB RAM,100 GB SSD,4.00 TB BW">$40/Month</option>
      <option value="16384 MB RAM,200 GB SSD,5.00 TB BW">$80/Month</option>
      <option value="32768 MB RAM,300 GB SSD,6.00 TB BW">$160/Month</option>
     </select>
    </td>
    <td>
     <label>Region</label>
     <select name="vultr[servers][0][region]" required="required">
      <option value="" disabled selected>Select your option</option>
      <option value="Atlanta">Atlanta</option>
      <option value="New Jersey">New Jersey</option>
      <option value="Chicago">Chicago</option>
      <option value="Dallas">Dallas</option>
      <option value="Los Angeles">Los Angeles</option>
      <option value="Miami">Miami</option>
      <option value="New Jersey">New Jersey</option>
      <option value="Seattle">Seattle</option>
      <option value="Silicon Valley">Silicon Valley</option>
     </select>
    </td>
   </tr>< ?php } ?></tbody>
  </table>
 
</fieldset>
<br />
<input class="submit" type="submit" value="Save Request &raquo;" />
</form><br />
</div>
</body>
</html>
09. November 2017 · Comments Off on YAML Templating Engine · Categories: Cisco, Firewall, Linux, Linux Scripts, Load Balancing, Networking · Tags: , , , ,

About a month ago I left my previous position and took a new engineering position. In my new role I am more focused on devops/orchestration for a new a network team. One major gap that I commonly see missing from most infrastructure/ops teams is the sharing of basic templates; ideally using a templating engine to maintain consistency. A couple years ago I stumbled upon a templating-engine by mxtommy on github that would read XML data and spit out a config. Aside from being a little rough, you would have to create your templates in a rough XML format plus it no longer seems to work with the latest version of PHP.

I decided that it was time to rewrite it; this time making it so it would read a YAML data input structure. YAML is a human-readable data serialization standard and is commonly used in orchestration tools like Ansible. Just like before this web script searches the current directory for template files (*.yaml). It then parses the questions to ask and creates the appropriate html input fields. The user answers the questions and submits them back to the script. The script then replaces the user responses with the placeholders to create a constant and repeatable configuration… For example, last year I posted a config snippet for configuring a Cisco UCS-E module on a 4000 series router. A template created for that config would make for constant and rapid deployments of UCS-E blades by infrastructure staff. Additionally you could use it to create a bootstrap config to quickly bring a host online so Ansible can take over management of it.

This web script requires the YAML-1.1 parser and emitter for PHP…

  • Debian: apt-get install php-yaml
  • RHEL: yum install php-pecl-yaml

template.php: Template Engine / PHP script.

< ?php
/*
   # yaml-templating-engine
 
   This web script searches the current directory for .yaml files. It will then ask
   questions based on questions defined in the template file, and then insert the info
   at the appropriate places in the template.
 
   Concept based on: https://github.com/mxtommy/templating-engine
 
   Rewritten to consume YAML and works with PHP7+.
 
   *** This is script not intended for use on a public server. ***
 
   2017 (v1.0) - Script from www.davideaves.com
*/
 
if (basename($_SERVER["PHP_SELF"]) == basename(__FILE__) && isset($_GET["source"])) {
  header("Content-type: text/plain");
  exit(file_get_contents(basename($_SERVER["PHP_SELF"])));
}
 
?>
< !DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8"/>
    <title>< ?php echo gethostname(); ?>: YAML Template Generator</title>
    <link href="template.css" rel="stylesheet"/>
  </head>
<body>
 
<table align="center"><tr>
	<td><h1>YAML Templating Engine</h1></td>
</tr></table>
 
<div id="TEMPLATE">
<fieldset>< ?php
 
// SELECT THE TEMPLATE
if (!isset($_REQUEST['template'])) {
  echo "<legend>Template Select\n";
  echo "<p>Please select one of the following templates:</p>\n<ul class=\"twocols\">\n";
  $templates = glob("*.yaml");
  foreach ($templates as $file) {
	print ("\t<li><a href='".$_SERVER['PHP_SELF']."?template=".$file."'>$file</a></li>\n");
  }
  echo "</ul>";
 
}
 
// ASK TEMPLATE QUESTIONS
elseif (!isset($_REQUEST['submitted'])) {
  echo "<legend>Template Input</legend>";
  echo "<h2>" . $_REQUEST['template'] . "</h2>\n";
 
  $yaml_template = yaml_parse_file($_REQUEST['template']);
  //print_r($yaml_template);
 
  echo "<form action=\"" . $_SERVER['PHP_SELF'] . "\" method=\"post\">\n";
  echo "<input type=\"hidden\" name=\"template\" value=\"" . $_REQUEST['template'] . "\"/>\n";	
 
  foreach($yaml_template['questions'] as $question) {
 
    // type is text
    if ($question['type'] == 'text') {
      echo "<div class=\"line\"><label>" . $question['description'] . ":</label><input type=\"text\" name=\"" . $question['name'] . "\" value=\"" . $question['default'] . "\"/></div>\n";
    }
 
    // type is select
    elseif($question['type'] == 'select') {
      echo "<div class=\"line\"><label>" . $question['description'] . ":</label><select name=\"" . $question['name'] . "\">\n";
      foreach ($question['option'] as $option) {
        echo "\t<option value=\"" . $option['name'] . "\"";
          if ($question['default'] == $option['name']) { echo " selected"; }
          echo ">" . $option['description'] . "</option>\n";
      }
      echo "</select></div>\n";
    }
 
    // type is checkbox
    elseif($question['type'] == 'checkbox') {
      echo "<div class=\"line\"><label>" . $question['description'] . ":</label><input type=\"checkbox\" name=\"" . $question['name'] . "\"";
      if ( isset($question['default']) ) { echo " checked"; }
      echo "/></div>\n";
    }
  }
  echo "<br /><input type=\"submit\" name=\"submitted\"/>\n</form>\n";
} 
 
// DISPLAY THE TEMPLATE
else {
  echo "<legend>Template Output</legend>";
  echo "<h2>" . $_REQUEST['template'] . "</h2>\n";
 
  $vars[] = "";
  $yaml_template = yaml_parse_file($_POST['template']);
 
  foreach ($yaml_template['questions'] as $i => $row) {
 
      $TMP = $yaml_template['questions'][$i];
 
      // Input type is text.
      if($TMP['type'] == "text") {
	$vars = $vars + array( "{{ ".$TMP['name']." }}" => "<mark>" . $_POST[$TMP['name']] . "</mark>" );
      }
 
      // Input type is select.
      elseif ($TMP['type'] == "select" ) {
        foreach ($TMP['option'] as $i => $row) {
          if($row['name'] == $_POST[$TMP['name']]) {
            foreach ($TMP['option'][$i]['option'] as $i => $row) {
              $vars = $vars + array( "{{ ".$row['name']." }}" => "<mark>" . $row['text'] . "</mark>" );
      } } } }
 
      // Input type is a checkbox.
      elseif ($TMP['type'] == "checkbox" ) {
        if(isset($_POST[$TMP['name']])) {
          $vars = $vars + array( "{{ ".$TMP['name']." }}" => "<mark>" . $TMP['checked'] . "</mark>" );
        } elseif (isset($TMP['unchecked'])) {
          $vars = $vars + array( "{{ ".$TMP['name']." }}" => "<mark>" . $TMP['unchecked'] . "</mark>" );
        } else {
          $vars = $vars + array( "{{ ".$TMP['name']." }}" => "" );
      } }
  }
 
  array_shift($vars);
  echo "<span class=\"config\">";
  echo strtr($yaml_template['config'], $vars);
  echo "</span>";
 
}
 
?></fieldset>
</div>
 
</body>
</html>

template.css: Example CSS file for the template generator.

/* TEMPLATE CSS */
#TEMPLATE {width:800px; margin:20px auto; background: #EEE; border: solid 1px #999; position: float; vertical-align: middle; border-radius: 5px;}
#TEMPLATE fieldset {border-radius: 5px; border-width:2px; border-style:dotted; margin:5px;}
#TEMPLATE h2 {margin:0; font-family:Arial, Sans-Serif; background:#5f9be3; color:#fff; white-space: nowrap; font-weight:bold; padding:0px; text-align: center;}
#TEMPLATE span.config {background-color:black; color:white; font-family:monospace; white-space:pre; display:block;}
#TEMPLATE mark {background-color:black; color:yellow; font-style:italic;}
#TEMPLATE form {margin-top: 15px; margin-left: 15px; margin-right: 15px;}
#TEMPLATE form label {display:inline-block; width:48%;}
#TEMPLATE form a {color: blue; text-decoration: none; margin: 0px;}
#TEMPLATE form input[type="text"] {width:50%;}
#TEMPLATE form input[type="submit"] {width:100%;}
#TEMPLATE form select {width:50.6%;}
#TEMPLATE form .line {clear:both;}
 
/* twocols layout */
.twocols {
  -webkit-column-count: 2; /* Chrome, Safari, Opera */
  -moz-column-count: 2; /* Firefox */
  column-count: 2;
}

Overall the templates are very easy to read and create making it so even non-technical support staff can contribute to the template repository this web script reads from.

example.yaml: Example yaml template file.

---
questions:

  - name: "hostname"
    description: "Hostname"
    type: "text"
    default: ROUTER

  - name: "mgmt_net"
    description: "Management Network"
    type: "select"
    default: dev
    option:
    - name: corp
      description: "Corporate"
      option:
      - name: VLAN
        text: 1000
      - name: VLAN-NAME
        text: PROD_VLAN
      - name: GATEWAY
        text: 192.168.50.1
    - name: dev
      description: "Development"
      option:
      - name: VLAN
        text: 2000
      - name: VLAN-NAME
        text: DEV_VLAN
      - name: GATEWAY
        text: 192.168.100.1

  - name: "service_encrypt"
    description: "Password Encryption"
    type: "checkbox"
    default: checked
    checked: service password-encryption
    unchecked: no service password-encryption

config: |
  {{ service_encrypt }}
  !
  h0stname {{ hostname }}
  !
  vlan {{ VLAN }}
   name {{ VLAN-NAME }}
  !
  ip default gateway {{ GATEWAY }}