04. February 2018 · Comments Off on Collect and archive all runtime information, statistics and status on F5 systems · Categories: F5, Linux, Linux Scripts, Load Balancing, Networking · Tags: , , , , , , , , , , ,

Last march I posted a TCL/Expect script (rtrinfo.exp) to backup configs and regularly collect runtime information via show commands on Cisco devices for archival purposes. It’s proven itself to be very useful and replaces the need to purchase convoluted commercial software to archive device configs. Not only do I use it, but I know of a few companies that have adopted it as well. Recently I needed something similar that could collect and archive all runtime information, statistics and status on F5 systems; the following is that script.

The script works by pushing a small, base64 encoded, command string to the F5 to be executed. The command string simply does a “tmsh -q show \?” to get a list of all show commands based on the enabled modules. The available runtime information is collected and piped to a for loop that runs all available show commands.

echo "Zm9yIE1PRCBpbiBgdG1zaCAtcSBzaG93IFw/IHwgc2VkIC1uIC1lICcvTW9kdWxlczovLC9PcHRpb25zOi9wJyB8IGF3ayAnL14gIC97cHJpbnQgJDF9J2A7IGRvIHRtc2ggLXEgc2hvdyAkTU9EIDI+IC9kZXYvbnVsbDsgZG9uZQ==" | base64 -d
for MOD in `tmsh -q show \? | sed -n -e '/Modules:/,/Options:/p' | awk '/^  /{print $1}'`; do tmsh -q show $MOD 2> /dev/null; done

All the output from the F5 is collected and some awk-foo is used to determine appropriate output destination on a per-line basis. The while loop appends the line to the appropriate file. Additionally all previous output archived to an appropriately named tar.gz file. I have also added the ability to silence the output, specify an output path override and to use root’s ssh private key instead of using the password (for running via cron).

f5info.sh: Bourne-Again shell script, ASCII text executable

#!/bin/bash
## Collect and archive all runtime information, statistics and status on F5 systems.
## 2018 (v1.0) - Script from www.davideaves.com
 
OUTDIR="."
 
### Script Functions ###
function USAGE () {
 # Display the script arguments.
 printf "Usage: $0 -d bigip -i id_rsa -p path\n\n"
 printf "Requires:\n"
 printf "\t-d: Target F5 system.\n"
 printf "Options:\n"
 printf "\t-i: Private id_rsa of root user.\n"
 printf "\t-p: Destination of output directory.\n"
 printf "\t-q: Quiet, do not show anything.\n"
}
 
function CLEANUP {
 # Cleanup after the script finishes.
 [ -e "${IDENTITY}" ] && { rm -rf "${IDENTITY}"; }
}
 
### Get CLI options ###
while getopts "d:i:p:q" ARG; do
 case "${ARG}" in
  d) F5="${OPTARG^^}";;
  i) trap CLEANUP EXIT
     IDENTITY="$(mktemp)"
     chmod 600 "${IDENTITY}" && cat "${OPTARG}" > "${IDENTITY}";;
  p) OUTDIR="${OPTARG}";;
  q) QUIET="YES";;
 esac
done 2> /dev/null
 
### Display USAGE if F5 not defined ###
[ -z "${F5}" ] && { USAGE && exit 1; }
 
### Archive & Create OUTDIR ###
if [ -d "${OUTDIR}/${F5}" ]
 then ARCHIVE="${OUTDIR}/${F5}_$(date +%Y%m%d -d @$(stat -c %Y "${OUTDIR}/${F5}")).tar.gz"
 
  [ -e "${ARCHIVE}" ] && { rm -f "${ARCHIVE}"; }
  [ -z "${QUIET}" ] && { echo "Archiving: ${ARCHIVE}"; }
  tar zcfP "${ARCHIVE}" "${OUTDIR}/${F5}" && rm -rf "${OUTDIR}/${F5}"
 
fi && ssh -q -o StrictHostKeyChecking=no `[ -r "${IDENTITY}" ] && { echo -i "${IDENTITY}"; }` root@${F5} \
'bash -c "$(base64 -di <<< Zm9yIE1PRCBpbiBgdG1zaCAtcSBzaG93IFw/IHwgc2VkIC1uIC1lICcvTW9kdWxlczovLC9PcHRpb25zOi9wJyB8IGF3ayAnL14gIC97cHJpbnQgJDF9J2A7IGRvIHRtc2ggLXEgc2hvdyAkTU9EIDI+IC9kZXYvbnVsbDsgZG9uZQ==)"' |\
 awk 'BEGIN{
  FS=": "
 }
 // {
  gsub(/[ \t]+$/, "")
 
  # LN buffer
  LN[1]=LN[0]
  LN[0]=$0
 
  # Build OUTPUT variable
  ## LN ends with "{" - special case header
  if(substr(LN[0],length(LN[0]),1) == "{") {
   COUNT=split(LN[0],FN," ") - "1"
   for (i = 1; i <= COUNT; i++) FILE=FILE FN[i] "_"
   OUTPUT=substr(FILE, 1, length(FILE)-1)
  }
  ## LN does not contain "::" but is a header
  else if(OUTPUT == "" && LN[0] ~ /^[a-z,A-Z]/) {
   OUTPUT=LN[0]
  }
  ## LN contains "::" and is a header
  else if(LN[0] ~ /^[A-Z].*::[A-Z]/) {
   gsub(/::/,"-")
   DIR=gensub(/\ /, "_", "g", tolower($1))
   FILE=gensub(/[ ,:].*/, "", "g", $2)
   if(FILE != "") {
    OUTPUT=DIR"/"FILE
   } else {
    OUTPUT=DIR
   }
  }
 
  # Print OUTPUT & LN buffer
  if(OUTPUT != "") print(OUTPUT"<"LN[1])
 }
 END{
  print(OUTPUT"<"LN[0])
 }' | while IFS="<" read OUTPUT LN
  do
 
     if [ ! -w "${OUTDIR}/${F5}/${OUTPUT}" ]
      then [ -z "${QUIET}" ] && { echo "Saving: ${OUTDIR}/${F5}/${OUTPUT}"; }
           install -D /dev/null -m 644 "${OUTDIR}/${F5}/${OUTPUT}"
     fi && echo "${LN}" >> "${OUTDIR}/${F5}/${OUTPUT}"
 
 done
07. September 2016 · Comments Off on Backing up your Citrix Netscalers. · Categories: Linux, Linux Scripts, Load Balancing, NetScaler, Networking · Tags: , ,

The following is an older script I wrote to automate the backup of a bunch of Citrix NetScaler appliances. Previously I posted an F5 backup script; which was based on this original script. NetScalers are awesome appliances! Not only are they insanely easy to manage; their configs are very straight forward to backup and restore. Very similar to the F5 backup script, posted earlier, we rely on SSH in this script. Except here I use SSHFS to mount the NS:/nsconfig directory and create an archive of it. The reason why I decided to use SSHFS was originally was that I intended to grep out the configured hostname from the config before creating tarball output; below is an example…

DEST=”$BACKUPDIR/$(grep ^”set ns hostName” /tmp/nsbackup/ns.conf | awk ‘{print “ns-“$NF”__”}’ | sed ‘s/__$/'” [$(echo $NS | cksum | awk ‘{print $1}’)] $(date +%F)”‘.tar.xv/’)”

Just like the previous script, this can ran automatically from cron…
@weekly [ -f /srv/nsbackup/nsbackup.sh ] && { /srv/nsbackup/nsbackup.sh; } > /dev/null

For further reading please reference the following Citrix Support Documentation:

Feel free to review, modify or use this script however you see fit. Remember you do so at your own risk!

#!/bin/bash
## Backup /nsconfig directories against a list of Citrix Netscalers.
## 2016 (v1.0) - Script from www.davideaves.com
 
NSHOSTS="ns01 ns02"
NSPW="nsroot"
BACKUPDIR="/srv/nsbackup"
 
# FUNCTION: End Script if error.
DIE() {
 echo "ERROR: Validate \"$_\" is installed and working on your system."
 exit 0
}
 
# Validate script requirements are meet.
type -p sshfs > /dev/null || DIE
 
# Main Loop.
for NS in $(echo $NSHOSTS | tr [:lower:] [:upper:]); do
 
 # Create backup directory and mount nsconfig using sshfs.
 mkdir /tmp/nsbackup && echo "$NSPW" | sshfs nsroot@$NS:/nsconfig/ /tmp/nsbackup -o password_stdin,StrictHostKeyChecking=no
 
 if [ -f "/tmp/nsbackup/ns.conf" ]; then
 
  # Figure out backup destination file.
  DEST="$BACKUPDIR/$NS$(echo $NS | cksum | awk '{print "_"$1}') ($(date +%F)).tar.xv"
 
  # Delete backup files older than 90 days.
  find "$BACKUPDIR" -maxdepth 1 -type f -name "*$(echo $NS | cksum | awk '{print "_"$1}')\ *.tar.xv" -mtime +90 -exec rm {} \;
 
  # Create backup file.
  if [ ! -f "$DEST" ]; then
   cd /tmp/nsbackup
   tar cfJ "$DEST" * && sync
   cd ..
  else
   echo "$DEST: Backup already exists..."
  fi
 
 fi
 
  # Unmount and remove backup directory.
  [ -d "/tmp/nsbackup" ] && { fusermount -u /tmp/nsbackup; }
  [ -d "/tmp/nsbackup" ] && { rmdir /tmp/nsbackup; }
 
done
26. August 2016 · Comments Off on Backing up your F5 load balancers. · Categories: F5, Linux, Linux Scripts, Load Balancing, Networking · Tags: , , , , ,

The following script is for performing scheduled backups of F5 load balancers. The Script initiates a backup against the F5 via SSH and then SCP’s the UCS output file off the box. It is meant to be ran in the crontab, on a Linux box, against the F5’s in an environment.

For further reading please reference the following F5 Support Documentation:

Feel free to review, modify or use this script however you see fit. Remember you do so at your own risk!

#!/bin/bash
## Create/Backup a UCS file against a list of F5 loadbalancers.
## 2016 (v1.0) - Script from www.davideaves.com
 
F5HOSTS="bigip01 bigip02"
BACKUPDIR="/srv/f5backup"
 
# FUNCTION: End Script if error.
DIE() {
 echo "ERROR: Validate \"$_\" is installed and working on your system."
 exit 0
}
 
# FUNCTION: Fetch the UCS or private id_rsa keyfile.
UCSFETCH() {
 if [ -e "$BACKUPDIR/.$F5.identity" ]
  then
        printf "$F5 "
 
        # Delete backup files older than 90 days.
        find "$BACKUPDIR" -maxdepth 1 -type f -name "$F5*.ucs" -mtime +90 -exec rm {} \;
 
        # Create the UCS backup file.
        ssh -q -o StrictHostKeyChecking=no -i "$BACKUPDIR/.$F5.identity" root@$F5 "tmsh save /sys ucs $(echo $F5) > /dev/null 2>&1"
 
        # Copy down the UCS backup file.
        scp -q -o StrictHostKeyChecking=no -i "$BACKUPDIR/.$F5.identity" root@$F5:/var/local/ucs/$F5.ucs "$BACKUPDIR/" && UCSRENAME
 else
        printf "\n$F5 "
 
        # Copy down the F5's private id_rsa keyfile for root user.
        scp -o StrictHostKeyChecking=no root@$F5:/var/ssh/root/identity "$BACKUPDIR/.$F5.identity" 2> /dev/null
 fi
}
 
# FUNCTION: Rename the UCS file.
UCSRENAME() {
 mv "$BACKUPDIR/$F5.ucs" "$BACKUPDIR/$F5$(echo $F5 | cksum | awk '{print "_"$1}') ($(date +%F -d "$(file "$BACKUPDIR/$F5.ucs" | awk -F': ' '{print $NF}' | awk -F',' '{print $1}')")).ucs"
}
 
# Validate script requirements are meet.
type -p scp > /dev/null || DIE
 
### Main Loop ###
for F5 in $(echo $F5HOSTS | tr [:lower:] [:upper:]); do
 
 # Validate host is pingable before fetching UCS file.
 ping -c1 $F5 > /dev/null 2>&1 && UCSFETCH
 
done; echo
20. March 2015 · Comments Off on Installing Rancid w/ViewVC under Debian/Ubuntu · Categories: Cisco, Linux, Linux Admin, Networking · Tags: , , , , , ,

When managing a network there are tools out there like Solarwinds, Cisco NCS or even Cisco Works that allow engineers to backup configurations; but tools like them can sometimes be unwieldy and/or clunky at their best. In the enterprise I general run ARCHIVE on Cisco routers and switches to automatically back up my configs to a Linux TFTP server. For other devices like the ASA and non-cisco gear, that do not have ARCHIVE functionality, I use a tool called RANCID and ViewVC for config backup and change tracking purposes.
There is a lot of documentation on the internet about setting up rancid, so the following is a very condensed step-by-step on getting Rancid up and running on a Debian distro.

Package Installation

sudo apt-get install rancid viewvc

Configure Rancid

Set the CVS folder

sudo pico /etc/rancid/rancid.conf

Add LIST_OF_GROUPS=”rancid to the config file.

sudo -u rancid /usr/lib/rancid/bin/rancid-cvs

Configure /etc/cron.d/rancid

sudo pico /etc/cron.d/rancid
MAILTO=root
 
# Run config differ hourly
0 23 * * *  rancid /usr/lib/rancid/bin/rancid-run
 
# Clean out config differ logs
50 23 * * * rancid /usr/bin/find /var/log/rancid -type f -mtime +2 -exec rm {} \;

Configure ~rancid/.cloginrc

sudo -u rancid pico ~/.cloginrc && chmod 600 ~/.cloginrc

add user        *       rancid
add password    *       RANCIDPW RANCIDPW
add method      *       ssh telnet

Configure device database

sudo -u rancid pico /var/lib/rancid/rancid/router.db

EX: DEXTER-CS01;cisco;up;Dexters Lab Core Switch
* Remember! For IPv6 compatibility RANCID now uses semicolons as a deliminator.

Create Symbolic Lync for clogin

sudo ln -s /usr/lib/rancid/bin/clogin /usr/local/bin/clogin

Configure ViewVC