Using spacecmd - Seamus Murray's Notes about Linux and System Engineering

Installation on RHEL 5

wget http://fedora.mirror.uber.com.au/epel/5/x86_64/spacecmd-1.9.4-1.el5.noarch.rpm
sudo yum install python-simplejson.x86_64
sudo rpm -i spacecmd-1.9.4-1.el5.noarch.rpm

Configure account credentials

[USER@SERVER ~]$ cat .spacecmd/config
[spacecmd]
server=localhost
username=admin
password=PASSWORD
nossl=1

Running spacecmd

spacecmd can be run either interactively (with tab completion and a help menu) or non interactive both modes enable you to run a single spacecmd-command plus wither a single or multiple operand

Interactive mode

[USER@SERVER ~]$ spacecmd
Welcome to spacecmd, a command-line interface to Spacewalk.

Type: 'help' for a list of commands
      'help <cmd>' for command-specific help
      'quit' to quit

INFO: Connected to http://localhost/rpc/api as admin
spacecmd {SSM:0}> system_details server1.localhost
spacecmd {SSM:0}> system_details server1.localhost server2.localhost

NonInteractive from the shell (bash, csh etc...)

[USER@SERVER ~]$ spacecmd system_details server1.localhost   ## output goes to standard out
[USER@SERVER ~]$ spacecmd system_details server1.localhost server2.localhost > {ouput file}   ##redirected the output to a file

Group creation and manipulation

Interactive mode

USER@SERVER ~]spacecmd
<log in>
spacecmd {SSM:41}> group_create newgroup
spacecmd {SSM:41}> group_addsystems newgroup server1.local server2.local 
spacecmd {SSM:41}> group_addsystems newgroup search:name:server

NonInteractive from the shell (bash, csh etc...)

USER@SERVER ~]spacecmd group_create newgroup   # you will be prompted for a group description
USER@SERVER ~]spacecmd group_addsystems newgroup server1.local server2.local
USER@SERVER ~]for i in ``cat list-of-servers`` ; do spacecmd group_addsystems newgroup  $i ;done

Note:

Using the output redirection is handy when dealing with lots of data, if you use sed/awk/grep/perl to parse the output you can save your self a lot of time.--

list all systems registered to the spacewalk server

USER@SERVER ~]spacecmd system_list > all-systems

List all systems with a certain package+version installed

USER@SERVER ~]spacecmd package_listinstalledsystems fakepackagename-1.2.3-4.x86_64 > fakepackagename-1.2.3-4.x86_64_installed

Comparing the installed packages on 2 servers

[USER@SERVER ~]$ spacecmd system_comparepackages server1.local server2.local
INFO: Connected to http://localhost/rpc/api as admin
Package                               This System              Other System            Difference
------------------------------------  -----------------------  ----------------------  ----------
initscripts                           8.45.42-1.0.1.el5        8.45.42-1.0.1.el5_8.1   Newer there
kernel                                2.6.18-308.24.1.0.1.el5  None                    Only here
kernel-fake                           None                     2.6.32-400.11.1.el5     Only there

listing negative delta lists..

Unfortunately you cannot query satellite/spacewalk for a list of servers where something does not exist

I work around this limitation by in 3 steps

Full-list - With-list = WithOut-list

list all systems registered to the spacewalk server

USER@SERVER ~]spacecmd system_list > all-systems

list all systems with a certain package+version installed

USER@SERVER ~]spacecmd package_listinstalledsystems fakepackagename-1.2.3-4.x86_64 > fakepackagename-1.2.3-4.x86_64_installed

Generate a list of the servers that do not have the package installed

for i in `cat all-systems` 
do if grep -q $i fakepackagename-1.2.3-4.x86_64_installed 
   then true
   else echo $i 
   fi 
done > fakepackagename-1.2.3-4.x86_64_not_installed

you now have a delta list

## note before you compare outputs from spacecmd you have to clean up the files first eg...remove the extra headers from the output files and the MS dos line feeds \^M etc....

scheduling - keeping track of a scheduled job initiated from the spacewalk web interface

Interactive mode

[USER@SERVER ~]$ spacecmd
Welcome to spacecmd, a command-line interface to Spacewalk.

Type: 'help' for a list of commands
      'help <cmd>' for command-specific help
      'quit' to quit

INFO: Connected to http://localhost/rpc/api as admin
spacecmd {SSM:0}> schedule_details 243013

NonInteractive from the shell (bash, csh etc...)

[USER@SERVER ~]$ spacecmd schedule_details 243013   ## output goes to standard out
[USER@SERVER ~]$ spacecmd schedule_details 243013 > {output file}   ##redirected the output to a file

Interactive mode

[USER@SERVER ~]$ spacecmd
Welcome to spacecmd, a command-line interface to Spacewalk.

Type: 'help' for a list of commands
      'help <cmd>' for command-specific help
      'quit' to quit

INFO: Connected to http://localhost/rpc/api as admin
spacecmd {SSM:0}> schedule_getoutput 243013

NonInteractive from the shell (bash, csh etc...)

[USER@SERVER ~]$ spacecmd schedule_getoutput 243013   ## output goes to standard out
[USER@SERVER ~]$ spacecmd schedule_getoutput 243013 > {output file}   ##redirected the output to a file

Running scripts on multiple severs.

Instead of trying to parse relatively unstructured output I prefer to structure the the output before it goes into spacewalk

My preferred method is echo the server name followed by the output of the command, this way it is easy to grep out the server you want

eg... if you want to run the following

script

#!/bin/sh
/usr/sbin/dmidecode | grep -m 1 Product

output from spacemcd_

spacecmd {SSM:0}> schedule_getoutput 245295
System:      server1.local
Start Time:  20131118T15:22:52
Stop Time:   20131118T15:22:52
Return Code: 0

Output
------
        Product Name: VMware Virtual Platform


##############################

System:      server2.local
Start Time:  20131118T15:24:38
Stop Time:   20131118T15:24:38
Return Code: 0

Output
------
        Product Name: VMware Virtual Platform

script

#!/bin/sh
product=`/usr/sbin/dmidecode | grep -m 1 Product`
servername=`hostname`
echo $servername  $product

output from spacemcd_

spacecmd {SSM:0}> schedule_getoutput 245295
System:      server1.local
Start Time:  20131118T15:22:52
Stop Time:   20131118T15:22:52
Return Code: 0

Output
------
server1.local        Product Name: VMware Virtual Platform


##############################

System:      server2.local
Start Time:  20131118T15:24:38
Stop Time:   20131118T15:24:38
Return Code: 0

Output
------
server2.local        Product Name: VMware Virtual Platform

__with the second method it is very easy to just grep for server*local or Vmware__

script to capture the IP details across all Ethernet interfaces

run across all hosts to capture all IP's on all hosts

#!/bin/bash

HOST_NAME=`hostname -s`
NICS=`ls /sys/class/net | grep -v lo`

for i in $NICS
  do echo -e $HOST_NAME $i `ifconfig $i | grep "inet addr" | awk -F" " '{print $2 " " $4}' | sed s/addr\://`
done

you could achieve the same results by querying the oracle database directly or using the satellite API. I chose to use spacecmd, grep and awk because that’s what I'm more comfortable with

this method seems to put less load on the satellite server and the back end database

(it creates hash tables locally in .spacecmd/localhost/) which I assume reduces hits on the database.

If the batch "list_PackageName-6.1_not_installed" is too large to run in one go..

you can divide it up based on server name prefix/suffix or simply batch size eg....

name based divide

grep servername list_PackageName-6.1_not_installed > list_PackageName-6.1_not_installed_servernames
grep '[0-9]pr' list_PackageName-6.1_not_installed > list_PackageName-6.1_not_installed_pr

batch size divide

STARTNUM=100
LASTNUM=200
awk '$1 == '$STARTNUM' , $1 == '$LASTNUM'' list_PackageName-6.1_not_installed > list_PackageName-6.1_not_installed-numbered-$STARTNUM-$LASTNUM

Create a system group to run the job against

GROUP_NAME=PackageName_install 
spacecmd group_create  $GROUP_NAME

Populate the group

           for i in `cat list_PackageName-6.1_not_installed-numbered-$STARTNUM-$LASTNUM
           do spacecmd group_addsystems $GROUP_NAME $i 
           done

schedule the job from the GUI

you can either keep track of the scheduled job from the GUI or from spacecmd

spacecmd schedule_details {schedule number}

If you need to schedule a job from satellite at a precise time, you can use satellite to run an arbitrary script to set up an at job for the client to check in with the satellite server at a specific time (within 60 seconds)

You can schedule a second job in satellite to perform the the actual work.

Example.

Suppose you want to execute the the following command at 6pm today

ps -Aef

Schedule an "at" job to execute the /usr/sbin/rhn_check at your desired time

Note: To ensure that all the clients have polled the satellite server and picked up this job you will have to schedule this at least an hour before your "desired time"

#!/bin/sh
PreciseTime=1800
cat > /tmp/sat-precision-strike.sh <<EOF
echo `date` " starting the at job" >> /tmp/sat-precision-strike.log
/usr/sbin/rhn_check
echo `date` " finished the at job" >> /tmp/sat-precision-strike.log
rm -f /tmp/sat-precision-strike.sh
EOF
sleep 5
at -f /tmp/sat-precision-strike.sh $PreciseTime
echo "added at job at -f /root/sat-precision-strike $PreciseTime" >> /tmp/sat-precision-strike.log
sleep 5

Schedule a second Job detailing the command that you need to run

ensure you set the "Schedule no sooner than:" time to 1 minute before your desired time

#!/bin/bash
/bin/ps -Aef