I recently had to update about 30 different servers with the identical software. This is an all-too-common occurance for an administrator, and there are many different systems available to do this. However, most of them do too much and are generally non-trivial to set up and manage. I wanted a system which would be easy to install, and easy for a non-administrator to use. Enter PullIt, a simple system which solves these requirements:
pullit.tar.gz (32.5 KiB, 313 hits)
PullIt Software Distribution System
2. Installation Details
4. Parameter Definitions
6. Error Codes
Pullit is a small system which gives administrators the ability to remotely install files on various systems. It has been designed to be simple to understand and simple to operate. Based on the KISS principle, it doesn’t try to do everything. There are many other packages which provide much more extensive services, if you need more than this can provide.
This system does not have any security attached. The code for all packages will be copied to all systems, only the required packages will be installed. What this means is that if you have a malicious user who wants to install a package and they are not supposed to, they will be able to copy the files and install them by hand.
PullIt consists of two parts, pullit.sh and pullitd.sh The first, pullit.sh, is the shell by which the administrator/user queues up a set of file to be distributed. The second, pullitd.sh, is a daemon which is run periodictly by cron.
The basic idea is that there is a master server, on which the users/administrators will place files for distribution. The files will be packaged up into a shar file and placed into a distribution directory.
The clients will have the pullitd.sh script running in a cronjob. The script firsts uses rsync to syncronize it’s local directory of packages with those on the server. This design allows each client to also function as a server, which both relieves the load on the main server in cases where either there are a large number of clients, or if the server is a very low-powered machine. It also allows remote distribution of packages
The system is flexible enough to allow for multiple architectures, and multiple distribution groups.
So for example, you can have a “manager” and “employee” group, and inside each group there could be both 32 bit, 64 bit, and noarch architectures. Packages intended for the manager group will not get installed on the employee group, and vice-versa. Currently a client can only have one group active; this will be modified in the future to allow for multiple groups.
Also, when installing files, there is a configuration option which says that a backup should be kept. When this is set, if the file exists then the system will first createa subdirectory in the destination directory and then MOVE (not copy) the files to it. The reason for the move is in case the file is currently being executed, it will be able to continue to execute without having to worry about the file being overwritten.
When installing a client, the group the client belongs to is specified in the configuration file.
2. Installation Details
On the server, the file pullit.sh will be installed to /usr/local/sbin (or whereever you specify), a configuration directory will be created in /etc, and a configuration file installed in the configuration directory.
On the client, the file pullitd.sh will be installed to /usr/local/sbin, a configuration directory will be created in /etc, and a configuration file installed in the configuration directory. Also, a file will be installed into /etc/cron.d
It is your responsibility to make sure that rsync is working properly on all systems.
To install, unpack the tar file into a directory and run the script:
The install script can optionally use the “dialog” command to have a curses- based installer. If you would like to use the text-based mode even if you have the dialog command available, invoke the installer as follows:
The installer should usually be run as root, however it will allow you to run as a regular user. Obviously, a regular user will not be able to change system files or directories, so if a regular user is installing this then it will be installed local to that user.
The installer has the ability to create an installation script for the clients. I strongly recommend you use this, since the installation script will also create ssh keys on the client and install them on the server. This makes the complete installation process extemely easy. The client installation creates the necessary ssh keys, and also asks for the group that the client is in, and the architecture of the client.
The system runs as a simple shell with a limited number of commands. The system is actually a bash shell script, and does not do any fancy screen manipulation, so you can very easily have this run in a batch mode.
The command to run is:
The valid shell commands are:
|done||[ packagename [groupname]]|
|noarch||(shortcut for “arch noarch”)|
|arch||32 | 64 | noarch (anything will work, as long as all clients use the same)
noarch is special, it means that ALL platforms in the group will install the package
|add||[ file [file]] destinationDirectory|
|del||file [file...] | all|
|commit||all | packagename [group]|
|list||[ files | packages | groups | archtypes ] (defaults to files)|
|runbefore||filename [in directory] [rc expected]|
|runafter||filename [in directory] [rc expected]|
|prune||packagename [packagename...] | all|
|at||[ yyyy-mm-dd [ hh:mm:ss ] ]|
4. Parameter Definitions
|packagename||Name of the package being worked on.|
|groupname||Name of the group the current package is intended for|
|filename||Name/path of the file being installed/deleted|
You can use quotes, either single or double, to enclose a parameter with spaces. Also, you can use a backslash to escape the next character; this will allow you to embed a quote inside a single parameter. All quotes, except for a quoted parameter at the end, must be closed with the same quote which started the quoting.
|start||[ packagename ]||Use this command to start building a package. Specify the name of the package with the command. The packagename needs to obey the rules for directory names. The packagename is optional at this point, but will need to be specified on the “end” command if not entered here.|
|group||groupname||An alternate method of specifying what group the current package will be distributed to. If specified with this command, it will not need to be specified with the “end” command|
|done | end||[ packagename [ groupname ] ]||This command finishes a package. However, the package is NOT released for distribution yet, use the “commit” command for that. The packagename and groupname are optional here, in that if they were specified using the “start” and “group” command you wouldn’t need to specify them here.|
|arch||archtype | noarch||The archicture specifies the destination system. While it would be advisable to use the same arch as the basic OS, there is no technical reason to do so. This allows you to specify any number of archictures. One reason to do so is to use a single server to distribute files to 32 bit, 64 bit, x86 x86_64, arm avr32 blackfin m68k m68knommu microblaze mips powerpc. However, if you specify any architecture types during the server install, only those types will be allowed.|
|noarch||This is a shortcut for “arch noarch”|
|(archtype)||If the archtype is simply typed on a line by itself, and it doesn’t match a command name, then the architecture will be set to the entered archtype. This is similar to the “noarch” command as a shortcut. This relies on the default architectures being specified in the configuration file.|
|add||[ filename [ filename ]…] destinationDirectory||Multiple files can be specified here. This command adds files to the package; the last paramenter is the destination directory on the client computers. The files don’t have to be in the current directory, the full paths will be stripped before building the package.|
|del||filename | all||Delete the specified file from the current package. If “all” is specified, then delete all files from the current package.|
|commit||all | packagename [ groupname ]||Commit a package for distribution. If a groupname is specified, then only commit packages which are ready for the specified group. If “all” is specified, then commit all available packages for distribution.|
|list||[ files | packages | groups | archtypes ]||This will list the following types of information:
files All files in the current package
|runbefore||filename [ inDirectory [ expectedReturncode ] ]||Specify a command or program to run before the files are installed. These commands are run BEFORE any files are renamed for backup. If inDirectory is specified, then the command will be run in the specified directory. If specified, the return code from the command must match the expectedReturncode, otherwise the install will abort. If the directory the command is run in is not important, but you do want to specify the returncode, then use a . as the directory.|
|runafter||filename [ inDirectory [ expectedReturncode ] ]||Specify a command or program to run after the files are installed. These commands are run BEFORE any files are renamed for backup. If inDirectory is specified, then the command will be run in the specified directory. If specified, the return code from the command must match the expectedReturncode, otherwise the install will abort. Note that since the files have been installed and backups made if specified, that an abort at this point could leave your systems in an unexpected state.|
|prune||packagename [packagename...] | all||Prune specified packages from the cache directory, or all packages|
|at||[ yyyy-mm-dd [ hh:mm:ss ] ]||Install the package at or after the specified date/time|
|quit||Both quit and exit will exit the shell.|
6. Error Codes
The following error codes can be returned by the pullit.sh script. These may be useful if you are calling it from a script:
0 All ok
1 Missing configuration file
2 Missing bash_arrays.sh file
3 Unable to create libdir
4 Unable to install sharutils with yum
5 Niether yum nor apt-get detected
6 unable to install sharutils with apt-get
7 Unable to create the cache dir
The following error codes can be returned by the pullitd.sh script:
0 All ok
1 Missing configuration file
2 Unable to create libdir
3 Unable to create the top journaldir
4 Unable to create the cachedir
5 Transfer program missing
6 Unable to create the lower journal directory
7 Not running as specified user
Lets say that you have a number of web servers, all being load-balanced together in a colo facility. You need to update some files in a directory on all the servers. A problem is that some of the servers are always down for maintenance or other problems.
Here is the situation:
|Main HTML directory:||/var/www/html|
|Directory containing files to be updated:||/var/www/html/mydir|
|Files to be updated/added:||index.php .htaccess config.php|
First, copy the files to the master server:
scp index.php .htaccess config.php jbb@master:
Then log in to the master:
Now start the pullit shell:
The shell gives the following prompt:
Enter the following commands, one per line:
start mydir group webserver arch noarch add index.php .htaccess config.php /var/www/html/mydir runbefore "/etc/init.d/httpd stop" runafter "/etc/init.d/httpd start" end commit all quit
Note that because there are spaces in the commands you want to run, you need to surround the command with quotes.
The commands do the following:
Start a package called "mydir"
This package is intended for the group called "webserver"
No architecture, so this will be installed on all servers in the group
add index.php .htaccess config.php /var/www/html/mydir
Add the three files to the package, they will be installed to the /var/www/html/mydir directory
runbefore "/etc/init.d/httpd stop
Stop the webserver before doing the installation
runafter "/etc/init.d/httpd start
|start the webserver after the install is done|
End the package, ready for commit
Commit all packages. At this point, the packages are available for the servers to pull them down and install
exit the pullit shell