#-- # Project: pkgman # File: pkgman/README # Revision: $Id$ # Home: http://code.google.com/p/pkgman/ # Author: Derek Olsen # Copyright: (c) 2007 by Derek Olsen # License: GNU General Public License v2 #++ contents DESCRIPTION OVERVIEW SOLARIS PACKAGES REQUIREMENTS INSTALLATION EXAMPLE USAGE LOGGING CONFIG FILE RBAC AND SUDO ZONES JRUBY FEEDBACK CREDITS DESCRIPTION ====================================================================== Pkgman is used to maintain the desired software package state on all of your Solaris hosts. Add the package names to the configuration file and indicate what hosts the package applies to and whether it should be installed or removed. Run pkgman on a host and point it to the configuration file and pkgman will determine which package records apply to the executing host. OVERVIEW ====================================================================== Once a host is deployed into the wild the procedures for managing packages are very limited. Some solutions exist but the ones I've tried and used for years are very cumbersome and require expert level knowledge of the underlying framework. I wanted to create a process which only requires that the admin know what packages should be installed/removed from which hosts. As long as the admin knows this high level information they are ready to leverage pkgman to simplify software package management. You should be able to integrate pkgman into your existing host management procedures regardless of how you manage your hosts. If you want to run pkgman from cron or a configuration management tool like cfengine or puppet that's great. If you do your package management manually during a maintenance window you can use pkgman to ensure consistency and minimize the amount of manual typing per host. Please see the requirements section to see if pkgman will work in your environment! SOLARIS PACKAGES ======================================================================= This section is here to provide some information on Solaris packages. Pkgman requires a specific format for packages. This will also show you how to extract the information from packages that you will use in the pkgman config. Just for fun here is the Sun definition of the term "package" "A collection of files and directories required for a software application." Solaris packages come in two different formats. The two varieties are referred to as "datastream format" and "file system format". The pkgman tool currently supports only packages in the datastream format. The simplest way to identify the format is to check if your package is a file or a directory. In this example we can see that the package SUNWzsh is a directory so that means it's a "file system format" package. # file SUNWzsh SUNWzsh: directory In this other example we can see that the package file named jmeter-2.3RC-sol9-sparc is actually reported as a "datastream package" by the file command. # file jmeter-2.3RC3-sol9-sparc jmeter-2.3RC3-sol9-sparc: package datastream So in short if the package is a file it's in datastream format and if it package is a directory it's a file system format. The Solaris distribution provides packages in file system format. From my personal experience most 3rd party and in house packages are created in datastream format as it's typically easier to transfer single files around and or compress a single file compared to a directory. If you have file system format packages that you need to convert to datastream format you can do the following. The arguments following the "-s" are the directory which contains the file system package, the file name of the new datastream pacakge, and the pkginst name of the package. # pkgtrans -s /Solaris_10/Product/ /var/tmp/SUNWzsh SUNWzsh Transferring package instance # file /var/tmp/SUNWzsh /var/tmp/SUNWzsh: package datastream Now we have a datastream package! Whoo! The pkgman config file uses the "pkginst" variable to refer to packages. This can be queried using the pkginfo command. In this example we'll query the datastream SUNWzsh package. # pkginfo -d /var/tmp/SUNWzsh system SUNWzsh Z shell (zsh) The pkginst, in the second column of the output, is the same as the file name in this package. This is not true of all packages. In this next example we will query the jmeter package we looked at earlier # pkginfo -d jmeter-2.3RC3-sol9-sparc application jmeter Jmeter with test cases So in this case the pkginst string is "jmeter" which is different than the file name. You will need to find out the version of a package as well. You can query the installed host package database or query a package file itself. In this example I'll query the host package database with pkgparam to find out what version of SUNWzsh I currently have installed. # pkgparam SUNWzsh VERSION 11.10.0,REV=2005.01.08.01.09 Now let's query the jmeter package file to see what version it is. # pkgparam -f jmeter-2.3RC3-sol9-sparc VERSION 2.3RC3 Now you have the information you need to populate a pkgman config file. The name of the package comes from the pkginfo and the version comes from pkgparam. The Sun document "Application Packaging Developer's Guide" is a great resource for further information. http://docs.sun.com/app/docs/doc/817-0406?l=en&q=sun+package+developer REQUIREMENTS ======================================================================= Currently the requirements to run pkgman are 1. ruby 1.8.X or higher 2. datastream packages available via http. The package files can be compressed with gzip as long as the file ends in ".gz". 3. the config file needs to be distributed to each host or made available via http. INSTALLATION ======================================================================= To download pkgman you can grab the tarball marked "latest stable version" from the download list http://code.google.com/p/pkgman/downloads/list. Currently the name for the tar ball is pkgman-.tar.gz. When you unpack the tar ball you will end up with a directory called pkgman-. You can also use subversion to check out the pkgman repository. The following command will check out the pkgman subversion repository into a directory called "pkgman". svn checkout http://pkgman.googlecode.com/svn/trunk/ pkgman To install pkgman you need to cd to the root of the pkgman directory. The setup script will use /usr/local as the prefix for deploying pkgman. So the executable goes into /usr/local/bin and the configuration file goes into /usr/local/etc. The setup script also changes the shebang line to match the path to ruby on your host. ruby setup.rb ---> bin <--- bin ---> conf <--- conf ---> bin <--- bin ---> conf <--- conf rm -f InstalledFiles ---> bin mkdir -p /usr/local/bin/ install pkgman.rb /usr/local/bin/ <--- bin ---> conf mkdir -p /usr/local/etc/ install pkgman.yaml /usr/local/etc/ <--- conf If you want to install pkgman into a different directory you would do something like this. Here I am changing the directory prefix from /usr/local to my home directory. ruby setup.rb all --prefix=$HOME ---> bin <--- bin ---> conf <--- conf ---> bin <--- bin ---> conf <--- conf rm -f InstalledFiles ---> bin mkdir -p /users/deet/bin/ install pkgman.rb /users/deet/bin/ <--- bin ---> conf mkdir -p /users/deet/etc/ install pkgman.yaml /users/deet/etc/ <--- conf You can also pkgman as a Solaris package. I have supplied a script in the test directory called "make_pkgman_pkg.rb". You can either call the rake target in the root of the pkgman directorya. rake mkpkg (in /home/dolsen/dirreorg) You'll find the pkg at /home/dolsen/dirreorg/test/pkgs/pkgman-257-sol10-any Or you can change to the test directory and manually execute the script. ./make_pkgman_pkg.rb You'll find the pkg at /home/dolsen/dirreorg/test/pkgs/pkgman-257-sol10-any Currently the version of the package will reflect the Subversion version of the pkgman.rb script. EXAMPLE USAGE ======================================================================= In these examples I will show the command line execution of the script as well as the associated configuration file. I will use very basic configuration files without any supporting details as the configuration file layout will be discussed later. You must execute pkgman.rb as a user who is allowd to run pkgman and pkgrm commands. Typically this will be the user root unless you have setup permissions for other users to add or remove packages on your host. The examples will only use 1 or 2 packages for demonstration purposes but you can manage all of your packages in a single configuration file. In this example we are upgrading to version 132 of the pkgman package. This is the corresponding configuration file. --- - :http_host: pkgrepo :http_port: 1080 :http_uri: /pkgs :logging: syslog :syslog_facility: daemon :syslog_level: notice :cmd_prefix: none - :file: pkgman-132-sol10-any :name: pkgman :version: 132 :action: install :hosts: default In the following pkgman execution I am running pkgman and instructing it to use a configuration file which is located locally at /usr/local/etc/pkgman.yaml. The pkgman package that was installed previously was version 127 which is why you see that old package being removed and then the desired version being added. [root@hostcool ~]$ /usr/loca/bin/pkgman.rb -f /usr/local/etc/pkgman.yaml Removal of was successful. Installation of was successful. In this next example we are installing test package ZZZpkgman-upgrade-test version 1.0. This is the corresponding configuration file. --- - :http_host: pkgrepo :http_port: 1080 :http_uri: /pkgs :logging: syslog :syslog_facility: daemon :syslog_level: notice :cmd_prefix: none - :file: ZZZpkgman-upgrade-test-1.0 :name: ZZZpkgman-upgrade-test :version: 1.0 :action: install :hosts: default In the following line I am instructing pkgman to use a configuration file which is located remotely at http://pkgrepo:1080/pkgs/install_example.yaml. [root@hostcool ~]$ pkgman.rb -f http://pkgrepo:1080/pkgs/install_example.yaml Installation of was successful. In this example we are executing pkgman in dry-run mode to see what would occur on the host if we ran pkgman normally. This is the corresponding configuration file. --- - :http_host: pkgrepo :http_port: 1080 :http_uri: /pkgs :logging: syslog :syslog_facility: daemon :syslog_level: notice :cmd_prefix: none - :file: ZZZpkgman-remove-y0 :name: ZZZpkgman-remove-y0 :version: 1.y075 :action: remove :hosts: default - :file: ZZZpkgman-install-a2 :name: ZZZpkgman-install-a2 :version: 1.a25 :action: install :hosts: db0.*, hostc.*, app09 [root@hostcool ~]$ pkgman.rb -n -f \ http://pkgrepo:1080/pkgs/remove_and_install_example.yaml DRYRUN: Remove pkg ZZZpkgman-remove-y0 DRYRUN: Install pkg ZZZpkgman-install-a2 So we can see what work would be done on this host. Now if we execute the command without the "-n" this is what happens. [root@hostcool ~]$ pkgman.rb -f \ http://pkgrepo:1080/pkgs/remove_and_install_example.yaml Removal of was successful. Installation of was successful. If you want to try pkgman without affecting the packages installed on your host you can use the test packages I have created. In the pkgman/test directory is a script called "make_pkgs.rb". If you run this program it will create some test packages and put them in the pkgman/test/pkgs directory. These pkgs don't actually contain any files or programs so they wont add any content to your hosts but will allow you to exercise pkgman. Additionally I have included some configuration files you can use for your testing. The config files are in the pkgman/test/conf directory. Currently the test config files are pkg_install_and_remove.yaml pkg_upgrade.yaml prep_remove.yaml prep_upgrade.yaml In order to setup the test environment first run pkgman using the config files that start with "prep". These install various test pkgs so that when you run pkgman using the other config files (pkg_install_and_remove.yaml, pkg_upgrade.yaml) the desired pkgs which will either be removed or upgraded exist on the host. To make the pkgs easy to spot in pkginfo output they all start with "ZZZpkgman". Once you are done installing the various test packages you should create your own config file which you can use to remove all of the test pkgs. LOGGING ======================================================================= You can either have the script output logged to the console or to syslog. Your logging choice is set in the config file which is covered in the "CONFIG FILE" section. When you set the logging option to "console" everything worth saying is logged to standard out. In this scenario you will need to make sure you capture standard output if your running pkgman under cron or some other service. The logging option "syslog" will send all logs to the syslog server. By default the DAEMON facility and NOTICE level are used as the stock Solaris syslog.conf is setup to log daemon.notice to /var/adm/messages. You can adjust the facility or level to work with your custom syslog setup. CONFIG FILE ======================================================================= The config file is used to set various values to support the execution of pkgman as well as all of the references to the packages that should be installed or removed. The config file is constructed using YAML (http://www.yaml.org/) as it seemed the easiest format which would support both human and script parsing. Additionally YAML support is one of the built-in ruby libraries. The minimum pkgman.yaml file contains the settings keys which are the configurations for the script. This record is distinguished from package records by the key "http_host". Upon opening the config file we peak inside each of the records which are nameless hashes. Once we find the key "http_host" we know we have located the script configuration settings. NOTE: The lines containing "---" and "-" are actually part of the config file. They will be covered further down. --- - :http_host: "pkghost.domain.net" :http_port: 1080 :http_uri: "/pkgs/" :logging: syslog :syslog_facility: daemon :syslog_level: notice :cmd_prefix: none This next example illustrates a pkgman.yaml file which contains the minimum script config entries as well as a package install and package remove reference. --- - :http_host: "pkghost.domain.net" :http_port: 1080 :http_uri: "/pkgs/" :logging: syslog :syslog_facility: daemon :syslog_level: notice :cmd_prefix: none - :file: jmeter-2.3RC3-sol10-sparc :name: jmeter :version: 2.3RC3 :action: install :hosts: default - :name: SMCtidy :action: remove :hosts: host18 Here are the details of what the settings keys mean. - # The "-" is a required record separator :http_host: "pkghost.domain.net" # Host serving up pkgs via http :http_port: 1080 # Port webserver listens on :http_uri: "/pkgs" # Directory containing pkgs :logging: syslog # Logging. either syslog or console :syslog_facility: daemon # the syslog facility to use :syslog_level: notice # the syslog level to use :cmd_prefix: none # sudo, pfexec or none Here are the package references from above with supporting comments indicating what the particular lines mean. - # The "-" is a required record separator :file: jmeter-2.3RC3-sol10-sparc # Filename of datastream package :name: jmeter # Name of the Solaris package :version: 2.3RC3 # The version of the package :action: install # Install or remove the package :hosts: default # Which host(s) record applies to - # The "-" is a required record separator :name: SMCtidy # Name of the Solaris package :action: remove # Install or remove the package :hosts: host18 # Which host(s) record applies to Package files can be plain files or compressed with gzip. If the file is gzipped it must end in the ".gz" extension. Here is an example which refers to a gzipped package file. - :file: jmeter-2.3RC3-sol10-sparc.gz :name: jmeter :version: 2.3RC3 :action: install :hosts: default Now that you have seen a basic pkgman.yaml example I'll go over a few important YAML details. Note that I'm no YAML expert and to use pkgman I don't believe you need to be either. At the start of the examples you'll see 3 hyphens "---". This entry indicates that start of the YAML markup. The single hyphens "-" are used as record separators. The lines following a "-" are a hash and each one gets added to an array which holds all of the package references and the supporting script configurations. Each line that is a key,value pair must start with 2 spaces per YAML requirements. In the following line :action: install The key ":action:" is a ruby symbol. All of the key's in pkgman.yaml file are ruby symbols. The ruby docs I have read typically say to use a symbol if you only care about the value associated with the key and not the key itself. In the example :action: install The value "install" is a string. All of the values in the pkgman.yaml file are strings. Currently the required keys for packages that you wish to install are file name version action hosts Currently the required keys for packages that you wish to remove are name action hosts The action applies to the comma delimited list of systems in the hosts key. Additionally you can use the string "default" and that would mean the action applies to all hosts. Now you should be able to populate the pkgman.yaml file with a list of packages that you wish to have installed and removed on your Solaris hosts. RBAC AND SUDO ======================================================================= RBAC and sudo can be used to give the minimum amount of privileges to get a unit of work accomplished. If you want to run pkgman as a non root user then you'll need to configure RBAC or sudo to allow the user to execute pkgadd and pkgrm. I will give some brief examples on how you can configure RBAC and sudo to allow a non root user to run pkgadd and pkgrm. My examples will use a user called "operator". The "operator" account is a user and not a role. First let's cover the RBAC configuration to allow the operator user to run pkgadd and pkgrm. This is pretty easy because the Solaris RBAC configuration includes the profile called "Software Installation". This profile is defined to allow users to run pkgadd and pkgrm, now how great is that! As the operator user we run the profiles command to see what profiles our account is [operator@funk ~]# /usr/bin/profiles -l All: * So you will need to update the /etc/user_attr file to link the "Software Installation" profile to the user operator. Append the following line to /etc/user_attr (keep in mind "defaultpriv=basic" needs to be at the end of the line). operator::::profiles=Software Installation,defaultpriv=basic Now let's rerun the profiles command and see what the output looks like. [operator@funk ~]# /usr/bin/profiles -l Software Installation: /usr/bin/ln euid=0 /usr/bin/pkginfo uid=0 /usr/bin/pkgmk uid=0 /usr/bin/pkgparam uid=0 /usr/bin/pkgproto uid=0 /usr/bin/pkgtrans uid=0 /usr/bin/prodreg uid=0 /usr/ccs/bin/make euid=0 /usr/sbin/install euid=0 /usr/sbin/patchadd uid=0 /usr/sbin/patchrm uid=0 /usr/sbin/pkgadd uid=0, gid=bin /usr/sbin/pkgask uid=0 /usr/sbin/pkgchk uid=0 /usr/sbin/pkgrm uid=0, gid=bin All: * So now the operator user is configured to run pkgman and pkgrm. The operator user needs to use the pfexec command to utilize the new privileges when running pkgadd or pkgrm. [operator@funk ~]# pfexec /usr/sbin/pkgadd The following packages are available: 1 ZZZpkgman-exit0 test package (all) 1.00 Select package(s) you wish to process (or 'all' to process all packages). (default: all) [?,??,q]: [operator@funk ~]# pfexec /usr/sbin/pkgrm ZZZpkgman-upgrade-test The following package is currently installed: ZZZpkgman-upgrade-test test package (all) 1.1b Do you want to remove this package? [y,n,?,q] You could configure RBAC in other ways to gain the same results but this was the cleanest for my example. To configure pkgman to use pfexec during pacakge installation or removal you would set the key ":cmd_prefix" to "pfexec" in the pkgman.yaml file. :cmd_prefix: pfexec While sudo does not come stock with Solaris many sites install and use it for a variety of reasons. To configure sudo you'll need to add the following entries for pkgadd and pkgrm. These lines will need to be put into the sudoers file to allow the operator user the required capabilities. The most important thing to note is that "NOPASSWD:" is required because pkgman cannot supply a password to sudo. Without the "NOPASSWD:" flag sudo (and pkgman) will just hang their waiting until the end of time for some input. Cmnd_Alias PKGMAN = /usr/sbin/pkgadd, /usr/sbin/pkgrm operator ALL = NOPASSWD: PKGMAN As the operator user you can verify that sudo is configured correctly through running "sudo -l". [operator@fun ]# sudo -l User operator may run the following commands on this host: (root) NOPASSWD: /usr/sbin/pkgadd, /usr/sbin/pkgrm :cmd_prefix: pfexec To configure pkgman to use sudo during pacakge installation or removal you would set the key ":cmd_prefix" to "sudo" in the pkgman.yaml file. :cmd_prefix: sudo Make sure that the sudo command is in the $PATH of the user who executes pkgman. You don't need to use sudo or pfexec in which case you would leave the ":cmd_prefix" setting in it's default state. :cmd_prefix: none ZONES ======================================================================= The ability to run virtual host instances was added to the Solaris 10 release. Hosts running earlier versions of Solaris will not be impacted by zone handling within pkgman. Pkgman will detect what the release is and execute pkgadd accordingly. If the release is Solaris 10 or later then pkgadd will be executed with the flag "-G". To paraphrase the pkgadd man page the "-G" flag will install the package in the zone in which pkgadd is executed. So if you run pkgman in the global zone then the package will be added only to the global zone. If you execute pkgman in a zone named "zone9" then the packages will only be added to zone "zone9". Zones present various difficulties to deal with. It is not possible to query the package database of a non global zone from the global zone. Additionally if a zone is not in the correct state the package and patch management command cannot be used without the "-G" flag in the global zone. You can read further about the management of patches and packages on hosts with zones installed at. http://docs.sun.com/app/docs/doc/817-1592/z.pkginst.task-31?l=en&q=pkgadd&a=view JRUBY ======================================================================= As of this writing the current release of jruby is version 1.0.3. I have done some basic testing and have not been able to get jruby to run pkgman. The first issue is that jruby does not support the ruby syslog library so you would need to remove the "require 'syslog'" entry from pkgman and make sure to set console logging in the config file. I imagine java has a syslog alternative that could be used in place of the ruby syslog library. I tried disabling syslog and configured the unit test to run pkgman under jruby but the first test failed during the execution of "pkgadd". Once i got the following error I stopped my investigation into using jruby. /usr/local/bin/pkgman.rb:385:in `pkg_install': undefined method `waitpid2' for Process:Module (NoMethodError) from /usr/local/bin/pkgman.rb:505:in `popen' from /usr/local/bin/pkgman.rb:379:in `pkg_install' from /usr/local/bin/pkgman.rb:505 from :1:in `each' from /usr/local/bin/pkgman.rb:488 If enough interest exists for jruby support I'll dig further into the above problems. Otherwise I'll just randomly try new jruby releases to see if the errors magically go away. FEEDBACK ======================================================================= Please! Give me feedback! Validate my life! Whooooo! Ok now that I have met my question mark daily allowance. If you encounter any problems (perceived or real) please drop me a line with the following information. 1. description of problem and/or any error messages 2. pkgman.yaml used when problem was encountered 3. output from 'uname -a' 4. output from 'ruby -version' 5. Anything else you can think of that will help me understand the problem. If you have any feature requests I would love to hear about them. Please take a moment to describe your feature request and how it would work. An example pkgman.yaml file that demonstrates your desired feature will help me visualize your request. You can check the list of open issues and enhancements by visiting http://code.google.com/p/pkgman/issues/list I have started marking feature requests as type "Enhancement" and everything else is either a defect or a placeholder for something I need to research (such as configuration RBAC to allow non root users to run pkgadd/pkgrm). Anyhoo as your emails justifiy my existence I'm open to any feedback! If your in Portland Oregon and want to get a beer let me know. CREDITS ======================================================================= Tajmon N. Thanks for testing and making this file readable. Mark "Jazzy" B. Thanks for testing and providing feedback.