summaryrefslogtreecommitdiffstats
path: root/extra/slacktrack/OVERVIEW
diff options
context:
space:
mode:
Diffstat (limited to 'extra/slacktrack/OVERVIEW')
-rw-r--r--extra/slacktrack/OVERVIEW595
1 files changed, 595 insertions, 0 deletions
diff --git a/extra/slacktrack/OVERVIEW b/extra/slacktrack/OVERVIEW
new file mode 100644
index 000000000..5e0f5d612
--- /dev/null
+++ b/extra/slacktrack/OVERVIEW
@@ -0,0 +1,595 @@
+##############################################################################
+# Document: OVERVIEW
+# Purpose : Provide an overview of the Slackware package system incorporating
+# the use of 'slacktrack'
+# Please note that this is *not* a guide to the use of slacktrack;
+# the man page and SlackBuild scripts in the 'examples' directory
+# aim to fulfill that requirement.
+# Date....: 15-May-2009
+# Author..: Stuart Winter <mozes@slackware.com>
+# Homepage: http://www.slackware.com/~mozes
+##############################################################################
+
+
+CONTENTS:
+---------
+1.0 Preamble
+2.0 Slackware packages
+ 2.0.1 Format of official Slackware packages
+ 2.0.1.1 Package names
+ 2.0.1.2 Version numbers
+ 2.0.1.3 Architectures
+ 2.0.1.4 Build numbers
+ 2.1 An overview of the creation of official Slackware packages
+ 2.1.1 'Clean' and 'dirty' builds
+ 2.1.1.1 Clean builds
+ 2.1.1.2 Dirty builds
+ 2.1.1 SlackBuild and .build scripts
+ 2.1.1.2 SlackBuild scripts
+ 2.1.1.3 .build scripts
+ 2.2 slacktrack in the scheme of things
+ 2.2.1 Using slacktrack with official .build scripts
+ 2.2.2 Writing your own .build scripts for slacktrack
+ 2.2.2.1 Making use of slacktrack's environment variables
+3.0 slacktrack in operation
+ 3.1 How slacktrack finds which files have been installed
+ 3.1.1 installwatch's author
+4.0 Example scripts
+ 4.0.1 non-slackware example build scripts
+ 4.0.2 slackware example wrapper build scripts
+5.0 Known problems/features of slacktrack
+6.0 New features
+7.0 Licence
+
+
+
+1.0 Preamble
+ ---------
+
+I used to work for a company that provided web hosting and shell accounts.
+When I started there were a number of shell servers all running various
+releases of RedHat Linux, in various states of repair. I managed to convince
+the management to let me try Slackware on there instead because I have a much
+stronger understanding of how to maintain and build server using the
+Slackware distribution. This trial went well and within a few months all
+servers were converted to Slackware version 8.1.
+
+In order to ease the speed of installation (and to prevent against
+forgetting to install or configure something critical), I wrote a
+set of post installation scripts that upgraded Slackware packages,
+configured numerous config files and copied pre-compiled software
+(such as courier IMAP) onto the new system.
+For other software that I could not simply copy, I'd compile it on the
+new server.
+
+However, it soon became clear that due to security updates and so on,
+it became incredibly time consuming (not to mention seriously boring)
+to compile PHP, Apache and so on on every server.
+
+At this point, I began to investigate how to create packages for Slackware.
+
+
+2.0 Slackware packages
+ ------------------
+
+ The Slackware Linux distribution consists of a variety of 'packages'
+ that make up the distribution.
+
+ Traditionally, all packages are in '.tgz' format (a gzipped tar archive).
+ Starting with Slackware v13, new compression formats are supported which
+ are represented by three additional file extensions:
+
+ .tgz - Gzip
+ .tbz - Bzip2
+ .tlz - LZMA
+ .txz - XZ
+
+ In this document, we use the file extension '.t?z' to represent
+ one of the above file formats.
+
+ Once you have initially installed Slackware from the installer screen,
+ you have the facilities to install, remove or upgrade new or existing
+ packages using the package management tools:
+
+ installpkg <package.t?z> - install 'package.t?z'
+
+ upgradepkg <package.t?z> - upgrade existing 'package' with the
+ new version specified at the command line.
+
+ removepkg <package> - remove specified package
+
+
+ Whilst the Slackware package system is not especially feature rich,
+ Slackware's user base (including me) like it because it is simple.
+ If we want to write our own package utilities then we can easily do so
+ by examining the existing package tools and querying and amending the
+ package database (text files).
+
+ 2.0.1 Format of official Slackware packages
+ -------------------------------------
+
+ In Slackware 8.1 and up, each package file is named as follows:
+
+ packagename-version-architecture-buildnumber.t?z
+
+ 2.0.1.1 Package names
+ -------------
+
+ The package name is either the name of the single program
+ or the name of a collection of utilities that fall under
+ a single umbrella.
+ For example:
+ autoconf-2.54-noarch-1.tgz
+
+ That is the name of the autoconf package I have on my
+ Slackware 8.1 box.
+ 'autoconf' is the name of the the entire collection of
+ binaries and associated documents that are extracted from
+ the autoconf source distribution archive.
+
+ However, if we consider another example:
+ tcpip-0.17-i386-15.tgz
+
+ There is no single piece of software called 'tcpip'.
+ This package contains a number of associated utilities
+ written by different authors but bundled into one single
+ 'package'.
+
+ 2.0.1.2 Version numbers
+ ---------------
+
+ If the package is the name of a particular piece of software
+ such as 'autoconf' from the example above, then the version
+ number represents the version number that its authors distribute.
+
+ If the package is a 'bundle' such as 'tcpip' then the version
+ number increases as and when you add a new piece of software to
+ the package, or upgrade a particular piece of major software
+ contained within the package.
+ For example, with 'tcpip' above, the 0.17 refers to the version of
+ Linux Netkit. However, there are other programs included within
+ the Slackware tcpip package that are not part of 'Linux netkit'.
+
+ 2.0.1.3 Architectures
+ -------------
+
+ The architecture is just that -- it tells you which architecture
+ the package is for.
+
+ The current values in use are:
+
+ ----- [ Official Slackware architecures ]
+
+ noarch - Architecture independent files such as config files
+ i386 - packages for the x86 (Slackware v8.1 & 9)
+ i486 - packages for the x86 (Slackware 9.1+)
+ i586 - packages for the x86
+ s390 - packages for the IBM s/390 mainframe
+
+ Note: Whilst Slackware v10 is primarily built for i486, you may
+ find that there are some packages whose architecture versions
+ are higher than i486. This is for two reasons:
+
+ [a] There is no source for the package - it is a repackaged
+ binary distribution (SUN's j2sdk is an example).
+
+ [b] The package is not required or otherwise not suitable for
+ earlier revisions of the architecture (this is especially
+ the true with ARM and SPARC).
+
+ ----- [ Unofficial ports of Slackware ]
+
+ arm - packages for the ARM architecture
+ sparc - packages for the SUN Sparc architecture
+ powerpc - packages for the PowerPC architecture
+
+ 2.0.1.4 Build numbers
+ -------------
+
+ A build number suplements the version number and is changed
+ when the *package maintainer* makes a change to the package but
+ the original source code and version number remains the same.
+
+ For example, I build a package of 'foo' version 0.14 for the
+ sparc. I have never built this package before, thus it becomes
+ foo-0.14-sparc-1.tgz
+ However, I realise that I haven't configured
+ the 'bar' parameter correctly in /etc/foo.conf. I fix it
+ and re-build the package. The package is now named
+ foo-0.14-sparc-2.tgz
+
+
+ 2.1 An overview of the creation of official Slackware packages
+ -----------------------------------------------------------
+
+ This section gives a brief introduction of the two methods of
+ used when building the official Slackware packages.
+
+
+ 2.1.1 'Clean' and 'dirty' builds
+ --------------------------
+
+ I am assuming the reader has some experience with Makefiles
+ and has compiled and installed a number of software packages.
+
+ 2.1.1.1 Clean builds
+ ------------
+
+ I term a 'clean' package one where you can specify a variable
+ to the 'make install' which contains the directory you wish to install
+ the package in, rather than installing it properly over the root file system.
+ For example:
+ # ./configure --prefix=/usr --sysconfdir=/etc
+ # make
+ # make install DESTDIR=/tmp/package-foo
+
+ With a 'Makefile' that honours the DESTDIR variable, this will
+ install the whole package into /tmp/package-foo. This directory
+ effectively is the 'root' directory '/' as far as the Makefile
+ is concerned.
+
+ From here you can use the Slackware 'makepkg' program and build
+ a package.
+
+ This is by far the safest and most preferred method by all
+ users that make packages.
+
+ You will find that DESTDIR is called prefix, TOPDIR and other names;
+ you need to examine the Makefile scripts in order to determine whether
+ it contains this functionality and if it does, then discover what
+ the variable name is.
+
+ 2.1.1.2 Dirty builds
+ ------------
+
+ A 'dirty' build is the name I give to source distribution archives
+ whose Makefile scripts do not have any provisioning to install
+ in an alternate directory other than root.
+
+ For these type of builds, you will typically do:
+ # ./configure --prefix=/usr --sysconfdir=/etc
+ # make
+ # make install
+
+ The package will then be installed on the root filesystem.
+
+ So how do you know what files were installed where and
+ even if you did, how do you pull all these files together in order
+ to run makepkg over them ?
+ That's the purpose of slacktrack! :-)
+
+
+ 2.1.1 SlackBuild and .build scripts
+ -----------------------------
+
+ Slackware has a number of packages by a great number of authors.
+ Some of the packages's source Makefiles honour a DESTDIR type
+ variable but others do not.
+
+ 2.1.1.2 SlackBuild scripts
+ ------------------
+
+ SlackBuild scripts can be 'interesting'. They are
+ scripts that install into a 'clean' environment (usually /tmp).
+
+ Some of the scripts follow the make install DESTDIR=
+ style, if the Makefile permits.
+
+ Others have a 'framework' or 'controller tarball' which is
+ a _packagename.t?z (note the prefixing underscore).
+
+ The SlackBuild script uses the Slackware 'explodepkg' script
+ to untar the contents of the _.t?z into the /tmp-package<name>
+ directory.
+ Slackbuild then runs 'make' to compile the binaries
+ and libraries, but then uses the 'cat' program such as:
+ # cat foobar > /tmp/package-foobar/usr/bin/foobar
+
+ By using 'cat', the *new* version of 'foobar' retains
+ the original permissions and ownerships that are in the
+ controller tar ball.
+
+ However, you may be wondering how, if the package does not
+ have a facility to install into somewhere other than root,
+ do we get the file names and permissions for the
+ controller _.t?z in the first place.
+ The answer is simple:
+ [a] find all files on the file system and dump to a file.
+ [b] compile and install the software
+ [c] find all files on the file system and compare the file
+ produced by the first 'find'. After a little pruning, you
+ have the list of files for the controller tar ball.
+
+
+ 2.1.1.3 .build scripts
+ ---------------
+
+ For those software distributions whose Makefile does not hounour
+ the DESTDIR type system, there are Slackware's .build scripts.
+
+ These scripts literally ./configure ; make ; make install
+ and copy docs and other goodies into the root file system.
+
+ One of the problems with these scripts is that they are
+ often incomplete -- they build and install the package but
+ do not gzip the man pages or strip the binaries and libraries;
+ this is done manually afterwards.
+
+ *These* are the scripts that slacktrack and altertrack were
+ written for.
+
+ * Note: Whilst some software's Makefiles may appear to honour
+ the DESTDIR variable, the Makefile *may* be broken which can
+ result in files missing or corrupt within your new package.
+ For example: I built Apache v2.0.48 and built a package using
+ make install DESTDIR. However, a problem exists in that some of
+ the Perl scripts it installs have *temporary build paths*
+ hard coded into the resulting scripts.
+ This is why you *may* find a .build instead of a SlackBuild
+ script within Slackware's source tree.
+
+ However, the primary reason is because the build script just
+ hasn't been updated to make use of DESTDIR. *
+
+
+ 2.2 slacktrack in the scheme of things
+ ----------------------------------
+
+ I follow Slackware-current quite closly. Often I want to
+ 'back port' a -current package to an earlier release of Slackware .
+ I can't simply upgrade with -current's package because it was
+ compiled for a newer GLIBC than Slackware 8.1's, for example.
+ For packages that use a 'clean' 'SlackBuild' script, this is
+ an easy job -- I simply run 'SlackBuild' on an 8.1 box.
+
+ However, for .build scripts, I needed a way of building packages
+ using Slackware's .build scripts.
+
+ I found a great program called 'CheckInstall' which fulfilled most of my
+ requirements. However, as time went on and I started building
+ more of Slackware's packages and writing my own build scripts, I found
+ that checkinstall was lacking some features that I required.
+ At this time I was also considering porting Slackware to run on
+ the ARM architecture and helping with the Splack (Slackware on SPARC project),
+ and therefore wanted a robust program that can deal with every .build script
+ I threw at it, and if it can't handle it, I needed to be able to make modifications.
+ The easiest way of doing this was to write my own script; thus
+ 'slacktrack' was born.
+
+ slacktrack is based on the *idea* behind 'CheckInstall', but uses
+ only my own code (and contributions from other users), and only contains
+ Slackware-specific facilities -- it can not build Debian or RedHat packages
+ and never will.
+
+ slacktrack does not have all of the facilities of CheckInstall either,
+ but then these are not required for building Slackware packages
+ from .build scripts.
+
+ Also, slacktrack only works with 'official' Slackware directory locations
+ and /usr/local.
+ For example, if your make install installs binaries in /opt/packagename/bin
+ and the man pages in anywhere other than /usr/man or /usr/local/man, then
+ slacktrack's relevant options (eg stripping libs, bins, gzman) will
+ not detect them.
+
+
+ 2.2.1 Using slacktrack with official .build scripts
+ ---------------------------------------------
+
+ Building a replicar Slackware package from a .build script is
+ typically fairly trivial with slacktrack.
+
+ If we wanted to build slackware-9.1's elvis, we could do
+ # cd slackware/slackware-9.1/source/a/elvis
+ # chmod 755 elvis.build
+ # slacktrack -jefkzp "elvis-2.1-i386-2.tgz" ./elvis.build
+
+ The resulting package (by default) be stored in
+ /tmp/built-slackwarepackages/
+
+ As already mentioned, some of the Slackware .build scripts
+ are incomplete with regard to gzipping man pages, stripping binaries
+ and so on -- fetchmail.build is one of them.
+ Therefore you can specify various options to slacktrack that
+ will take care of this for you.
+ The options in the example above :
+ j - compress libraries
+ e - chown root:bin /bin,/sbin,/usr/bin,/usr/sbin directories
+ f - chown root:bin files in the binary dirs listed above
+ k - strip binaries found in the binary dirs listed above
+ z - gzip man pages
+ p - the resulting Slackware package .t?z name
+
+ The way I re-create the packages is to build a 'trackbuild' script that
+ launches slacktrack with the required options and the name
+ of the Slackware .build script. You can find examples of such
+ scripts within the docs directory after installing slacktrack:
+
+ /usr/doc/slacktrack*/buildscript-examples/
+
+ You will also find that in Slackware versions 12.0 and upwards,
+ the .build scripts are accompanied by 'trackbuild' scripts because
+ slacktrack is used by the Slackware team to produce the packages.
+
+ 2.2.2 Writing your own .build scripts for slacktrack
+ ----------------------------------------------
+
+ There isn't any specific that you have to do to write a build
+ script for use with slacktrack -- the script name specified to
+ slacktrack doesn't even have to be a script - it can be a binary -
+ as long as it is executable, it is acceptable.
+
+ You can see some of my own build scripts that I have written
+ for use with slacktrack by looking in the documents directory
+ after installing slacktrack:
+
+ /usr/doc/slacktrack*/buildscript-examples/
+
+ 2.2.2.1 Making use of slacktrack's environment variables
+ -------------------------------------------------
+
+ slacktrack exports two environment variables:
+ SLACKTRACKFAKEROOT and SLACKTRACKSCRATCHDIR
+
+ SLACKTRACKFAKEROOT:
+ ````````````````````
+
+ The purpose of this to allow your .build script to access the
+ 'root' directory of the resulting package.
+ There are two scenarios where you may want to use this
+ variable:
+
+ 1. During the build:
+
+ The reason you may wish to do this is to pre-populate
+ the package with files that you may not wish to place directly
+ onto the root filesystem.
+ The package contents will only appear inside this directory
+ once your build script has finished, and slacktrack has determined
+ what to put into this directory.
+
+ In previous slacktrack versions which used a pseudo-root filesystem
+ (where the package contents were populated *during* the build), this
+ made sense, but in slacktrack v2.00, it is unlikely that you'd want
+ to use this vairable from the build script.
+
+ 2. Post-build -- cleanups after the build:
+
+ The most likely use of this script is to perform package cleanup
+ tasks after the build. This is mainly to perform tasks that slacktrack
+ does not handle itself - such as deleting known files/directories that
+ creep into packages (due to a system daemon such as CUPS), or setting
+ some file and directory permissions.
+
+ An example post build script is below.
+ A post build script can be specified by slacktrack's -R option:
+
+ # Incase you had CUPS running:
+ rm -rf etc/cups etc/printcap
+ # crond:
+ rm -rf var/spool/cron
+ rmdir var/spool
+
+ # perllocal.pod files don't belong in packages.
+ # SGMLSPL creates this:
+ find . -name perllocal.pod -print0 | xargs -0 rm -f
+
+ # Some doc dirs have attracted setuid.
+ # We don't need setuid for anything in this package:
+ chmod -R a-s .
+
+
+ SLACKTRACKSCRATCHDIR:
+ `````````````````````
+
+ The purpose of this variable is to provide some temporary
+ space to untar your source archives and so on. slacktrack
+ will manage the creation and deletion of this directory.
+
+ For example:
+ # cd ${SLACKTRACKSCRATCHDIR}
+ # tar zxvvf ${ORIGPATH}/source/foobar.tar.gz
+
+ You can see in some of the example 'non-slackware' scripts
+ how I have used these variables
+
+
+3.0 slacktrack in operation
+ -----------------------
+
+ The basic event flow is as follows:
+
+ [1] Parse command line arguments
+ -> Check they are OK, if not bomb out
+ [2] Launch the supplied build script
+ [3] Run any specified functions (eg gzman, strip bins, chowns) over the
+ package 'root' directory and contents
+ [4] Run Slackware's 'makepkg' program over the package contents
+ [5] Move the .t?z to the specified build store path
+ [6] Scan for any hard links that may be in the package
+ -> If any found, alert the user on screen and also
+ log to a packagename.hardlinks.log file in the build store path
+
+ The slacktrack shell script is fairly simple and well commented; it should be
+ relatively easy for anybody who understands bash to quickly comprehend what
+ is happening and why.
+
+ 3.1 How slacktrack finds which files have been installed
+ ----------------------------------------------------
+
+ In order to track the files and directories that have been installed
+ or changed, slacktrack follows this ordered process:
+
+ [1] Scans a pre-set list of directories on the filesystem and
+ logs the contents.
+ [2] Launches build script which installs the package onto the
+ filesystem
+ [3] Scans the filesystem once again
+ [4] Compares the differences in the before and after snapshots
+ [5] Copies the differences into a 'package root directory' and
+ runs makepkg over them.
+
+ In slacktrack version 1, we used 'installwatch' which overloaded
+ some of glibc's filesystem calls, creating new files and directories
+ into a pseudo-root filesystem, avoiding the need to install onto
+ the host's filesystem (and also allowing to build as a non-root user).
+ However, this library is ill maintained and even in the early days
+ when it worked well, it still had problems and workarounds were required.
+
+4.0 Example scripts
+ ---------------
+
+ Included with the slacktrack binary distribution are a number of example
+ scripts that hopefully should provide you with a basis of how to use slacktrack
+ to build from 'dirty' source distributions.
+
+ The examples are bundled in the documentation directory available
+ after installing slacktrack:
+
+ /usr/doc/slacktrack*/buildscript-examples/
+
+
+5.0 Known problems/features of slacktrack
+ -------------------------------------
+
+ Current problems:
+
+ [1] slacktrack doesn't have sufficient error checking.
+
+ I'm in two minds about *where* to put error checking, you see.
+ Do I go no further if the supplied build script exits with a non-zero
+ value?
+ No, that's a bad idea because what if you didn't write the build script?
+ it might be one of these qmail type binary installer programs that
+ you can't (easily without hacking the source) fix. The author may
+ have their own systems, and the program exits with a value that their
+ own controller script recognises as non-failure.
+
+ What should I do if makepkg has failed? You see it on screen
+ and in the logs anyway, plus makepkg is one of the last things
+ that slacktrack calls -- how can you recover?
+
+ That said, version 1.03 now has some error handling. There's still
+ not enough though, imo.
+
+ [2] No support for a default config file
+
+ I don't see the point. Each .build script is different
+ and although I typically use the same options to slacktrack
+ for most of my build scripts, I don't see why I'd need a
+ config file just to save 4 bytes or so in a trackbuild wrapper :-)
+
+
+6.0 New features
+ -------------
+
+ See the TODO file in the docs directory.
+
+ If you have any specific features you would like to see
+ included, or have found any bugs, please
+ email me <mozes@slackware.com>
+
+7.0 Licence
+ -------
+
+ slacktrack is distributed under BSD Licence.