Slay::Maker(3p)

NAME
Slay::Maker - An perl make engine using perl code for rules

STATUS
Beta. Pretty stable, though underdocumented.

DESCRIPTION
Slay::Maker is a make engine that uses perl declaration syntax for rules, including regular expressions for targets and anonymous subs for targets, dependencies, and actions.

This allows you to tightly integrate a make engine in an application and to exercise a large amount of control over the make process, taking full advantage of Perl code at any point in the make cycle.

RULE SYNTAX
The rulebase syntax is:

   [ @targets1, ':', @dependencies1, '=', @actions1, { option => val } ],
   [ @targets2, ':', @dependencies2, '=', @actions2, { option => val } ],
   [ @targets3, ':', @dependencies3, '=', @actions3, { option => val } ],
   [ @targets4, ':', @dependencies4, '=', @actions4, { option => val } ],
   ...

Each item in any of the three arrays may be a literal string or a subroutine (CODE) reference. A literal string is pretty much the same as using a literal string in a regular makefile. You may also use regular expression ('Regexp') references ("qr/.../") in @targets and the $1, $2, ... variables inside strings in @dependencies:

   [ qr/(.*).pm/, ':', "$1.pm.in", '=', sub { ... } ],

Subroutine references are evaluated as lazily as possible when the make is being run, so any CODE refs in @targets will be called each time a make is run, CODE refs in @dependencies will only be called if a target matches, and CODE refs in @actions are only called if the rule is fired.

  TARGET SUBS
** NOT IMPLEMENTED QUITE YET **. It's simple to do, just haven't needed it yet.

Aside from strings and Regexps, you will be able to use CODE refs in the target list. These are called each time the rule is evaluated, which will usually happen once per target or dependency being checked when the make is run.

A target sub declaration might look like:

   sub {
      my ( $maker ) = @_ ;
      ...
      return @targets;
   },

(if target subs were implemented already).

  DEPENDENCIES
Dependencies may be strings or CODE references. Plain strings have $1, $2, ... interpolation done on them (remember to \ escape the $1, etc.).

CODE refs will be called if the target matches and must return a possibly empty list of strings containing the names of dependencies. Variable interpolation will not be done on the returned strings. That would be obscene.

A dependency sub declaration might look like:

   sub {
      my ( $maker, $target, $matches ) = @_ ;
      ...
      return @dependencies ;
   },

where

   $maker    refers to the Slay::Maker (or subclass) being run
   $target   is the target that matched (in the event of multiple targets)
   $matches  is an ARRAY of the values extracted from $1, $2, etc.

.

  ACTIONS
If an $action is a plain string, it's passed to "sh -c '$string'". If it's an ARRAY ref, it's run without interference from or benefit of a shell (see "run" in IPC::Run for details). If it's a CODE ref, it's called.

An action sub declaration might look like:

   sub {
      my ( $maker, $target, $deps, $matches ) = @_ ;
      ...
      return @dependencies ;
   },

where

   $maker    refers to the Slay::Maker (or subclass) being run
   $target   is the target that matched (in the event of multiple targets)
   $deps     is an ARRAY of the expanded dependencies.  There's no way
             of telling which are freshly rebuilt, but you can track that
             yourself in the action rules of the dependencies, if you
             like.
   $matches  is an ARRAY of the values extracted from $1, $2, etc.

TARGET BACKUPS
A target may be moved off to a backup location before it is rebuilt, so that it may be restored if rebuilding fails. This is also used for the optional restoration of modification times described below.

Restoration needs to be done manually by calling the "restore" method, and you can call the "backup" method, too.

The "backup" method will be called automatically if modification time restoration is enabled for a target.

MODIFICATION TIME RESTORATION
One unusual option is that a target file's modification time can be restored if it is unchanged after being updated. This can be useful when checking files out of a repository: the files' mod times won't be affected if their contents haven't changed.

This can be done by a (fast but possibly misleading) check for a change in file size or by executing 'diff --brief' between a target's backup and it's new version. Other methods, such as hashing or block-by-block binary comparison will be implemented in the future if needed.

This is controlled by the "detect_no_diffs" option passed to the base class constructor:

   my $self = Slay::Maker->new( ..., options => { detect_no_diffs => 1 } ) ;

and can be passed as an option to any rule.

AN EXAMPLE
Here's a real example, which will have to stand in for documentation until further notice. If you need more, mail me (barries@slaysys.com) and get me to do something productive for a change.

This is a subclass that compiles a set of builtin rules at module compilation time. It declares a method for spawning the cvs command first, then builds some rules.

   package Safari::Cvs::Make ;

   use Class::Std ;
   use base qw( Slay::Maker ) ;

   use strict ;
   use IPC::Run ;

   sub cvs {
      my Safari::Cvs::Make $maker = shift ;

      my $stdout_to ;
      if ( $_[-1] =~ /^\s*>(.*)/ ) {
         $stdout_to = $1 ;
         pop ;
      }

      my $cvs_out ;
      run [ qw( cvs -q -l -z9 ), @_ ], \undef, \$cvs_out or die $! ;

      return $cvs_out ;
   }

   my $builtin_rules = Safari::Make->compile_rules(
      [  'meta/cvs_modules',
         '=', sub {   ## The action that occurs when the rule fires.
            ## We could just do the cvs co -c here, but many pservers don't
            ## have that implemented.  so, check out the modules file and
            ## parse it.
            my ( $maker, $target ) = @_ ;
            $maker->cvs( qw( checkout -p CVSROOT/modules ), ">$target" ) ;
         },
      ],
      [ 'update',
         ':' => sub {
            my ( $maker ) = @_ ;

            my $context = $maker->{CONTEXT} ;

            my @modules ;

            my %args = $context->request->args ;
            if ( defined $args{modules} ) {
               @modules = split( ',', $args{modules} ) ;
            }
            elsif ( defined $args{module} ) {
               @modules = $args{module} ;
            }
            else {
               eval {
                  ## A recursive make
                  $maker->make( 'meta/cvs_modules', { force => 1 } ) ;
               } ;
               if ( $@ ) {
                  warn $@ ;
               }

               if ( ! open( F, "<meta/cvs_modules" ) ) {
                  warn "$!: meta/cvs_modules, globbing" ;
                  @modules = map {
                     s{^meta/HEAD/}{}; $_
                  } grep {
                     -d $_
                  } glob( 'meta/HEAD/*' ) ;
               }
               else {
                  my $line ;
                  my %modules ;
                  while (<F>) {
                     next if /^\s*#|^\s*$/ ;
                     chomp ;
                     $line .= $_ ;
                     redo if $line =~ s{\\$}{} ;
                     $modules{$1} = 1 if $line =~ m{^\s*(\S+)} ;
                     $line = '' ;
                  }
                  close F ;
                  @modules = sort keys %modules ;
               }
            }

            debug 'modules', \@modules ;
            die "No modules found\n" unless @modules ;
            return map { "source/HEAD/$_/CVS/" } @modules ;
         },
         '=' => sub {
            my ( $maker, $target, $deps ) = @_ ;

            my @dirs = map { s{/CVS/}{} ; $_ } @$deps ;

            ## We go ahead and update after creating modules for a couple of
            ## reasons:
            ## 1. It's rare that we've just checked out a module
            ## 2. It's simpler this way
            ## 3. If we just created a *big* module, then we might need to
            ## update anyway.

            ## We set $logs{$filename} = 1 if we must replace the current log file,
            ## or = 0 to just ensure that the log file is fetched.
            my %logs ;
            my $force_all ;

            my $cwd = cwd() ;
            for ( @dirs ) {
               chdir $_ or die "$!: $_" ;
               my $module = $_ ;
               $module =~ s{.*/}{} ;
               ## -P: Prune empty dirs
               ## -d: Add new dirs that we don't have
               for ( split /^/m, $maker->cvs( qw( update -d -P ) ) ) {
                  chomp ;
                  if ( /^[UP]\s+(.*)/ ) {
                     $logs{"meta/HEAD/$module/$1.log"} = 1 ;
                  }
                  elsif ( /^\?\s+(.*)/ ) {
                     my $log_file = "meta/HEAD/$module/$1.log" ;
                     eval {
                        rmtree( [ $log_file ] ) ;
                     } ;
                     warn "Error removing ;$log_file'" if $@ ;
                  }
                  else {
                     warn "Unexpected CVS file mode: $_" ;
                  }
               }
               chdir $cwd or die "$!: $cwd" ;
            }

            for ( sort keys %logs ) {
               $maker->make(
                  $_,
                  {
                     force => $force_all || $logs{$_}
                  }
               ) ;
            }

         },
         {
            force => 1,   # Always remake this target
         }
      ],
   ) ;

METHODS

new Constructor.

      my $rules = [
         # ...
      ] ;
      my $maker = Slay::Maker->new( { } );
      my $maker = Slay::Maker->new( { rules => $rules } ) ;
      my $maker = Slay::Maker->new( { rules => $rules,
                                      options => { option => 1 } } ) ;

    options (which can also be defined on a per-rule basis) are:

    auto_create_dirs
        Creates directories that targets will be created in before executing a target's actions.

    detect_no_diffs
        Copy the target before executing a rule, then restore the original modification and access times if the contents of the target are the same after the rule.

    detect_no_size_change
        Look for changes in the size of a file after executing a rule and restore the original modification and access times if it changes.

    force
        Always remake target, even if it does not appear to be out of date

    Warning: options are not checked for spelling errors.

    Options may be passed to new(), make(), build_queue(), and to rules themselves. Options passed to make() or build_queue() take precedence over rules' options, and rules' options take precedence over those passed to new().

add_rules
    Add rules (compiled or not) to the rule base.

atime
    This returns the atime of a file, reading from the stat cache if possible.

build_queue
    Builds a new queue of rules to be exec()ed to make a target

builtin_rules
    Returns [] by default. This is provided so that subclasses may overload it to provide sets of rules. This is called by new() before adding any rules passed to new().

canonpath
    Cleans up the path, much like File::Spec::Unix::canonpath(), but also removes name/.. constructs.

chdir
    Calls system chdir(), die()s on failure, and uses the parameter as the current directory. The last is the main reason for this sub: if you chdir() to a symbolic link, then we want to know the symbolic directory, not the real one returned by cwd().

check_targets
    Checks targets and adds them to queue if need be. Does not integrate Slay::Maker options: this is left to the caller (usually the original build_queue() call).

clear_caches
    Clears the stat cache, so the filesystem must be reexamined. Only needed if Slay::Maker is being called repetitively.

clear_stat
    Clears the stat cache for a given path, so the next stat() on that path will read the filesystem.

compile_rules
    Returns a rulebase compiled from the arguments. Rules that are already compiled are passed through unchanged. This is a class method, so

       Slay::Maker->compile_rules(
          [ 'a', [qw( b c )], 'cat b c > a' ],
          ...
       ) ;

    can be used to compile a rulebase once at startup

backup
    Copies a file so that it can be restored later or checked for changes.

    If the target will only ever be replaced by the make, then it will not be altered in-place, and the "move" option may be passed:

       $maker->backup( $target, { move => 1 } ) ;

    If the target is an file which always changes size when it is changed, you may pass the "stat_only" option:

       $maker->backup( $target, { stat_only => 1 } ) ;

    The return value can be passed to restore(), target_unchanged(), and remove_backup().

cwd Returns the current working directory, from the cache if that is possible.

e   Returns true if the file exists, but uses the stat_cache if possible.

exec_queue
    Executes the queued commands.

find_rule
    Given a target, finds a rule.

get_rule_info(<target>)
    Given a target that has already been processed with "check_targets", either directly or indirectly, returns the rule that is used to produce the target, a reference to an array of dependencies of the target, and a reference to an array of the matches. Thus, you would call

        ($rule, $deps, $matches) = get_rule_info($target);

    Returns an undefined rule if there is no processed rule to produce the target.

make
    Makes one or more target(s) if it is out of date. Throws exceptions if the make fails. May partially make targets.

make_level
    Returns 0 if make() has not been called (well, actually, if recurse_in() has not been called). Returns number of recursive calls otherwise, so this is equal to 1 when making something but not recursing.

mtime
    This returns the mtime of a file, reading from the stat cache if possible.

options
    Sets / gets a reference to the options hash.

push
    Adds a ( target, rule ) tuple to the exec queue. Will not add the same target twice.

recurse_in
    Sets up for a recursive make. Called automatically by make() if make() is already running.

recurse_out
    Restored after a recursive make. Called automatically by make() if make() is already running.

queue_size
    Number of rules that need to be made.

remove_backup
       my $backup = $maker->backup( $target ) ;
       ## ...
       $maker->remove_backup(
          $backup,
          {
             restore_if_unchanged => 1,
             deps                 => \@deps,
          }
       ) ;

    Removes a backup of the target created with backup_target().

replace_rules
    Replaces the rule for a target (or targets). The targets passed in must exactly match those of the rule to be replaced.

restore
       my $backup = $maker->backup( $target, { move => 1 } ) ;
       ## Try to recreate target, setting $error on error
       $maker->restore( $backup )
          if $error ;
       $maker->restore( $backup, { deps => \@deps } )
          if ! $error && $maker->target_unchanged( $backup ) ;
       $maker->remove_backup( $backup ) ;

    Note that you only need this in case of an error. You can pass the restore_if_unchanged => 1 and deps => \@deps options to remove_backup().

    When backup() has been called, it's return value can be passed to restore_target() to restore the original target, timestamps and all.

    NOTE: restoring a target that's not changed is likely to cuase it to be remade every time once a dependency's timestamp becomes more recent. The "deps" option allows the timestamps to be set to the newest of the original timestamps and the dependencies' timestamps. This should not be done if there was an error generating the file.

rules
    Gets or replaces the rules list

size
    This returns the size of a file, reading from the stat cache if possible.

stat
    Looks in the stat cache for the stat() results for a path. If not found, fills the cache. The cache is shared between all instances of this class, and may be cleared using clear_stat_cache().

target_unchanged
    Takes the result of backup_target() and checks to see if the target has been changed or removed.

TODO

•   Propagate effects of restored timestamps.

    If a target has it's timestamps restored as a result of detecting no change (see options detect_no_size_change and detect_no_diffs), then there may well be no need to actually execute later rules.

    One way to do this is to re-check the mtime dependencies when rebuilding. Another is to subscribe later items in the queue to earlier items and have the earlier items set a flag that tells the later items to go ahead and execute. Items could flag themselves to execute regardless, which we might want to do if a dependency is not present when make is run.

•   Don't really call diff(1) for detect_no_diffs

AUTHOR
Barrie Slaymaker <barries@slaysys.com>

LICENSE
Copyright 2000, R. Barrie Slaymaker, Jr., All Rights Reserved.

That being said, do what you will with this code, it's completely free.

Please let me know of any improvements so I can have the option of folding them back in to the original.