Salt file.recurse source file not found (file encoding issue)

So I was running this:

/var/www/jj-web-1-www.jj5.net-sixsigma:
  file.recurse:
    - clean: True
    - user: root
    - group: root
    - dir_mode: 755
    - file_mode: 644
    - source: salt://inst/mediawiki-1.29
    - require:
      - pkg: apache2

And getting an error like this:

----------
          ID: /var/www/jj-web-1-www.jj5.net-sixsigma
    Function: file.recurse
      Result: False
     Comment: #### /var/www/jj-web-1-www.jj5.net-sixsigma/vendor/james-heinrich/getid3/getid3/module.audio.ac3.php ####
              Source file 'salt://inst/mediawiki-1.29/vendor/james-heinrich/getid3/getid3/module.audio.ac3.php?saltenv=base' not found

              #### /var/www/jj-web-1-www.jj5.net-sixsigma/vendor/james-heinrich/getid3/getid3/module.audio-video.mpeg.php ####
              Source file 'salt://inst/mediawiki-1.29/vendor/james-heinrich/getid3/getid3/module.audio-video.mpeg.php?saltenv=base' not found
     Started: 14:27:18.352264
    Duration: 134735.945 ms
     Changes:
----------

The issue was that the source files mentioned weren’t in UTF-8 format. To convert the files I ran, e.g.:

$ iconv -f WINDOWS-1252 -t UTF-8//TRANSLIT < module.audio-video.mpeg.php.bak > module.audio-video.mpeg.php

(Actually I couldn’t get the ‘iconv’ command to work so I edited manually in Vim)

Debugging JINJA in Salt Stack

So I was trying to figure out how to report the template source file within the template itself and I found myself wanting to know what was available in the JINJA globals for a Salt Stack JINJA template.

Turns out you use the show_full_context() function, like this:

Context is: {{ show_full_context() }}

You can parse the output to see what top-level items are available. In my case I found the ‘source’ setting for the JINJA template in use.

Hot tip: if you’re using Vim to inspect the show_full_context() output, try using ‘%’ or ‘]}’ to move between matching braces.

Bonus: while I was learning about reporting the JINJA context I discovered that you can call Salt functions from JINJA templates, like this:

# The following two function calls are equivalent.
{{ salt['cmd.run']('whoami') }}
{{ salt.cmd.run('whoami') }}

Installing Subversion on Mac OS X with WebDAV support (serf library)

cd ~/Development/svn-install
wget http://prdownloads.sourceforge.net/scons/scons-2.3.4.tar.gz
wget http://mirror.ventraip.net.au/apache/apr/apr-1.5.1.tar.gz
wget http://mirror.ventraip.net.au/apache/apr/apr-util-1.5.4.tar.gz
wget http://mirror.ventraip.net.au/apache/subversion/subversion-1.8.10.tar.gz
tar xzf scons-2.3.4.tar.gz
tar xzf apr-1.5.1.tar.gz 
tar xzf apr-util-1.5.4.tar.gz
tar xzf subversion-1.8.10.tar.gz
cd scons-2.3.4
sudo python setup.py install
cd ../apr-1.5.1
./configure --prefix=/usr/local
make
sudo make install
cd ../apr-util-1.5.4
./configure --with-apr=/usr/local --prefix=/usr/local
make
sudo make install
cd ../subversion-1.8.10
sh get-deps.sh serf
cd serf
scons APR=/usr/local APU=/usr/local OPENSSL=/usr/local PREFIX=/usr/local
sudo scons PREFIX=/usr/local install
cd ..
./configure --with-openssl --with-serf=/usr/local --prefix=/usr/local
make
sudo make install

Nuff said!

Oh, wait… there’s a problem with misconfigured CA certificates…

cd ~/Development/svn-install
wget https://distfiles.macports.org/MacPorts/MacPorts-2.3.3-10.10-Yosemite.pkg

Run the MackPorts*.pkg…

xcode-select --install

Then…

sudo /opt/local/bin/port install curl-ca-bundle

Then…

sudo -s
cd /System/Library/OpenSSL/certs/
ln -s /opt/local/etc/openssl/cert.pem cert.pem
cd /usr/local/etc/openssl/certs
ln -s /opt/local/etc/openssl/cert.pem cert.pem

Testing:

openssl s_client -connect www.progclub.org:443 -CApath /opt/local/etc/openssl/

Using MacPorts OpenSSL:

sudo port install openssl
cd ~/Development/svn-install/subversion-1.8.10/serf/
scons APR=/usr/local APU=/usr/local OPENSSL=/opt/local PREFIX=/usr/local
sudo scons PREFIX=/usr/local install

Done!

Changing parent’s directory from a subshell

I was trying to figure out how to have a child process change the current directory of a parent process. Turns out you can’t quite do that. One thing I guess you could do is export a global variable and then have the parent check that when the child returns, but I decided on another approach. What I do is write a file to ~/bin/errdir that contains a single line in the format:

  pushd "/path/to/directory"

I can then source this as a command file after my script has run to change directory into the directory where the last erroneous file was encountered with the command:

  $ . errdir

When I’m finished I can just type ‘popd’ to return myself to wherever I was before I changed into the error directory.