<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>mind the explanatory gap &#187; Snippets</title>
	<atom:link href="http://explanatorygap.net/category/snippets/feed/" rel="self" type="application/rss+xml" />
	<link>http://explanatorygap.net</link>
	<description>many a slip 'twixt mind and lip...</description>
	<lastBuildDate>Tue, 11 May 2010 19:03:25 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.0</generator>
		<item>
		<title>python subprocess over shell</title>
		<link>http://explanatorygap.net/2010/05/10/python-subprocess-over-shell/</link>
		<comments>http://explanatorygap.net/2010/05/10/python-subprocess-over-shell/#comments</comments>
		<pubDate>Mon, 10 May 2010 04:16:54 +0000</pubDate>
		<dc:creator>nigel kersten</dc:creator>
				<category><![CDATA[*nix]]></category>
		<category><![CDATA[Snippets]]></category>
		<category><![CDATA[python]]></category>
		<category><![CDATA[ramblings]]></category>

		<guid isPermaLink="false">http://explanatorygap.net/?p=256</guid>
		<description><![CDATA[One of my common gripes is when people struggle with complicated shell scripts that would be much simpler in a scripting language like Python, Ruby or Perl. I used to abuse PHP for this, but saw the light. If you&#8217;re talking about replacing shell scripts, all of these are pretty much equivalent, but I don&#8217;t [...]]]></description>
			<content:encoded><![CDATA[<p>One of my common gripes is when people struggle with complicated shell scripts that would be much simpler in a scripting language like Python, Ruby or Perl. I used to abuse PHP for this, but saw the light.</p>
<p>If you&#8217;re talking about replacing shell scripts, all of these are pretty much equivalent, but I don&#8217;t really do Perl for no particular reason, I&#8217;m a big fan of Python for general purpose work just because of the rich module system, and at least <a href="http://python.org/about/quotes/">&#8220;dozens of engineers&#8221; use it at work.</a>.</p>
<p>The reason we generally write shell scripts is because we want to execute a bunch of external processes in an automated way. </p>
<p>In this area you can whip something basic up fastest in shell, yes, but at some point you&#8217;re going to have to repay that <a href="http://explanatorygap.net/2009/10/02/the-complexity-debt/">technical debt</a> if you need to get past a certain point.</p>
<p>Besides, <a href="http://mywiki.wooledge.org/BashPitfalls">it&#8217;s not like shell scripting always stays simple and easy</a>&#8230; and the overhead of moving to a more powerful language isn&#8217;t that huge.</p>
<p>If I&#8217;m sure the scope of a script will be small, or I don&#8217;t have the option of moving to structured data format for input and output with external commands, or I don&#8217;t have to futz around with arrays, I&#8217;ll stick with bash.</p>
<p>But, if I want to work with dictionary objects or talk in a protocol like LDAP or model things as objects, or need complex handling and passing around of stdin/stdout/exit statuses, or know some module handles lots of edge cases for me, I&#8217;ll move to Python or Ruby.  I quite like both, but feel that Python is more utilitarian, and simply due to whitespace enforcement and <a href="http://www.logilab.org/857">extensive linters</a> is a good fit for code that may need to be picked up and understood quickly by a co-worker.</p>
<p>When it comes to getting started with Python, I still suggest <a href="http://www.diveintopython.org/">Dive into Python</a> for people. Just flipping through Chapters 1-3 equips you with an awful lot.</p>
<p>Anyway, some people think Python is hard, but it&#8217;s not really. I think of Python as being extremely utilitarian, which makes it a great fit for sysadmin work.</p>
<p>Someone posted on the <a href="http://www.macenterprise.org">MacEnterprise</a> list a question about working out what PPD was in use by each printer on the system in OS X, and someone gave a good shell example using the usual suspects of for, grep and awk.</p>
<p>There&#8217;s nothing really wrong with doing this, but I&#8217;ve come to distrust parsing non-structured output if I need to keep this solution working across many multiple versions of operating systems, or even between OS X major versions given Apple&#8217;s history.  One of the big advantages of moving to Python is being able to parse and manipulate property lists, which can get really painful in shell.  You end up writing out lots of temp files or giving up on dealing with error conditions. </p>
<p>Anyway, so lets see what it&#8217;s like executing a command in Python with the subprocess module. We&#8217;re asking System Profiler for printer info, and telling it to return the output in XML plist format.</p>
<div class="dean_ch" style="white-space: wrap;"><span class="co1">#!/usr/bin/python</span></p>
<p><span class="kw1">import</span> <span class="kw3">subprocess</span></p>
<p>
command = <span class="br0">&#91;</span><span class="st0">&#8216;system_profiler&#8217;</span>, <span class="st0">&#8216;-xml&#8217;</span>, <span class="st0">&#8216;SPPrintersDataType&#8217;</span><span class="br0">&#93;</span><br />
task = <span class="kw3">subprocess</span>.<span class="me1">Popen</span><span class="br0">&#40;</span>command,<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; stdout=<span class="kw3">subprocess</span>.<span class="me1">PIPE</span>,<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; stderr=<span class="kw3">subprocess</span>.<span class="me1">PIPE</span><span class="br0">&#41;</span></p>
<p><span class="br0">&#40;</span>stdout, stderr<span class="br0">&#41;</span> = task.<span class="me1">communicate</span><span class="br0">&#40;</span><span class="br0">&#41;</span><br />
<span class="kw1">print</span> stdout</div>
<p>To skim through those lines, we&#8217;re</p>
<ul>
<li>setting the python shebang</li>
<li>importing the subprocess module</li>
<li>defining a list called &#8216;command&#8217; to store the command we want to run</li>
<li>creating a subprocess Popen object to run our command called &#8216;task&#8217;</li>
<li>setting standard out and standard error to go to our own pipes</li>
<li>getting standard out and standard error from the task.</li>
<li>printing standard out</li>
</ul>
<p>This really isn&#8217;t much work, and if you really only wanted this functionality, there are other <a href="http://docs.python.org/library/subprocess.html#convenience-functions">convenience functions</a> that you can use to make this even shorter, or write your own convenience function.</p>
<p>But we have all sorts of options now.</p>
<p>We can send standard input to the task:</p>
<div class="dean_ch" style="white-space: wrap;"><span class="br0">&#40;</span>stdout, stderr<span class="br0">&#41;</span> = task.<span class="me1">communicate</span><span class="br0">&#40;</span>stdin<span class="br0">&#41;</span></div>
<p>We can ask what the exit status is easily:</p>
<div class="dean_ch" style="white-space: wrap;">status = task.<span class="me1">returncode</span></div>
<p>and if we get None we know the process hasn&#8217;t terminated yet.</p>
<p>And if we want to do something a bit more complicated we can search through the structured data plist output easily like:</p>
<div class="dean_ch" style="white-space: wrap;"><span class="co1">#!/usr/bin/python</span></p>
<p><span class="kw1">import</span> plistlib<br />
<span class="kw1">import</span> <span class="kw3">subprocess</span></p>
<p>
command = <span class="br0">&#91;</span><span class="st0">&#8216;system_profiler&#8217;</span>, <span class="st0">&#8216;-xml&#8217;</span>, <span class="st0">&#8216;SPPrintersDataType&#8217;</span><span class="br0">&#93;</span><br />
task = <span class="kw3">subprocess</span>.<span class="me1">Popen</span><span class="br0">&#40;</span>command,<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; stdout=<span class="kw3">subprocess</span>.<span class="me1">PIPE</span>,<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; stderr=<span class="kw3">subprocess</span>.<span class="me1">PIPE</span><span class="br0">&#41;</span></p>
<p><span class="br0">&#40;</span>stdout, stderr<span class="br0">&#41;</span> = task.<span class="me1">communicate</span><span class="br0">&#40;</span><span class="br0">&#41;</span><br />
printers = plistlib.<span class="me1">readPlistFromString</span><span class="br0">&#40;</span>stdout<span class="br0">&#41;</span><br />
printers = printers<span class="br0">&#91;</span><span class="nu0">0</span><span class="br0">&#93;</span><span class="br0">&#91;</span><span class="st0">&#8216;_items&#8217;</span><span class="br0">&#93;</span></p>
<p><span class="kw1">for</span> printer <span class="kw1">in</span> printers:<br />
&nbsp; <span class="kw1">print</span> <span class="st0">&#8216;Name: &#8216;</span> + printer<span class="br0">&#91;</span><span class="st0">&#8216;_name&#8217;</span><span class="br0">&#93;</span><br />
&nbsp; <span class="kw1">print</span> <span class="st0">&#8216;PPD: &#8216;</span> + printer<span class="br0">&#91;</span><span class="st0">&#8216;ppd&#8217;</span><span class="br0">&#93;</span><br />
&nbsp;</div>
<p>This is easier to extend than the standard for/grep/awk/sed equivalents tend to be, and much easier for a co-worker to pick up and understand. It comes close to documenting itself, just needs some comments about the structure of Apple&#8217;s output, and some try/except blocks in case the output is malformed or does change.</p>
]]></content:encoded>
			<wfw:commentRss>http://explanatorygap.net/2010/05/10/python-subprocess-over-shell/feed/</wfw:commentRss>
		<slash:comments>3</slash:comments>
		</item>
		<item>
		<title>useful screen bash function</title>
		<link>http://explanatorygap.net/2008/05/14/useful-screen-bash-function/</link>
		<comments>http://explanatorygap.net/2008/05/14/useful-screen-bash-function/#comments</comments>
		<pubDate>Tue, 13 May 2008 14:52:22 +0000</pubDate>
		<dc:creator>nigel kersten</dc:creator>
				<category><![CDATA[*nix]]></category>
		<category><![CDATA[Snippets]]></category>

		<guid isPermaLink="false">http://explanatorygap.net/2008/05/14/useful-screen-bash-function/</guid>
		<description><![CDATA[So I pretty much live in screen permanently these days, and there are a gazillion awesome introductions out there to screen that describe the amazing advantages of screen. One thing that has been a minor niggle has been the clunky way you pick which screen session you wish to reattach to if you have multiple [...]]]></description>
			<content:encoded><![CDATA[<p>So I pretty much live in <a href="x-man-page://screen">screen</a> permanently these days, and there are a gazillion awesome introductions out there to screen that describe the amazing advantages of screen.</p>
<p>One thing that has been a minor niggle has been the clunky way you pick which screen session you wish to reattach to if you have multiple sessions active, so I whipped up this bash function that I&#8217;ve been finding really useful.</p>
<pre>
function scr() {

  if [ ! $1 ]; then

    screen -D -R

  else

    i=1

    for j in $(screen -list | awk --posix '/^[[:space:]]*[[:digit:]]{2,}.*$/{print $1}'); do

      screens[${i}]=$j

      let i=i+1

    done

    # test if integer

    if ! [ $1 -eq $1 2&gt; /dev/null ]; then

      echo "You must supply an integer as the argument"

    elif [ $1 -eq 0 ]; then

      echo "screens list count starts at 1, not zero."

    elif [[ ${1} -gt ${#screens[@]} ]]; then

      echo "only ${#screens[@]} screens exist"

      screen -list

    else

      screen -D -R ${screens[$1]}

    fi

  fi

}</pre>
<p>So you can basically go <code>scr</code> to reattach to a single screen session, <code>scr 1</code> to reattach to the first, <code>scr 2</code> to reattach to the second, etc etc.</p>
<p>I&#8217;m kind of expecting someone to pipe up now and point out a much easier way&#8230; :)</p>
]]></content:encoded>
			<wfw:commentRss>http://explanatorygap.net/2008/05/14/useful-screen-bash-function/feed/</wfw:commentRss>
		<slash:comments>8</slash:comments>
		</item>
		<item>
		<title>Creating stable and unstable branches with Apple&#8217;s Software Update Server.</title>
		<link>http://explanatorygap.net/2007/08/23/creating-stable-and-unstable-branches-with-apples-software-update-server/</link>
		<comments>http://explanatorygap.net/2007/08/23/creating-stable-and-unstable-branches-with-apples-software-update-server/#comments</comments>
		<pubDate>Wed, 22 Aug 2007 15:18:28 +0000</pubDate>
		<dc:creator>nigel kersten</dc:creator>
				<category><![CDATA[Snippets]]></category>

		<guid isPermaLink="false">http://explanatorygap.net/2007/08/23/creating-stable-and-unstable-branches-with-apples-software-update-server/</guid>
		<description><![CDATA[Creating stable and unstable branches with Apple&#8217;s Software Update Server @ afp548.com]]></description>
			<content:encoded><![CDATA[<p><a href="http://www.afp548.com/article.php?story=20070821170126681">Creating stable and unstable branches with Apple&#8217;s Software Update Server @ afp548.com</a></p>
]]></content:encoded>
			<wfw:commentRss>http://explanatorygap.net/2007/08/23/creating-stable-and-unstable-branches-with-apples-software-update-server/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Heading to MacWorld&#8230;</title>
		<link>http://explanatorygap.net/2006/12/22/heading-to-macworld/</link>
		<comments>http://explanatorygap.net/2006/12/22/heading-to-macworld/#comments</comments>
		<pubDate>Thu, 21 Dec 2006 21:44:18 +0000</pubDate>
		<dc:creator>nigel kersten</dc:creator>
				<category><![CDATA[Snippets]]></category>
		<category><![CDATA[ramblings]]></category>

		<guid isPermaLink="false">http://explanatorygap.net/2006/12/22/heading-to-macworld/</guid>
		<description><![CDATA[Just as a quick note&#8230; After spending four (maybe five if England can hold out that long) glorious days watching Shane Warne&#8217;s last Test match at the SCG, I&#8217;ll be flying straight out to San Francisco to do some presentations at MacWorld. It&#8217;s always good to put faces to names, so if you&#8217;re around make [...]]]></description>
			<content:encoded><![CDATA[<p>Just as a quick note&#8230; </p>
<p>After spending four (maybe five if England can hold out that long) <b>glorious</b> days watching <A href="http://aus.cricinfo.com/australia/content/story/273509.html">Shane Warne&#8217;s last Test match</a> at the SCG, I&#8217;ll be flying straight out to San Francisco to <a href="http://www.macworldexpo.com/live/20/events/20SFO07A/SN363608/bio//CMONYA00BHX1">do some presentations at MacWorld</a>.</p>
<p>It&#8217;s always good to put faces to names, so if you&#8217;re around make sure to say hi, and even more importantly, make sure you make it to the <a href="http://www.afp548.com/article.php?story=20061106113640489&#038;query=macworld">AFP548 Kick-Off Session at Macworld</a>.</p>
<p>We can play the <a href="http://www.afp548.com/comment.php?mode=view&#038;cid=6614">drinking game every time Joel&#8217;s voice does that squeaky Jay Leno thing&#8230;</a> :)</p>
]]></content:encoded>
			<wfw:commentRss>http://explanatorygap.net/2006/12/22/heading-to-macworld/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Typos can be fatal&#8230;</title>
		<link>http://explanatorygap.net/2006/10/18/typos-can-be-fatal/</link>
		<comments>http://explanatorygap.net/2006/10/18/typos-can-be-fatal/#comments</comments>
		<pubDate>Tue, 17 Oct 2006 21:49:46 +0000</pubDate>
		<dc:creator>nigel kersten</dc:creator>
				<category><![CDATA[Snippets]]></category>
		<category><![CDATA[ramblings]]></category>

		<guid isPermaLink="false">http://explanatorygap.net/2006/10/18/typos-can-be-fatal/</guid>
		<description><![CDATA[So last night I was playing around with the MBR on an external drive&#8230; and completely without thinking managed to wipe out my laptop. /dev/rdisk1 Nigel, not /dev/rdisk0 &#8230;. bleagh. Anyway, it&#8217;s made me realise just how much I love HomeSync, as 95% of everything that matters has been syncing up to the server&#8230; and [...]]]></description>
			<content:encoded><![CDATA[<p>So last night I was playing around with the MBR on an external drive&#8230; and completely without thinking managed to wipe out my laptop.</p>
<p><b>/dev/rdisk1</b> Nigel, not /dev/rdisk0 &#8230;.</p>
<p>bleagh.</p>
<p>Anyway, it&#8217;s made me realise just how much I love HomeSync, as 95% of everything that matters has been syncing up to the server&#8230; and as I&#8217;ve had two requests now for a post on Mobile Accounts and HomeSync best practices, this is as good a time as any&#8230; working on it now while watching OS X and Win XP reinstall&#8230;. </p>
]]></content:encoded>
			<wfw:commentRss>http://explanatorygap.net/2006/10/18/typos-can-be-fatal/feed/</wfw:commentRss>
		<slash:comments>5</slash:comments>
		</item>
		<item>
		<title>Airport Extreme base stations do not do Radius Accounting.</title>
		<link>http://explanatorygap.net/2005/10/13/airport-extreme-base-stations-do-not-do-radius-accounting/</link>
		<comments>http://explanatorygap.net/2005/10/13/airport-extreme-base-stations-do-not-do-radius-accounting/#comments</comments>
		<pubDate>Thu, 13 Oct 2005 23:35:25 +0000</pubDate>
		<dc:creator>nigel kersten</dc:creator>
				<category><![CDATA[Snippets]]></category>

		<guid isPermaLink="false">http://explanatorygap.net/2005/10/13/airport-extreme-base-stations-do-not-do-radius-accounting/</guid>
		<description><![CDATA[This is mainly for any weary souls treading the same Google path that I have been lately&#8230; The Airport Extreme (and I assume the Express as well) base stations do not do Radius Accounting. They do happily work with Radius Authentication, but not Accounting. This means that if you&#8217;re looking for a solution for network [...]]]></description>
			<content:encoded><![CDATA[<p>This is mainly for any weary souls treading the same Google path that I have been lately&#8230;</p>
<p>The Airport Extreme (and I assume the Express as well) base stations do not do Radius <b>Accounting</b>.  They do happily work with Radius <b>Authentication</b>, but not Accounting.<br />
<br />
This means that if you&#8217;re looking for a solution for network quotas on a wireless network, you&#8217;ll have to find some other way.</p>
]]></content:encoded>
			<wfw:commentRss>http://explanatorygap.net/2005/10/13/airport-extreme-base-stations-do-not-do-radius-accounting/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

<!-- Dynamic Page Served (once) in 0.993 seconds -->
