So some of the people I know who read this already know… but it looks like I’ll be moving the family over to Silicon Valley in the next couple of months.

I’ve accepted a position at Google HQ in Mountain View (huzzah!) doing Mac/Linux sysadmin stuff, which means several things…

We’re finally getting married tomorrow, as we’ll be doing it on an E-3 visa, and the US doesn’t recognize de facto/common law relationships… I think our daughter is the most excited person involved… :) The process doesn’t look too bad, and luckily Andrew Pollock has put up some useful info about his experiences going through the same situation. I’ll add some more posts here if anything has changed…

We’re not doing much of a ceremony, but we’ll combine something with our farewell party here in Sydney before we leave.

We’ll also be looking for someone to replace my current position as Senior Technical Officer at the College of Fine Arts, UNSW here in Sydney, Australia. Here’s the post I sent to macos-x-server@lists.apple.com about it. Please contact me if you think you might be interested…

This probably means I’ll be a bit light on technical articles for this blog and afp548.com for a while, but hopefully that won’t be a permanent state of affairs…

I’m not sure we can afford to live in Mountain View itself, and at this stage it looks like we might be looking around Sunnyvale, Santa Clara or San Jose. Useful suggestions for an ex-pat moving to the US and looking for housing would be much appreciated… :) Someone pointed me to housingmaps.com which is rather cool, but it doesn’t seem like the local newspapers have sites like domain.com.au and realestate.com.au from poking around…

So I was getting annoyed that I couldn't claim my weblog via Technorati without faking it, as blojsom requires a trailing slash at the end of the URL to go to the correct blog, and Technorati strips the slash from your url when you submit it.

Googling around brought up an entry by andwest talking about using UrlRewriteFilter to solve the problem. He'd set it up for a single case, but I wanted to get a general solution going, as I've got nearly 2000 blogs running on the staff and student blog servers here.

After faffing around a bit in Regular Expressions land with Java Pattern Matching, I ended up with this rule that seems to do the job happily enough.

        <rule>
        <from>^/blog/([a-z]*)([^/])$</from>
        <to type="redirect">/blojsom/blog/$1$2/</to>
        </rule>

This is with UrlRewrite just installed into the blojsom context, not the whole server, so it will rewrite an url like:

http://staff.cofa.unsw.edu.au/blojsom/blog/nigelkersten

to:

http://staff.cofa.unsw.edu.au/blojsom/blog/nigelkersten/

We mustn't have tested this thoroughly enough when we were forced into rapidly deploying 10.4 on our labs of new iMac G5s, but we started having major problems with DropBoxes on our 10.3.9 AFP servers.

Network users logging in on a 10.4.x client were unable to write files to DropBoxes, although when using a 10.3.x client, there were no problems.

After tearing my hair out over it for a little while, I posted to the very useful MacEnterprise.org mailing list, and Greg Neagle came up with a solution for me.

On your AFP server, do:

serveradmin set afp:noNetworkUsers = y

This moves you from mapped permissions (where everything appears to be owned by the user and permissions granted are effective rather than real) to showing the real underlying FS perms, as long as the RecordName and UniqueID of the logged in user match those reported by the client.

Apparently this may not be an optimal solution if you have a mix of network and local users, but as all of our users are either straight up network users, or mobile accounts, this has worked seamlessly for us.

So we've been investigating a new CMS for the COFA website, and we've settled upon OpenCMS, which is quite excellent so far, especially considering the fact that it's open source and free.

The only problem is that we needed LDAP authentication which isn't a part of the distribution.

Ah, but there is a commerical LDAP plugin available from Alkacon, as part of their OpenCMS Enterprise Extensions package, which also includes a Transaction Manager, an Accelerator and a VFS Doctor.

The problem I quickly ran into is that while user authentication works happily, groups are another matter. The solution .. ?

The LDAP Connector expects group records to look like:

dn: ou=cmsgroup,ou=groups,dc=mydomain,dc=edu,dc=au
cn: cmsgroup
objectClass: posixGroup
objectClass: extensibleObject
uniqueMember: uid=michelfoucault,cn=users,dc=mydomain,dc=edu,dc=au
uniqueMember: uid= gottlobfrege,cn=users,dc=mydomain,dc=edu,dc=au

which isn't how Apple do it with Open Directory, for their groups look like:

dn: cn=cmsgroup,cn=groups,dc=mydomain,dc=edu,dc=au
cn: cmsgroup
objectClass: posixGroup
objectClass: extensibleObject
memberUid: michelfoucault
memberUid: gottlobfrege

It's not the container types (cn, ou) that are the problem, for the LDAP connector is more than capable of allowing that, it's the way that members are specified that is the problem.

Apple simply list the username as a 'memberUid' property, but the LDAP connector expects things to be the way lots of other LDAP implementations do it, where the full user record is specified, not just the username.

After I was thinking about this for a while, Mike Bartosh suggested something that I really should have picked up straight away.

Simply construct another branch of group records in my LDAP directory that contains the information in the appropriate format.

I decided to take all Open Directory groups whose name started with “cms” and construct the appropriate group record in another branch.

So this script will take all such groups, and construct the appropriate group records in “ou=cmsgroups,dc=mydomain,dc=edu,dc=au”. I made them as ou rather than cn, mainly because I've gotten into the habit of making non-Apple LDAP data as that container type. It's much of a muchness really.

#!/bin/sh

ldiffile="/tmp/cmsgroups.ldif"
odserver="odserver.mydomain.edu.au"
odadmin="myodadmin"
odpass="myodpassword"
ldapdomain="dc=mydomain,dc=edu,dc=au"

rm -f $ldiffile

groups=$(/usr/bin/ldapsearch -LLL -w $odpass -h $odserver \
        -D "cn=$odadmin,$ldapdomain" \
        -b "cn=groups,$ldapdomain" \
        '(&(objectclass=posixgroup)(cn=cms*))' \
        cn 2> /dev/null | /usr/bin/grep "cn:" | /usr/bin/sed 's|cn: ||g'  )

echo "dn: ou=cmsgroups,$ldapdomain" >> $ldiffile
echo "objectClass: top" >> $ldiffile
echo "objectclass: organizationalUnit" >> $ldiffile
echo "ou: cmsgroups" >> $ldiffile
echo "" >> $ldiffile

for group in $groups
do
        echo "dn: ou=$group,ou=cmsgroups,$ldapdomain" >> $ldiffile
        echo "cn: $group" >> $ldiffile
        echo "objectClass: posixGroup" >> $ldiffile
        echo "objectClass: extensibleObject" >> $ldiffile
        members=$(/usr/bin/ldapsearch -LLL -w $odpass -h $odserver \
                          -D "cn=$odadmin,$ldapdomain" \
                          -b "cn=$group,cn=groups,$ldapdomain" \
                          memberUid 2> /dev/null | /usr/bin/grep memberUid | /usr/bin/sed 's|memberUid: ||g');
        for member in $members
        do
                echo "uniqueMember: uid=$member,cn=users,$ldapdomain" >> $ldiffile
        done
        echo "" >> $ldiffile
done

/usr/bin/ldapdelete -w $odpass -D "cn=$odadmin,$ldapdomain" -r "ou=cmsgroups,$ldapdomain"
/usr/bin/ldapadd -w $odpass -D "cn=$odadmin,$ldapdomain" -f $ldiffile

So with this running each half an hour, my other support staff can still use Workgroup Manager to create groups with the prefix “cms”, add users to them, and the script will create the necessary groups and membership info as required, in a completely separate LDAP branch.

The OpenCMS LDAP connector can then happily read the data out of the “ou=cmsgroups” branch, rather than “cn=groups”. This is easily configured in the LDAP connector config setup.

Using this approach, we can maintain the integrity of group records the way Apple create and use them, and yet we can still get the flexibility of being able to use LDAP plugins that may expect group records to be in a different format.

There are a bunch of annoying bugs to do with Office versions earlier than 2004 if you have network home directories.

Actually, the bugs will hit you even if you have your home directory on a different local partition to the one that Office is running from.

This simple script gives you a workaround.

cd /Volumes/Homes/staff
for user in $(ls /Volumes/Homes/staff | grep -e "^[a-z]")
do
 /usr/bin/chflags nouchg $user/Library/Preferences/Microsoft/Office\ Registration\ Cache\ X
 /bin/rm -f $user/Library/Preferences/Microsoft/Office\ Registration\ Cache\ X
 /bin/mkdir -p $user/Library/Preferences/Microsoft
 /usr/bin/touch $user/Library/Preferences/Microsoft/Office\ Registration\ Cache\ X
 /usr/sbin/chown -R $user $user/Library/Preferences/Microsoft
 /usr/bin/chflags uchg $user/Library/Preferences/Microsoft/Office\ Registration\ Cache\ X
done

Basically you just need to trash the file “Office Registration Cache X”, touch it to create an empty file, make sure the user is the owner of it, and then lock it.

The reason I have my loop look like:

for user in $(ls /Volumes/Homes/staff | grep -e "^[a-z]")

is because all my staff home directories start with a lowercase letter, and various group sharepoints exist at the same level, but I start them with either an uppercase letter or an underscore.

The grep command just makes sure that only folders starting with a lowercase letter are looped, which also gets rid of those annoying .DS_Store etc files.

So another thing people seemed keen on is my system for setting up appropriate WebDAV realms for iCal publishing automatically for personal web pages on Mac OS X Server. This kind of follows on from the previous entry, but you could easily put the two scripts together.

It assumes:

• ~/Sites for personal web pages.
• The actual file server for home directories is also running Apache for personal web pages. If you instead have personal web pages on another server, you'll have to use the mount in /Network/Servers rather than the local path. If anyone needs to do this instead, post a comment and I'll modify the script for you.
• ~/Sites/calendars already exists, and is where iCal publishes to. See the previous entry on setting up WebDAV areas for how to automatically do this. You could roll both scripts together, but I like to set the ~/Sites permissions more freuently than I set up new WebDAV realms.
• Each user is the only person who can publish to this location.

Again, you'll notice that I'm doing this by looping over the contents of the network home directory sharepoint, rather than collecting users from Directory Services. You might also notice that the actual path that Apache serves out is the mount in /Network/Servers, not the local filesystem path. I like having my staff webpages being served from the same server as my staff home directories, but this isn't a requirement.

To start with, I set up a comment line in the relevant apache config file that will mark off where the WebDAV realms start. You'll find this file in /etc/httpd/sites/, and I use a comment like this:

# Start automatic realms for staff calendars

Then I have this script run, which basically just looks for the comment line, trashes everything below it, and generates the right syntax for WebDAV realms to be appended.

Then I run apachectl configtest, and check the exit status to make sure that Apache is happy with the config. If it isn't happy (which hasn't happened yet, but better safe than sorry), it copies the original config file back, and emails me to let me know that something has gone wrong. If it is happy, it restarts apache with the new config file.

In this example, /Volumes/Homes/staff is the sharepoint, and this is mounted at /Network/Servers/server.domain/Volumes/Homes/staff, so change those to match your setup.

#!/bin/bash

CONFFILE="/etc/httpd/sites/0000_any_443_your.server.apache.config.file.conf"

STARTNUM=$(cat $CONFFILE  |grep -n "# Start automatic realms for staff calendars" | sed 's|:.*||g')
STARTNUM=$(expr $STARTNUM - 1 )

rm -f /tmp/site.conf
head -n $STARTNUM $CONFFILE > /tmp/site.conf
echo "# Start automatic realms for staff calendars" >> /tmp/site.conf

cd /Volumes/Homes/staff
for user in $(ls | grep -e "^[a-z]")
do
        echo "<Directory \"/Network/Servers/server.domain/Volumes/Homes/staff/$user/Sites/calendars/\">" >> /tmp/site.conf
        echo "<IfModule mod_dav.c>" >> /tmp/site.conf
        echo "DAV On" >> /tmp/site.conf
        echo "</IfModule>" >> /tmp/site.conf
        echo "  AuthName \"$user\"" >> /tmp/site.conf
        echo "  <Limit PUT DELETE PROPPATCH MKCOL COPY MOVE LOCK UNLOCK>" >> /tmp/site.conf
        echo "          Require user $user" >> /tmp/site.conf
        echo "  </Limit>" >> /tmp/site.conf
        echo "  AuthType Digest" >> /tmp/site.conf
        echo "</Directory>" >> /tmp/site.conf
done

echo "# End automatic staff realms" >> /tmp/site.conf
echo "</VirtualHost>" >> /tmp/site.conf

mkdir -p /etc/httpd/backup
cp $CONFFILE /etc/httpd/backup
cp /tmp/site.conf $CONFFILE

/usr/sbin/apachectl configtest

if [ $? -ne 0 ]
then
        mv /etc/httpd/backup/*.conf /etc/httpd/sites
        echo "Something went wrong with the auto realm script" | mail -s "ERROR: auto realm" your.email@your.domain
else
        /usr/sbin/apachectl graceful
fi      

So as always, buyer beware, backup your stuff before you try it, and make sure you get the paths right, as that's where Apache serves personal web pages from.

More requests from X-World…

We use ~/Sites for personal web pages. I don't really like the way that by default other users can browse this folder via AFP, and you can't fix this by modifying the User Template folder, so I run this script every night on all network home directories.

This same script also makes sure that each user has a folder ~/Sites/calendars that has the correct permissions such that they can use iCal to publish to this location.

You'll notice that I'm doing this by looping over the contents of a network home directory share point. I do this because my users are treated differently based upon where their home directories are, and not all sharepoints contain users that will automatically get a ~/Sites/calendars folder.

You could modify this to instead loop over all users from Directory Services, but that's unnecessary overhead in my case. Post a comment if you'd like to see what that script would look like.

#!/bin/bash
#
cd /Volumes/path/to/staff/homes
for user in $(ls | grep -e "^[a-z]")
do
        chown -R $user:www $user/Sites
        chmod -R 750 $user/Sites
        mkdir -p $user/Sites/calendars
        chown -R www:www $user/Sites/calendars
        chmod -R 770 $user/Sites/calendars
done

NB: The next entry I'll have up tomorrow or maybe tonight will describe how I'm automatically creating Apache WebDAV realms for the ~/Sites/calendars location, as that's the bit people seem to be mailing me about…

A few people at X-World seemed interested in this simple one-liner, which will recursively unlock files from the command line.


find /Volumes/Transit -flags +uchg -print0 | xargs -0 chflags nouchg


The command above would look at the path /Volumes/Transit, recursively find any locked files, and unlock them. The '-print0' and '-0' bits will deal with any files or folders that have spaces in them.

NB: It seems like the 'rm' command has changed in Tiger such that if you use 'rm -Rf' with elevated privileges, it will automatically unlock the files. This isn't the case in Panther, which is why I initially needed to do this on a temporary sharepoint that gets wiped nightly.

So I've been getting a lot of emails asking for RapidMetaBlog to support other blogging APIs, and so I'm thinking about doing a v2.0 that supports blog servers that don't do the MetaWeblog API. If you have any other suggestions for features or interface, feel free to leave some comments.

Been nice to see a bunch of links come up in Google about it, and since I put it on the Apple Dashboard Downloads page, there has been a lot more people downloading it.

So far I'm definitely planning to add:

• Authenticated Proxy support.
• non-Aqua scrollbars.
• WYSIWIG like editing. (would people prefer an option to compose in straight HTML?)
• Movable Type support

and I'm thinking about adding stuff like:

• editing posts.
• deleting posts.

but I'm not sure whether that starts to go against the whole point of Dashboard… I have been working on a fully featured blog editor like the very very excellent ecto, but with more Tiger Weblog Server specific features, like the ability to actually have the preview look just like your actual post.

Anyway, if people have suggestions, feel free to fire away… I've trackbacked a couple of sites that are commenting on this and some who are still hosting old versions, so hopefully people will get up to date on their versions and some of you who are users will get this…

My aim is always going to be to support blojsom and thus Tiger Weblog Server primarily. A few people have asked for the ability to post an entry in multiple categories, but blojsom doesn't really work that way, so that's probably unlikely to happen, plus I can't think of a nice interface way of doing that.

Oh, and I would like to do localized versions, as there are a few Germans and Japanese using it. If anyone wants to help out with that, just send me an email.

The chicken was ok, but the salad needed some work.

not too bad though.