So after I hit somewhere in excess of 850 COFA Student blogs on blojsom (we're well over 1000 now, be interesting to see how many of them actually start using it…), I started running into problems with the amount of memory Tomcat was using by default.

On your standard Tomcat install, you'd increase the memory settings with something like:

JAVA_OPTS="-Xms128M -Xmx512M"

to set the minimum and maximum memory footprint, and you'd do this at your 'catalina.sh' file, which on OS X Server is in /Library/Tomcat/bin/catalina.sh

The problem here is that this file isn't what is actually called when you do stuff like:

serveradmin start appserver

only if you call catalina.sh directly like:

/Library/Tomcat/bin/catalina.sh start

which means you need to faff around if you have to reboot the server.

Anyway, some digging around found this file, which is what serveradmin uses to start/stop the appserver service.

/usr/share/servermgrd/bundles/servermgr_appserver.bundle/Contents/Resources/run.sh

which essentially contains the same stuff as catalina.sh if you're just running Tomcat in standalone mode and not using JBoss at all.

This will probably get wiped out in software updates, but at least I don't have to come up with some clunky startup item which stops the appserver service and starts it again via catalina.sh.

Be nice if we could customize these settings somewhere without having to worry about this though.

So not only does dsimportexport not actually export as well as import, but it doesn't even import properly.

It turns out that you can't use the -O overwrite flag and expect to get the same behaviour as when you import exactly the same file using Workgroup Manager. sheesh…

so anyway, as I was using php anyway for the automation scripts, it turns out it is far faster to just use php ldap functions than even thwacking dscl around to put the data in yourself, and much easier if you're already constructing arrays from SQL queries.

So say we're wanting to completely replace the membership list for a group called 'thisgroup'.
It will look something like this:

<?php

$od['server'] = 'your.odmaster.com';
$od['port'] = '389';
$od['v3'] = '1'; // By default OS X Server uses LDAPv3.
$od['tls'] = '1';   // if you're using TLS
$od['user'] = 'uid=your-od-admin-username,cn=users,dc=mythical,dc=edu,dc=au';
$od['pass'] = 'your-od-admin-password';
$od['base'] = 'dc=mythical,dc=edu,dc=au';

$groupname = 'testgroup';
$groupmembers = array('andyclark', 'jerryfodor', 'danieldenntt', 'gottlobfrege');

$update_connection = ldap_connect($od['server'], $od[port])
   or die("Could not connect to LDAP server.");
if ($od['v3'])
{
   ldap_set_option($update_connection, LDAP_OPT_PROTOCOL_VERSION, 3);
}
if ($od['tls'])
{
   ldap_start_tls($update_connection);
}
$update_bind = ldap_bind($update_connection, $od['user'], $od['pass']);
if (! $update_bind)
{
   die("LDAP bind failed");
}
else
{
   $update_array[memberuid] = $groupmembers;
   // In this example, $update_dn will be "cn=testgroup, cn=groups, dc=mythical, dc=edu, dc=au"
   $update_dn = "cn=$groupname, cn=groups, $od['base']";
   $update_results = ldap_mod_replace($update_connection, $update_dn, $update_array);
   if ($update_results)
   {
      echo "update successful";
   }
   else
   {
      echo "Unable to update group";
   }

}

ldap_close($update_connection);
?> 

So really you want some sanity checking around the values you're whacking into LDAP. Remember, that this kind of stuff is faster than dsimportexport because it doesn't check the values…. you're basically doing exactly the same thing as enabling Inspector in Workgroup Manager and putting your own values in. You'll want to make sure your 'shortnames' in the $groupmembers array don't contain spaces, evil characters etc.

It's fast though. Several orders of magnitude faster than dealing with a broken disimportexport…

(edit): I should note that you really want a unique array for work like this, but there are some caveats.

You can't just do something like:


$update_array[memberuid] = array_unique($groupmembers);

due to the way php makes unique arrays by not messing with the index.

You instead need to make sure you have a continuous index for the unique array, like this:


$update_array[memberuid] = array_merge (array_unique($groupmembers));


You wouldn't expect it, but array_merge, when given only one array as an argument, will reindex the array in a continuous way.

You know, I love php, but I wish they'd get some more consistency in their function names….

I was banging my head against a brick wall trying to get aspell support working with PHP.

The php module make stage kept failing with errors like:

ld: ext/pspell/pspell.o illegal reference to symbol:
_aspell_error_number defined in indirectly referenced
dynamic library /usr/local/lib/libaspell.15.dylib

Turns out this is a known bug to do with aspell and php under OS X.

The fix is quite simple.

Before you attempt to run “./configure –whatever” for php, edit the 'configure' file inside the php source folder, and look for the line:

LIBS="-lpspell $LIBS

Change it so that it looks like:

LIBS="-laspell -lpspell $LIBS"

and bob's your uncle.

I've beeing using groff to nicely format man pages to plain text like so:

groff -mandoc -Tascii  /usr/share/man/man1/ps.1 > man_ps.txt

but the problem is that you need to find the actual man page file, which adds another step to the process. This tip landed in my inbox the other day from the Unix Guru Universe Hot Tip of the Day list.

To convert a man page to text, simply do:

man ps | col -b > man_ps.txt

Much easier than messing around with groff and the mandoc options.

groff is still more useful for doing stuff like exporting to straight html like:

groff -mandoc -Thtml  /usr/share/man/man1/ps.1

You may have heard about the upcoming integration of blojsom in Tiger Server when it comes out.
I really like blojsom, and have been running it for a while now. It is far easier to administer for multi-user and multi-blog setups than anything else I've experimented with, and offers some awesome features.

Well, recently an LDAP authentication plugin has been contributed to blojsom, which means that right now you can install it and set up blogs for your Open Directory users such that they authenticate using their OD username and password. Installing and configuring all of this is a little bit of work, especially if you haven't set up Tomcat on your OS X Server box yet, but hopefully the step by step instructions will make it easy enough.

mkdir ~/blojsom_files
cd ~/blojsom_files
curl -O http://optusnet.dl.sourceforge.net/sourceforge/blojsom/blojsom.war
curl -O http://optusnet.dl.sourceforge.net/sourceforge/blojsom/blojsom-2.22-addons.zip
unzip blojsom-2.22-addons.zip

if you can't get to that direct download link, or want to use a SourceForge mirror closer to you, go to the following two links:

http://prdownloads.sourceforge.net/blojsom/blojsom.war?download

http://prdownloads.sourceforge.net/blojsom/blojsom-2.22-addons.zip?download

Step 1: Configuring blojsom.

sudo mkdir /Library/Tomcat/webapps/blojsom
cd /Library/Tomcat/webapps/blojsom
sudo jar xvf ~/blojsom_files/blojsom.war
sudo cp ~/blojsom_files/radeox-1.0b2.jar /Library/Tomcat/webapps/blojsom/WEB-INF/lib/
sudo cp ~/blojsom_files/blojsom-addon-core-2.22.jar /Library/Tomcat/webapps/blojsom/WEB-INF/lib/
sudo cp ~/blojsom_files/netscape-ldapjdk-4.17.jar /Library/Tomcat/webapps/blojsom/WEB-INF/lib/

Now edit the file:

/Library/Tomcat/webapps/blojsom/WEB-INF/blojsom.properties

and change:

blojsom-authorization-provider=org.blojsom.authorization.PropertiesAuthorizationProvider

to:

blojsom-authorization-provider=org.blojsom.authorization.LDAPAuthorizationProvider

and

blojsom-blog-home={user.home}/blojsom-blogs/

to:

blojsom-blog-home=/usr/local/blog-data/

Now edit

/Library/Tomcat/webapps/blojsom/WEB-INF/web.xml

insert the following text into each servlet entry that you wish to use LDAP authentication for:

<init-param>
    <param-name>blog-ldap-authorization-server</param-name>
    <param-value>DNS/IP Address of your LDAP server</param-value>
</init-param>
<init-param>
    <param-name>blog-ldap-authorization-dn</param-name>
    <param-value>DN of your Open Directory User branch</param-value>
</init-param>

ie, say your Open Directory master is “odmaster.mydomain.com”, it is more than likely that your OD User branch is something like “cn=users,dc=mydomain,dc=com”, so it would look like:

<init-param>
    <param-name>blog-ldap-authorization-server</param-name>
    <param-value>odmaster.mydomain.com</param-value>
</init-param>
<init-param>
    <param-name>blog-ldap-authorization-dn</param-name>
    <param-value>cn=users,dc=mydomain,dc=com</param-value>
</init-param>

so, at a minimum, you'll want to set up LDAP authentication for the main blojsom servlet.
Look for the lines:

    <servlet>
        <servlet-name>blojsom</servlet-name>
        <servlet-class>org.blojsom.servlet.BlojsomServlet</servlet-class>

and insert your LDAP configuration from above just after this, like so:

    <servlet>
        <servlet-name>blojsom</servlet-name>
        <servlet-class>org.blojsom.servlet.BlojsomServlet</servlet-class>
<init-param>
    <param-name>blog-ldap-authorization-server</param-name>
    <param-value>odmaster.mydomain.com</param-value>
</init-param>
<init-param>
    <param-name>blog-ldap-authorization-dn</param-name>
    <param-value>cn=users,dc=mydomain,dc=com</param-value>
</init-param>

If you're planning on using the XML-RPC API of blojsom so that you can use the Blogger, MetaWeblog and MovableType APIs, you'll also want to add the LDAP configuration to the appropriate servlet.

Look for the lines:

    <servlet>
        <servlet-name>blojsomxmlrpc</servlet-name>
        <servlet-class>org.blojsom.extension.xmlrpc.BlojsomXMLRPCServlet</servlet-class>
        

and insert your LDAP configuration just after, like so:

    <servlet>
        <servlet-name>blojsomxmlrpc</servlet-name>
        <servlet-class>org.blojsom.extension.xmlrpc.BlojsomXMLRPCServlet</servlet-class>
<init-param>
    <param-name>blog-ldap-authorization-server</param-name>
    <param-value>odmaster.mydomain.com</param-value>
</init-param>
<init-param>
    <param-name>blog-ldap-authorization-dn</param-name>
    <param-value>cn=users,dc=mydomain,dc=com</param-value>
</init-param>

If you're planning on using the Atom API then add your LDAP config to the to Atom servlet:

Look for the lines:

    <servlet>
        <servlet-name>blojsomatomapi</servlet-name>
        <servlet-class>org.blojsom.extension.atomapi.AtomAPIServlet</servlet-class>
        

and again, insert your LDAP configuration just after it, like so:

    <servlet>
        <servlet-name>blojsomatomapi</servlet-name>
        <servlet-class>org.blojsom.extension.atomapi.AtomAPIServlet</servlet-class>
<init-param>
    <param-name>blog-ldap-authorization-server</param-name>
    <param-value>odmaster.mydomain.com</param-value>
</init-param>
<init-param>
    <param-name>blog-ldap-authorization-dn</param-name>
    <param-value>cn=users,dc=mydomain,dc=com</param-value>
</init-param>

The comments servlet doesn't do authentication, so we don't need to modify it.

We need to set up an area for blog data to be stored. These are text files that contain the entries themselves.
I'm going to use /usr/local/blog-data, and set up the default blog here as well.

mkdir /usr/local/blog-data
mkdir /usr/local/blog-data/default

Step 2: Setting up a blog

now, we need to create our first blog for an LDAP user.

First, let's make sure that Tomcat has permission to update the settings for the default user, and for the resources folder, which is where images and theme data are stored.

chown -R appserver /Library/Tomcat/webapps/blojsom/WEB-INF/default
chown -R appserver /Library/Tomcat/webapps/blojsom/resources

I prefer to create my own template directory based upon the default that I can then use for all my users like so:

cd /Library/Tomcat/webapps/blojsom
sudo cp -Rp WEB-INF/default WEB-INF/default_template
sudo cp -Rp resources/default resources/default_template

Edit the file “blog.properties” in the WEB-INF/default_template folder so that it looks like this:

#
# blojsom user properties configuration file
#
blog-directory-depth=6
blog-language=en
blog-country=US
blog-name=BLOGOWNER
blog-description=BLOGOWNER - blog
blog-home=/usr/local/blog-data/BLOGOWNER
blog-base-url=http://myserver.mydomain.com/blojsom/
blog-url=http://myserver.mydomain.com/blojsom/blog/BLOGOWNER/
blog-file-extensions=.*html, .*txt, .*textile
blog-entry-meta-data-extension=.meta
blog-properties-extensions=.properties
blog-entries-display=15
blog-default-category-mapping=
blog-directory-filter=.*/CVS
blog-owner=BLOGOWNER
blog-owner-email=BLOGOWNER@mydomain.com
blog-comments-enabled=true
blog-comments-directory=.comments
blog-trackbacks-enabled=true
blog-trackbacks-directory=.trackbacks
blog-email-enabled=false
blog-file-encoding=
blog-blacklist-file=blacklist.filters
blog-xmlrpc-entry-extension=.html
blog-banned-ip-addresses=
blog-default-flavor=html
plugin-comment-autoformat=true
linear-navigation-enabled=false

#
# MetaWeblob extension properties
#
blojsom-extension-metaweblog-accepted-types=image/jpeg, image/jpg, image/gif, image/png, img

We've set things up so that we will be using Apache with mod_jk for blojsom, so that users behind strict firewalls can still view the pages. If you don't want to do this, and you just want to serve these pages out using Tomcat without Apache, change the lines:

blog-base-url=http://myserver.mydomain.com/blojsom/
blog-url=http://myserver.mydomain.com/blojsom/blog/BLOGOWNER/

so that they look like:

blog-base-url=http://myserver.mydomain.com:9006/blojsom/
blog-url=http://myserver.mydomain.com:9006/blojsom/blog/BLOGOWNER/

You may also want to change the blog-owner-email property if this isn't the format of your email addresses.

Now we're going to copy our template blog to set up a blog for a user who exists in Open Directory with the short name 'odusername'. Replace this with the username you wish to create a blog for.

sudo cp -Rp WEB-INF/default_template WEB-INF/odusername
sudo cp -Rp resources/default_template resources/odusername
sudo mkdir /usr/local/blog-data/odusername
sudo chown -R appserver /usr/local/blog-data/odusername

Edit odusername/blog.properties and replace all instances of 'BLOGOWNER' with the equivalent of 'odusername'. If you're wondering why we've done it like this, it is so that you can easily script creating a new user with a simple sed script, which I'll put at the end of this entry.

Edit odusername/authorization.properties and again, replace all instances of 'BLOGOWNER' with the equivalent of 'odusername'.

You also need to add 'odusername' to the blojsom-users property in:

/Library/Tomcat/webapps/blojsom/WEB-INF/blojsom.properties

like so:

blojsom-users=default,odusername

Step 3: Setting up mod_jk

Now, before we start up blojsom, we need to configure Apache to use mod_jk with Tomcat so that visitors can browse the blogs over port 80. If you're not planning on doing this, you can skip this whole section.

Open up Server Admin for your server, and in the Web service, go to the Modules tab. Enable the “jk_module” module.

edit the file:

/etc/httpd/httpd.conf

and change the section:

<IfModule mod_jk.c>
        JKWorkersFile /etc/httpd/workers.properties
        JKLogFile /var/log/httpd/mod_jk.log
        JKLogLevel error
        JKMount /*.jsp JBoss1
        JKMount /servlet/* JBoss1
        JKMount /examples/* JBoss1
</IfModule>

to:

<IfModule mod_jk.c>
        JKWorkersFile /etc/httpd/workers.properties
        JKLogFile /var/log/httpd/mod_jk.log
        JKLogLevel error
        JKMount /*.jsp JBoss1
        JKMount /servlet/* JBoss1
        JKMount /examples/* JBoss1
        JKMount /blojsom/* blojsomworker
</IfModule>

edit the file:

/etc/httpd/workers.properties

and make the following changes:
Change the line:

worker.list=JBoss1, JBoss2, loadbalancer

to

worker.list=JBoss1, JBoss2, loadbalancer, blojsomworker

and after the line:

worker.loadbalancer.balanced_workers=JBoss1, JBoss2

add:

# -----------------------
# Blojsom worker
# -----------------------

worker.blojsomworker.port=9007
worker.blojsomworker.host=127.0.0.1
worker.blojsomworker.type=ajp13
worker.blojsomworker.lbfactor=100

Now check your apache config:

sudo apachectl configtest

If no errors were reported (spurious [warn messages about WebObjects are ok, just make sure you have a "Syntax Ok" result returned) then do:

sudo apachectl graceful

Try starting Tomcat. If you haven't set it up before, go to the Application Server service in Server Admin, and in the General tab, choose "Tomcat only". Save, and Start Service. It takes a little while to start up, so wait a few seconds, and hit Refresh to check that it is working. If you've already got Tomcat working, you can skip to the next section "Testing Blojsom".

To check that Tomcat is working properly, go to this page:

http://myserver.mydomain.com:9006

and click on "Servlet Examples" on the left hand side.

If this all seems ok, then Tomcat is fine.

Now to test mod_jk, try:

http://myserver.mydomain.com/examples/servlets/

If this gives you the same page, then mod_jk is working happily.

Step 4: Testing Blojsom

Now try:

http://myserver.mydomain.com/blojsom/blog/default/

to see the default blog.

If that all looks ok, try the blog of the Open Directory user you've set up.

http://myserver.mydomain.com/blojsom/blog/odusername/

If that all looks ok, try logging into the admin interface for your blog.

http://myserver.mydomain.com/blojsom/blog/odusername/?flavor=admin

You should be able to login with your Open Directory username and password.

If that works, try adding some entries, and try switching the theme,
(Plugin Settings -> Configure Available Plugins -> Theme Switcher Plugin)

If that all works, then it looks like everything is working. You now have a multi-user blog server set up, and have create a blog for a user that authenticates against Open Directory.

Now, if you want an easy way to have this automatically set up blogs for your OD users, you might want to look at a script like this:

This version loops over the listing of a folder. I've done this one this way, as this allows me to quickly loop over all these users who are logically grouped, without having to actually interrogate the Open Directory server for group memberships. The 'grep -e' statement that the ls gets piped through is there to preclude those pesky ".DS_Store" and ".VolumeIcon.icns" files, as well as my group shared folders, which all start with uppercase characters.

#!/bin/sh
#
BLOJHOME=/Library/Tomcat/webapps/blojsom
for user in $(ls /Volumes/raid/mystaff | grep -e "^[a-z]“)
do
  if [ ! -e $BLOJHOME/WEB-INF/$user ]
  then
    cp -Rp $BLOJHOME/WEB-INF/default_template $BLOJHOME/WEB-INF/$user
    cp -Rp $BLOJHOME/resources/default_template $BLOJHOME/resources/$user
    mkdir -p /usr/local/blog-data/$user
    cat $BLOJHOME/WEB-INF/default_template/blog.properties | sed “s|BLOGOWNER|$user|g” > $BLOJHOME/WEB-INF/$user/blog.properties
    cat $BLOJHOME/WEB-INF/default_template/authorization.properties | sed “s|BLOGOWNER|$user|g” > $BLOJHOME/WEB-INF/$user/authorization.properties
    cat $BLOJHOME/WEB-INF/default_template/permissions.properties | sed “s|BLOGOWNER|$user|g” > $BLOJHOME/WEB-INF/$user/permissions.properties
    chown appserver $BLOJHOME/WEB-INF/$user/blog.properties
    chown appserver /usr/local/blog-data/$user
    if [ ! $(cat $BLOJHOME/WEB-INF/blojsom.properties | grep $user) ]
    then
      BLOJUSERS=$(cat $BLOJHOME/WEB-INF/blojsom.properties | grep blojsom-users)
      cat $BLOJHOME/WEB-INF/blojsom.properties | grep -v blojsom-users >  /tmp/blojsom.properties
      echo “$BLOJUSERS,$user” >> /tmp/blojsom.properties
      cp /tmp/blojsom.properties $BLOJHOME/WEB-INF/blojsom.properties
    fi
  fi
done

serveradmin stop appserver && serveradmin start appserver

If you really do want to loop over all the users in your Open Directory setup, you could change the for loop to something like:

for user in $(dscl /LDAPv3/odmaster.mydomain.com -list /Users)

but you'll probably want to filter out some users like unknown and maybe your OD Dir Admin account.

for user in $(dscl /LDAPv3/odmaster.mydomain.com -list /Users | grep -v unknown | grep -v myodadmin)

You could do that more elegantly in the script by checking the username, rather than just grepping, but that works...

This is kind of why I like looping over sharepoints. I know that all the users in a certain group who should have this service will be in one of several sharepoints, and it's kind of easier. If they don't have a home directory yet, they shouldn't be blogging yet either...

So this is how I'm running this blog, and have set up blogs for all my staff. As always though, buyer beware, especially when it's free... standard not responsible for destroying your server disclaimer here.