Monthly Archives: May 2009

Special Characters and MediaWikis Do Not Mix

I have run a MediaWiki wiki for my house on campus for close to two years. When I set it up, I gave it the name “Saint Claude de La Colombière Men’s Catholic House,” and since MediaWiki accepted this name complete with two accented characters, I assumed it wasn’t a problem. For an unknown reason, it became a problem today. It started throwing the annoying “Fatal error: Allowed memory size of 20971520 bytes exhausted (tried to allocate 122880 bytes) in /home/www/claude/w/includes/Revision.php on line 361″ errors. I tried increasing the memory allocated to PHP, looking through Apache log files, and upgrading the version of the MediaWiki software.

After at least an hour of frustration accumulated throughout the day, it occurred to me that this only happens on one page: the “About” page. This page, in standard MediaWiki fashion, is actually called: “Saint Claude de La Colombière Men’s Catholic House: About.” Then it hit me that perhaps those accented characters weren’t such a good idea after all. As soon as I changed the value in the LocalSettings.php to something more reasonable and accent free:

$wgSitename = "Saint Claude de La Colombiere Men's Catholic House";

everything started working again as normal.

I would be very curious to know why this worked in the past. I suppose it could have to do with my versions of Ubuntu Server, Apache, PHP, MediaWiki, etc. However, it’s one more thing to check when PHP starts throwing out of memory exceptions.

Edit: It turns out that increasing the memory to PHP was the solution as today I started experiencing the same error on other pages. I increased the LocalSettings.php memory limit from 20 MB to 32 MB.

ini_set( 'memory_limit', '32M' );

and this (actually) solved the problem. Before, I was editing the php.ini file, and it would appear that the LocalSettings.php was overriding the global value even though it was higher. Special characters may still be a bad idea.

URIBL SpamAssassin Settings

I have been receiving a lot of emails that contain web links that are getting marked as spam. According to URLBL.COM, these are links that appear in spam and not links where the spam originates. Therefore, about all I can do is whitelist the senders or dial down the scores on the rules for these filters. After adding a handful of senders to the whitelist, I decided to alter the rules.

I found all of the URIBL rules in the /usr/share/spamassassin/50_scores.cf file. I copied them to the /etc/spamassassin/local.cf file where I could change the values to something more reasonable:

score URIBL_AB_SURBL 0 0.800 0 0.900 # n=0 n=2
score URIBL_JP_SURBL 0 1.400 0 0.700 # n=0 n=2
score URIBL_OB_SURBL 0 1.000 0 0.700 # n=0 n=2
score URIBL_PH_SURBL 0 1.000 0 0.800 # n=0 n=2
score URIBL_RHS_DOB 0 0.400 0 0.500 # n=0 n=2
score URIBL_SBL 0 1.200 0 0.700 # n=0 n=2
score URIBL_SC_SURBL 0 1.200 0 0.200 # n=0 n=2
score URIBL_WS_SURBL 0 1.000 0 0.700 # n=0 n=2
score URIBL_BLACK 0 0.900 0 0.900 # n=0 n=2

Hopefully decreasing the scores for these rules will decrease false positives that I have been receiving in my inbox.

OpenVPN on Mac OS X

I have been using Tunnelblick to connect to the VPN I have configured on my Linksys WRT54GL router running DD-WRT. DD-WRT provides several different tutorials for configuring OpenVPN on your router. Tunnelblick works fairly well on my laptop running Mac OS X most of the time, but every now and then I have had issues connecting. Therefore, I decided to investigate alternatives.

I decided to try some variation of what was described in this post on Tony’s Cafe. The first step was to install the TunTap driver for Mac OS X. This provides the necessary network interfaces to connect to an OpenVPN. It appears that a reboot is no longer required. For command line simplicity and for scripting purposes, you can use the following to install the TunTap driver.

cd /tmp/
curl -O http://superb-east.dl.sourceforge.net/sourceforge/tuntaposx/tuntap_20080804.tar.gz
tar xzvf tuntap_20080804.tar.gz
installer -verbose -pkg tuntap_20080804.pkg -target /
rm -Rf tuntap*

The post on Tony’s Cafe then shows the steps necessary to build OpenVPN from source. You are welcome to do this and you can check it out in the post. However, the simply way is to make use the openvpn binary inside of the Tunnelblick application package. I simply copied it out of the .app and into /usr/sbin. The following again automates the entire process for installing the openvpn binary.

cd /tmp/
curl -O http://tunnelblick.googlecode.com/files/Tunnelblick_3.0b10.dmg
hdiutil attach Tunnelblick_3.0b10.dmg
cp /Volumes/Tunnelblick/Tunnelblick.app/Contents/Resources/openvpn \
    /usr/sbin/openvpn
hdiutil detach $( df | grep Tunnelblick | awk '{print $1}' )
rm -Rf Tunnelblick*

I then created a new OpenVPN configuration file using the example provided in the referenced Tony’s Cafe post. The example I used is provided below.

client
dev tap0
# In the next line, specify the IP address and subnet mask of the tap0 device
ifconfig [IP Address] [Subnet Mask]
proto udp
remote [OpenVPN Server DNS Address] [port]
resolv-retry infinite
# Downgrade privileges after initialization (non-Windows only)
user nobody
group nobody
persist-key
persist-tun
;http-proxy-retry # retry on connection failures
;http-proxy [proxy server] [proxy port #]
mute-replay-warnings
# If using SSL Certificates, use the following 3 lines:
ca /etc/openvpn/ca.crt
cert /etc/openvpn/client.crt
key /etc/openvpn/client.key
# If using a SSL static key, use the following line:
secret “/etc/openvpn/vpn.key”
# Use the next line if utilizing LZO compression:
comp-lzo
verb 3
mute 20

I then plugged in the host to connect to, the IP address and subnet mask for my laptop to use while it is on the road, and the path to the certificates and keys stored on my machine. All that was left was to launch OpenVPN from the command line:

/usr/local/sbin/openvpn --config /path/to/openvpn.conf

I haven’t used this new method of connecting to my VPN extensively; however, so far it appears to be working well.

Finger Daemon on Ubuntu

Since I set up my Ubuntu servers and desktops at home I have wanted to be able to remotely finger the different computers on my network. By default, a remote finger request will look like this:

zac@dakara:~$ finger @athos
[athos.lund]
finger: connect: Connection refused

The solution is to install the required packages:

apt-get install inetutils-inetd fingerd

Then start the Internet super server:

/etc/init.d/inetutils-inetd start

According to the package details: “Inetd is the daemon that listens on various TCP and UDP ports and spawns programs that can’t or won’t do it for themselves.”

Now a remote finger request will look like:

zac@dakara:~$ finger @athos
[athos.lund]
Login     Name           Tty      Idle  Login Time   Office     Office Phone
zac       Zachary Lund   pts/2          May  4 20:12 (dakara.lund)

The contents of the file /etc/inetd.conf show what is necessary for finger:

finger		stream	tcp	nowait	nobody	/usr/sbin/tcpd	/usr/sbin/in.fingerd

It simply takes an incoming finger request and spawns the appropriate process.

Configure SpamAssassin with Postfix on Ubuntu

I’ve been running a mail server for the last year and a half. When I initially set up my Postfix mail server on Ubuntu, I knew that eventually I would need to add a spam filter. I recently decided that SpamAssassin was the best choice to filter email on my mail server.

I now receive on average more than one spam message each day. Interestingly, all of my spam is sent to an email address that I have only given out to Marquette University. I guess that means they have either sold my email address or poorly secured it in their database. Neither would surprise me.

I used the content from two different tutorials to get SpamAssassin up and running on my server.

First, I installed SpamAssassin.

apt-get install spamassassin spamc

Next, I created the spamd user and group. You can specify a specific uid and gid if you want.

groupadd spamd
useradd -g spamd -s /bin/false -d /var/log/spamassassin spamd

Then I created the spamd home directory and set the permissions.

mkdir /var/log/spamassassin
chown spamd:spamd /var/log/spamassassin

Then I set up some configuration for SpamAssassin. You can edit the file directly, but I use Sed so that I can automate the installation process in a script. This enables SpamAssassin, Cron, and some other options.

DEFAULT_SPAMASSASSIN=/etc/default/spamassassin
mv $DEFAULT_SPAMASSASSIN $DEFAULT_SPAMASSASSIN.default
sed '
    s/ENABLED=0/ENABLED=1/
    s/CRON=0/CRON=1/
    s/^OPTIONS.*/SAHOME="\/var\/log\/spamassassin"\nOPTIONS="--create-prefs --max-children 5 --username spamd -H ${SAHOME} -s ${SAHOME}\/spamd.log"/
' $DEFAULT_SPAMASSASSIN.default > $DEFAULT_SPAMASSASSIN

Then I set up the rest of the configuration for SpamAssassin. I initially set the required score to 2.0, but this caused a lot of legitimate emails (ham) to be marked as spam. The following configuration will rewrite subjects of spam messages to identify them as spam.

SA_LOCAL_CF=/etc/spamassassin/local.cf
mv $SA_LOCAL_CF $SA_LOCAL_CF.default
echo "
rewrite_header Subject [***** SPAM _SCORE_ *****]
required_score           5.0
# to be able to use _SCORE_ we need report_safe set to 0
# If this option is set to 0, incoming spam is only 
# modified by adding some \"X-Spam-\" headers and no 
# changes will be made to the body.
report_safe     0

# Enable the Bayes system
use_bayes               1
use_bayes_rules         1
# Enable Bayes auto-learning
bayes_auto_learn        1

# Enable or disable network checks
skip_rbl_checks         0
use_razor2              0
use_dcc                 0
use_pyzor               0
" > $SA_LOCAL_CF

Now that I have been running the spam filter for a couple weeks, I have had to whitelist some email addresses that send me emails with strange headers or get sent from “shady” IP addresses. This goes into the same local.cf file.

whitelist_from *@hq.acm.org

I find it amusing that emails from the ACM keep getting marked as spam. Next I started SpamAssassin.

/etc/init.d/spamassassin start

Next, I modified Postfix to send emails through the SpamAssassin filter.

POSTFIX_MASTER_CF=/etc/postfix/master.cf
mv $POSTFIX_MASTER_CF $POSTFIX_MASTER_CF.default
sed 's/smtp      inet  n       -       -       -       -       smtpd/smtp      inet  n       -       -       -       -       smtpd\n\t-o content_filter=spamassassin/'  \
$POSTFIX_MASTER_CF.default > $POSTFIX_MASTER_CF
echo 'spamassassin unix -     n       n       -       -       pipe
  user=spamd argv=/usr/bin/spamc -f -e    
  /usr/sbin/sendmail -oi -f ${sender} ${recipient}' >> $POSTFIX_MASTER_CF

Next, reload Postfix so it will use SpamAssassin.

/etc/init.d/postfix reload

Once SpamAssassin is running, you can train it by passing it spam and ham emails.

sa-learn -u spamd --spam --mbox /path/to/spam_mbox
sa-learn -u spamd --ham --mbox /path/to/ham_mbox

After adjusting the spam threshold, training the filter with spam messages that I have acquired over the last year, and whitelisting a few problematic senders, my spam filter has been doing a good job of marking spam as spam. At this point it is easy enough to sort through the email manually and confirm that they are spam. In the future, if it ever gets bad enough, I will be able to automatically delete the messages or filter them into a different mailbox on delivery.

Configure a Reverse Proxy with Apache

In a previous post, I discussed the details required to set up Subsonic on Ubuntu Server. The problem with Subsonic is that it runs on Tomcat on a non-standard port. The rest of my web applications run in Apache on port 80. Therefore, it would be nice if instead of having to go to http://athos.zacintosh.com:8180/subsonic/, I could simply go to http://athos.zacintosh.com/subsonic/. The solution is called a reverse proxy. Reverse proxies can do things like load balance between multiple web servers or simply make resources on an internal web server available externally. In this case, I am using a reverse proxy to make a web application available on a different port available on the standard port 80.

The set up is fairly simple. On Ubuntu, it should be as simple as issuing this command (as root) to enable the proxy modules:

a2enmod proxy_http

Next I inserted the following into /etc/apache2/conf.d/subsonic.conf to make it available on all websites running on my server. If you only wanted it to be available for a single website, you could insert this into a site inside of /etc/apache2/sites-available/.

ProxyRequests Off
ProxyPreserveHost On
<Proxy *>
    Order allow,deny
    Allow from all
</Proxy>
ProxyPass /subsonic/ http://localhost:8180/subsonic/
ProxyPassReverse /subsonic/ http://localhost:8180/subsonic/

The configuration above says to not proxy all requests (not act as a forward proxy server). The ProxyPass and ProxyPassReverse set up the actual retrieval of data from Subsonic running on Tomcat. You need to replace the localhost and the port number with the specific items for your configuration. Finally, you need to reload the Apache configuration:

/etc/init.d/apache2 force-reload

Now Subsonic should be available on port 80 just like everything else running on your web server with Apache.

Install Subsonic on Ubuntu Server

Subsonic is a free, web-based media streaming application. After evaluating it alongside Jinzora and Sockso, I selected it for use on my servers. Internally, I have been using Firefly Media Server (previously called mt-daapd) to make music available to different desktop music players. Subsonic had several features that made it a good choice including allowing music from several different directories, Last.fm scrobbling support/integration, and multiple users with passwords and support for LDAP integration.

I installed Subsonic on a server running Ubuntu Hardy Heron Server. I built on a previous tutorial for installing Subsonic on Ubuntu.

First, I install the necessary Java packages, Tomcat, and Lame. I broke it into three installs to be safe. It could probably be done in one.

apt-get install java-common sun-java6-bin sun-java6-jdk sun-java6-jre
apt-get install tomcat5.5
apt-get install lame

Next, I stopped Tomcat before changing the Tomcat configuration and installing the Subsonic web application.

/etc/init.d/tomcat5.5 stop

Next, I disabled the Tomcat security stuff. It’s required for Subsonic to work. You could do this by editing the file directly (/etc/init.d/tomcat5.5), but I use Sed so that I can automate the installation process in a script.

cd /etc/init.d/
mv tomcat5.5 tomcat5.5.bak
sed "s/TOMCAT5_SECURITY=yes/TOMCAT5_SECURITY=no/" tomcat5.5.bak > tomcat5.5
chmod +x tomcat5.5
rm tomcat5.5.bak

Next, I downloaded, decompressed, and installed the subsonic application.

cd /tmp/
wget http://superb-east.dl.sourceforge.net/sourceforge/subsonic/subsonic-3.6-war.zip
unzip subsonic-3.6-war.zip -d subsonic
mv subsonic/subsonic.war /var/lib/tomcat5.5/webapps/

Subsonic expects certain directories to exist. The following creates those directories and sets the permissions.

mkdir /var/subsonic
mkdir /var/subsonic/playlists
mkdir /var/subsonic/transcode
chown -R tomcat55:nogroup /var/subsonic

Subsonic uses Lame to transcode/compress mp3s for streaming. The next two lines make Lame available to Subsonic.

cd /var/subsonic/transcode
ln -s /usr/bin/lame lame

Now we can start Tomcat again.

/etc/init.d/tomcat5.5 start

I always like to clean up after my installs. This removes the files we downloaded and no longer need any more.

rm -R /tmp/subsonic*

Subsonic should now be running and available at http://localhost:8180/subsonic/. The port might vary depending on the installation of Tomcat. Log in. You can change the password by clicking on “Settings” and then “Users” and then selecting the admin account.

Next, I configured LDAP authentication. It was as simple as clicking on “Settings” and then “Advanced” and then “Enable LDAP authentication.” I then filled in the following:

  • LDAP URL: ldap://athos.lund:389/dc=lund
  • LDAP search filter: (uid={0})

I left the “LDAP manager DN” and “Password” blank. I checked “Automatically create users in Subsonic” so that any user in LDAP can automatically log into Subsonic.

If you would like to make Subsonic available on port 80 like most web applications instead of port 8180, check out my post on Reverse Proxies.