An African Travelogue

Recently (28 June 08), my 64-bit CentOS machine started sending me error reports from cron jobs running perl scripts that had been running fine until now. There was a problem coming from Scalar::Util:

Use of uninitialized value in concatenation (.) or string at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/Scalar/Util.pm line 30

I’ve seen this problem before, as on this comp.lang.perl.misc post from Tim Boyer.

A force re-install of the Scalar::Util package in cpan does the trick:

cpan> force install Scalar::Util

However, while looking at Scalar/Util.pm around line 30, I found another bug which was hinted at by the first bug, but still only runs if there really is something to croak about:

Scalar/Util.pm version 1.19 lines 28-31:

if (grep { /^(dualvar|set_prototype)$/ } @_ ) {
    require Carp;
    Carp::croak("$1 is only available with the XS version");
}

If the array @_ contains either ‘dualvar’ or ‘set_prototype’, croak should be called with $1 filled with the string that matched. Unfortunately, this doesn’t happen. The ‘grep { /pattern/ } @_’ construct returns an array of values from @_ that matches the pattern. The if statement around it is true if the array returned from grep contains at least one element, meaning at least one element in @_ matched the /pattern/. Capturing within this construct does not work outside the { /pattern/ } block, and therefore the $1 will not get filled with the first (or any) string that matched the pattern.

This could be fixed by separating tests, as follows, although there is probably a much sexier way to do it:

if (grep { /^dualvar$/ } @_ {
    require Carp;
    Carp::croak("dualvar is only available with the XS version");
}
if (grep { /^set_prototype$/ } @_ {
    require Carp;
    Carp::croak("set_prototype is only available with the XS version");
}

I’ve emailed the author of Scalar::Util to see if this change can be made.

19 July 2008: Update
I found the proper place to report bugs in perl modules available on cpan, which is http://rt.cpan.org. Then I found a previous bug report that was for this exact problem.

It’s at: http://rt.cpan.org/Public/Bug/Display.html?id=31054

As I was writing my previous post about using Apple Mail’s rules and AppleScript to block spammy email messages, I realized there was a much easier way to do it, without using AppleScript. When I was going through the headers added by SpamAssassin, I noticed that the X-Spam-Bar header’s length represented the spaminess score. So instead of using AppleScript to parse the X-Spam-Score header as a number, I could just use rule conditions to weed out particularly spammy emails.

The following rule definition does almost exactly the same thing as the AppleScript in the previous post:

If [all] of the following conditions are met:
    [Subject] [Begins with] [***SPAM***]
    [X-Spam-Bar] [Begins with] [+++++++++]
Perform the following actions:
    [Delete Message]
    [Mark as Read]
    [Stop evaluating rules]

Two things might be of note here:

• The ***SPAM*** tag in the message’s subject is added by our SpamAssassin setup automatically. We don’t actually need it to make this rule work, but I figure it probably cuts down on some processing time.
• The X-Spam-Bar header is not initially a choice in the rule adding/editing dialog. You have to add it to the list by choosing the [Edit Header List] option from the bottom of the [Subject] select menu.

This rule takes advantage of the X-Spam-Bar header, which, if the spam score is 9.0 or greater, will contain at least 9 ‘+’ symbols, and will thus be deleted.

The one thing it doesn’t do is mark the message as spam, which helps teach Mail’s spam analysis engine. If anyone knows how to do that with Mail’s rules, please leave me a comment and let me know!

At work, our mail server uses SpamAssassin to block undesired messages. We’ve got the spam threshold currently set at 6, meaning that any message with a “spaminess” score of 6.0 or more is determined to officially be spam. But since we don’t want to miss a customer’s email coming from a recently blacklisted domain, we don’t have SpamAssassin automatically delete the spammy messages, rather it just tags their subjects and delivers them to their recipients.

SpamAssassin also adds a few headers into each email it passes on. An example of each header is included after it’s description.

• X-Spam-Status: A line indicating whether the message is spam or not, and it’s spaminess score. [Yes, score=19.2]
• X-Spam-Score: The message’s spaminess score, multiplied by 10. [192]
• X-Spam-Bar: A series of ‘+’ symbols, each one representing 1 spaminess point. [+++++++++++++++++++]
• X-Spam-Report: The full spaminess detection process report of the message. [Example too long to include.]
• X-Spam-Flag: A simple yes/no saying whether the message is spam. [YES]

Since all the spam got through, I wanted to at least delete the emails which were definitely spam. Since a spam threshold of 6 was doing almost a perfect job of keeping the real emails clean and marking the spam emails as such, I decided that a spaminess score of 9 or more would indicate that an email was definitely spam, and could without consequence be deleted.

Everybody at work uses Apple’s Mail program for their emailing, so I wrote an AppleScript to be triggered from a rule that will delete any email with an X-Spam-Score header value of 90 or more. The AppleScript is shown here:

on perform_mail_action(theData)
  tell application "Mail"
    try
      set theMessages to |SelectedMessages| of theData
      set theRule to |Rule| of theData

      repeat with theMessage in theMessages
        set theHeader to the header "X-Spam-Score" of theMessage
        set theSpamScore to the content of theHeader as integer
        if theSpamScore ≥ 90 then

          -- Make sure the message is marked as junk
          if the junk mail status of theMessage is false then
            set junk mail status of theMessage to true
          end if

          -- Don't let the message show up as unread
          set the read status of theMessage to true

          -- Delete the message
          delete theMessage

          -- No need to check this message against anything else
          if stop evaluating rules of theRule is false then
            set stop evaluating rules of theRule to true
          end if
        end if
      end repeat
    end try
  end tell
end perform_mail_action

And to attach this script to a rule in Mail, we used this
definition:

If [any] of the following conditions are net:
    [Subject] [Begins with] [***SPAM***]
Perform the following actions:
    [Run AppleScript] [/path/to/my/script.scpt]
    [Stop evaluating rules]

One thing which might be of note is that SpamAssassin automatically prepends the ***SPAM*** string to the subject line of every email it believes is spam.

Recently I spent a day trying to get Archive::Zip to open an existing zip file which was already loaded into memory (and stored in a scalar variable). It just wouldn’t work, so I tried the simplest possible example in a separate file, and it still wouldn’t work. Eventually I discovered a solution, though not perfect, it works for me, and I hope you can use it.

The simplest example of the functionality was included as one of the example files, readScalar.pl. This file uses IO::File to load a zip file into a scalar variable, and then uses IO::Scalar to open a seekable filehandle to that scalar which Archive::Zip can use with it’s readFromFileHandle method to load in the zip file.

Unfortunately, even this example wasn’t running on both my Mac OS X box or my CentOS box. I was getting an error that looked like this:

error: file not seekable
at /Library/Perl/5.8.6/Archive/Zip/Archive.pm line 437
Archive::Zip::Archive::readFromFileHandle('Archive::Zip::Archive=HASH(0x18e0700)', 'IO::Scalar=GLOB(0x1917dd8)') called at ./readScalar.pl line 20

After some Google searching, I came upon some bug reports on CPAN that dealt with this:

http://rt.cpan.org/Public/Bug/Display.html?id=7855
http://rt.cpan.org/Public/Bug/Display.html?id=7633

They both promised that the problem would be fixed in the next major release of Archive::Zip, but here we are, a couple years later, and it still doesn’t work.

The solution that worked for me was a combination of the user-submitted fixes. At the end of my perl script, I overrided Archive::Zip::Archive’s _isSeekable method with code suggested by NEDKONZ, and it looks like this:

# Override the _isSeekable function in Archive::Zip
no warnings 'redefine';
package Archive::Zip::Archive;
sub _isSeekable {
    my $fh = shift;
    if ( UNIVERSAL::isa( $fh, 'IO::Scalar' ) )
    {
        return $] >= 5.006;
    }
    elsif ( UNIVERSAL::isa( $fh, 'IO::String' ) )
    {
        return $] >= 5.006;
    }
    elsif ( UNIVERSAL::can( $fh, 'stat') )
    {
        return -f $fh;
    }
    return UNIVERSAL::can( $fh, 'seek');
}
use warnings 'all';

This function living in the bottom of my code ended the “error: file not seekable” problem. Of course, you could also just edit your Archive/Zip/Archive.pm file and fix it forever, rather than including this function in every file that needs to read a Zip file from memory.

Recently at work we had a printer network card fail. It had happened in the past a couple of times, and we just assumed that since the network cards were two years old or so that we should just buy some new ones.

I decided to go online to check out which cards are recommended by people who’ve had similar experiences. What I found instead was that this particular model of printer network card had a serious design flaw, and instead of screwing their customers and saying “buy a new one,” HP decided to extend the product warranty for 5 years, whether it was purchased from them or not, and they are replacing my card with a newer card which doesn’t have the same problem. This resolution was achieved with an 8 minute phone call.

I just wanted to take a blog post to give three cheers to HP for fully supporting a lemon product, even though I purchased it more than 4 years ago!

If you’ve got an HP JetDirect 615n printer network card, and it failed, you should do this:
0. Note your new warranty information is good until the end of October 2008 (http://www.hp.com/pond/jetdirect/j6057a.html)
1. Call 1-800-HP-INVENT
2. Navigate the prompts to tech support > JetDirect.
3. Tell them about the problem.

If you’ve got the same problem I had, they’ll send you a new card, with next-day shipping! If not, good luck! Here’s a link to HP’s site describing the bad cards:

http://www.hp.com/pond/jetdirect/j6057a.html

A foreigner looking for a job in Namibia has a difficult task. It’s actually very easy to find a job, but to start legally working (i.e. getting paid for it) you’ve got to get a work visa. To get a work visa, you’ve got to somehow prove or explain why the job you want can’t be done by any Namibian, or for some reason no Namibian wants that job. Sometimes this can involve advertising the job in the newspaper and allowing 6 months for Namibians to respond to it.

Now, TIA (this is Africa), so of course there are other ways of getting a work visa, which (thank God that Namibia is pretty civilized) usually involves knowing the right person rather than bribing someone. But still you’ve got to almost follow all the rules. Math, IT, and science are fields that are pretty open to foreigners right now because there’s a shortage of well-trained Namibians. So the chances are that I could probably find a job in one of those fields legally, but if I wanted to be a cashier at Shop Rite, probably not.

In my last week of pounding the pavement in Windhoek and throwing out the network, I’ve met some really interesting people and places. BEN (Bicycling Empowerment Network Namibia) is an organization which collected donated bicycles from all over the world, fixes them up, and sells them at about half the local retail price. They’ve currently got an American volunteer who has been developing a bicycle ambulance project. The idea is that you attach a sort of wheeled stretcher (complete with awning) to your bicycle like a trailer, an injured person can be placed on that stretcher, and you peddle them out of the village to a place where better transport to a hospital or a clinic can be found.

There’s also SchoolNet Namibia, which has taken upon itself a huge set of tasks. Their primary goal is to empower rural schools and community centers with computers and internet access. Previous to a redundant-yet-stagnant government effort, they had been giving out labs of around 6 computers in a well-polished fashion, complete with tables and UPS’s. They even negociated great internet rates with the local telecom company (~ US$42 for an always on modem connection, which is 10% of one teacher’s monthly takehome salary) which the school can obviously divide up amongst the people who use it. SchoolNet has even made it possible for very rural schools to get on the internet. They’ve installed 23 completely solar-powered computer labs at schools where electricity isn’t available, and for those schools which are close enough to towns, radio modems rather than telephone modems. They are also the country’s stronghold for spreading the goodness of open-source software.

I also walked into a solar electricity store, as I had heard the prices on solar electricity have come down. Previously, I had spec’d and priced out a system according to what my local brother had told me about his system. I wanted to be able to use my computer at my village house (no electricity grid) for about 12 hours a day. Since it’s a laptop, I thought something like that should be small enough to keep costs down. I had priced the required elements of the system to something around US$500. When I walked into the store, I knew I was in the right place. They have everything related to solar electricity, and knew exactly how to put together a system to meet my electricity usage needs. Tremendously knowledgable and ready to put your request together on the spot. The only problem was that when they were finished, they handed me a quote for US$2400. I just sort of looked at it, and them, and wondered how they could possibly have any customers. For such a small system, certainly no Namibian, at least the village people I had lived with for 2 years, would ever think of paying that much for electricity! They tried to tell me that many villagers had purchased systems costing them US$900 which would be just enough to power two lightbulbs for about 4 hours a night, and I promptly informed them that their claim was preposterous. If my village house was on the grid, a monthly electricity bill for my computer usage would go for about US$14.50, which means their US$2400 system would be the equivalent of paying upfront for 165 months of electricity (more than 13 years). And they’re trying to tell me that people do it all the time. They later confessed that convincing people of the price is the hardest part. Consider the “village light” system for US$900. A candle in Namibia, which can be used for about two nights, sells for around US$0.145. Let’s say you use two of them to light up your house at night. That means your nightly cost of light is US$0.145 (because each candle lasts two nights). The cost of candles would catch up to the solar system in 6206 days, or 17 years. And they are trying to tell me that villagers are willing to pay for 17 years of electricity upfront. Ha!

Lastly, I’ve been pushing a proposal for the Ministry of Education. I’ve got an idea to analyze factors of learner’s home situations, teacher demographics, and school resources to see how they impact a school’s performance. It’s moving forward, and I’ve got the name of the right person to meet, but he’s been out of town all this week. So, next week, we’ll see if I can get an appointment with him and his stat team.

Holy crap! I’m writing this post on my computer which is connected to the internet via a bluetooth connection to my mobile phone, which is using GPRS to transmit data over the cellphone network to the internet. You can do this in the States of course, but here in Ghana, not everyone has a DSL or cable connection to the internet… This connection will work anywhere my phone will, and the speed is about that of a 28.8 Kbps modem (remember those days?). The price? About ₡5 (≈ US$0.0005) per kilobyte. What does that mean? I can check my email using between 10 kilobytes (no mail) – 500 kilobytes (lots of mail, no photos attached), which costs between US$0.005 and US$0.27. Suck on that, T-Mobile!

When you think of traveling through Africa, and trying to communicate with your family, what comes to mind? Maybe some dingy internet café – a hole-in-the-wall that has 5 computers smooshed together, all sharing a single modem line, with everybody except you downloading MP3s?

I thought I was being slightly ridiculous by taking my laptop all over Africa with me, but I found my second home here in Accra, called Busy Internet. This place is absolutely amazing, from an American standpoint. They’ve got around 120 terminals split between two rooms, a large space for their 4 X-Boxes, and even a room where you can bring in your laptop, sit at a desk, and plug directly into their internet connection. And regarding their internet connection, they’ve got redundant connections to lessen the down time, a fiber-optic cable to London, and another to the national phone company. Holy crap!

Needless to say, if I need to get anything computer-wise done, I can do it there.