Emboldened by that, I decided to take a stab at fixing two bugs in the core Thunderbird code that have been driving me crazy. That, too, required a steep learning curve, but in the end, I was able to submit fixes for two bugs, one quite old and one new in Thunderbird 3.1, affecting a whole bunch of people:
It was impossible to remove attachments from some MIME messages, including MIME messages generated by the Mac Mail client (Mozilla bug #351224). This bug has been reported by at least 30 different people and was first reported almost four years ago. Fixing it required rewriting pretty much an entire module within C++ source code for Thunderbird.
Thunderbird was incorrectly inserting a couple extra spaces at the beginning of some sent email messages (Mozilla bug #564737). This bug was first reported just a few months ago and has already been reported by at least 56 different people. This bug is in the core code that is shared between all Mozilla applications, which means that the fix will impact Firefox, Seamonkey, etc. as well as Thunderbird.
Needless to say, there are other things I should have been working on when I got distracted by fixing these bugs. But I’d almost forgotten how rewarding it is to be able to contribute to open-source software in ways that benefit a lot of people.
The old version is not compatible with Thunderbird 3, and its author and maintainer appears to have abandoned it.
I’d love for him to integrate my changes into his version and resume maintaining it, but in the meantime, for the sake of making it available to people, I’ve released the new version myself.
Here’s a picture:
Interestingly, it’s been less than two days since I released it, and it’s already been downloaded by 71 people. Sweet!
If you administer a GNU Mailman installation, you are probably aware that message footers don’t always work quite right: if a message submitted to a list is entirely plain text with no attachments, then the footer is fine, but if it’s formatted in HTML or has attachments, then the footer is added to the message as a separate message part, and some email clients display it as an attachment which must be clicked on to view, rather than displaying it as part of the message text.
This is a significant problem, since Microsoft Outlook, which has by far the biggest market share of any email client, is one of the clients that displays Mailman footers incorrectly.
Many people have complained about this problem to the maintainers of GNU Mailman, but they have declined to address it. I don’t agree with their reasoning, but it is of course their prerogative as the volunteer maintainers of free software to decide that they’d rather maintain some sort of vision of purity in their code rather than actually make the it do what their users want it to. Jan Ploski also has some interesting thoughts about this.
Fortunately, it’s our prerogative as users to fix it ourselves if they don’t . Adrian Bye did this with a patch to Mailman way back in 2005, but the maintainers rejected his patch and it’s now out-of-date and incompatible with the current stable Mailman release. Others have hacked together site-specific solutions using mimedefang, but no one has implemented a generic solution that can be deployed on top of a standard Mailman installation. Until now, that is.
I’ve just released a script that can be deployed easily into a mimedefang installation to automatically reformat outbound Mailman messages to insert the footer into the text and/or HTML bodies of the message rather than as a separate attachment. All you need to do to use it is install it into your mimedefang installation using the provided instructions, then modify the msg_footer setting inside Mailman to add a couple of special tokens which tell the script to reformat your footers.
Share and enjoy! And hey, if you find this useful, maybe you can show your appreciation.
I am an email pack-rat. I have saved just about every email message I’ve sent or received at work or at home in the past twenty years (excluding messages sent to public mailing lists). That’s a lot of email, over 400,000 messages.
I don’t just do this for the heck of it. I look for things in my archived email on a regular basis. Most of the time I’m only looking back a few months or perhaps a couple of years, but I do occasionally find it necessary for one reason or another to go digging through the really old stuff. To make that feasible, I need to be able to quickly search hundreds of thousands of messages.
Over the years I have imposed a lot of requirements on acceptable solutions for this problem:
I want to be able to store the email in a compressed form, so the search engine needs to understand how to decompress the email archives when indexing or retrieving messages from them.
To achieve decent compression, the email has to be stored in files that hold multiple messages, rather than storing each message in its own file. Therefore, the search engine needs to understand how to break up the mailbox files into separate messages, index them separately, and retrieve them separately during searches.
I don’t want to use a proprietary or binary mailbox format — I want to be able to look at the mailboxes in a text editor and manipulate them easily with tools such as Perl. I used to store my email in BABYL Files, but now I use mbox format (which Thunderbird and Eudora also use for local folders).
I don’t want to be locked into a GUI — I need to be able to update the index, do searches, and retrieve results through the command line.
For many years, I was unable to find any actively maintained open-source software package that satisfied all of these requirements. I was therefore stuck using freeWAIS 0.5, one of the very first Internet search engines, which was developed and released by Thinking Machines Corporation over 20 years ago. I was an active developer on the project; my efforts were focused on making the indexing code faster and less of a disk-hog and fixing a myriad of bugs and memory leaks (freeWAIS was written in C; its primary authors, all of them Lisp programmers, were so used to automatic garbage collection that they were very bad about cleaning up after themselves). Every since the freeWAIS project went defunct, I’ve maintained my own personal version of the code, layering hack on top of hack to keep it compiling and running on new versions of Linux. It was gross, but it was good enough.
That is, it was good enough until I upgraded my Linux box at home about a month ago and went from 32-bit Linux to 64-bit Linux. The freeWAIS code is very dependent on things like the size of an integer, the size of “time_t” and “off_t”, etc. Furthermore, to say that the code is not particularly clean or portable would be a gross understatement. When I rebuilt it for 64-bit, it stopped working. After spending several hours trying unsuccessfully to yet again nurse it back to health, I decided to take another look around to see if any new search engines that would do what I need had come onto the scene.
I was delighted to discover mairix, a package written and maintained by Richard Curnow which bills itself as “a program for indexing and searching email messages stored in maildir, MH or mbox folders.” Wow, someone went and wrote exactly the tool I needed. w00t!
Well, actually, it’s not exactly the tool I needed, because when I set it up last night I discovered some minor issues with its parsing of mbox files. But I fixed those issues and sent Richard my patches, and I hope they’ll be incorporated into the next release so I don’t have to go down the maintain-my-own-software road again . (If you’re trying to use version 0.21 of mairix to index mbox files, email me and I’ll send you my patches.)
Mairix indexed my >400,000 email messages in 12 minutes (3.5GHz CPU, 7,200 RPM SATA hard drive). The mairix index consumes only 144MB of space, despite the fact that my email archives take up 1.1GB compressed. It takes mairix less than 0.2 seconds to do an AND search for two search terms and save the 8 matching messages into an mbox. That is simply incredible.
One of the ways Richard made mairix so fast is by using another tool he wrote, dfasyn, which he describes as “a tool for building general deterministic finite automata (DFAs) given a description as a non-deterministic finite automaton.” You use a high-level syntax to describe the various legal state transitions, and dfasyn compiles that into some big numeric arrays and a very trivial function which transitions between machine states using nothing but pointer and integer arithmetic. The nerd in me thinks this is très cool.
I’m posting about mairix both to let other people know about this great tool, and to give Richard the kudos he deserves for implementing it.
I file my email in different IMAP folders, grouped by topic rather than the “received messages in one folder, sent in another” model imposed nowadays by most mail clients. It’s easy to keep the messages I receive organized by topic — when I’m done handling a message, I simply move it into the appropriate folder.
On the other hand, organizing the copies of message I’ve sent is less straightforward. Visiting my “Sent Items” folder every time I send a message and filing the message I’ve just sent is gross. What I’ve done in the past is to leave all the copies in “Sent Items” and periodically file all of them one at a time. This is time-consuming and error-prone, and what’s more, it could be at least partially automated — if I file a message I received in a particular folder, then something ought to be smart enough to figure out that the response I sent should go into the same folder.
After spending over an hour this morning filing “Sent Items” messages, my annoyance threshold was finally exceeded and I decided to automate the process as much as I could. The result is file-sent-items.pl. In a nutshell, this script reads each message in your “Sent Items” folder (or whatever it’s called), looks for an “In-Reply-To” and/or “References” header pointing at earlier messages in the conversation, looks for those earlier messages in your other folders, and if one is found, moves the sent message into its folder. There’s a bit more too it than that, of course, which you can find out by reading the comment at the top of the script and running it with the “–help” option to get a usage message.
Of course, this script won’t be able to file messages that aren’t part of conversations already filed into other folders, but it still reduces by a lot the amount of filing you have to do by hand.
I’ve been using Vonage for telephone service for a couple of months, and I’m quite happy with it. Their feature set is quite competitive, and I’m paying them significantly less than I’d be paying Verizon, Comcast or RCN for equivalent service. (Shameless plug: if you’d like to give Vonage a try, send me email and I’ll send you a referral. Both you and I will get a free month of service.)
You can configure Vonage to email you about voicemail messages. The email contains the actual voicemail message as an audio attachment, but what it doesn’t have, inexplicably, is the name of the caller pulled from caller ID. This has annoyed me ever since we switched to Vonage, and I recently finally got annoyed enough to finally do something about it. I’m posting my solution here on the off chance that it’ll be useful to others.
To take advantage of this hack, you have to have the ability to filter the contents of your incoming email, e.g., with procmail. If you don’t know what that means, then you should probably stop reading now. :-/
My solution consists of two scripts and two configuration changes. First, the scripts:
The script vmail-cid.pl fetches the caller ID information for recent calls from www.vonage.com and saves it in a CSV file.
The script vonage-vmail-filter.pl reads the aforementioned caller ID CSV file, as well as a CSV file you’ve exported from your Outlook contacts, and uses the information in those files to filter a voicemail notification email message on stdin and send the (possibly modified) notification to stdout. You can configure the script which of the two CSV files to read, i.e., you can take advantage of either the caller ID information from vmail-cid.pl, or the Outlook export CSV, or both.
Now, the configuration changes:
You need to set up a scheduled task, cron job, or whatever to run vmail-cid.pl periodically to keep the caller ID CSV file up-to-date.
You need to tie vonage-vmail-filter.pl into your email delivery, e.g., by editing your .procmailrc file (for which there is an example in a comment at the top of the script), sieve configuration, or whatever.
When all of this is done properly, then every voicemail notification you receive will be updated with the caller ID or Outlook Contacts name for the calling phone number, if it’s available, before it lands in your mailbox.
Click here to download a zip file containing the scripts. See the comments at the top of the scripts for additional details. As always, please feel free to send me any questions, comments and suggestions you might have.
For those of you who use Cygwin to make Windows slightly more bearable…
Suppose you’re happily working in a Cygwin window, and you need to load one of the files you’re working on into a Windows app. You could select File > Open… in the app and then browse to the ten-levels-deep Cygwin folder to find the file you need, or instead you could put this in your .bashrc:
e() { explorer $(cygpath -w $(pwd)) & }
Then, in the Cygwin window, just type “e <ENTER>”, and an Explorer window will pop up, opened to the current directory. You can now double-click on the file, open it from the right-click menu, or drag and drop into a running app.
I’m responsible for coordinating weekly new-functionality demos for my company’s R&D organization. Each week, a developer or product manager presents a half-hour demo of some new functionality to a local audience in one of our offices, using a big 1920×1080 digital TV as the local display, with a GoToMeeting session set up so that people in other offices can watch the demos. We record the GoToMeeting session, and then after the demo we publish it internally for people who weren’t able to watch it live or who want to watch it again.
A 30-minute 1920×1080 WMV is very large, like around 50MB. We want it to be smaller before we publish it. One way to make it smaller is to simply scale it down, but when the entire 1920×1080 canvas is shrunk down to a reasonable size, text which was perfectly legible before becomes unreadable. Since the text is usually a crucial part of the demo, that’s unacceptable.
Usually, the presenter isn’t using the whole screen at any given time. Rather, he’ll be focused on the content in one particular window for an extended period of time, and that window often takes up only around half of the screen. Therefore, cropping the WMV to the active area is a possible way to the video file smaller without sacrificing legibility. There’s just one problem — what happens when the presenter jumps around from window to window during the demo?
There are various expensive commercial video editors which will let you crop different segments of the video differently and then glom them all together into one video at the end. However, I don’t like using commercial software when I can do it for free (this is probably about half on principle and half out of stubbornness). Therefore, I set out to figure out how to do this with free tools. The result is a script called wmv-to-panned-mov.pl.
This script uses the excellent free tool ffmpeg to slice and dice the input video with the appropriate cropping and produces a much smaller MOV file (i.e., QuickTime video) as its output. You feed the script a CSV file indicating the start times, end times, and cropping areas of the various slices, and it does the rest. There’s more information in the comment at the top of the script about how to use it.
Please comment here to let me know if you find this useful. If the time I spent writing this script saves you some time or money, please consider sending a little something to my tip jar (but if you don’t, that’s OK too).
About a month ago, I dived into the world of Mozilla add-on development by adopting the abandoned Thunderbird “Send Later” add-on and porting it to Thunderbird 3.1. The learning curve was pretty steep, and it took a lot more work than I expected to stabilize the add-on, but I think it was worth it, considering that in the two weeks since I released it, almost 2,000 people have downloaded it and at least 444 of them are using it.![[Digg]](http://blog.kamens.us/wp-content/plugins/bookmarkify/digg.png)
![[Facebook]](http://blog.kamens.us/wp-content/plugins/bookmarkify/facebook.png)
![[Email]](http://blog.kamens.us/wp-content/plugins/bookmarkify/email.png)
. 
