Apple Mail likes to prompt about self-signed certificates when connecting to imap or pop servers over ssl. Unfortunately this gets rather annoying after a while so here’s how to import the self-signed certificate into your system.
This is based on a hint from Mac OSX hints but it seems they didn’t quite get a couple of the commands right. Notably the certtool line – you import the .der file not the .pem and you have to make sure you include it from the right directory as well.

Assuming the file imapd.pem is in your home directory, these are the steps I took to actually get the certificate imported.
Convert .pem to .der
$ openssl x509 -in imapd.pem -inform pem -out imapd.der -outform der
Copy the system X509Anchors file to your local Keychain directory.
$ sudo cp /System/Library/Keychains/X509Anchors ~/Library/Keychains/X509Anchors
$ cd ~/Library/Keychains/
$ certtool i ~/imapd.der k=X509Anchors d
You should see a message “…certificate successfully imported.” If not then make sure you specified the imapd.der file correctly.
Finally copy it back to the system location.
$ sudo cp ~/Library/Keychains/X509Anchors /System/Library/Keychains/