Postfix+Dovecot with LDAP

I’ve been running a Postfix+Dovecot with MySQL as the backend for eSource for some time now. It’s on old CentOS 5, but it really did wonders for our ability to manage Virtual email accounts. We’ve been using postfixadmin to manage the MySQL database. Recently, I’ve been getting into LDAP, and have found some success migrating to CentOS 6.

The goal here is to simply replace MySQL with LDAP. (This isn’t a “tutorial”, so I can’t guarantee that copying and pasting anything will work. I frequently copy something real, and then massage the text to eliminate sensitive data. I may or may not be consistent with my changes.)

First, I have to duplicate my vmail user that does all the email transactions.

useradd -u 101 vmail -g mail -s /sbin/nologin -d /var/vmail

Then I copied over my postfix config, which is already configured for Dovecot and for MySQL. If you are starting from scratch, I pity you, and I’m sorry I don’t have time to start postfix from scratch. (On the plus side, the dovecot stuff below is “from scratch”.) So I left all the dovecot stuff the same, and found all the mysql entries and changed them to ldap in /etc/postfix/main.cf:

virtual_mailbox_domains         = proxy:ldap:$config_directory/ldap_virtual_domains_maps.cf
virtual_mailbox_maps            = proxy:ldap:$config_directory/ldap_virtual_mailbox_maps.cf
virtual_alias_maps              = proxy:ldap:$config_directory/ldap_virtual_alias_maps.cf

I picked up a really great tip from Postfix’s VIRTUAL_README page. It suggests, “The reader is strongly advised to make the system work with local files before migrating to network databases, and to use the postmap command to verify that network database lookups produce the exact same results as local file lookup.” I’ll say it right now, “postmap -v” saved me HOURS of troubleshooting. The LDAP README article is an excellent starting position and explains some of the basic configuration settings. My setup turned out to not include some of the attributes that postfix defaults to, so I had to make some adjustments. First, the virtual_domains list is what postfix uses to determine if it should handle the mail locally, or lookup the domain’s MX record, and send it over the internet. I created an LDAP “Organisational Unit” and named it “Domains”, and put all my domains underneath, like so:

dn: ou=Domains,dc=example,dc=com
description: Domains is used for Postfix as it's list of locally hosted doma
 ins.
objectclass: organizationalUnit
objectclass: top
ou: Domains

And then add a domain:

dn: dc=example.com,ou=Domains,dc=example,dc=com
dc: example.com
objectclass: dNSDomain
objectclass: top

In LDAP, there are two main parts of the schema, the ObjectClass and the Attribute. The LDAP database is a tree, so everything is a parent or a child of another entry. ObjectClasses define which attributes are available, and if they are required or not. You can only add attributes to an entry if the ObjectClass lists it.

Now, in ldap_virtual_domains_maps.cf, I filter the query to be just dNSDomain objectclasses that have a “dc” attribute equivalent to the domain being looked up. The attribute that carries the value I’m interested in is “dc”.

ldap_virtual_domains_maps.cf:
server_host = ldap://localhost/
search_base = ou=Domains,dc=example,dc=com
version = 3
bind = no
query_filter = (&(ObjectClass=dNSDomain)(dc=%s))
result_attribute = dc

If you are familiar with SQL, the postfix variable “result_attribute” is like your “SELECT” clause (but your limited to only one column), and the postfix variable “query_filter” is an LDAP query_filter and is like your WHERE clause. In LDAP, the query_filter syntax here is doing an AND logical operation on both conditions. I don’t have a good reference for the filter syntax, but google does return some good results for “ldap filter syntax”.

With out having to even bother restarting/reloading postfix, you can test the configuration file with postmap, like this:

postmap -v -q example.com ldap:/etc/postfix/ldap_virtual_domains_maps.cf

If you leave out the -v, you either get a result (and a return value of 0), or no result (and a return value of 1). With -v, you get to see all the variables that postfix uses to query the LDAP server, including default values, in lines that begin with “cfg_get_str”, “cfg_get_int”, “cfg_get_bool”, and so on. What’s really nice is that the syntax is perfect, and you can copy and paste it into your .cf file. This is how I figured out how to use result_attribute correctly. Now that postfix knows which domains it is hosting email for, we teach it which mailboxes exist. We start with more LDIF:

dn: ou=People,dc=example,dc=com
objectclass: organizationalUnit
objectclass: top
ou: People

and

dn: cn=Kai Meyer,ou=People,dc=example,dc=com
cn: Kai Meyer
cn: Master of the Universe
gidnumber: XXX
givenname: Kai
homedirectory: /home/kaiuser
loginshell: /bin/bash
mail: kaiuser@example.com
maillocaladdress: kaiuser@example2.com
maillocaladdress: kaiuser@example3.com
o: eSource
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
objectclass: inetLocalMailRecipient
objectclass: posixAccount
sn: Meyer
telephonenumber: 555-555-5555
uid: kaiuser
uidnumber: XXX
userpassword: CENSORED

Notice, I have a “mail” and “maillocaladdress” attribute. “maillocaladdress” is a list of email aliases that will deliver to the “mail” address. With those LDAP entries in place, you can configure ldap_virtual_mailbox_maps.cf:

server_host = ldap://localhost/
search_base = ou=People,dc=example,dc=com
version = 3
bind = no
query_filter = (&(objectclass=inetOrgPerson)(mail=%s))
result_attribute = mail

So now I’m starting in the “People” tree (which means LDAP won’t even look in my other “Domains” tree) for all entries that are ObjectClass “inetLocalMailRecipient” and have a “mailLocalAddress” set to the lookup value. This is where email will actually attempt to deliver. IN my case above, it’s “kaiuser@example.com”. Again, before bothering postfix with a reload, you can do another postmap:

postmap -v -q kaiuser@example.com ldap:/etc/postfix/ldap_virtual_mailbox_maps.cf

Again, eliminate the -v if you just need a simple test. If you leave out the query_filter and result_attribute, postfix goes looking for objectclasses and attributes that don’t exist on my default openldap install, so again I’m modifying the default behavior to get the information I want it to look for. Hurray, mailboxes done. Now let’s do aliases. We don’t need any more LDAP entries. We created one with fields we can use for aliases already. Here’s ldap_virtual_alias_maps.cf:

server_host = ldap://localhost/
search_base = ou=People,dc=example,dc=com
version = 3
bind = no
query_filter = (&(objectclass=inetLocalMailRecipient)(mailLocalAddress=%s))
result_attribute = mail

Here, the result attribute is the same “mail”, but the filter is for “mailLocalAddress”. Doing another postmap for example2.com should return the email address for example.com, based on my LDIF above. Again, I wish I had LDAP attributes the way I want them (and I could modify/create schema to get it if I really really wanted to), but this is what I have to work with. So, postfix is done (for me anyway.) I don’t have any other MySQL lookups in my existing server. So now we need to plug in Dovecot.

CentOS 6 comes with Dovecot 2.0. I know that Dovecot2 is very different than Dovecot1, so I’m sorry if you’re stuck on 1. The changes to dovecot were actually quite minimal. Start in /etc/dovecot/conf.d/10-auth.conf, and un-comment the ldap include:

!include auth-ldap.conf.ext

Then copy the example ldap config to /etc/dovecot.conf:

cp /usr/share/doc/dovecot-2.0/example-config/dovecot-ldap.conf.ext /etc/dovecot

Then modify the following variables (they are all there either in comments or not):

hosts = localhost
auth_bind = no
ldap_version = 3
base = dc=example,dc=com
deref = never
scope = subtree
user_attrs =
user_filter = (&(objectclass=inetOrgPerson)(mail=%u))
pass_attrs = mail=user,userPassword=password
pass_filter = (&(objectclass=inetOrgPerson)(mail=%u))
default_pass_scheme = SSHA

With LDAP done, we just need to teach dovecot about how to dump the email the same way it was don the last machine, in /var/vmail/<email dress>. In /etc/dovecot/conf.d/10-mail.conf, add:

mail_location = maildir:/var/vmail/%u

Then we need to enable postfix-auth and auth-userdb correctly. In /etc/dovecot/conf.d/10-master.conf, change your “service auth” settings to this:

service auth {
  # auth_socket_path points to this userdb socket by default. It's typically
  # used by dovecot-lda, doveadm, possibly imap process, etc. Its default
  # permissions make it readable only by root, but you may need to relax these
  # permissions. Users that have access to this socket are able to get a list
  # of all usernames and get results of everyone's userdb lookups.
  unix_listener auth-userdb {
    mode = 0640
    user = vmail
    group = mail
  }

  # Postfix smtp-auth
  unix_listener /var/spool/postfix/private/auth {
    mode = 0666
    user = postfix
    group = mail
  }

  # Auth process is run as this user.
  #user = $default_internal_user
}

And we’re done! ( I think ). I may have missed a step here or there. Leave a comment if you are running into a problem. Chances are, it’s one I hit, and solved.

Next on the chopping block, SVN and SuPHP.

Kai vs LDAP

I’ve previously posted about the success I’ve had with my RHEL6 workstation joining the Windows Domain at the office. This has in-part inspired me to attempt to learn LDAP. I currently use a number of services that would be nice to have a unified authentication mechanism. I frequently use SSH (all the time) on multiple servers for administration work for eSource. I also run the eSource Mail server on Postfix/Dovecot + MySQL, using postfixadmin as my administrative tool. Lastly, my little SVN server that hardly gets any updates, especially now that I’m out of school. Unifying the Authentication across these three services would provide a great deal of flexibility for eSource, as well as my own personal stuff. So here we go.

First, my LDAP server will be CentOS 6 (another motivation for the move to LDAP is that I have to move mail, svn, and web services anyway.) It doesn’t take much to get a slapd service running, but you have to be careful. I thought it was as easy as editing slapd.conf, but it’s not….RHEL6 moved to a new slapd configuration format. Once I figured out where to stick the stupid password, slapd config was done. I installed phpldapadmin, and haven’t had any problems since. The trick has been learning what LDAP is all about. I’m still very confused, but I’m at last limping along. I’ve been able to successfully create an ldap entry, and use it to log in via ssh to my new server. Here’s an LDIF entry that I’ve exported from my running LDAP server, and slightly modified to obfuscate any information I’m concerned about. This isn’t a tutorial, so I can’t guarantee any of this will work cut-and-paste for you.

dn: cn=Kai Meyer,dc=example,dc=com
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
objectclass: posixAccount
objectclass: inetLocalMailRecipient
uid: kaiuser
givenname: Kai
sn: Meyer
cn: Kai Meyer
cn: Master of the Universe
telephonenumber: 555-555-5555
mail: therewasan@emailhere.com
maillocaladdress: therewasan@emailhere.com
userpassword: CENSORED
uidnumber: 1000
gidnumber: 1000
homedirectory: /home/users/kaiuser
loginshell: /bin/bash

I did other sorts of things, like create an Organizational unit, and add the object to the OU by modifying the “dn:” to include the OU after the cn. (If that made sense to you, you should probably be doing this for me.) The only other thing I needed to do to enable SSH + LDAP is to configure authentication for the machine to allow LDAP. This is CentOS 6, so I did it like so:

yum -y install nss-pam-ldapd
authconfig --enablemkhomedir --enableldap --enableldapauth --ldapserver=localhost --ldapbasedn="dc=example,dc=com" --updateall

It ended up looking like this:

[kai@example.com ~]$ ssh kaiuser@localhost
kaiuser@localhost's password:
Creating directory '/home/users/kaiuser'.
[kaiuser@example.com ~]$ pwd
/home/users/kaiuser

Next up, Postfix/Dovecot + LDAP. Then after that, SVN + LDAP.

RHEL6 vs Windows Domain logins

I started my new job at StorageCraft this week. We produce backup and disaster recovery software for Windows servers and desktops. They have grown big enough to organically grow their development team, and pursue new markets. I think I’m under NDA not to disclose the nature of the projects I’m working on, but given my history, it shouldn’t be difficult to figure some of it out.

One of the nice perks they offered was to purchase books or software that I think would make me more effective at work. So I asked for a RHEL6 Workstation License, and was successful in justifying the purchase. Since I am one of only 4 brand-new employees that have any substantial experience with Linux at all, their entire infrastructure uses Active Directory for authentication. Workstation logins, Version Control, Issue Trackers, WiFi, the VPN, and Email all use it. As a sort of last hurrah as a Systems guy, I decided to get my RHEL workstation “on the domain.” What that really boils down to underneath is allowing PAM to delegate authentication out to the Domain Controllers, so things like GNOME and SSHD can authenticate users “on the domain.”

The process really just boils down to popping up a little GUI, joining the domain, and installing winbind-server. The only thing the GUI does that I had to update the smb.conf file for was to use the default Domain for logins. Gnome didn’t like having the backslash in the username, ie: “WDM\kai.meyer”. With winbind running properly, running the command “id kai.meyer” returns valid user information.

Once I figured out how to authenticate (running “ssh WDM\\kai.meyer@kai-rhel6” just felt wrong), what services were needed (winbind, smb, nmb), and which config files were used (winbind actually uses smb.conf), I feel like I could easily teach our IT guys how to deploy a RHEL6 Workstation for Developers, or a RHEL6 Desktop for other positions like Technical Support, and give them warm fuzzies about eliminating those pesky Windows Viruses.

One more benefit of having my RHEL6 workstation on the domain is configuring samba shares to use Domain Authentication to control permissions to files. My local files can be shared over the domain, and access read-write from any other Windows workstation that I’ve logged into with my account.

I realize all these benefits from joining the domain are fairly small in reality. All of the “features” it provides can be done in so many less-convoluted ways. What really makes it worth it is having the IT guys go, “You can do what?!?”

FQDN Regular Expression

I was asked to help create a regular expression to validate that a string is a Fully Qualified Domain Name. Google searching didn’t give me a direct result, but it gave me something close. I found a clever website dedicated to sharing regular expressions called regexlib. On their site, someone posted a regex for MS FQDNs, which aren’t quite the same as regular FQDNs. The rules are a little different. In RFC 1035 section “2.3.1. Preferred name syntax”, we read:

The labels must follow the rules for ARPANET host names.  They must
start with a letter, end with a letter or digit, and have as interior
characters only letters, digits, and hyphen.  There are also some
restrictions on the length.  Labels must be 63 characters or less.

Section “2.3.4.Size Limits” reads:

Various objects and parameters in the DNS have size limits.  They are
listed below.  Some could be easily changed, others are more
fundamental.
labels          63 octets or less
names           255 octets or less
TTL             positive values of a signed 32 bit number.
UDP messages    512 octets or less

Given those rules, I’ve modified the regular expression from regexlib, to be:

(?=^.{1,254}$)(^(?:(?!\d|-)[a-zA-Z0-9\-]{1,63}(?<!-)\.?)+(?:[a-zA-Z]{2,})$)

The differences between the one on regexlib and mine are fairly subtle. Theirs excludes any label that is comprised of all digits, but the RFC only specifies that the first character can’t be a digit (or hyphen.) They also allow an underscore character as part of a label, which is not part of the RFC specification.

The only deviation to the RFC rules that I make is the extra rule that the top level domain (the part that comes after the last ‘.’) must be characters only, and must be 2 or more (.com, .net, .org, .eu, .uk, ect). I can’t find where that is documented though.

Multi-Value parsing with Awk and Bash

I do one-liner shell scripts all the time. One of the things that has always bugged me is having to write a perl script just for it’s parsing capabilities when it’s just a 3 column text file. I always found it difficult to handle parsing lines in Bash when I need to use multiple values in each line. When there’s only one value in each line, I typically use “| while read line; do”. This week, I made good friends with the ‘eval’ bash command. Try this little exercise. I always end up doing grocery lists for this type of stuff, since it’s easy to keep in your head. Create a file (ie: values.txt) with the following text:

apples .24 50
oranges .54 21
grapes .05 100

Now run the command:

$ awk '{printf "fruit[%d]=%s; value[%d]=%s; quantity[%d]=%s;\n", NR,$1,NR,$2,NR,$3} END {printf "count=%s\n", NR}' values.txt

This prints out some handy bash assignment operations. If you eval the result of this command, you can loop around the array with a for loop; like so:

eval `awk '{printf "fruit[%d]=%s; value[%d]=%s; quantity[%d]=%s;\n", NR,$1,NR,$2,NR,$3} END {printf "count=%s\n", NR}' values.txt`; for num in `seq 1 $count`; do echo ${fruit[$num]} will cost \$`echo ${value[$num]} \* ${quantity[$num]} | bc -l`; done

In the old days (a week ago) I would have needed to revert to a perl script to parse each line, just for a little math. This one-liner (even though it’s really long) uses only shell scripting tools that I use on a regular basis. I can crank this bad boy out a whole lot faster than it’s equivalent in perl.