Regression in pam_listfile module in RHEL5.9

Hot on the heels of the recent support ticket we opened with RedHat that has been finally resolved. For those who have RHN access below is the direct link: https://access.redhat.com/knowledge/solutions/328433

In a nutshell. After upgrading to RHEL 5.9 we lost the ability to ssh into any of our servers where pam_listfile was configured and the following error messages were registered in the logs instead:

Feb 27 13:51:41 host1 sshd[2649]: pam_listfile(sshd:account): Refused user abc for service sshd
Feb 27 13:51:41 host1 sshd[2649]: fatal: Access denied for user abc by PAM account configuration

pam_listfile configuration was configured as below:

account    required     pam_listfile.so onerr=fail item=group sense=allow file=/etc/security/groups.allow

Historically we had a duplicate local group in /etc/group with same id and name as ldap group and as we initially suspected that was the root cause:

In pam-0.99.6.2-6.el5_5.2, pam_listfile used the getgrent glibc function to fetch group information. This function call is non-selective, it fetches information about all groups from all NSS sources.
With the update to pam-0.99.6.2.12, pam_listfile uses the getgrnam_r glibc function call to fetch group information. This function call is selective, and it fetches information about only one group. Since it is a requirement that group names and group IDs should be unique across all identity sources, it stops once a single instance of the group name is found.

If you’re facing a similar behavior just double check for duplicate entries in the identity database.