Illegal passwords list

[aurbjo]

It would be great to be able to specify a list of illegal passwords in the config file, or even illegal words in password.

[r2evans]

(Caveat: I'm not a dev with LTB.) Are you talking about a simple blacklist? It sounds like you mean substring matches, meaning !password123! should fail, since either password or (at least) password123 should not be a substring.

On a cursory search, there are API-based subscription services (such as https://www.passwordrbl.com). I think this is different than what you are suggesting (and considerably different to implement).

I assume you mean a "static" blacklist ("static" in that its updates are either manual (by the sysadmin) or manual/versioned (by the maintainers of LTB-SSP). There are several known lists (such as http://web.archive.org/web/20130605082350/http://www.isdpodcast.com:80/resources/62k-common-passwords/, among many), so finding a reliable and reasonable source should just take a little time to research. I don't have much experience looking for them, so that link is just an example; I can't speak to it.

Some discussion questions:

• should this be case-insensitive?
• should it look for (simple) 1337-speak transliteration?
• what about looking for reversals? (321wrodssap)
• if substring matching is done, is there a point at which leading/trailing characters make it alright?
  (difference between aaPASSWORD123aa and &13as!&8sd@@PASSWORD123@@f7y*sd)

Obviously the combinations of all this needs to be considered for performance: a great password-matching blacklist does nothing if it takes too long to process and the page times out.

I suggest a first cut on this would be literal string matching, with later extensions to include substrings (with padding limits) and reversals.

What do you think, @coudot, do you find utility in having a blacklist? (I am confident that a lot of my authenticating-users will curse at me for enabling this :-)

@aurbjo, if he likes the idea (and doesn't have a starting point), it might help if you found an acceptable starting point for the list. Think: freely distributed, possibly updated (not antiquated), small enough, and easily parsed (perhaps just a literal text file, one-per-line).

[coudot]

Yes, we could indeed use a simple text file as a blacklist, this file could then be updated by any means outside SSP. We can provide a simple blacklist at installation.

[r2evans]

As a thought, would it be a bit robust to use something like cracklib for checking more than just a dictionary? It supports more than just blacklists, though I admit I don't know how to configure its various checks.

[aurbjo]

In my case i am just thinking about a simple list like @coudot is suggesting, this would deny users setting passwords like "qwerty12345", "password123" and so on. I also see users that have their company name and the current year in their password and it's not a good practice.

[coudot]

Cracklib usage can be another feature.

By the way, the support of cracklib in PHP seems experimental: http://php.net/manual/en/ref.crack.php

[r2evans]

(Essentially the same link I posted earlier.) Yes, and furthermore it was removed from base-php (around php-5.0) and pushed to be an extension, so that says that it is not something everyone will need but it has been in the ecosystem for many php versions.

I'll counter with this: I recommend against reinventing the wheel. Even if it has more features than originally suggested, php-crack has fast lookups and attempts to address the performance issues that large lists will bring. As a bonus, it can comment on password-strength (admittedly a contentious and subjective metric). The biggest detriment in my mind is that it isn't a flat text file (it needs to be compiled into the files, as commented on the link you provided), so in that vein it has a little more required investment on the part of the user.

If you think php-crack is too much (for now), I suggest finding another already-existing php package that permits fast lookups in a text file. There's no need to reinvent "lookups" (and therefore less work for coudot to test and maintain).

[coudot]

Indeed, sorry, I did not see the previous link.

If some of you want to propose some code, I'll be happy to review it.

[r2evans]

@coudot, do you have a sample LDAP image (e.g., docker) for testing? It's much easier when testing on something other than a production environment.

[r2evans]

Hehe, @coudot, I guess I didn't read all of the page I originally provided ... last "update" to php-crack was in 2005. Doesn't really inspire confidence ...

So perhaps the wheel can be reinvented a little here, simplistically.

Some thoughts, let me know if you have reservations:

• Using $out = system("grep -q ...", $retval). My primary motivation is efficiency, since I know of no methods that will be able to do yes/no pattern "a match exists" as quickly as grep and family. Using -q means that it can short-circuit: as soon as a match is found, it returns with no other searching, perhaps a slight efficiency. (This also means we need to look at $retval instead of $out.)
• Considering allowing compressed lists, thereby requiring zgrep and bzgrep; this can be determined by filename and runtime.
• In order to protect against code-injection and/or command-injection when using system(), I'm thinking it may be safer to write the new password into a temp file and use grep -q -F -f ptnfile dictfile (the -F is to ensure it is used as a literal string so that regex chars aren't mistakenly taken as patterns).

[plewin]

If the goal is to implement a blacklist of passwords (and not a blacklist of words) in efficient way I would suggest filtering out the lines not conforming to the password policy, save the list sorted in a cache file, then read the cache file and find if the password is present using a binary search.

For example banning all passwords of this list of "one million of the most common passwords" ~ 9 Mio uncompressed

1. load the whole file with file() (< 70 Mio RAM usage in php 7)
2. filter out all passwords not conforming to password policy
3. sort the array (can be already sorted)
4. serialize() (serialized string < 24 Mio RAM in php 7 if nothing is filtered out)
5. dump to file blacklistcache.txt

a) read and unserialize blacklistcache.txt (I'm not sure if file_get_contents + unserialize is better than a file() EDIT: no it's not, see benchmark below);
b) perform the binary search with a function adapted/similar to this one

edit: renamed steps
edit2: added "no it's not, see benchmark below"

[r2evans]

@plewin, you suggest doing all of that for each password change? I haven't benchmarked it, but that seems a bit extraneous. Perhaps the first five steps could be avoided since the program is already checking if the new password meets the password policy (otherwise no dictionary lookup is required).

I understand the suggestion to use a binary search; assuming that the file is sorted (which, given your first five, would be assured), this would most certainly be faster. However, are you sure even this will be faster than using an external grep?

Back to your first point: the OP did request blacklisting passwords; I wonder if it makes sense to be that literal, since defeating it is as simple as adding one character to the password (from password123 to password123! might work).

Thanks! I'll see if I can muster up some benchmarking between external grep and an in-memory binary search in PHP.

[plewin]

@r2evans Steps 1, 2, 3, 4, 5 are done only one time to compile the cache. It will require a script that can be launched manually or automatically if we detect that the cache is older than the dictionnary. Steps a) b) must be done each time there is a password change and only after the new password is valid according to the password policy.

With very basic tests on my side it looks like the binary search idea is not a such good idea after all. It's better only if the dictionnary is already in ram or the call of the file() to load the dictionnary in php is faster than spawning grep with exec.

Is it okay to add a dependency on a external command ? exec isn't portable.

IMHO it's okay if "password123!" passes the blacklist with "password123". An attacker would use the dictionnary, if "password123!" isn't in a dictionnary of the N most common passwords it should be ok to use it.