A new year, and a new lease of life for Lil Pwny, which gets some love with the release of version 2.0.0
The new version can be found on GitHub
Or can be installed from PyPI:
python3 -m pip install --upgrade gitlab-watchman
About Lil Pwny
Lil Pwny is a Python application that provides offline auditing of Active Directory account passwords against Have I Been Pwned. It returns users who’s NTLM hashed AD password matches a hash in the HIBP compromised password list. Such auditing is recommended in the NIST 800-63 Password Guidelines.
Additional features of Lil Pwny also allow:
- Search AD user hashes against your own custom list of passwords. You can curate this yourself, and could include passwords commonly known to be used in your company, as well as ones that contain variations on the companies name and related phrases.
- Return accounts using the same passwords. Useful for finding users using the same password for administrative and standard accounts, as well as multiple service accounts using the same passwords (telltale signs of a lazy sysadmin using the same standard password for every account they create).
Exporting NTLM hashes from Active Directory is explained in this post
Instructions on how to use Lil Pwny are in the GitHub repository.
My other (and now somewhat out of date) posts on Lil Pwny are part 1 | part 3. They give an idea behind the usage of Lil Pwny, even if this post updates the functionality.
Updates
Version 2.0.0 brings the following updates
Faster… a lot faster
Version 1 of Lil Pwny was designed to be functional and, while it got the job done, it did require high resources in spite of using multiprocessing. As with lots of projects when you revisit them after a year, I realised I could do better.
For version 2.0.0 there was one simple aim: Run fast on consumer level hardware.
What started as a goal turned into an obsession for speed, and the result is that Lil Pwny 2.0.0 runs a lot faster than version 1.
From the release notes of version 1.0.0, I can see that auditing ~6800 AD users against 550 million HIBP hashes took 48 minutes (on an instance with 80 cores and 80GB RAM). Running Lil Pwny 2.0.0 on my test devices searching ~8500 AD hashes against the updated HIBP list of 613,584,246 hashes took:
- 6 logical cores – 00:05:57.640813
- 12 logical cores – 00:04:28.579201
Just under 6 minutes with 6 cores… that’s faster than you can make a cup of coffee
This is achieved via much more efficient use of multiprocessing. Before, the application became I/O bound opening up and searching the large HIBP file by each process for each user hash. In version 2.0.0 the logic is switched around by creating blocks of 100MB size for the large HIBP file, reading these blocks in parallel using multiprocessing (the more cores, the more blocks read at once), and searching for matches in the given AD user hashes. This, combined with dictionary processing of users and a few other tricks, has lead to Lil Pwny version 2.0.0 being multiple times faster than version 1.
Logging
Rather than logging results to text files, Lil Pwny now outputs logs in JSON format like a grownup:
{"localtime": "2021-00-00 00:00:00,000", "level": "NOTIFY", "source": "Lil Pwny", "match_type": "hibp", "detection_data": {"username": "RICKON.STARK", "hash": "0C02C50B2B08F2979DFDE12EDA472FC1", "matches_in_hibp": "24230577", "obfuscated": "True"}}and for matches for users using the same password:
{"localtime": "2021-00-00 00:00:00,000", "level": "NOTIFY", "source": "Lil Pwny", "match_type": "duplicate", "detection_data": {"hash": "BD08FEF052D3810335FE1B3D9724EEE9", "users": ["EDMURE.TULLY", "ASHA.GRAYJOY", "OBERYN.MARTELL", "ELLARIA.SAND"], "obfuscated": "True"}}*In hindsight yes, I do regret my GOT inspired test domain after the mess of the final season. It still breaks my heart, but I’m too far in to change it now…
This formatted logging is perfect for ingesting into a SIEM or other log analysis tool, and can be fed to other scripts or platforms for automated resolution actions.
This log output contains some additional information:
- The hash is now included, as well as whether it has been obfuscated or not (see below)
- The number of times the password has been seen in HIBP is now included as well. This can help you make distinctions between high and low risk user passwords. Perhaps you only want to contact users using passwords seen over 100 times, and aren’t so worried about ones who are using a password that is in HIBP only once.
Obfuscating hashes
Lil Pwny now gives you the option to obfuscate NTLM hashes in the logging, for if you don’t want to store or process live user password hashes. This is done by further hashing with a randomly generated salt at runtime. Just run Lil Pwny with the `-o flag set.
Of course, if you do want to see the NTLM user hashes, you can leave the obfuscated flag out of the command. I’m not here to judge what you want to do with them…
Excluding AD computer accounts from input
When collecting NTLM hashes from an IFM dump, the output includes Active Directory computer accounts as well as user accounts. A computer account exists for each device bound to Active Directory and, like users, it has a sAMAccountName and a password that is stored as a hash. This password is managed locally on each machine and is rotated roughly every 30 days, and the new password is synchronised back to the domain controller.
Computer accounts are easily distinguishable from user accounts, as the sAMAccountName contains a $ symbol at the end of the string. As there is little value in auditing these passwords, Lil Pwny now excludes computer accounts when importing user hashes, helping the audit to run even faster.
Future plans
Lil Pwny has had quite a facelift with this version, and it fulfils every requirement I had when I set out to create it. I’ve always wanted to keep password auditing offline so that no password hashes get sent offsite, so I doubt any interaction with the HIBP API will get added to Lil Pwny, but who knows how things may change in the future.
As ever, any questions or if you do have suggestions for new features, feel free to raise an issue or pull request, or ping me directly on Twitter: https://twitter.com/_PaperMtn
