Sponsored by:

Enhancing Pwned Passwords Privacy by Exclusively Supporting Anonymity

When I launched Pwned Passwords in August, I honestly didn't know how much it would be used. I made 320M SHA-1 password hashes downloadable and also stood up an API to query the data "as a service" by either a plain text password or a SHA-1 hash. (Incidentally, for anyone about to lose their mind over SHA-1, read that launch post as to why that hashing algorithm is used.) But the service did become quite popular, although that was just the beginning...

I launched V2 in February and pumped the number of passwords up to just over half a billion. The big difference, however, was the introduction of the k-Anonymity model developed by Cloudflare (special hat-tip to Junade Ali in particular for his work on this). This is what really drove the adoption and within a day, big names were using the service. 1Password was a very early adopter and since then, time after time, I've seen all sorts of organisations using the service. Even just this morning, I was delighted to see it in action at EVE Online:

The anonymity API isn't just great for privacy, it's also great for speed. In that blog post, I showed how I got the speed of the "Range-GET" operation down to only 54ms whilst the old API that pulled a result by either plain text of full SHA-1 hash sat around 81ms. That was a 24-hour report but with the benefit of time, I've now been able to run a full month of data through the report:

Pwned Passwords Volume and Duration.jpg

Pwned Passwords Operations

So execution time of the range search is now taking less than half as long as the old search, it's being used 5 times as much and if Cloudflare already has a request for that range in cache (which it usually does about 80% of the time), it's even faster again. The k-Anonymity range search is absolutely smashing it in every way which makes me very happy 😀

It's also why I'm going to support the anonymity model exclusively to the exclusion of the old models which searched by plain text password or full SHA-1 hash.

On 1 June 2018, all Pwned Password API endpoints that don't enforce anonymity will be retired.

This shouldn't come as a surprise, the Pwned Passwords API documentation has very specifically said "searching by hash may be deprecated in future" and "avoid using this API and refer to searching by range instead" for some time now. Now, some people may well ask "why not just continue to support them both?" and it's a good question, so let me address that in point form:

  1. Obviously anonymity. I don't want to see other people's passwords being sent to this service; I don't want the responsibility if I don't need to have it and frankly, the API consumers shouldn't want to send them to me if they have a better alternative.
  2. The range search has awesome cacheability. Cloudflare is sending most responses back directly from their edge which is good for consumers in terms of speed, and good for me because I don't need to support (and pay for) those requests on the Azure end.
  3. The range search is also much faster on the back end. Read that earlier post about performance for the reasons why, the bottom line is that I pay for Azure Function execution based on how long it takes so faster is always better.
  4. Integration with the range search is dead simple. For anyone already using the old API, it's just a different endpoint then a matter of picking the hash suffix out of the response.

I've left a month and a bit of notice to give everyone heaps of time and I'll be sending out constant reminders via social, on my blog and in my weekly updates. If anyone has any specific concerns not addressed here, leave a comment below or reach out directly.

Have I Been Pwned Pwned Passwords