Leaked Password Analysis – 2011-06 Edition

As most of you likely know, several months ago saw a shift in how a certain type of attack was being done on the Internet. Instead of breaking into a website and simply stealing information, people began breaking into sites to steal information and then release it publicly on the Internet. It is not my intent to discuss the choice of targets or the motivations of these groups. Others have written plenty on this topic and really, if you’re not working for a target or one of the attackers, anything you can say about their motivations is likely to be guesswork at best.

Instead, I want to talk about the passwords. I’ve been following these leaks and collecting password information. My goal is not to break into people’s accounts or to discuss whether or not the leaked data supports the claims of either side. I only have one goal in doing this. I want to find out what I can about people and passwords so I can help everyone choose better ones. So here is my initial analysis. If time permits, I hope to come back to this and do the analysis with more rigor and dive more deeply. However, since my initial rough analysis is done, I wanted to share my preliminary findings. I think they’re interesting, so I hope that you will as well.

 

Data Set

I’ve combed leaked data for all the cleartext passwords I could fine. Realistically, this means that the passwords I’ve analyzed here fall into two categories. The first category is passwords that were stored unencrypted or very weakly. The second is passwords that were weak to begin with and were easily cracked by those who released the data sets or analyzed them later. So, the important takeaway here is that this is not and analysis of typical passwords used on the Internet. This is an analysis of bad passwords used on the Internet combined with passwords that were stored poorly. Still, since I want to learn what not to do, this seems like a worthy use of my time.

The data set exceeded half a million passwords… but likely involved some duplicate records. I hope to tighten up the analysis in my next go-around.

 

Common Passwords

Everyone starts these analysis with a list of the most common passwords. I do not wish to disappoint, so here is what I found.

So what can we learn from this? First of all, note the number of passwords that are just numbers. 123456, 12345678, 12345, 111111, 1234, 1234567, and 123456789 were seven of the top 20 bad passwords. This is ridiculous. Who on earth thinks this is a good idea? A lot of people, apparently.

Second, notice the surprisingly large number of people who thought that trustno1, baseball and superman were good choices. Perhaps choosing passwords based on popular culture is unwise.

 

Password Lengths

I then looked at the average password length. There’s not much of a surprise here, but here’s the graph if you’re interested:

What I found most interesting was how relatively few passwords were seven characters long. I expected six and eight to be large, but not for seven to be so short. Also, note how quickly it drops off after 8. Nine characters and up are ridiculously small.

 

Keyspaces

This is where things get interesting. We have been talking for years about how people should use a mix of lower case, upper case, numbers and symbols in their passwords. I don’t want to bore you with math, but the reason is that the more characters you have to pick from, the longer it’s going to take to guess the password. If, for example, your password is one character long, if you use a lowercase letter and the attacker tries those first, it will only take 26 tries to get it. If you use a character from any of these sets, it will take 26 (lower case) + 26 (upper case) + 10 (numbers) + 32 (symbols) = 94 tries. If your password is longer, then it will be increasingly harder.

Let’s use a few pictures to make this easier to talk about.

 

This is what we’d like to think people are doing. We know that not everyone is following our advice, but at a guess, we’d expect there to be a reasonable mix of people doing it the right way and some overlaps within the other spaces.

 

Our ideal, of course, would be to widen the overlapping space. This way, more people are using more complex passwords and would be safer.

 

… and this is where we actually are today. The spaces aren’t the same size, which isn’t terribly surprising I guess. However, I didn’t expect not only for the special characters space to be so small, but I also didn’t expect the overlap to be so tiny. In fact, of the 519,229 I analyzed, only 315 had a mix of lower case letters, upper case letters, numbers and special characters. No wonder they got hacked. This means that 0.06% of all the passwords were considered minimally secure.

Really… is it so hard to add an exclamation point or question mark in there somewhere?  Here, I’ll even give you some you can use.  I mean, really!?!?!?!?!?!?

 

Other Metrics of Interest

When I compared the list of passwords to itself and weeded out the duplicates, I found that 65.71% of the passwords overlapped. I must say, folks are just not as creative as I had hoped.

For those that follow math, the average entropy score of the password set was 29.63. I hope to make a neat graph comparing entropy to things like length and commonality, but will apparently have to get more proficient with better graphing tools first. My existing tools found graphing 500,000+ data points somewhat challenging.  :)

When I ran the list of passwords against the standard Linux word list, I got 85,196 hits out of 178,049 unique passwords. That’s a 47.85% rate of people that aren’t even trying. Again, we’re talking about the easily-cracked passwords, so this number is inflated… but it’s still much too high.

Surprisingly, I did not see many passwords that were just dates. Those stories of people using their kids’ birthdays as passwords seem to have been exaggerated… or perhaps people today don’t care about their kids very much.  :)

So What Do We Do?

Given that this was a set of easily broken passwords, the key things to do to prevent your password from being broken is to make them not fit these patterns. This means:

1. Use a mix of lower case letters, upper case letters, numbers and special characters. Use at least one of each.
2. Make your passwords longer than eight characters. To lay outside of this data set, 10 would be fine. Personally, I’m going up to 16. After all, if you can remember an eight character password, you should be able to remember two of them stuck together.
3. Avoid basing your password on popular culture, sequences of numbers (or keys on the keyboard) or sports. Those passwords are much more common than you’d think.

That’s it. If you do these three steps, you’ll be well outside of this data set and therefore, much less likely to get your password stolen. Of course, the one thing I couldn’t measure was how much these passwords are shared between accounts of the same person. The 65.71% overlap rate suggests that there is a lot of this going on, but I can’t prove it. Still, it’d be a good idea not to do that.

Do these suggestions sound familiar? They should. If you’re still not following them, maybe you should. We don’t suggest them to be annoying or to help protect against some amorphous threat in the future. We suggest it because if you don’t follow these rules, you will be hacked.

We’ve just seen it happen.

Over half a million times in the last six months.

Firefox and Facebook

I am involved in a great many groups that are (ostensibly) focused on technology or security to some extent.  One somewhat disturbing trend that I’ve noticed in recent months is people complaining about their significant others and how they constantly put their shared system at risk through Facebook.  Now, I could make this post about how being with someone means accepting their flaws along with their virtues or even go so far down the path of “all you ever do is complain, why on Earth did you marry them in the first place?”, but this isn’t that sort of blog.  Instead of doing that, I’ll point out that we have all the tools we need to secure someone else’s connection and you’re having issues isn’t because your spouse is stupid, it’s because you’re lazy.

Here’s how to be less lazy… involving Firefox profiles and Facebook.

This is not another “how to secure Facebook” post.  To do that, please see this post from Sophos.  This is also not about basic Internet security.  No, this is about how to use some built-in functionality in Firefox to create walls between dangerous sites.  By itself, it will help a lot against account takeovers and complex leveraged attacks… but if you don’t follow basic security practices like using complex passwords and not sharing them between systems, the benefit will be limited.  Keep this in mind as we go through this process.

Profiles

Firefox uses profiles to separate different settings.  They are amazingly powerful, and yet, shockingly, people hardly use them at all.  What we’re going to do here is create a specific profile for Facebook use and then adjust the default profile to block Facebook.  The important thing to remember here is that this technique can be used to protect ANY website, not just Facebook.

Let’s start by installing Firefox if it is not already installed.  To do that, just go over to Mozilla.com and download and install Firefox. Once it’s installed, we have to launch the profile manager. The way you do this is going to vary based on operating system. Under Windows 7 and Vista, go to the search box at the bottom left of the start menu and type firefox.exe -ProfileManager -no-remote. If you are running Windows XP, go to the start menu, click Run… and in the dialog, type firefox.exe -ProfileManager -no-remote. If you are running an older version of Windows, just give up now. Those operating systems are dead and cannot be secured. Either upgrade to Windows 7 or look at running an alternate operating system like Ubuntu Linux.

If you are running Linux, you can just open a shell and run firefox -ProfileManager -no-remote

Now we need to create a new profile for Facebook use.  To do this, go to Create Profile -> Next -> Enter Facebook for the profile name and click on Finish.

Now you can just select the Facebook profile and click on Start Firefox.  This will launch a basic web browser for you.  Now we need to configure the appropriate add-ons.

Add-ons

Firefox supports “add-ons” (also called “extensions”) which supply additional functionality to the browser.  Each profile maintains its own set of add-ons, so if you like any of the one’s we’re adding here and want to use them in your regular browsing, you’ll have to add them into the default profile as well.

To select your add-ons, you should open the Firefox menu and select the Add-ons link over to the right.  For the rest of this section, we will be adding each add-on by searching for the name in the search box at the top right and then clicking the Install button by the Add-on name.  The links provided are so you can read about the add-on before adding it if desired.  However, please add them through the Firefox interface so that they will be automatically updated for you.

• RequestPolicy – This prevents the so-called “like-jacking” attack by explicitly allowing the browser to connect to specific sites.
• Web of Trust – This connects your browser to a free service that compares sites you try to visit to known list of bad sites.
• NoScript – This prevents your browser from running scripts except for the ones that you explicitly allow.
• AdBlock Plus – Prevents ads from displaying, however, this may break some games.  If you play games, please see note 1 at the bottom of this post.
• Certificate Patrol – Improves the HTTPS security within Firefox.
• Force-TLS – Allows Firefox to refuse to connect to a site if it is not secure.

Once these are installed, you will have to restart Firefox to activate them.  Either click on of the Restart Firefox links or close the browser and re-launch it using the -ProfileManager -no-remote trick above.

Automatic Tuning

Once you’ve restarted Firefox, it will launch into the automated tuning process and you’ll have to specify some configuration options.

The first thing that will come up is the RequestPolicy configuration window.  By default, it allows for some automated tuning… but this makes it less secure than we really want here.  Uncheck the “International” checkbox and click on “OK”.  We’ll tune the rest of this add-on shortly.

The next dialog is Web of Trust (WOT).  The WOT add-on just needs you to accept the EULA before you proceed.  Read the EULA and then click on “Accept” if you accept the terms of the EULA.

Now you should have four to five tabs open.  The order will likely depend on the order in which you added the add-ons.  We will be tuning the NoScript and ForceTLS later in this process, so just close those tabs.

Web of Trust
This is where things start to get complicated.  The RequestPolicy addon, by default, will conflict with WOT.  You can tell because there is a red flag icon in the bottom right corner.  You need to click on the flag and go up to the “Temporarily allow all requests” option.

NOTE:  This is something you should do only during the tuning process.  Allowing all requests basically turns off the protection that Request Policy allows, and since this is the key protection for Facebook, it should usually be on.

Once this is selected, the page should reload and give you a configuration page for Web of trust. Basic is good enough for us, so just click on Next.

The next option is to register.  You do now have to do this, but if you wish to do so, fill out the form and click Finish.  Otherwise click on the little red X at the top of the “window” in the browser.  Then close the tab.

Adblock Plus

If you chose to install Adblock Plus, this tab will appear.  If you chose not to do this, just skip to the next sub-section. On this pane, you select the subscriptions you want.  Most users will be fine with just EasyList which should be selected by default, so click on Add subscription and that tab will close.

Options

Now we need to tell Firefox that this profile is to launch Facebook.  To do this, click on the Firefox menu and then go to Options and select the top Options option.  (And please accept my apologies for that sentence.)  You should be in the General tab (far left) of the options dialog.  In the area where it says Home Page, please enter in https://www.facebook.com.

Now click on the Content tab.  Where it says Enable JavaScript look over at the right and click on the Advanced button.  In the tiny little window that comes up, uncheck each checkbox and click on OK.  This will help prevent Javascript-based attacks, which are very common on Facebook.  We will protect against the rest of them shortly when we configure NoScript.

Now click on the Privacy tab and select Never remember history in the drop down.  The less data you store, the less there is for an attacker to steal.

Now click on the Security tab.  For the most part, the defaults are good, except that it defaults to storing passwords.  Remember that every password you store is a password that could be stolen by an attacker.   Uncheck the Remember passwords for sites checkbox.  If you have used this profile in the past, you may also wish to click on Saved Passwords and select Remove All.

Now for the complicated step.  By default, most browsers choose user convenience over security.  We discovered this problem back when Comodo was hacked a few months ago, and this is what you need to do to fix it.  Select the Advanced tab.  Then select the Encryption sub-tab at the far right of the list of tabs below the primary tabs across the top.  Click on Validation and click the bottom-most checkbox.  Then click OK to close the sub-dialog and then OK to close the options dialog.  The drawback to this is that if Facebook’s OCSP server goes down, you will not be able to connect.  The upside is that if Facebook is attacked, you won’t be able to connect to a compromised site.

Now it’s time to restart Firefox again.  This will clear the temporary setting change we made and get us to where we can start tuning the system.  Run Firefox with the -ProfileManager -no-remote trick again and select the Facebook profile.  You should automatically-connect to Facebook and be prompted to log in.  Just log in as usual and we can start the manual tuning process.

Manual Tuning

Request Policy

This is going to be the most annoying aspect of accessing Facebook this way, but it is very much worth the extra time it takes.  When you start Facebook, you will see a bunch of missing images and some grey flags in their place.  This is because RequestPolicy doesn’t yet know which sites are safe, so it blocks everything.

To fix this, click on the little red flag icon in the bottom right of your browser (this is in the status bar of the browser window, not in the Facebook section).  This will allow you to let RequestPolicy know which sites can talk to other sites. First we need to go to the Preferences option at the top of the RequestPolicy menu.  Click on the Advanced tab over at the right and then select Allow permanent whitelisting when using Private Browsing.  Now click on the red flag again and allow the two sites akamaihd.net and fbcdn.net to be accessed by Facebook by selecting the option in bold at the top of each sub-menu.

NoScript

Now we need to allow Javascript so more of Facebook will work.  To do this, click on the Options button on the yellow bar along the bottom of the screen.
This works much like RequestPolicy.  Just click on the Allow Facebook.com option in bold and the page should refresh.

At this point, Facebooks should be looking pretty normal.

ForceTLS

Now we need to force secure connections in the event that Facebook changes that option (again).  To do this, right-click on the Facebook page and select View Page Info, click on the Permission tab, scroll down to where it says Secure Connection and click on both checkboxes.

Games

At this point, things should be more or less working securely for basic Facebook services (reading and posting to walls, getting messages, etc).  If you play games, you may have to go through the RequestPolicy and NoScript steps above to allow different sites, but be aware that for every site you add, you increase your risk significantly.

Default Profile

Now we have to tweak the default profile.  Restart Firefox again (sorry), and run Firefox with the -ProfileManager -no-remote trick again.  This time, select the default profile and go into the Add-ons section as before.  This time, we will be adding only one add-on.

LeechBlock

• LeechBlock – Prevents access to specific sites.

After it’s installed and you’ve restarted Firefox, it should come back to the Add-ons Manager.  If it does not, you can get there by going to the Firefox menu, then to Add-ons then the Extensions.  Now click on Options next to LeechBlock.  The Block Set 1 tab should be selected.  Under What to Block enter *.facebook.com.  Then click on Next to go to the When to Block tab.  Just click on the All Day and Every Day buttons as you never want to access Facebook from the default profile.  Now, click on OK to activate this change.

Note, if you are doing this to protect someone other than yourself, you may wish to turn on some other options in this add-on to prevent them from unblocking Facebook.  You may also wish to replace the standard page with one that says that Facebook is only available via the dedicated Facebook profile.  These steps are out of scope for this little How To guide.

Optional Others

If you plan to do anything risky in the default profile, consider using the other add-ons that we used on the Facebook profile.  After you’ve used them a bit for Facebook, it should be pretty easy to adapt them to other uses.  You may also wish to load LeechBlock into the Facebook profile to prevent people from using that profile to go to other common sites (online banking, webmail, etc) from that profile.

You can also create a dedicated Firefox profile for each of these common uses, if you wish.

Desktop Configuration

Now for the final step.  You don’t want people to have to manually type in -ProfileManager -no-remote every time they need to access this profile.  Instead, we’ll modify the Firefox icon on the desktop to do this automatically.  To do this, right click on the Firefox icon on your desktop and add  -ProfileManager -no-remote to the end of the Target section (outside the quotes).  Then click OK to save your change.  Now when you double-click on the icon, you will be prompted for which profile you wish to run.

If you wish, you can read a bit more about Firefox profiles and make an icon that launches the Facebook profile, but this How To is long enough already, so I won’t be getting into it.

There you go, that’s it!  While there’s no “Safe” on the Internet, if you take these steps, you’ll be a whole lot safer than the vast majority of Facebook users.

---

Notes:

1. It would be best if you don’t play games at all on Facebook.  There have been numerous problems with game developers being less than trustworthy… and you probably have better things to do with your time anyway.  ;)   If you must play games, consider using two Facebook accounts and creating a second “Facebook Games” profile to access them in.  This way, if you have your friends in one account and your games in another, a bad game won’t put all of your friends at risk.
2. You should still use a strong password on your account and not share it anywhere else.  If you have a weak password, an attacker can figure it out without your involvement at all, and none of these protections will help.  If you share passwords, an attacker can use your password to steal a lot more from you.  You can generate strong passwords over at Strong Password Generator.
3. You may wish to add different Firefox themes to each profile so there is a visual reminder where you are and what you can do.  You can find lots of Firefox themes at the Firefox Themes Site.
4. If you are technically-skilled, both RequestPolicy and NoScript allow you to export your configuration so you can import it elsewhere.  If you have to set up multiple computers, this can be a time saver (or you can just copy the profile directories).  In case it’s useful, here are my exported policy files:
   • RequestPolicy
   • NoScript

Presentations Posted

I have finally gotten around to purging sensitive information from my presentations and they are now posted. You can see them from the menu by going to “Professional” and then to “Writings and Presentations”. For your reference, the following are new:

• Malware (2009)
• Malware and Organized Crime (2010)
• Natural PCI (2010)
• Telco Security (2010)
• Natural HIPAA (2011)
• Malware and Identity Theft (2011)
• Natural Security (2011 version, originally in 2010)
• Senior Scams (2011)