Hashing Email Addresses For Web Considered Harmful 155
cce writes "The MicroID standard, despite getting thrashed soundly by Ben Laurie two years ago, has since been recommended by the DataPortability Project and published on the user profiles of millions of users at Digg and Last.fm. MicroID is basically a hash calculated using a user's profile page URL and registered email address, producing a token that makes the email address vulnerable to dictionary attacks.
To see how easy it was to crack these tokens, I conducted a small study, choosing 56,775 random Digg users, and cracking the email addresses of 14,294 of them (25%) using just their MicroID, username, and a list of popular email domains. Digg has more than 2 million users, and that means half a million of them — mostly people who had never heard of MicroID, and had probably not logged in for a long time — had their email addresses exposed to this trivial attack. I also applied this attack to Last.fm (19%) and ClaimID (34%).
Digg and Last.fm have since removed support for MicroID, but the lesson is clear: don't publish a hash of my email address online, guys!"
Re:Solution: salt your emails (Score:5, Insightful)
Re:Solution: salt your emails (Score:4, Insightful)
Except that once the salted email is found, everything between the @ and the + will just be discarded.
They already have your email address (Score:5, Insightful)
This concern that you may have your email address *discovered* by spammers because you post it on a web page is so 5-years-ago. They already have your email address, and they probably didn't get it by scraping web pages.
When you have sent a couple emails out with a given address, you can figure that at least one of them will to sit around in someone's Outlook mailstore for the next couple years. (Someone you know uses Windows!) When that person's computer gets infected with spam gang malware (as they all do), they have your address.
Once of them has it, they probably all have it.
Re:They already have your email address (Score:3, Insightful)
> Once of them has it, they probably all have it.
But they don't know that it is yours. They can spam you with it but they can't use it for anything else.
Re:A better solution? (Score:4, Insightful)
Use gmail. I'll get a thousand or so spams a month, but I've had maybe four make it to my inbox in the past three years.
It obviously doesn't eliminate the problem of spam, but in theory if it didn't make it to anyone's inbox, idiots would stop acting on it and suddenly spam wouldn't be profitable and would fizzle away.
Flawed study? (Score:2, Insightful)
Re:They already have your email address (Score:2, Insightful)
I'd argue that the added value of a spammer getting an email address connected to your online "identity" -- your user profile, recently-played Last.fm songs, favorite Digg articles, etc -- makes getting your email from a MicroID a little more valuable than the ordinary harvested email address. Plus, they don't have to bother confirming the address to see if it's still active (Digg already did).
Re:They already have your email address (Score:4, Insightful)
They can spam you with it but they can't use it for anything else
Actually, in addition to spamming you, they can use your email address in the from and reply-to field for their next spam run.
Ask me how I know.
Re:Solution: salt your emails (Score:5, Insightful)
Spammers aren't bright? So spam filtering is easy, right?
One (partial) solution is to have large providers provide alternate domains that you can register throw-away addresses. For instance, under Google Account settings, you might have the option to generate an address from cephelo@gmail.com and assign d785jd47fj@southeast.gmail.com and allow you to record a note that you intend to use d785jd47fj@southeast.gmail.com as your Amazon.com user ID.
As time progresses, Gmail can show you stats that, for example, 100% of e-mail on d785jd47fj@southeast.gmail.com is spam - "Do you want to delete this account?" and poof - the spam stops. Now that address automatically becomes a honey pot.
Why is this a big deal? (Score:5, Insightful)
You are worried because someone, if they really wanted to send you some mail, could go to the trouble of doing a CPU-intensive search against some hash shown on a website and find out that ultimate, embarassing secret: your *email address*??
What gives? Email addresses are designed to be public. If you don't want people you do not know to be able to contact you, then you are free to drop all mail from unrecognized addresses. If you want to set up some kind of secret knowledge that people must have in order to contact you, then ask them to put a particular word in the subject line when first sending you a message. Either of these does not rely on keeping the address secret, which just isn't likely to happen.
The only thing more broken than trying to keep an email address secret is trying to make a 'private' web page by keeping the URI secret. Again, the system is designed so that the address itself is not sensitive, but other information such as a password or PGP key can be.
Actually, what it reminds me of most is the crazy situation in the US where a basically public identifier, the social security number, is abused as some kind of secret token. Hence all the fuss made when it is possible to find out someone's SSN. The answer is not to add more and more baroque means to stop the SSN from leaking out: one breach, and it's no longer a secret.
I understand the desire to stop spam address harvesters, but really, there are hundreds of web sites which display email addresses with only light obfuscation, enough to stop a harvester bot but not a determined human being (or someone determined enough to use an OCR engine). The kind of hashing talked about here is way more difficult to undo than that. If you are even more paranoid, you need to revisit your assumptions of what is public and what is secret.
microid doesn't seem to factor into it (Score:3, Insightful)
It seems like the attack is just taking user names and other publicly-known data trying to determine an email address from them. Spammers don't need microid to confirm that their guess is correct; they'll just send to all 50 or 100 top email domains, hoping to get a hit.
The whole point of MicroID is that if someone knows your email address, they can tell that you are the author of the page. If your email address is easy to guess, then your email address will be revealed, _whether_or_not_ there's a microid here, there, or anywhere.
If an email address is easy to guess, then the email address is easy to guess. Not clear what new ground we're covering here.