Hashing Email Addresses For Web Considered Harmful

cce writes "The MicroID standard, despite getting thrashed soundly by Ben Laurie two years ago, has since been recommended by the DataPortability Project and published on the user profiles of millions of users at Digg and Last.fm. MicroID is basically a hash calculated using a user's profile page URL and registered email address, producing a token that makes the email address vulnerable to dictionary attacks. To see how easy it was to crack these tokens, I conducted a small study, choosing 56,775 random Digg users, and cracking the email addresses of 14,294 of them (25%) using just their MicroID, username, and a list of popular email domains. Digg has more than 2 million users, and that means half a million of them — mostly people who had never heard of MicroID, and had probably not logged in for a long time — had their email addresses exposed to this trivial attack. I also applied this attack to Last.fm (19%) and ClaimID (34%). Digg and Last.fm have since removed support for MicroID, but the lesson is clear: don't publish a hash of my email address online, guys!"

Solution: salt your emails

[pwnies]

I suppose this is yet another reason why it's nice that a few email services (most notably gmail) allow you to append a string to your email address using the + symbol (e.g. [e-mail] will go to the inbox of [e-mail]). In effect it allows you to "salt" your email, which adds a layer of complexity when trying to match these hashes with valid email (not to mention it allows you to check which site compromised your email if you use different 'salts' for each site you use your address on). If more email services start to allow this (doubtful), more sites start realizing that a + in your email is still a valid email (more doubtful), and more users start using it effectively (even more doubtful still), then I don't think the MicroID will be a huge problem.

Re:Solution: salt your emails

[Rinisari]

Maybe that FOAF could attack ESPN.com, too. I tried registering there for a fantasy football league at work and used [e-mail]. The damned system took the + out, making the address invalid!

superparanoid? regexp

[Tmack]

If you are superparanoid, you can run your own mta, like qmail or postfix, and specify your own delimiter to regexp out of the address in one of the pre-processing filters. With qmail, I believe you could even just edit the qmail-smtpd config/run file (iirc, been a while) and add a pipe through sed to do the dirty work with the addy before the normal pipe through qmail.

tm

Re:Solution: salt your emails

[geekgirlandrea]

Yeah, this can happen, but I dunno that this is as big a problem as you think. Spammers just plain aren't all that bright, and they don't care very much if they miss the tiny proportion of addresses that geeks try to protect like this when there are so many totally unprotected addresses so easy to obtain. It seems like a lot of the time, when they try to harvest addresses, the harvester doesn't realize + is a valid character in an address and only gets the part after the plus sign. I bounce a lot of spam sent to addresses like [e-mail] and [e-mail].

Re:Solution: salt your emails

[statemachine]

And the few times a harvester is correctly written? What then? That's the address that gets spread around. Obscurity doesn't work on the Internet. Just don't post it at all.

But you seem fine with it because you're also posting your personal domain name here, which links to your name and your photo, along with a street address and phone number (which I hope are only P.O. box and a voicemail-only phone service). You're a hell of a lot more comfortable with it than I am. (At least I hope you knew that all that info was very publicly available.)

Re:Solution: salt your emails

[cduffy]

Obscurity doesn't work on the Internet.

So why bother?

Someone who was serious could get into public records and get my address anyhow (owning a house generates lots of public records). Someone who isn't serious presumably doesn't pose a threat. I think the worst thing that's actually likely to happen is 4chan-style harassment, and (1) it's not particularly likely, as I don't hang around those types enough for them to care about me, and (2) if it did happen, countermeasures are certainly available. And, again, (3) if anyone were serious enough about it, they could find all the relevant information through other channels anyhow.

Being nymous online is a Good Thing -- it means people I know IRL can recognize me (I've run into ex-coworkers and old friends I didn't think I'd see again) and it gives me a chance to build a reputation that follows me into Real Life (so potential employers find plenty to recommend me when googling my name). Further, it acts counter to the tendency for anonymous communication to degrade into... well, you're on slashdot; you know exactly what I'm talking about. :)

Postfix Solution

[bill_mcgonigle]

Assuming you're using postfix and virtual, you can do something like this:

main.cf:

recipient_delimiter = +
virtual_alias_maps = hash:/etc/postfix/virtual, regexp:/etc/postfix/virtual-regexp

virtual-regexp: /(.*)\-(.*)@example.com/ ${1}+${2}@example.com

and then you can do:

    [e-mail]

this works for every site I've tried but oracle.com, who apparently doesn't want you tracking their mail. :)

It's not a secret: [e-mail]

[Charles Dodgeson]

I fully agree with the parent. The idea of keeping an email address that you actually use private is several orders of magnitude sillier than thinking your credit card number and social security number hasn't been stolen a dozen times already.

But there is one place I won't "publish" my email address ([e-mail]), and that is in the From line of a Usenet posting. Reply-to is fine, and there absolutely no problem in the body of messages, but tests have shown that putting something in the From line of a Usenet posting will give you a very noticeable increase in spam.

Re:Solution: salt your emails

[cduffy]

Hey -- we didn't arrange our schedules that way on purpose; it just happened as a happy accident. Likewise, I mess with Asterisk first and foremost because I think it's fun, and only secondarily because I dislike phone spam. (We did decide to do the large-dog thing as a security measure, but that was for late-night walks outside, not protection of the household proper -- and any weaponry we may have usable for home defense would have been purchased primary for recreational hunting; that said, I don't disclose the presence or lack of such online). I don't put myself through a whole bunch of hassle because I'm paranoid about security, and I'd probably still decide to be as easily identifiable online as I am had things not worked out that way, on account of the benefits I gave earlier (ability to translate online reputation-building into real-life interactions, which I really do think is a serious and compelling advantage)... that said, when it comes down to defending my decision, the set of happy accidents comes in handy.

I agree with you that paranoia is contrary to happiness -- that's part of why I'm comfortable with having my identity online; if I had to live in a mental state such that I believed people as a whole to be an irresponsible set (or such irresponsible people to be numerous enough to be worth thinking about), that mode of thought would, in and of itself, make me less happy.

Re:They already have your email address

[coryking]

The spammer (or actually, botnet owner who wrote a spam program) has already figured that out by putting a shim inbetween you and your network card. They just sniff your traffic for anything that looks interesting. In fact, I wouldn't be suprised at all that the botnet software will "turn on" when you use hit up gmail.com and can screen scrape the page while you check your email. I would even bet that it can update its screen scraping rules from some kind of distributed network.

Somebody in this thread said spammers are dumb. That might have been the case five years ago but it is not the case now. The "spam industry" has really evolved to the "botnet industry". These botnet people are smart, smart people. Almost as smart as the P2P people in terms of getting around "damage". Shame they couldn't apply their skill and talent to doing something positive for our society though.