I've accepted that it's best to treat people like grown-ups and if there's '@' and '.' and it's retyped then it passes. Someone can easily submit a fake name or phone number or street address, and e-mail's no different.
If they get it wrong, intentionally or not, then they don't get their receipt, confirmation, validation link, etc. and I believe in most cases the incentive is there for them to get it right.
In the rare case where there's some incentive to circumvent the system and this has some measurable impact on a site, then more validation may be warranted. Otherwise, why worry about it?
Your recommendation is almost as wrong as the regexp in the article, as both will reject the perfectly valid postmaster@ai address (check it out, "dig -t mx ai" will return a result, so that address must exist). The only thing you can be sure about in a non-local email address is that it contains at least one @ sign. More are allowed (for source routing), but hopefully no server is still configured to honor that, so you might get away with requiring exactly one @. Everything else is evil.
Oh, and don't forget to make sure that one component in your spam^W email processing chain correctly encodes unicode charaters in the domain part into punycode.
In which you are assuming you will receive a proper bounce, which is not warranted. Checking bounces is something you can do in addition to checking whether the user didn't screw up his email address.
True, but you have to balance that against the small but nonzero number of people put off by an extra text field. Plus, I would find email repetition more annoying if I didn't always do Cmd-A/Cmd-C/Tab/Cmd-V, and in this case the repeated field won't catch any errors.
The fact that you know the shortcuts for select all, copy, and paste puts you in the top percentile of users. Most people don't even know that's possible, and certainly not with keyboard shortcuts.
(The point being that for most users the faster approach that requires less thinking is to type it twice. Sometimes I do things the "slow" or "long" way when coding because it doesn't require a mental shift from the task at hand.)
The point being that for most users the faster approach that requires less thinking is to type it twice.
Spoken like a touch-typist with a short email address. I feel confident in claiming that annabelle.t.johnson@woodandplaster.co.uk will be looking for that copypaste button. It's not like copypaste is a new technological innovation - it's practically the most used feature of personal computing, after backspace.
That's true. My main point was the first one. The second point was mainly to illustrate that annoyance isn't hypothetical; I'm annoyed by an extra text box.
I agree they are annoying. I guess I'm just less annoyed than you are. Cmd-a Cmd-c Tab Cmd-v has a very negligible impact on the amount of keystrokes I make in a day. Typing my address again doesn't even make a real difference now that I think about it, 3 seconds or so.
Have you ever abandoned a sign-up because of the e-mail confirmation?
You'd need something more along the lines of a spell checking algorithm to do that sanely. Don't abuse regular expressions for things they're not good at.
Yeah, that python script (from StackOverflow London - http://norvig.com/spell-correct.html) that learned what was correct from a large number of words would work. Analyse your DB of users' email address url (after @) that have registered fully and provide a "Did you mean.." type response if a new users email url was not found and something close (within 2 alterations) to it was.
To let users revisit their email addresses without adding a second field for copy-pasting, Russ Unger of UserGlue and Jonathan Knoll found a nice solution:
My routine for dealing with a second e-mail field is shift+tab, ctrl+a, ctrl+c, tab, ctrl-v. Usually auto-fill takes care of the first fields anyway, never of the second one. It doesn't help me in any way.
Most forms asking for an email address use the same field name (or one of a limited number of field names), so the browser remembers what you typed last time, and it'll suggest it to you in a pop-up for autocompletion. All it takes is to type one letter, press down, and enter.
My thoughts exactly. There are so many websites that tell me my valid email address is invalid that it's not even funny. These people then have to deal with phonecalls and lost business, because their form won't even submit without a valid email address (and why should I change if they're the ones that suck).
BTW, the email address that doesn't work is "jon-whatever@jrock.us". The .us confuses people and the - confuses people. WTF!?
The have a lot more problems to deal with if they have to deal with all the people that include a stray space in the address.
There are regexes out there that catch all valid addresses that people reasonably use (including those with dashes and ending in .us) and the fact that all kinds of incompetent developers use a homebrewn regex is no reason to lower the best case scenario to "just don't validate". Just do it right.
If they get it wrong, intentionally or not, then they
don't get their receipt, confirmation, validation link,
etc. and I believe in most cases the incentive is there
for them to get it right.
Well, I hope you have a large enough support team, because with any reasonable amount of customer growth, you'll soon be swamped with support emails that say "I didn't get my ..., where is it? You suck!". Validating the email address (which includes checking for common typos in domains) is a service that reduces frustration and the amount of customer support needed.
Even if they've typed it correctly it still can be blocked for various reasons.
You will still need a support mechanism (doesn't have to get all the way to a person) to sort out undelivered verification mails. It's just how it goes.
Still, some rudimentary validation can prevent cases of someone entering the wrong value (e.g., just their username) or an incomplete value (e.g., scott.trudeau@gmail ) -- which I see with some regularity ~10k signups/month. If this lowers support email volume by even just 10%, that's not insignificant.
Along those lines, I've settled on the following overly permissive regex: /^[^\s@]+@[^\s@]+\.[^\s@]{2,}$/ -- it makes sure it looks something like an email address (a@b.cd)
RFC 5321 section 2.3.5 specifically prohibits TLDs from receiving email. Other RFCs back that up, actually including RFC5322 (not directly, but in wire format).
That said, we see some TLDs run MX's. I think at least some portion of them sell the mail received there to spammers. Seriously.
You obviously know a lot more on this topic... but where exactly does it say in that section that TLDs are prohibited from receiving emails?
From my reading, it seems to suggest the opposite, e.g. "In the case of a top-level domain used by itself in an email address, a single string is used without any dots."
I've re-read that section many times and forgive me if I missed it — it's late! =)
People have been posting conflicting responses throughout this thread suggesting that TLDs can/cannot receive emails, and I'd really like to know which should be valid... thanks!
interestingly i count 19 TLD's with mx records. but judging from your earlier post you know as well as i do (i use <single character>@w.tf for many purposes) that there are pretty much no web forms on the planet that would take such an address. tis a shame, but such addresses while pretty nifty are also frequently unusable. not to mention trying to explain to someone that that is really your email address and if it doesn't work they should talk to their ISP/mail client author/etc...well, suffice it to say they won't be getting in touch with you that way anyhow. (be that a blessing or curse, it's just reality)
There are many other uses for this regex than registration systems. We have at least two situations where a staff member would type in a customer mail address, and some get it wrong. Confirming won't help, and getting it wrong costs lots of time to fix. Using a better regex costs very little.
There are more reasons to check for an email than simply validation. Perhaps you want to detect emails in a field and let people click/tap on them to send that person an email? Don't want to let people send email off to foo@bar.baz now, do we?
You assume an email regex is only used for validating addresses. It can also be used to parse text for email addresses (for formatting or mining purposes maybe).
Nothing is finished, nothing is permanent, and nothing is perfect.
In particular, one of the evaluation tests used here is wrong: it requires failure-to-match on a TLD with a digit in it:
numbersInTLD@domain.c0m
In fact, IDN TLDs will have digits in them. An internet-draft is in the works to replace RFC1123's IDN-unfriendly implication that digits in TLDs are illegal:
A nice try at pedantry, which I would usually respect, but in the domain of internet standards with which I familiar, you are wrong.
The existing specs are in conflict, with the more recent ones (such as IDN) allowing digits. Internet authorities, including ICANN, have enabled domains with digits in TLDs; software which is far more foundational than any web-app's email validation regex has been updated.
Registration of names in some of these digited-TLDs has begun; you can visit these TLDs with your browser; your users can have functioning email addresses on these TLDs.
If your app rejects such email addresses because of slavish compliance with imprecise language in a 31-year-old RFC, you'd be the one violating prevailing standards, which are a function of more than just formal IETF RFCs.
That the Internet-Draft I referenced may soon become an RFC is just cleaning up loose ends on a change that's already happened. This final step isn't even strictly necessary for the de facto standard to have changed by consensus among practitioners. Plenty of vibrant well-understood standards never reach RFC status, nor pass through any formal standards body. The standard is ultimately what people do, not what someone once-upon-a-time decreed.
At the risk of sounding flip, not supporting punycode sounds like a feature. Getting internationalization right is clearly important, but punycode as a means of doing so? It's one of the many things that make me weep for my industry.
In fact, I often wonder if punycode is a prank that got out of hand.
UTF-8, on the other hand, would have been excellent for this purpose.
True. It doesn't. But as far as I know, there is no deep technical reason why it can't. The dots in domain names are '\0', and a few characters ('@', in particular) need to remain reserved, but other than that, what stops UTF-8 from being used?
It does not look like any of these can detect or were tested against quoted-local-part addresses. As I understand it, the local part can be quoted to allow illegal characters to be used, e.g. "John Doe"@example.com
I fully understand that these are not in common use, but they are part of the RFC and may be in use somewhere.
His test also considers the % sign invalid, yet I've both sent and received email that required the % for the mail to be routed/relayed correctly (think "in care of" or c/o).
Granted, that was 17 years ago, but who's to say it's not in use somewhere?
I never understood the purposes of the "perfectly valid by the RFC" email regex. You may be able to with 100% accuracy say that something should be an email address, but you'll never be able to tell if it's a valid account on the server, or if the server even exists.
Email validation is user input validation; you are protecting the user from some cases of erronous input, and yourself from stuff that doesn't even look like an email.
I find these large catch-all email regexps silly for two reasons:
1. They are hard to write, hard to understand, and hard to maintain.
2. Most importantly, they are difficult to understand for users. "You entered an invalid email". Now what? the user asks.
This is why e-mail validation should be done in steps. Here is some Rails pseudocode:
validates_format_of :email,
:with => /@/,
:message => "Needs to contain an @."
validates_format_of :email,
:with => /\.[^\.]+$/,
:message => "Has to end with .com, .org, .net, etc."
validates_format_of :email,
:with => /^.+@/,
:message => "Must have an address before the @"
validates_format_of :email,
:with => /^[^@]+@[^@]+$/,
:message => "Must be of the format 'something@something.xxx'"
Much easier to write, much easier to maintain, and much better error messages to the users.
So many web services do this wrong, that it isn't even worth doing it right: no one is going to complain that your service doesn't accept their "wacky! quoted"@email.address
Sometimes, you aren't validating a whole string, you are searching for email addresses in a sea of text, or an arbitrarily delimited, user-entered list of contacts.
Support usernames with alphanumerics, dashes, underscores, periods, and plus signs; Require a single @; Support domains with alphanumerics, dashes, and at least one period. Screw anyone with something more complex than that. Done deal.
That is part of Lepl - http://www.acooke.org/lepl/ - and although it's implemented in a recursive decent parser, much is compiled to regular expressions for efficiency. So you get the best of all worlds: regexp efficiency; parser accuracy; standards based.
Seriously, I can't think of a single good reason why you would want to check whether an email address is "valid". What you should be concerned about is whether or not the address works (and usually, can/does the person who just signed up actually read and reply to email to that address).
Hypothetically, if an invalid address works (due to bugs in mail systems) -- then it works, and the only problem with accepting such an address is that the bugs might get fixed. If an address is valid, there's no reason to assume that it will work or that it belongs to the person who signed up. It isn't even a good way of detecting typos; transpose two characters in an email address and it will most likely still pass your validation.
After this I almost stopped paying attention: "It's my philosophy that it's better to accept a few invalid addresses than reject any valid ones, so I'm shooting for 0 false-positives and as few false-negatives as possible."
But then I looked at the regexps and they miss an absolutely trivial fact: valid email addresses can end in a dot. "jemfinch@supybot.com." is just as valid (more so, in fact) than "jemfinch@supybot.com".
What's the deal with using a port number in an e-mail address? I can't imagine many systems supporting that. Anyone got any more info on it that doesn't involve me digging through 101 pages of RFCs? :-)
I don't think that's valid. I think they misread the RFC. First of all, an IP address (address-literal) is supposed to be enclosed in [square brackets]. The colon is only allowed to designate address family, as in "foo@[IPv6:XXXXXX]".
It thinks baddomain@300.0.0.1 is valid, and thinks decimaldomain@2130706433 (localhost) is not valid.
It also thought user@3ffe:1900:4545:3:200:f8ff:fe21:67cf as invalid.
I wrote an article "Validating Email Address in Web Forms - The Hazards of Complexity" that discusses several comprehensive email validation libraries and common problems with complex methods of validation.
As a typical optimizer, I was always trying to reduce my source code a few more bytes or speed up my processes a few nanaseconds here and there. I was so proud of myself. Until I tried to maintain my own slickness.
There's a fine line between clever and practical. Why do I have a gut feeling that this approach is way over that line?
I think a basic token based parser might be more appropriate for thorough validation of email addresses. I see it similar to trying to match XML or HTML with regexps, very common but so very easy to break. You can't go to wrong parsing an address token by token, can you?
AFAIK a naked IP address as domain like IPInsteadOfDomain@127.0.0.1 doesn't work (at least in some mail servers like Postfix they are bounced with a "bad address syntax error). The address has to be like IPInsteadOfDomain@[127.0.0.1] and I don't see this case covered.
I mean... Why do this? Just, why? It's almost unreadable.
Just write a short 20-line function which validates an email address. Use if statements. Write comments. Then verify that your algorithm does in fact handle all corner cases, just like the regexp does. (Your verifications will be in the form of short, simple unit test case functions.)
To encourage regexp abuse like that is to encourage bad programming.
I've often wondered why we don't have any standardized test cases for email validation. A simple list of known good, and known bad addresses to use as test cases would really help.
You could almost make a game out of finding valid addresses that are not matched, or invalid addresses that are false positives, or optimizing candidate regexps. The list could be continuously growing as new variations are discovered, and tested against previously submitted candidate regexps.
Another know-it-all web developer guy who thinks he got his regex right...
These are valid addresses that he rejects:
user@ua (.ua = Ukraine)
user@km (.km = Comoros)
user@ne (.ne = Niger)
Many ccTLDs have MX or A records pointing to real MTAs.
Based on the HN title I thought it was going to be an article describing a post-it found on Fermat's dressing table mirror.
Instead it's a list of mostly correct regexes. As Miracle Max might have observed, mostly correct is partly imperfect, and partly imperfect is Not Perfect.
practically speaking, most respectable email services will have an MX record. i tested it on about 30,000 emails & a simple MX lookup caught about 40 gmial's and only 1 false positive.
Merely looking up MX records is something you do for a _domain_ and it's no different than the DNS requests your computer makes when you go to a website.
Regex is the only real sensible way to validate strings until something better is found. Even if you just wrote code to do it manually, you'd really just be writing a verbose and poorly implemented finite state machine that globbed symbols together, which in the end, would just be inferior to writing a well tested Regex string.
Regex can be easier to read if you have something do a graphical expansion for you. Otherwise, it's write-once, read-never.
> Regex is the only real sensible way to validate strings until something better is found.
There are plenty of ‘better’ (in the sense of ‘more powerful’) string-validation techniques. For example, lots of grammars are expressed in BNF; the languages that can be so expressed are (if I remember my Chomsky hierarchy correctly) the context-free grammars, a strictly larger class than the regular languages. The extra power comes from the fact that they have the expressive power of a finite-state automaton augmented by an (infinite) stack. (It's fair to argue that it's not ‘really’ infinite, since a computer's memory is finite; but, in that sense, real-life computers will never be Turing complete.)
(Of course, common ‘regular-expression’ libraries aren't actually regular any more, because of added features like capture groups. I don't know if they recognise all CFG's, though; I suspect not.)
Given this, why would we use regular expressions? Well, by intentionally sacrificing power, we can achieve faster matching (http://swtch.com/~rsc/regexp/regexp1.html) and, probably, lower memory usage. Sometimes this trade-off is worth it, even if it means that the match must be somewhat fuzzy; but sometimes one needs a precise match, and regexes just aren't up to the job.
> Given this, why would we use regular expressions?
If I can get a way with a simple regexp I will use it. If it starts getting complicated I would use something else.(e.g. top down parser or a combination)
> If I can get a way with a simple regexp I will use it. If it starts getting complicated I would use something else.(e.g. top down parser or a combination)
That's just silly. Depending on what's in the string, writing a parser might be much better than a regex. Lots of parser libraries already out there, too.
I can agree with this sentiment, there's also nothing wrong with writing a parser that breaks the input up into pieces and validates those individual parts using specialized regular expressions.
True, though it's EBNF with a bunch of explanatory text and annotations.
What would be interesting, but I can't find with some googling: Has someone implemented a parser-generator based on the spec? The ideal would be that the parser specification looks a lot like the RFC, since then you'd have more confidence it was actually correct (and it'd be easier to maintain for future changes).
There was a long post on the topic of 'regex-vs-parser' for email on reddit a while back. I hope I'm pointing to the correct person, but he wrote a parser in Haskell that validates against RFC5322: http://hackage.haskell.org/package/email-validate
If I remember correctly my theory of languages course a parser (implemented in a turing machine) is more powerful than a reggex (which is a finite state machine).
If they get it wrong, intentionally or not, then they don't get their receipt, confirmation, validation link, etc. and I believe in most cases the incentive is there for them to get it right.
In the rare case where there's some incentive to circumvent the system and this has some measurable impact on a site, then more validation may be warranted. Otherwise, why worry about it?
Also: HTML5. ;)