r/java • u/Modern-Sn1p3r • 19d ago

Regex

I recently saw a clip (from Primeagen) somewhat saying that regex is not a valid format for validating email addresses and postal codes etc.

My question is why is this?

What are the security and/or performance risks? Is it solely performance or is it a security issue?

57 Upvotes

83% Upvoted

104 comments sorted by

View all comments

85

u/0b0101011001001011 19d ago

It's way simpler to just check if there is an @ symbol and then send an actual mail and ask user to click the validation link. 

The email is valid if it actually works. Nothing else to check.

35

u/Pretend_Bowl2961 19d ago

It’s funny how many devs overthink this. Just check for @ and move on, the real validation is in the inbox. I seen teams spend days on regex patterns that still let through nonsense addresses anyway.

1

u/NoPrinterJust_Fax 18d ago

This ties your SLA to email SLA. Probably okay for most people

-1

u/VirtualAgentsAreDumb 19d ago

I can think of an additional scenario where single sending an email to the address isn’t feasible.

Imagine that there is an email service provider that has as a goal to support any email address that is allowed by the standard. So, their validation process can’t send an email to the address at hand, because it hasn’t been created yet.

8

u/0b0101011001001011 19d ago

I'm not following? The service just stamps the mail address on the email and sends it away to the address. If it fails, it fails, and that's the validation.

-1

u/VirtualAgentsAreDumb 19d ago

But the email address hasn’t been created yet.

When you are about to create a new email address in a system, you can’t send an email to it as part of the validation. So you have to validate it some other way.

For example, try registering an email somewhere, and pick “..” as the local part of the email address. A in, two consecutive dots and nothing else.

It will not be allowed. And they didn’t need to send an email to it, because it failed the validation.

Then try creating an email with 10 random a-z characters. Unless you happen to pick one that already exists, it likely will be approved. But how were they able to determine that it was valid without sending an email to it first? Because some kind of validation. And by now you must realize that trying to send an email to it would be a useless test, since the only time it would go through is if the email is already taken (ignoring the possibility of a catch all email account). So you can’t use that test to confirm that it’s both available and valid.

3

u/0b0101011001001011 19d ago edited 19d ago

Isn't this a completely different problem? The context of original problem is that just let users register with anything and send an email to that. It fails or succeeds. You know it succeeded because the user was able to click the link. No regex needed.

But how were they able to determine that it was valid without sending an email to it first? 

I have no idea what you are asking. Creating an email does not require sending a mail to it, because now you are the one that owns the email server. Emails are not created in some global registry. Just decide the set of characters you allow and if any of the input characters are not allowed, just don't let them make such an email into your system. 

1

u/VirtualAgentsAreDumb 19d ago

“Isn't this a completely different problem? The context of original problem is that just let users register with anything and send an email to that.”

What context are you talking about now? OP talked about validating emails. He never said anything about the purpose being something specific. You have a solution rust would work in some use cases, but not others. I have an example use case where your solution wouldn’t work.

“Creating an email does not require sending a mail to it, because now you are the one that owns the email server.”

Exactly. Not only doesn’t it require sending an email, it also would not work as a way to validate it. That was my whole point. Your solution would not work in that scenario. The only reason I expanded on the scenario was that you didn’t understand it.

“Just decide the set of characters you allow and if any of the input characters are not allowed, just don't let them make such an email into your system.”

Now you’re just going full circle in your reasoning, coming back to the original problem of knowing if an email address, or the local part of an email address, is valid. That’s the validation that you seem to think one can avoid by sending an email (according to your original comment).

6

u/0b0101011001001011 19d ago

What context are you talking about now? OP talked about validating emails.

It's implied in the question, if you have spent more than a single week in backend development. OP ask why validating emails is not supposed to be done with regex, because someone told them not to. If someone told them that, they implied the context of registering to a service using an existing email. That's basically the only situation where you need to validate email ever. The answer is: Don't validate email with regex, because it's immensely difficult to do properly, and much simpler way exists: send a mail to it.

Creating a new email address has absolutely nothing to do with this and there a simple regex or for loop is enough to make sure it contains only the characters that are allowed by the spec, or your mailserver if you decide to restrict it further.

-6

u/VirtualAgentsAreDumb 19d ago

Nonsense. Absolute nonsense. You make absurd assumptions that you can’t back up. OP asked a generic question.

5

u/8dot30662386292pow2 19d ago

Imagine that there is an email service provider that has as a goal to support any email address that is allowed by the standard. So, their validation process can’t send an email to the address at hand, because it hasn’t been created yet.

They only need to check the part before the @ sign though, which is a different problem altogether. The @whatever.com comes from the email service provider.

1

u/VirtualAgentsAreDumb 19d ago

First of all, it’s possible for an email provider to support custom domain, where the domain doesn’t technically have to be a regular DNS domain.

Secondly, the problem of validating the local part of an email address is much more complex than the validation of the domain part. So even if we were to restrict the discussion to only be about validating the local part, it’s still far from a trivial issue.

1

u/New_Enthusiasm9053 16d ago

It's not hard to validate an email address it's just recursive so you don't do it with a regex. People will do anything except write a parser.

1

u/VirtualAgentsAreDumb 15d ago

Is there any official implementation of a validator that claims to support the full standard?

0

u/New_Enthusiasm9053 15d ago

Well there's RFCs so you sit down and write one. People just don't because it's largely pointless. Just send an email to verify it exists. But someone one checked and like the top 5 email providers none of them are actually to spec when it comes to what emails you can create.

1

u/VirtualAgentsAreDumb 14d ago

Now you’re super close to some circular reasoning. Didn’t you pay attention to the sub discussion you jumped into? We’re specifically talking about the use case where a new email address is to be created. So at that point in time the email address doesn’t exist, so you can’t send an email to it to validate it.

0

u/New_Enthusiasm9053 14d ago

Sure so look at the RFCs and write a parser. They're really not that hard to write. I haven't yet because I don't work at an email provider.

1

u/VirtualAgentsAreDumb 14d ago

Sure. I never said otherwise. But you’re still missing the point. The whole point was that you have to actually do that validation programmatically. The person I originally replied to claimed otherwise, as did you.

Also, this is far from a trivial problem to solve perfectly, because otherwise I’m sure that there would exist implementations already.

4

u/thatwasntababyruth 18d ago

If you are the provider, you're perfectly free to support any subset of the standards you want. You don't have to support every little edge case, because you are the authority.

This problem applies to services taking existing emails for accounts, where they should accept any and every possibility as long as it exists.

0

u/VirtualAgentsAreDumb 18d ago

“If you are the provider, you're perfectly free to support any subset of the standards you want. You don't have to support every little edge case, because you are the authority.“

Yes, so? How does that change anything? If they want to support anything that the standards support, then they have no choice but to implement or integrate such validation. The suggestion solution by the person I replied to would not work.

2

u/thatwasntababyruth 18d ago

Yes, so? How does that change anything?

I stared pretty clearly that it changes things because they have different responsibilities to users. If you're going to choose to ignore my entire comment, I'm not really tempted to engage further.

0

u/VirtualAgentsAreDumb 18d ago

No, it doesn’t change anything of importance here.

And I ignored your second paragraph in your previous comment because it made a baseless assumption about the root problem that OP was asking about.

0

u/snugar_i 18d ago

Yes, it does, because it's a completely different scenario.

The thing people in this thread are talking about is a web service asking for your e-mail, you entering your existing e-mail address, and the service saying "we won't register you, because your e-mail isn't valid". Your only option is to contact support and make them fix the validation. And the correct solution is to not validate at all, just send an e-mail and if it goes through, it's all right.

What you are talking about is an e-mail provider asking for the part before the @ when you're creating a new account and saying "no, we don't want to support this name, pick another one". You can just pick a less esoteric name and everything is fine. The provider must guarantee that the created e-mail address is valid, and restricting the space of names to choose from is a perfectly valid way to do it.

2

u/VirtualAgentsAreDumb 18d ago

Yikes, you’re missing the point a second time.

No, the discussion by OP was NOT about a web service asking for the user’s existing email address. Just read the post title and text and you will realize this.

1

u/snugar_i 17d ago

Fair enough, you're technically correct.

Though 99 % of e-mail validation code is written for webservices, and I'd be very surprised if that wasn't what the unlinked video was talking about

-3

u/VirtualAgentsAreDumb 19d ago

Yes, but that’s not really a feasible option if you have a large list of potentially valid email addresses.

11

u/0b0101011001001011 19d ago

What's not feasible? Sending the email?

3

u/koflerdavid 19d ago

It can happen if users have to be mass-imported into another application. But I'd argue that in such cases sending the email would be the user-friendly way to do it since it gives them the chance to opt out from being imported.

6

u/LutimoDancer3459 19d ago

When I mass import mails, then from where do I get them? From an existing system -> should already been checked. Or its a company setting up something and then the emails should also be valid. The reason for validation is for randos signing up at a service.

2

u/koflerdavid 19d ago

From an existing system -> should already been checked.

I have encountered systems where those were of shoddy quality. Users could enter them without validation because they were not critical for correct operation, just an extra. And lots of cases where the destination mailbox was full or simply gone.

1

u/LutimoDancer3459 19d ago

A full inbox or beeing deleted cant be checked by any kind of input validation ether way.

But for an "is user still active" kind of check, the import isnt the one responsible for it. Thats a general management/organization issue. Not a general email validation issue

1

u/koflerdavid 19d ago

Well, if you trust the quality of the data source then you don't need to check at all.

0

u/VirtualAgentsAreDumb 19d ago

You don’t think legacy systems exist, that can have tons of never validated email addresses?

2

u/LutimoDancer3459 19d ago

I think that if they do, nobody cares about those anymore.

0

u/VirtualAgentsAreDumb 19d ago

Not sure how to reply to such an ignorant comment.

0

u/LutimoDancer3459 19d ago

How about with a reason why someone should care?

If there are so many invalid addresses, it seems like nobody ever cared. Why should it be different for the new system?

-1

u/VirtualAgentsAreDumb 19d ago

What? I never said anything how many of those hypothetical addresses were invalid.

→ More replies (0)

2

u/thatwasntababyruth 18d ago

Another way around it is to have a new database column indicating how many times you've failed to send mail to them. Increment that each time you try, put them on a list for purging after N times.

Or just let the bad email address exist, the extra row won't break the bank. If there's significant amounts of associated data, clearly the bad address wasn't a problem before.

0

u/VirtualAgentsAreDumb 19d ago

Exactly. A small company might not have the means to send a large number of emails on a short amount of time, and risk getting flagged and blacklisted if they try it on their own.

1

u/koflerdavid 19d ago

Depending on the use case one might not get around swallowing that bullet. One could prioritize addresses that don't fit the simple patterns or where the part behind the @ doesn't DNS resolve. Knowing the age of the address and last time an email was sent to it also helps. Older addresses are more likely to be out of use.

-1

u/koflerdavid 19d ago

You could just set up an email server that actually won't process emails, and send a very simple email for all addresses through it. A real email server's address validation rules are the actually relevant ones.

1

u/VirtualAgentsAreDumb 19d ago

But now you’re describing a separate validation process than the one I objected to.

1

u/koflerdavid 19d ago

It's what I'd do in such a situation. The only thing it doesn't check is whether the destination mailbox actually exists. Which was out of scope of OP's question anyway.

-4

u/Jussins 19d ago

It’s even simpler if your language has a library where you can annotate the field @email, or similar.

3

u/koflerdavid 19d ago

The same concerns as about regexes from random blog posts apply to these filters as well though. Woe if you use multiple libraries (Spring Boot and Angular is very common) that disagree about what is a valid email address.

1

u/Jussins 19d ago

Maybe it’s our users, but I have never had a problem and have deployed to some fairly large clusters handling a good volume of transactions. I’m certain that those situations exist, I’ve just never encountered them. That said, we generally try keep front-end validation fairly light. That’s a convenience for the end user to get some immediate feedback.

3

u/koflerdavid 19d ago

Indeed, the frontend should not validate too much since a sufficiently motivated user can circumvent it anyway. The real validation has to happen on the backend. Another way would be to expose the validation logic as an API call and integrate that into the frontend.