It sounds like your problem is with the way providers handle email and not email itself. Email is actually a really nice protocol. It’s got so much fault tolerance built into it. I could take my servers down for 24 hours, and none of my customers would miss an email.
Yes, there is definitely a spam problem, but overzealous spam filters are not the fault of email, they are the fault of email providers.
As much as I hate Gmail, at least they are pushing for everyone being required to use SPF and DKIM. That alone will eliminate a huge portion of the spam problem.
Also, email isn’t the only protocol with a spam problem. I get so many spam messages on SMS, Facebook (back when I used it), Telegram, etc. Basically anything that allows someone to send a message without two-party consent first (like scanning each other’s QR codes) is going to have a spam problem if it’s popular enough.
It sounds like your problem is with the way providers handle email and not email itself.
No. Providers handle mail this way because they have no choice to do so.
You are stuck between two major Issues.
On one hand you can have your anti-spam very lenient and receive pretty much everything. But if you do you will get more phishing and malware ridden mails. So the users will be exposed to one of the most dangerous vector of infection.
On the other hand you can have a super aggressive spam filter but some mail will be dropped. Whether an email notifications or the contract of the year for a business. It’s no matter. It might never be delivered.
And since we have to block millions of spam mail everyday we have to block them silently because if you respond to certain malicious SMTP server online they will just spam you.
In reality businesses are used to email so that’s what is commonly used.
But it’s far too unreliable to communicate with clients of that business. You can’t just have an important contract sent as an attachment by mail with some chance that it will be silently dropped at some point.
The simple fact that you can send an information to someone by email and it might be silently dropped without you ever being aware of it should IMO have led to the conclusion that it should never be used for anything remotely critical.
If it’s important it shouldn’t be an email. The reality is millions of dollars worth of business conducted solely through email conversations. And also a very lucrative business of spam.
Even businesses are often spammers or as they may call it “gray mail”.
No email providers will guarantee you a 0% fault spam filtering.
Not Gmail either.
As much as I hate Gmail, at least they are pushing for everyone being required to use SPF and DKIM. That alone will eliminate a huge portion of the spam problem.
It’s a good thing Gmail does that but it helps only their users right now (since February’s changes). If your business communicates with thousands of small domains on small providers it will take another decade for every SMTP server to fix their s***. And even then there will still be spam.
What’s the difference between a spammer going through all the hoops of creating a mail domain and a new business ?
They both would have “legit DNS records” but would both have trouble sending mail to Gmail at first.
Because Gmail cannot know if you are a spammer that setup a new disposable domain or a serious actor in email that just wants to communicate with you.
Truthfully Email is a terrible protocol that cannot be fixed with yet another layer of duct tape. You will never have any guarantee your mail is delivered. There is plenty of communication systems that’s will tell you it’s delivered or not.
Again, your problem is with the way providers handle email. It would be perfectly possible to deny email that’s flagged as spam, then the sender would get a bounce notification. “Dropping them silently” (which actually means accepting them and delivering them to a spam folder in this context) is a choice that providers make. It’s already general practice to deny email from an IP address that’s been blocklisted.
Also, spammers aren’t going to spend the money to buy and set up domains if each one is blocklisted before it makes a profit. My own email service will mark something as spam if it fails FCrDNS, SPF, and DKIM. Gmail went one step further and doesn’t even consider FCrDNS.
And again, any communication method will have a spam problem if it is popular enough and it allows non-two party consent messaging. Email’s popularity is the reason it has a spam problem, not its protocol design. And any distributed system cannot guarantee delivery. If my server tells your server it’s delivered, you just have to trust it, no matter what protocol you’re using.
By dropping silently I meant really litteraly. If you answer to SMTP commands, you are not silent. You essentially say a spammer server that you are a valid target and that they can go on.
It’s not even a question if spammer buy domains to spam.
It’s well known and the reason why commercial products provides a feature to filter too fresh domains.
There are procedures to “warm-up” an IP if you are a large provider and if you don’t do it and attempt to send a lot of mails to Gmail this will not work. It’s not just about DNS records. You could have donne everything perfectly DNS wise and still be blocked by Gmail servers.
You should take a look at the requirements of Gmail for large providers. As far as I recall Gmail does check FcrDNS since last month. On top of more requirements for authentication.
Still you can’t just buy an IP, a server, set MX, SPF, DKIM, DMARC, ARC?, FcrDNS and expect large amounts of mail to go through right away.
And again, any communication method will have a spam problem
The major issue here is that anybody can send any email to whoever. Most communication apps won’t let you do that certainly not like emails.
You can’t open WhatsApp and start spamming the whole world.
You basically can only do that with phone calls and emails ?
So no, SMTP/IMF has rotten foundations. No matter how many (optional) protocol you add on top, it will always be such an hassle to maintain and there will be always people who can’t afford that much effort.
Small businesses having to set that up just to reach Gmail is a big problem that they usually externalize with Outlook365 and so on.
Again, Gmail calls the shots because they are the leader. But on paper my fully unauthenticated mail from Barack.obama is perfectly RFC compliant and legit. These protocols that are essential are optional at the end of the day. They became virtually mandatory because of the spam issue and Gmail pushing in the (right) direction because they have leverage.
I don’t see your issue with dropping a connection before issuing any SMTP commands. Your problem is with not being able to determine delivery status, right? If your server never even gets to send the message, then you know with 100% certainty that the message wasn’t delivered. And if it’s denied, you know with near certainty that it wasn’t delivered. (I don’t know of any servers that will issue a hard deny after receiving the message and then still deliver it, but that’s technically possible.)
I have read Gmail’s requirements, and I’m familiar with IP reputation. I didn’t mean that they don’t check FCrDNS, I meant that only having that is not enough. They now require both SPF and DKIM. Whereas my service will still accept your messages and not automatically mark them as spam if you only pass FCrDNS.
Generally if you’re getting your emails denied right off the bat, it’s because your IP or the block your IP comes from already has a bad reputation (basically any IP a cloud provider will give you). But yeah, you don’t want to spin up a server on a brand new IP and start firing off 10,000 emails a day, just like you said you don’t want to fire off 10,000 messages a day on WhatsApp. That’s a bad idea for any platform.
WhatsApp is not distributed, nor is it an open protocol, so that’s right out. It will never be the standard.
Gmail only calls the shots for Gmail users. If you never interact with Gmail users, you don’t have to obey any of their requirements. Like imagine a system that you’ve set up to receive notification emails from your own servers. You don’t have to obey anyone’s rules.
Your spoof mail may be perfectly valid for the base ESMTP spec, but there is not one single email provider on the planet that only considers that spec. Email isn’t just one spec. It’s a system that’s made of many specs and common practices, some required, some de facto required, and some optional.
I work on email systems everyday.
Please don’t let this protocol survive.
Forget emails that is functionally a terrible communication tool.
You never know if it will be received by the recipient. There is always false positive false negative classification in spam.
SMTP is an outdated protocol that needs to die.
It sounds like your problem is with the way providers handle email and not email itself. Email is actually a really nice protocol. It’s got so much fault tolerance built into it. I could take my servers down for 24 hours, and none of my customers would miss an email.
Yes, there is definitely a spam problem, but overzealous spam filters are not the fault of email, they are the fault of email providers.
As much as I hate Gmail, at least they are pushing for everyone being required to use SPF and DKIM. That alone will eliminate a huge portion of the spam problem.
Also, email isn’t the only protocol with a spam problem. I get so many spam messages on SMS, Facebook (back when I used it), Telegram, etc. Basically anything that allows someone to send a message without two-party consent first (like scanning each other’s QR codes) is going to have a spam problem if it’s popular enough.
No. Providers handle mail this way because they have no choice to do so.
You are stuck between two major Issues.
On one hand you can have your anti-spam very lenient and receive pretty much everything. But if you do you will get more phishing and malware ridden mails. So the users will be exposed to one of the most dangerous vector of infection.
On the other hand you can have a super aggressive spam filter but some mail will be dropped. Whether an email notifications or the contract of the year for a business. It’s no matter. It might never be delivered.
And since we have to block millions of spam mail everyday we have to block them silently because if you respond to certain malicious SMTP server online they will just spam you.
In reality businesses are used to email so that’s what is commonly used.
But it’s far too unreliable to communicate with clients of that business. You can’t just have an important contract sent as an attachment by mail with some chance that it will be silently dropped at some point.
The simple fact that you can send an information to someone by email and it might be silently dropped without you ever being aware of it should IMO have led to the conclusion that it should never be used for anything remotely critical.
If it’s important it shouldn’t be an email. The reality is millions of dollars worth of business conducted solely through email conversations. And also a very lucrative business of spam.
Even businesses are often spammers or as they may call it “gray mail”.
No email providers will guarantee you a 0% fault spam filtering.
Not Gmail either.
It’s a good thing Gmail does that but it helps only their users right now (since February’s changes). If your business communicates with thousands of small domains on small providers it will take another decade for every SMTP server to fix their s***. And even then there will still be spam.
What’s the difference between a spammer going through all the hoops of creating a mail domain and a new business ?
Not much. Both mynewlegitEmailDomain.com and SpammerWho UnderstandsDNS.com are essentially the same for a spam filter.
They both would have “legit DNS records” but would both have trouble sending mail to Gmail at first.
Because Gmail cannot know if you are a spammer that setup a new disposable domain or a serious actor in email that just wants to communicate with you.
Truthfully Email is a terrible protocol that cannot be fixed with yet another layer of duct tape. You will never have any guarantee your mail is delivered. There is plenty of communication systems that’s will tell you it’s delivered or not.
Again, your problem is with the way providers handle email. It would be perfectly possible to deny email that’s flagged as spam, then the sender would get a bounce notification. “Dropping them silently” (which actually means accepting them and delivering them to a spam folder in this context) is a choice that providers make. It’s already general practice to deny email from an IP address that’s been blocklisted.
Also, spammers aren’t going to spend the money to buy and set up domains if each one is blocklisted before it makes a profit. My own email service will mark something as spam if it fails FCrDNS, SPF, and DKIM. Gmail went one step further and doesn’t even consider FCrDNS.
And again, any communication method will have a spam problem if it is popular enough and it allows non-two party consent messaging. Email’s popularity is the reason it has a spam problem, not its protocol design. And any distributed system cannot guarantee delivery. If my server tells your server it’s delivered, you just have to trust it, no matter what protocol you’re using.
By dropping silently I meant really litteraly. If you answer to SMTP commands, you are not silent. You essentially say a spammer server that you are a valid target and that they can go on.
It’s not even a question if spammer buy domains to spam. It’s well known and the reason why commercial products provides a feature to filter too fresh domains.
There are procedures to “warm-up” an IP if you are a large provider and if you don’t do it and attempt to send a lot of mails to Gmail this will not work. It’s not just about DNS records. You could have donne everything perfectly DNS wise and still be blocked by Gmail servers.
You should take a look at the requirements of Gmail for large providers. As far as I recall Gmail does check FcrDNS since last month. On top of more requirements for authentication.
Still you can’t just buy an IP, a server, set MX, SPF, DKIM, DMARC, ARC?, FcrDNS and expect large amounts of mail to go through right away.
The major issue here is that anybody can send any email to whoever. Most communication apps won’t let you do that certainly not like emails.
You can’t open WhatsApp and start spamming the whole world. You basically can only do that with phone calls and emails ?
So no, SMTP/IMF has rotten foundations. No matter how many (optional) protocol you add on top, it will always be such an hassle to maintain and there will be always people who can’t afford that much effort.
Small businesses having to set that up just to reach Gmail is a big problem that they usually externalize with Outlook365 and so on.
Again, Gmail calls the shots because they are the leader. But on paper my fully unauthenticated mail from Barack.obama is perfectly RFC compliant and legit. These protocols that are essential are optional at the end of the day. They became virtually mandatory because of the spam issue and Gmail pushing in the (right) direction because they have leverage.
SMTP on its own is trash.
I don’t see your issue with dropping a connection before issuing any SMTP commands. Your problem is with not being able to determine delivery status, right? If your server never even gets to send the message, then you know with 100% certainty that the message wasn’t delivered. And if it’s denied, you know with near certainty that it wasn’t delivered. (I don’t know of any servers that will issue a hard deny after receiving the message and then still deliver it, but that’s technically possible.)
I have read Gmail’s requirements, and I’m familiar with IP reputation. I didn’t mean that they don’t check FCrDNS, I meant that only having that is not enough. They now require both SPF and DKIM. Whereas my service will still accept your messages and not automatically mark them as spam if you only pass FCrDNS.
Generally if you’re getting your emails denied right off the bat, it’s because your IP or the block your IP comes from already has a bad reputation (basically any IP a cloud provider will give you). But yeah, you don’t want to spin up a server on a brand new IP and start firing off 10,000 emails a day, just like you said you don’t want to fire off 10,000 messages a day on WhatsApp. That’s a bad idea for any platform.
WhatsApp is not distributed, nor is it an open protocol, so that’s right out. It will never be the standard.
Gmail only calls the shots for Gmail users. If you never interact with Gmail users, you don’t have to obey any of their requirements. Like imagine a system that you’ve set up to receive notification emails from your own servers. You don’t have to obey anyone’s rules.
Your spoof mail may be perfectly valid for the base ESMTP spec, but there is not one single email provider on the planet that only considers that spec. Email isn’t just one spec. It’s a system that’s made of many specs and common practices, some required, some de facto required, and some optional.