Email Whiteout: March 1, 2004?
Nov. 23rd, 2003 10:35 am![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
doc_strangeMany clients of ours have been reporting a significant change in the rate of "spam" volume growth (in this case, unsolicited commercial email, mass-mailing worms, and the like) over the last two months. Some, who keep excellent records of filtered, accepted, and rejected mails, have been sharing their email growth numbers with us.
From Jan 2003 through the present, the volume of spam email vs. desired commercial and personal mail was on the rise. From 40% of total corporate mail to 70%, on a gently steepening curve. Projecting the curve, the best formulaic match was geometric: The trend predicted a hyperbolic curve. Indeed, many individuals I know are already reporting over 90% spam (and are using filters as a result). But, the large corporate sites (and AOL's often-cited figures) do a better job as a significant sample set.
The curve projection also showed the "total mail volume" (all mail including spam) would be exceeded by the "spam" curve around April, 2004. Obviously, Y cannot be > X+Y where X and Y are positive integers, so that graphical projection is kind of nonsensical. However, it indicates two trends: the growth in spam and a relative falling off of normal email use. It also indicates something I'll get to, below.
I asked around, and indeed, most people I talked with said the spam volume or effects therefrom discouraged them from using email as much as they used to -- from "exposing" their email address or dealing with erroneous filtering or delivery delays caused by heavy-duty mail scrubbers (that are trying to keep out the spam). Having to worry about picking through filtered mail (for false-positives) becomes a chore when for every 10 good mails, you receive 100 that go into the filter trap. A daily, or twice-daily chore, to pick through for any 'good' ones.
But that's not all. Indeed, just two weeks ago, from all accounts, it looks like we hit the "knee" in the hyperbolic curve -- slightly ahead of the prediction. A revised curve prediction puts us around March 1 as the point where the curves come together.
March 1 is effectively a ballpark prediction of an email whiteout. That is, the volume of spam, if it continues along its present trend, will so exceed the volume of other, normal email correspondence that in effect all email will be "spam." Systems will spend almost all their processing, bandwidth, storage, time, and other resources processing these unsolicited tidbits and worm junk, that direct, personal correspondence will disappear under the flood.
It's already starting to happen.
An alternative interpretation is that, rather than a whiteout, the growth will lead to a dollar-out -- a point at which the cost of processing one direct, personal business correspondence via Internet-based email will exceed its business value. Internal email, inter-partner ("whitelisted") email, and other internal communications (corporate instant messaging (IM), voicemail, and ever-less-expensive cellphone costs) will probably continue, and indeed, increase in volume. Indeed, expect to see large corporations set up inter-corporate messaging solutions including private IM solutions. Many business partners already have private line or VPN inter-connects for the conduct of business transactions. That interpretation is also already being contemplated by some commentators.
Similarly, personal email will take so much time, energy, and passed-on filtering overhead delays or costs from ISPs, that email will start to fall from favor as a personal correspondence medium (and because Internet service providers that filter take on much of the burden, there will be a delay in effect, here). Good olde phonecalls, voicemails, SMS, blogs, web bulletin boards, and so on are likely to push into the gap (and SMS and blogging are already significantly on the rise, replacing personal mailing lists in many groups with whom I have talked).
So where does this leave us? A tragedy of the commons, in effect, will play out. "Unless someone passes a solid, anti-spam law," I hear you say. Ah, no. Over half of the UCE we see right now comes from compromised personal computers -- i.e., a crime has already been committed -- or from overseas, or it's advertising already-illegal items... or all three. The law won't make much difference to this unauthenticated medium.
No, the only way to win may be not to play, at least until the mechanism catches up with the threat model.
From Jan 2003 through the present, the volume of spam email vs. desired commercial and personal mail was on the rise. From 40% of total corporate mail to 70%, on a gently steepening curve. Projecting the curve, the best formulaic match was geometric: The trend predicted a hyperbolic curve. Indeed, many individuals I know are already reporting over 90% spam (and are using filters as a result). But, the large corporate sites (and AOL's often-cited figures) do a better job as a significant sample set.
The curve projection also showed the "total mail volume" (all mail including spam) would be exceeded by the "spam" curve around April, 2004. Obviously, Y cannot be > X+Y where X and Y are positive integers, so that graphical projection is kind of nonsensical. However, it indicates two trends: the growth in spam and a relative falling off of normal email use. It also indicates something I'll get to, below.
I asked around, and indeed, most people I talked with said the spam volume or effects therefrom discouraged them from using email as much as they used to -- from "exposing" their email address or dealing with erroneous filtering or delivery delays caused by heavy-duty mail scrubbers (that are trying to keep out the spam). Having to worry about picking through filtered mail (for false-positives) becomes a chore when for every 10 good mails, you receive 100 that go into the filter trap. A daily, or twice-daily chore, to pick through for any 'good' ones.
But that's not all. Indeed, just two weeks ago, from all accounts, it looks like we hit the "knee" in the hyperbolic curve -- slightly ahead of the prediction. A revised curve prediction puts us around March 1 as the point where the curves come together.
March 1 is effectively a ballpark prediction of an email whiteout. That is, the volume of spam, if it continues along its present trend, will so exceed the volume of other, normal email correspondence that in effect all email will be "spam." Systems will spend almost all their processing, bandwidth, storage, time, and other resources processing these unsolicited tidbits and worm junk, that direct, personal correspondence will disappear under the flood.
It's already starting to happen.
An alternative interpretation is that, rather than a whiteout, the growth will lead to a dollar-out -- a point at which the cost of processing one direct, personal business correspondence via Internet-based email will exceed its business value. Internal email, inter-partner ("whitelisted") email, and other internal communications (corporate instant messaging (IM), voicemail, and ever-less-expensive cellphone costs) will probably continue, and indeed, increase in volume. Indeed, expect to see large corporations set up inter-corporate messaging solutions including private IM solutions. Many business partners already have private line or VPN inter-connects for the conduct of business transactions. That interpretation is also already being contemplated by some commentators.
Similarly, personal email will take so much time, energy, and passed-on filtering overhead delays or costs from ISPs, that email will start to fall from favor as a personal correspondence medium (and because Internet service providers that filter take on much of the burden, there will be a delay in effect, here). Good olde phonecalls, voicemails, SMS, blogs, web bulletin boards, and so on are likely to push into the gap (and SMS and blogging are already significantly on the rise, replacing personal mailing lists in many groups with whom I have talked).
So where does this leave us? A tragedy of the commons, in effect, will play out. "Unless someone passes a solid, anti-spam law," I hear you say. Ah, no. Over half of the UCE we see right now comes from compromised personal computers -- i.e., a crime has already been committed -- or from overseas, or it's advertising already-illegal items... or all three. The law won't make much difference to this unauthenticated medium.
No, the only way to win may be not to play, at least until the mechanism catches up with the threat model.
![[identity profile]](https://www.dreamwidth.org/img/silk/identity/openid.png){kind=small}
Re: Agreed, but slightly missing the point
Date: 2003-11-24 10:38 am (UTC)The random-email-address attacks were perhaps not properly thought through. I gave up generating bounce messages to spammers when my ISP sysadmin told me that these bounce in turn, which create more work for him.
The other hobby horse I ride on the antispam crusade is moving the fight to the analog domain. Flood the spammer with data that has to be sorted by analog means: phone calls, letters. The spammer then has to differentiate, manually, between real user data and fake data. This quickly reduces his profit margin towards zero. I saw a tool on the net that does this: the tool fills out spammer's forms with random but realistic data.
Remember the folks who played that Nigeria scammer for laughs? Imagine if there were a modified Eliza variant to that engaged each of these spammers... they'd have to spend days trying to sort through the real answers from he fake ones. If they received 100,000 realistic responses they'd never be able to find the real suckers.
The way to fight the RIAA is not to send them email; that's processed, sorted, and dumped in the digital realm. It's to call them on the telphone, politely, to complain about their policies. Ten thousand calls a day to them -- and their lawyers -- may not change their minds but will certainly get their attention.