Author: mike

This week Jim Allchin wrote on the Vista Blog a rebuttal to claims that some of the most recent Windows attacks will also be security holes in Vista. 

First off, for the record, I am a fan of Vista for security reasons – it makes it far more difficult for viruses to compromise the system.  It’s definitely a step forward.

However, I am a bit surprised by Jim’s reluctance to admit that Vista is vulnerable to security holes just like XP.  Sooner or later, Vista is going to be hit by a nasty virus or worm, despite the best that Jim’s team has been able to do so far.   He should not be ashamed of this, it is going to happen, and Vista SP1 will fix a whole series of problems. 

The one question I wanted to know which I haven’t seen answered yet, is if you take this same set of 10 attacks, how many will compromise a fully-patched XP system?  I think the answer is zero?  If so, it isn’t clear that this study proves any strength (or weakness) in Vista yet.

In my last article, we talked about spam as the problem which SMTP can never solve.  So what is the solution? 

First, let’s be clear on the goals:

1. Reliable transport of email
2. Efficient transport of email
3. No spam
4. Ability to send a message just by knowing a user’s address, nothing more.  If we make it more complicated, users will not switch.

Sender Challenge

My first choice for reducing spam is to use a CAPCHA or another challenge-response system the first time a sender sends email to a particular recipient.  This would mean that the first time you tried to send email to a Joe, you would be challenged with a CAPCHA-like system to verify that you are indeed human.  You don’t need to know anything about the Joe other than his email address, but you do need to be able to decipher this code.  This simple challenge would kill most all spam, as spammers would no longer be able to send millions of emails per day.  Once you’ve sent email to a recipient and passed the challenge, that recipient could add you to the list of “no challenge required”.  Communicating further with that user would not require additional challenges.  There is a drawback to this, however, which is that spammers will start trying to figure out who is on your whitelist, and forging emails from your approved contacts.  Still, this will be much more difficult than the current system where spammers can simply flood the net.

There are some of downsides to using a Sender Challenge.  But I think these are surmountable.  First, mail would need to be able to lookup challenges in real time, rather than the current store-and-forward technology which is used today.  With SMTP, “email relaying” is a core concept for delivery of mail.  After all, it was designed in the 70’s/80’s when the Internet was an interconnect of many different networks, many of which were disconnected for most of the day.  Most mail servers today intentionally disable mail relaying, because it is a spammer’s dream.  Relaying could still exist, but only if the challenge could still be done in real time.  Switching to a real-time protocol seems pretty practical at this point.

Another problem is how to receive automated email from legitimate sources.  For example, you really do want Amazon to send you an email when your purchase has shipped.  But Amazon’s computer won’t be able to pass your email-client’s challenge.  To overcome this, I think you’d need to specifically give Amazon a pre-approved token for sending email.  Your mail server could be configured to determine the lifetime of that token.  You could either allow Amazon to send you email via that token forever, or for a short period of time, or for just 3 emails.  The receiver would be in control. 

DiffMail – A Pull-based email approach

In researching new SMTP mechanisms, I came across DiffMail.  DiffMail is also a proposal for altering SMTP to make it better able to deal with spam.  It does have one fantastic recommendation – pull based email.

Today, SMTP pushes email from the sender to the recipient.  That is, when the sender sends email, the recipient’s server is required to accept it.  DiffMail proposes that we switch it around.  Sending an email would merely send an informational notice to the receiver saying “I have mail for you on my server if you want it.”  The sender would be required to keep the email on his server until the recipient picks it up.  (Keep in mind the user would never see this – this is just what the protocol is doing on behalf of the user) 

When SMTP was designed, a pull-based approach was not feasible – you couldn’t guarantee the sender’s server would be up when the receiver wanted to read mail.  Today, however, this is not an obstacle.  This model again clips the wings of the spammer, because it requires the spammer to remain online in order to deliver the message.  And of course, if the spammer’s content is online, it is much easier to pinpoint the spammer and stop him altogether.  Further, if we add a little cryptography to the format of email, we can force the spammer to either have a sophisticated spam server application or keep a lot of email stored on his own disk.  Both of which help reduce his ability to send spam at large scale.   The end user, of course, would never see any of this “push” or “pull” of email.  It would all happen under the covers of the email client.

An interesting feature emerges from switching to the pull model.  Have you ever wished to “retract” an email after you sent it but before it was read?  This model would allow you to edit the email as many times as you wish before the receiver downloads it.  Of course, if the receiver pulls it right away, you may not be able to retract. 

DiffMail proposes the pull-based email with the Sender Challenge as an optional component.  But I don’t think pull-based email is enough.  Imagine arriving at your machine and having to sort through 200 potential “new contact requests” to find the one person that is legitimate?  It is really the same problem as manually filtering out spam email, except that you would only have the sender’s email address to use for determining whether you would open the message.

There is a big risk with DiffMail, however.  With SMTP, the sender delivers the email content to the recipient immediately (disregard mail queues or relays for the moment).  In the new model, the sender most host a copy of that email on his server, waiting for the intended recipient to come back and fetch it.  How will it know if a bad guy isn’t trying to steal the email by pretending to be the real recipient?  DiffMail proposes to solve this by basically making the email contain a really big random number to hide the content.  But it isn’t a very safe solution.  I do believe the security issues can be overcome, but they are definitely not trivial.

Sender Challenge or Pull-based Email or Both?

The Sender Challenge system seems pretty straightforward to me to implement.  It fits within the email model that we have today, but introduces an optional challenge which the sender must be able to respond to.  Receivers of email need to be able to identify which people sending email to them need to be challenged and which do not.   That is a slightly tricky concept, but manageable.  The drawback to only using a Sender Challenge is that spammers could continue to exist.  Their new job would be to find ways to beat the challenge- either by random guessing or more sophisticated techniques.  If they can figure out ways to do this, we’ll have spam again.

The pull-based email system makes it costlier for the spammer to send mail at scale.  It also is a somewhat more logical way to communicate given universal access to the network.  However, it does represent a major architectural shift, and it will be a challenge for ISPs to deploy.

For the best spam protection, the combination of the two may be the best bet, but will obviously force a lot of changes to our existing systems.

Compatibility with Existing SMTP

It does seem attractive to make this system compatible with SMTP.  DiffMail attempts to this by proposing some modest additions to the SMTP protocol.  However, I think this is not the right approach for a couple of reasons. 

First off, even if we only change the protocol slightly, the mail servers all need to change radically.  With SMTP, we all deal with our outbound mail servers (SMTP) and our inbound mail storage (POP/IMAP/WebMail).  These are distinct servers.  Using any pull-based email mechanism will require the sender’s outbound mail server to now store data.  Further, routers and firewalls will need to be reconfigured.  Everything in the system is different from a deployment perspective, so why pretend it is the same via the protocol?   It does allow a recipient to receive both types of mail via a single server, which is helpful if your correspondents aren’t willing to move from SMTP.  But, in that model, you have no benefit over existing SMTP, as the spammers can still get to you.

Another problem with maintaining SMTP compatibility is that the protocol is not very efficient.   Sending each email via SMTP is at least 4 round trips to the server.  Using a model of sending all headers in one half trip and receiving the responses in the second half trip would allow for much lower cost server deployments.  If we’re breaking the world, we may as well do it right.

Comments? 

It seems to me that SMTP is showing enough strains with spam that it is time to invest in a new protocol.  If you don’t know what SMTP is, well, you probably aren’t interested in this article.  But maybe RFC 2821 can help.

I don’t take this proposal lightheartedly.  Replacing a protocol is extremely hard and expensive to do.  It’s as crazy as a plan to change all cars on the road to use a new style of brake lights.  With the billions of deployed mail clients and millions of deployed mail servers that are all working, it is virtually impossible to replace them all – unless there is a really compelling reason to do so.  Perhaps there is.

The problem with SMTP is spam, of course.  SMTP simply does not provide protection from spam.  It is so bad that today most users receive far more spam than legitimate email.  This has been a nuisance to end users, but is also a very costly problem for mail administrators and even for internet bandwidth as a whole.  Mail administrators today spend a significant amount of time and energy managing whitelists and blacklists for who can or cannot send email.  This is a manual process.  The spammers keep changing their addresses, and the cycle goes on.  So far, the spammers are winning, and there is currently no plan to change it.

I think we need to change it now.  SMTP was great for getting email started.  But it doesn’t work anymore, as defined by RFC 2821 itself:

“The objective of the Simple Mail Transfer Protocol (SMTP) is to transfer mail reliably and efficiently”

There is no doubt that today SMTP meets neither of these objectives. 

First, due to spam filters, email is no longer reliable.  Every email user has lost messages due to errant spam filters.  We don’t know how much we miss, but it is significant. And a mail transport is either reliable 100% of the time, or it is not reliable.  SMTP is not reliable.

Second, again due to spam, email is not efficient.  When a user must filter out 100 junk messages in order to find one real message, that is inefficient.  And most experts agree, this trend is getting worse, not better.

Our current path of improving spam filters ad-nauseam will never make SMTP able to accomplish it’s primary objective.  We need something better.  And while we’re at it, maybe we can finally add some interesting new mail features too.

More thoughts coming in my next article.

I’ve been a big fan of Foxit’s PDF reader for a while.  It is a 1.5MB download, and it is blazingly fast to display PDF documents.  I must admit, with version 1.3, I did notice some PDF documents (maybe 5%?) were unreadable for various reasons by Foxit.  This was frustrating, because I really didn’t want to download Adobe’s 60-ton Acrobat.

Good news – Foxit has a new version 2.0 available for download, and it seems to be solid.  I not a good enough PDF tester to know how good it is, but it is the only PDF viewer I have installed, and I haven’t seen any errors yet.  If you haven’t tried it, you should, it will change the way you feel about PDF!

Let me know if you’re a rock-solid programmer and would be interested in working with me on a fun, email-related project in C#.  Could turn into something great; it all depends on you.  I won’t post more details here, inquire if interested.

The Windows Live team just posted the top searches on Live.com for 2006.

Yahoo posted it’s top 2006 searches last week.

Google has yet to weigh in, but here are last year’s top searches.

Interestingly, Microsoft’s top search was for Ronaldinho, while Yahoo’s top search was for Britney Spears.  Ronaldinho doesn’t even rank on the Yahoo top-10, and Spears ranks #4 on Live.  Likewise, #10 on Live is Rebelde – apparently a popular Mexican television show.

I’m guessing these rankings are not very scientific.

Opt-in UIs are all the rage in software these days.  With prices dropping, software vendors need you to opt-in to the stuff they know you don’t want so that they can stay in business.  It may be an opt-in for the company’s XYZ service, or it may be an opt-in for the company’s XYZ mailer, or it may be an opt-in to having your kidney donated to monkeys.  How do you get users to opt-in when you know they don’t want to?

Today I ran across a sneaky UI which does this very well!  It is a simple matter of mixing metaphors between checkboxes and radio buttons.  What happens if you display checkboxes to the user, but make them behave like radio buttons?

In typical user interfaces, when you offer a user a choice of something, you’ll use a “radio button”.  The user can select one item from the list, and the user knows that when they select one choice, it will automatically unselect the other choices.  It’s like the old multiple choice tests in school. For example:

Would you like your burger:
With Cheese
Without Cheese

Similarly, checkboxes are used when you offer a user multiple choices, and the user can independently check off zero, one or many of the items they want. For example:

What condiments would you like on your burger:
Ketchup
Mustard
Mayonaise

What this tricky site did was to use checkboxes when it really needed a radio button.  This completely fools the user.  I saw the checkbox, clicked it, expecting it to uncheck, and then thought I must have miss-clicked when it didn’t uncheck.  Only by checking the second item in their list did the first uncheck!  Strange!

This is pure-genius!  It is so easy for the user to click the checkbox, expecting it to uncheck, and then proceed. The company providing the site not only gets you opted in when you didn’t want to be, but it also has a plausible deniability if ever questioned on the matter!  (Oh, we thought this was a good UI.  The user can opt-out, see, just check that box…)

Since a few of my colleagues have thought I’m too pessimistic on Microsoft, I thought I’d share with you my thoughts on Vista.  This weekend, I had to reinstall the OS, and I was somewhat lamenting that I don’t yet have a copy of Vista to install.  I don’t know if I’ll go right out and buy a copy (I’m just too cheap).  But here are some things I look forward to:

1) Starting your PC quickly.  Turning the PC on and off should be lightning fast.  The Vista team agrees, and they’ve been working on it.  Jim claims that a PC left on 24×7 will save about $50 per year in electricity.  Well, we already discussed that I’m a cheap bastard, so you know I’m excited about that.  There is also an interesting technology called ReadyBoost/SuperFetch which claims to more predictably know which applications you need loaded into memory before you need them.  It’s hard to know if it works, but it sounds like a good step.

2) Security.  As a developer, security tends to get in my way, so I don’t care for it much.  But there is no doubt in my mind that Vista’s addition of integrity checking is going to help.  The truth is that most hackers comprimise systems by using the same types of tricks over and over.  Eventually these exploits become known and tools are created so that ‘script kiddies’ can use them on their own.  Vista conquers several of these, and comprimising one area of the system is less likely to give you a ticket to other areas of the system.  I particularly like the fact that you can’t run anything which needs elevated privileges into the startup sequence (which hackers previously used to reinstall their malware on every reboot so it would never go away). 

Unfortunately, the basic problem of human error (“Wow, someone sent me an email with Free, Live, Hot, Naked Women!  Let me run this application!!”), remains unsolved.  The elevation prompt keeps the lawyers happy though.

3) Windows Movie Editing.  Like most everyone with a PC, I have tried the Pinnacle Movie Maker software that came in various bundle deals over the past 10 years.  But, like everyone else, each time I tried I discovered it was horrible and basically didn’t work.  I don’t know why video editing has remained elusive on the PC, but it has been broken for so long, I’m still a bit skeptical about Vista.  Sheepishly I admit, however, that I’m looking forward to trying this out.  Somewhat related, here is an article which says that Vista’s graphics/media upgrades will create $12 in economic growth for every $1 of vista earnings.  In other words, because it will be easier to create content, users will do more of it, and that will drive tech growth in other businesses, for peripherals, cameras, etc.  I hope he’s right – everyone is a winner!

(BTW – what do you think this gadget does after the countdown reaches zero?)

Every time I debug with someone else at their machine, they are surprised by this little trick.  So I will post it here.

In Windows programming, you often have Win32 calls which return an error, and you then need to call GetLastError() in order to find the real error number.  If you are in the debugger, you don’t need to modify your code to call GetLastError at that point. 

Instead, just add “@err” or “@err,hr” to your debug watch window.  Now it’s more like looking at errno.