Thursday, August 06, 2009

AppRiver: SaaS security provider sets standard for rapid response

On July 28th I was happily catching up on my RSS feeds before getting ready to head of to Las Vegas for DEFCON when a Dark Reading headline caught my eye.
Tim Wilson's piece, After Years Of Struggle, SaaS Security Market Finally Catches Fire, drew me in for two reasons.
I'm a fan of certain SaaS Security products (SecureWorks), but I also like to pick on SaaS/cloud offerings for not shoring up their security as much as they should.
The second page of Tim's article described AppRiver, the "Messaging Experts" as one of some smaller service providers who have created a dizzying array of offerings to choose from.
That was more than enough impetus to go sniffing about, and sure enough, your basic, run-of-the-mill XSS vulnerabilities popped up almost immediately.



Not likely an issue a SaaS security provider wants to leave unresolved, and here's where the story brightens up in an extraordinarily refreshing way.
If I tried, in my wildest imagination, I couldn't realize a better disclosure response than what follows as conducted by AppRiver AND SmarterTools.
Simply stunning.

Let me provide the exact time line for you:
1) July 28, 9:49pm: Received automated response from support at after disclosing vulnerability via their online form.

2) July 28, 9:55pm: Received a human response from support team lead Nicky F. seeking more information "so we can look into this".

3) July 28, 10:27pm: Received a phone call from Scott at AppRiver to make sure they clearly understand the issue for proper escalation.

4) July 29, 6:35am: Received an email from Scottie, an AppRiver server engineer, seeking yet more details.

5) July 29, 8:51 & 8:59am: Received a voicemail and email from Scottie to let me know that one of the vulnerabilities I'd discovered was part of 3rd party (SmarterTools) code AppRiver was using to track support requests.

6) July 29, 2:08pm: Received email from Steve M., AppRiver software architect, who stated that:
a) "We deployed anti-XSS code today as a fix and are using scanning tools and tests to analyze our other web applications to ensure nothing else has slipped through the cracks. We do employ secure coding practices in our development department and take these matters seriously. We appreciate your help and are going to use this as an opportunity to focus our development teams on the necessity and best practices of secure coding."
b) "Regarding XSS vulnerabilities you detected in the SmarterTrack application (the above mentioned 3rd party tracking app) from SmarterTools, one of our lead Engineers and myself called them this morning explaining the vulnerability and requesting an update to fix the problem. We also relayed to them that a security professional had discovered the vulnerability and would be contacting them to discuss it further."

NOTE: Less than 24 hours after my initial report, the vulnerabilities that AppRiver had direct ownership of were repaired.

7) July 29, 4:17pm: Received an email from Andrew W at SmarterTools (3rd party tracking app vendor) who stated "thank you for pointing this out to us... we will be releasing a build within the next week to resolve these issues."

8) August 4, 8:02am: Received another email from Andrew W at SmarterTools who stated "we plan to release our next build tomrrow morning. (Wednesday GMT + 7) I will let you know as soon as it becomes available for download on our site."

9) August 5, 9:37am: Received another email from Andrew W at SmarterTools stating that "a new version of SmarterTrack is now available via our website. (v 4.0.3504) This version includes a fix to the security issues you reported."

10) The resulting SmarterTools SmarterTrack vulnerability advisory was released yesterday on my Research pages: HIO-2009-0728

I must reiterate.
This is quite simply the new bar for response to vulnerability disclosures.
It is further amazing that such a process was followed by not one, but two vendors.
I am not a customer of either of these vendors but can clearly state this: if I required services offered by AppRiver and SmarterTools, I would sign up without hesitation.

AppRiver and SmarterTools, yours is the standard to be met by others. Should other vendors utilize even a modicum of your response and engagement process, the Internet at large would be a safer place.
Well done to you both. | digg | Submit to Slashdot

Please support the Open Security Foundation (OSVDB)


Rafal Los said...

@Russ - thanks for the write-up. Unfortunately like someone said yesterday... "unhappy customers write complains, happy customers pay their bills"... so it's rare to hear of a GOOD experience like this.

"When I do good, no one remembers, when I do bad, no one forgets"

We've got to do a better job making the people who do a GOOD job at these things more visible...


Anonymous said...

I'm a dev on an unreleased SaaS product. We've spent a lot of time on security. I hope that the communication team does a good job getting reports to us so that we can be as responsive...

Anonymous said...

keep patting urself on the back for finding xss its really hard

Russ McRee said...

@Anonymous2: You quite obviously miss the point. Of course it's easy to find XSS vulns; what's difficult is finding vendors as responsive as AppRiver and SmarterTools to repair them so that consumers aren't at risk.

Anonymous said...

Seriously... that just ROCKS! These guys not only listened and got out of their own way; they prompted the same effort with another vendor!

TravisV said...

Refreshing to see them "own it" with so much accountability. It seems so often that a vendor will try to turn a problem around to be a user config issue, or minimize its severity, or otherwise spin it. This does seem to be the most honest / effective method to handle a customer issue (i.e., not just for a security issue, but in general).

Moving blog to

toolsmith and HolisticInfoSec have moved. I've decided to consolidate all content on one platform, namely an R markdown blogdown sit...