A new breed of startups is helping hackers make millions :- legally

Shashank Kumar was in seventh grade when he was introduced to computer hacking. At first he had fun breaking in and defacing web sites, something he says he now regrets, but then he learned that he can get paid for reporting the weaknesses he was exploiting. Under the handle @cyberboyIndia, he says he has earned around $30,000 in so called bug bounties, enough to pay for a good portion of his college education.

These days the 19-year-old is supposed to be cramming for his final exams as he prepares for a degree in engineering. But many nights he finds himself awake too late, laptop humming away, hunting for software vulnerabilities on services run by firms like Yahoo, Paypal, and AT&T. On Twitter, Shashank catalogs the rewards he receives for reporting weaknesses, a highlight reel that ranges from a free hat, to a new smartphone, to a $1,500 check. The money is good, although it’s murder on his grades.

Shashank is part of a broader trend sweeping the security industry. Last week, Google announced that it was changing the rules on its bug bounty program, Pwnium. Instead of a respectable $2.7 million awarded once a year, the contest will now run year round, with a total prize pool of "∞ million*." In other words the money never has to stop flowing, although Google’s clever asterisk placement reserves the right to cancel at any time.

GOOGLE UPPED ITS REWARD POOL TO INFINITY

But that’s not likely to happen, at least if Google wants to stay competitive. Bug bounty programs used to work with informal rewards: a thank you letter, an online shoutout, a free t-shirt, or perhaps a few hundred dollars. But over the last five years, they have become a bonanza. Almost every major tech company has one running, and they have steadily increased the size and volume of rewards.

Most importantly, a new breed of start-ups like Crowdcurity, Bugcrowd, Synack, and Hacker One have made it possible for any company to launch its own bug bounty, dramatically expanding the size of the market. Instead of being something only large technology companies do, bug bounties are now proliferating across all industries and for companies big and small, allowing these start-ups to pay out millions in rewards over the past two years.

A new weapon in the endless war

"It’s changed the way we think about security," says Andrew Pile, the chief technology officer at Vimeo, which recently launched a bug bounty through Hacker One. "It would have been nearly impossible for us to build this kind of program in-house from scratch."

"IT CHANGED THE WAY WE THINK ABOUT SECURITY."

As Vimeo grew in size over the years, it would receive occasional submissions about security flaws in its system, but Pile never felt comfortable paying for them. "The only way we could thank them was a free account or t-shirt, which didn't really encourage the best people to come back."

Like most companies, Vimeo had a chicken and egg problem. Building up critical mass of trusted researchers required shelling out big bucks. But it was loath to open up its checkbook for people it didn't know and trust, especially a disparate and sometimes anonymous mob of teenage hackers.

Bug bounty start-ups essentially act as market makers, creating trust and liquidity so that smaller companies like Vimeo can tap into the supply of global hackers. "Paying people can be a real pain; they are all over the world, and they don’t have W-9s," says Pile. Hacker One handles the legal and logistical nightmare, taking care of billing and payment in exchange for a 20 percent commission on top of each bounty.

The system is working so well that companies, which are large enough to handle their own programs, like Yahoo and Twitter, have chosen to use third-party vendors like Hacker One instead. "The band-aid approach of throwing more and more hardware and software at the problem clearly is not working. We need an orthogonal approach, and one that is proven," says Bill Gurley, a venture capitalist and investor in Hacker One. "Bug bounty programs get in front of the problem instead of just playing defense."

Down with the dupes

For the bug hunters, the biggest risk is that they will invest hours in finding a flaw and writing a report that explains how it works and can be fixed, only to learn that it’s already been submitted by a rival. Shashank and his friends have crafted playful memes about the heartbreak of "duplicates," as they are known in the business.

"There is a huge amount of trust involved," says Vimeo’s Pile. "They spend a ton of time identifying and documenting these issues, and then the report goes into a black box. I closed out a significant number that were duplicates, and unfortunately we can only pay on a first come first serve basis."

"BUG BOUNTY LIKE ONLINE POKER, I THINK."

Interestingly, none of the half dozen researchers from bug bounty leader boards contacted by The Verge were doing this work full time. The haphazard nature made monthly earnings uneven. All referred to it as a passion, a hobby, or a part-time gig, although many said the pay was quite good when they happened to strike first. "Bug bounty like online poker I think [sic]," wrote a Russian hacker, Andrew. "You may be in luck and got a big prize [but] may be not in luck and during long time nothing to find [sic]." In a way, the robust market of researchers may be as much about outsourcing as crowdsourcing. There is a wealth of teenage talent in places like India and Pakistan willing to hunt bugs despite the risk of time wasted.

Dupes or low-value reports can also be a problem for the companies entering this market. "The downside of saying 'if you give us vulnerability we will give you money' is you get a lot of garbage reports," says Daniel LeCheminant, who recently launched Trello’s bounty program. "In the first week and a half we have gotten 200 submissions and maybe 10 were actionable."

FORMER NSA OPERATIVES ARE GETTING INTO THE BUG BUSINESS

Because of this inefficiency, Synack uses a much less open model. It’s the best funded of the bug bounty start-ups, and it was founded by veterans of the NSA who spent years looking for vulnerabilities that could be exploited by the government. "We are taking a different approach from these other managed bug bounty providers," says founder Jay Kaplan. "The number of people out there in developing countries eager to make even $50 can create a lot of noise." The companies that use Synack don’t pay hackers, but instead pay a flat fee for the service. Synack uses a limited pool of hackers who have passed its three-tiered test, vets all the submissions, and assigns the bounties, insulating its more conservative, largely non-tech clients from the chaos.

Avoiding Blowback

Alex Rice, the former head of Facebook security and now CTO of Hacker One, says that most hackers will choose an official program over the black market, even if the prices are not as high. "To sell something on the black market, you have to weaponize it. That can take months. And so you have to empower the majority. They have the software skills but not the malicious intent."

But some industry experts warn that if not handled properly, this dynamic can have repercussions for the companies running the bounty programs. "The problem is companies think that bug bounties are simply something they can announce and that will be enough," Ilia Kolochenko, CEO and founder of the security firm High-Tech Bridge, said on his company blog. He used the example of a hacker who reports a duplicate that has not yet been patched. "Okay, if you’re not interested in what we've discovered, we’ll swap our white hat for a grey / black hat and talk to someone else who may well pay us more."

""WHEN AN ORGANIZATION HAS A GUN TO ITS HEAD, IT STARTS WASTING TIME."

Gus Anagnos helped launch the bug bounty programs at Paypal and now works at Synack. "What doesn't get talked about much," he says, is the blowback from hackers who felt they didn't get enough money or that their bugs weren't disclosed in a reasonable amount of time. These hackers would often go public, creating a PR nightmare for the company involved. That leads to companies overreacting to every submission. "When an organization has a gun to its head, it starts wasting time on vulnerabilities that aren't very important," says Anagnos.

The start-ups who run these marketplaces have ameliorated the duplicate problem in part by awarding points that build a hacker’s reputation. "They give points for your duplicate bugs," Shashank told The Verge by email. Those points add up to invitations for private bounty hunts where rewards are more plentiful and competition is less crowded. "So here we get the fruit of our duplicate bugs," Shashank says. He currently sits on the leader boards for Bug-crowd and Crowdcurity, which highlight the top researchers of each month and all time.

These private bug hunts have benefits for both sides. "Large tech companies are comfortable with the chaos of inviting people to try and pick apart your system in public," says Casey Ellis, the CEO of Bugcrowd. "A bank, a big box retailer, they aren't built like that, they don’t have the same appetite for risk." Start-ups that offer this service typically start companies with a private program open only to a select group of trusted hackers. Slowly, as the companies learn the ropes, that can be expanded to a more public test, or the launch of an official bounty program on the client’s own website.

Making the best of a bad situation

Some security professionals who were critical of the proliferation in bug bounties a few years ago have since changed their tune. "I was afraid companies would start these programs and people would put up terrible bugs and demand money for them, and companies would waste time on them while real security vulnerabilities didn't get fixed," says Dan Kaminsky. "But I’m pleasantly surprised with how well they have worked in the field. There is a serious talent crunch, and programs like this help to maximize a company’s ability to tap all the expertise out there."

"THE MOMENT YOU SQUASH THEM, THE NEW ONE SURFACES."

Still, the programs have plenty of skeptics who feel bug hunting is largely housekeeping that doesn't address systemic security risks. "There is a huge culture around finding the bugs so that we can patch them," says Peter Herzog, a security analyst and creator of OSSTMM. "It has more of a basis in marketing than reality. The moment you squash them, the new one surfaces."

Herzog points out that high-profile hacks like Sony Pictures likely originated with an employee who was compromised, not a technical flaw. "Most of the attacks that happened were done through social engineering. How are we going to start patching the people?" asks Herzog. "What’s the point of looking for bugs when people are the ones downloading and installing the malware."

Even the founders of these start-ups are quick to admit that bug bounties are no magic bullet for security woes. "It’s not like you run a bug bounty program and now everything is secure," says Jacob Hansen, Crowdcurity’s co-founder and CEO. "It’s one item in the toolbox. You need code reviews, physical security, and employee training."

But studies of the Chrome and Firefox bounty programs have found they are cheaper than hiring full-time security researchers. "Organizations are coming to realize that the tools of the past don’t scale to the way they develop applications today," says Synack’s Kaplan. "Companies are pushing out code 10 times a day. To keep up with hackers you can’t rely on automated approaches or occasional consultants. You need a big group with a diverse set of skills constantly probing your system for weakness."