January 21, 2020
The number of ransomware attacks on MSPs mounted last year and more are likely to be targeted in 2020.
Dark Cubed, which provides cybersecurity solutions, procured a research study with data revealing that MSPs are fighting a losing battle when it comes to cyberattacks. MSP networks are under a barrage of attacks from malicious threat actors, and 100% of MSPs reviewed suffered either automated attacks, directed attacks or both.
To get a firsthand account of an MSP ransomware attack, we spoke with Darin Harris, COO of Remote Techs, which incurred a ransomware attack last year that nearly drove it out of business. The MSP works with clients across the western United States, and construction and transportation are its two biggest verticals.
Channel Futures: How did the ransomware attack unfold?
Darin Harris: We used two pieces of software that are very common in the industry. We used a remote management and remediation tool [from] Kaseya and then we used a ticketing and billing system [from] ConnectWise. ConnectWise had a plug-in essentially that connected the workstation data, the audit data back into ConnectWise so that you could connect tickets to workstations and things of that nature. They released a patch [at] the end of 2017 or the early part of 2018 that was to fix a vulnerability. We applied the patch, thinking we were safe, and then in … the early part of February of 2019, the exploit that existed and that was supposedly patched started to be used in the wild, and ConnectWise and Kaseya started to see MSPs becoming attacked. What it would do is essentially bypass your two-factor security, bypass your user passwords … to a direct sequel injection into the database to change a password, they would log in and then they would use Kaseya to start installing their ransomware using your servers to push the ransomware to all of the clients that were connected. Yeah, real friendly stuff.
So for us, it started at about 2:45 p.m. on a Sunday, and we have some customers that run pretty close to 24 hours a day, and so we started to get a few phone calls about 3:15 p.m. of servers being unavailable for one of our clients. We started to investigate and found clients that were ransomed. And we started to see that affect a couple of clients at the same time, at which point we quickly deduced that the issue was the Kaseya server itself. We looked into that and found that we couldn’t gain access to it like we used to be able to. And so we quickly took it offline, shut it down and then started the remediation process to fix everything. That was probably a good, solid, six-to-eight weeks, and we had – compared to other owners like myself that I’ve spoken to – manageable damage. We had about 14-16% of our connected devices become encrypted and more than half of those were servers. I know some owners and some other MSPs that had 100% encryption rate. Every single device was encrypted before they found out what was going on. So yeah, that was January. It took us two months to get everything kind of back to normal. We had our customers back up within just a few days, but … even if you can recover workstations and desktops, and servers you still have to go back and back up all the data, rebuild it from scratch and …
… put all the data back on it, reinstall. You still have to go through the steps, even if you can do a a recovery. You still have to go through and harden and protect everything the second time around just just to make sure.
CF: What was the financial impact of all of this? Was it costly?
DH: Oh gosh, we spent hundreds of thousands of dollars to recover from this and lost another hundreds of thousands of dollars in revenue for 2019. There was essentially three to six months [where] we couldn’t take any projects on; we were paying overtime. We were buying new tools. And you can’t bill a customer. When this happens you can’t call him up the next month and say, “Oh, by the way, your monthly bill is due. But you can’t use your system so send me my money, and I’ll try and connect” — it doesn’t work. So you know you have to work with your customers to get them fixed and get them comfortable again, hope that they don’t leave, and then you can start to rebuild the financial inflow.
CF: Did you lose any any customers?
DH: We lost one customer.
CF: Is your company fully recovered now? Is this all in the past or are you still dealing with any lingering issues?
DH: We don’t have any lingering issues from a technical standpoint. I am very blessed and very lucky to have a phenomenal staff who stepped up and worked literally around the clock for two weeks. We had shifts going. We brought in three consultants who helped us. We brought in a security management team – a company called Solis out of Texas – to do a review and an audit, and get to the bottom of everything and how it happened. So we spent a lot of money to make sure everything was right. I don’t know if I’m ever going to get that back. I mean, our insurance company picked up the tab for chunks of it – huge chunks of it – but ultimately even four months ago we would run across a desktop that had been turned off for five months, and oh yeah, you know it was encrypted. You know now you don’t have to pay for the encryption keys … one of the antivirus vendors has figured out a key that will work and you can run decryptors against it. And so we’ve encrypted old drives and things like that, but again, I can’t bill for that. It’s a lot of labor for my guys to do that — I can’t bill for it. I wouldn’t bill for it; it’s not morally right. It’s not the customer’s fault.
CF: Could this happen to you again or are things now in place that could prevent another attack?
DH: This exact attack could not happen again. We’ve broken the connection between ConnectWise and Kaseya so there’s no more integration. Unfortunately, that hurts us [because] that causes all kinds of problems with our billing and our tickets, and things like that. We’ve changed a lot of our systems to provide better protection. We moved away from standard antiviruses and antimalware applications. We moved toward the artificially intelligent … and no-trust type systems, so we went from an antivirus that essentially ran a white list to approve applications, and now we have an antivirus/antimalware application that essentially denies everything and requires a manual intervention to do even install software. And that is simply to …
… prevent the spread of these kinds of events.
We looked deeply into how these events spread, how they spread across internal networks, and we removed every option that they use as quickly as we could. We’ve had to move all kinds of printers off local workstations and … directly connect them to the network because we don’t want workstations to run admin shares or IPC shares, or anything like that. We turned on firewalls on everything across the environment to just keep an extra layer of protection. We run redundant systems, so it’s one antivirus on top of another. We run a system called Carbon Black, we run a system called Huntress that just does intrusion detection, and then we started to look and implement applications like NeuShield recovery applications. Not that I want to have to recover, but if I do, I want to be able to do it in groups quickly without having to be concerned that I’m going to let the virus reinfect a network. And unfortunately you do all of that for your customers, and while they are under contract, you can’t go back and raise their rates, even though you know you’re spending significantly more money trying to protect them and it’s what has to be done.
CF: Did this threaten the stability of your business? Were there times when you thought this was going to drive you out of business?
DH: Yeah, I think there was some concern in the first couple of days until we had a real handle on how many machines were infected and what the scope of the damage was. I think there was some concern that we weren’t going to survive it. I think, and I’ll be very honest, our insurance group and the attorneys that we brought in to help represent us, they were amazing, and they really stepped up and got involved, and had resources and had the right people available for us to talk to and help through that transition. And without them, I don’t believe we would have come out of it as well as we did. We’re a multimillion-dollar MSP. I don’t know how you would do that if you were a three-person shop, at a $500,000 or $800,000 revenue model. You can’t afford cyber insurance to the values that the bigger companies can. You can’t afford to pay some of these consultants $50,000 deposits in 24 hours to get their help. The acquisition of the smaller MSPs is I think going to become paramount to surviving these kinds of events.
Remote Techs found NeuShield after this event and is using it to avoid another ransomware attack. Steve Bottini, NeuShield’s channel chief, tells us “all of the companies that you find in the news that have been attacked by ransomware all had pretty extensive security and backup solutions in place.”
“There are always going to be the new vectors, new ways that hackers and attackers are finding a way into the environment,” he said. “NeuShield is really that last critical layer that says, ‘Hey, if something does get through, we’ll allow you to instantly recover that data as though the ransomware never occurred. You’ll always get your original data back.”
NeuShield’s anti-ransomware technology allows organizations to recover damaged data without a backup utilizing Mirror Shielding to protect files and ensure instant recovery of important data. This new approach allows customers to instantly recover from any unknown or zero-day threat because NeuShield protects the data, rather than try and find specific threats, without requiring continuous updates and without signatures.
Sophos Rolls Out New Mobile Threat Defense
Sophos has unveiled Intercept X for Mobile with new security capabilities for Chrome OS devices, and improved mobile threat defense for Android and iOS devices.
Leveraging the same deep learning antimalware technology used in Intercept X for Windows, macOS and server, Intercept X for Mobile protects users, their devices and …
… their data from known and never-before-seen mobile threats.
Sophos’ Petter Nordwall
Petter Nordwall, director of product management at Sophos, tells us by offering the “best next-generation protection technology” for all platforms – Windows, macOS, Servers, and now mobile devices – under the same Intercept X brand, Sophos is making it much easier for partners to offer mobile threat defense to their customer base.
“With the introduction of Sophos Intercept X for Mobile, partners can easily offer the same Intercept X technology as an upsell and cross-sell of mobile threat defense to their endpoint and server protection customer base, and of course increase the order value accordingly,” he said.
Intercept X for Mobile is managed in the cloud-based Sophos Central platform alongside Sophos’ entire portfolio of cybersecurity solutions. Sophos’ synchronized security approach allows these solutions to work together for real-time information sharing and threat response.
“Sophos Intercept X’s deep learning technology provides superior protection than the competition and is now available for mobile devices,” Nordwall said. “For customers, reducing the number of security vendors has a proven advantage in terms of TCO, and partners can benefit from this as Sophos is the only security vendor that can offer a cybersecurity platform that protects endpoints, network, data, email and mobile devices natively in the same console.”
Neustar Unleashes UltraThreat Feeds
Neustar has released UltraThreat Feeds, a new service that provides its enterprise customers with access to real-time threat data designed to help them better identify cyberthreats as they evolve.
Neustar’s Anthony Chadd
The service allows security and threat intelligence teams of all sizes to access threat feeds based on analysis and observations curated from Neustar’s DNS, OneID and IP decisioning data.
Anthony Chadd, Neustar’s senior vice president of global security, tells us his company’s partners now have access to “highly contextualized and actionable” intelligence, which will help them understand what specific threats their organizations face and act accordingly to limit or prevent attacks.
“Utilizing UltraThreat Feeds allows our partners to improve brand monitoring, increase brand protection and better prevent spam and phishing attempts,” he said. “Because UltraThreat Feeds are developed from our own data and from activities within our own organization, our data is unique, not aggregated from other sources. UltraThreat Feeds leverages Neustar’s unique position and visibility across the global internet ecosystem to provide users with rich, contextualized insights on threats impacting their business and industry. Neustar’s breadth and depth of data is unmatched, and our curated approach to threat data delivery provides users with highly contextualized and actionable intelligence, enabling them to quickly identify indicators of compromise or malicious activity and act accordingly to limit or prevent attacks.”
Read more about:MSPs
About the Author(s)
You May Also Like
AWS re:Invent Partner, Vendor News: Cisco, Salesforce, MoreDec 01, 2023
People on the Move: Comcast, Cisco, NICE, TPx, Barracuda, MoreNov 29, 2023
AWS re:Invent 2023 Partner News: Marketplace, Salesforce, Certs, MoreNov 29, 2023
AWS re:Invent Expo: VMware, Snyk, HPE, More Showcase Cloud, Security, AINov 28, 2023