By Brian Horton
What to do when the security threat threatens you.
In April of this year, I authored a Channel Executive article detailing basic steps MSPs should consider when transitioning to security-centric services. These steps included increased security training for team members, general legal concerns, framework considerations, and several other logistical variables.
I received numerous emails from MSP owners, all requesting specifics to better safeguard their organizations. In response, I thought it would be helpful to follow up and include tangible specifics for tech firms. Our penetration testing and incident response teams have identified five common vulnerabilities to MSP businesses.
#1 ANTIVIRUS SOFTWARE
OK, let’s get this one out of the way, as it’s typically the most contentious. Ripping off the bandage, the antivirus solutions that come recommended and integrated with popular RMM/PSA tools generally don’t get the job done. This can change at any time, but as of this article, it holds true. While these antivirus solutions offer ease of management, clean solution integrations, a single pane of glass, and scalability in price, they leave much to be desired in preventing basic targeted attacks. For example, in every penetration test conducted by Breadcrumb this year, in which such antivirus solutions were present, we were successful 90+ percent of the time. So much so, that once we understood an MSP was the IT resource, all it took was a quick perusing of their website for their “preferred antivirus product,” and we knew exactly how to peel the onion. Now, this isn’t a knock on the MSP; in fact, it’s quite the opposite. My previous firm used a lot of these same tools, too. When you’re immersed in the MSP/RMM single pane of glass world, you tend to lose sight of vetting and can be compelled to trust the platform. The reality? There is no easy answer. The products that are more successful are typically not associated with RMM platforms. Thus most answers would be a change to any given solution stack.
Recommendation: Directly vet your antivirus software. If this is beyond the skillset of your team or you simply don’t have the time, hire a penetration testing firm to conduct testing in an isolated environment. This may sound a bit extreme, but considering your firm is recommending the product for thousands or tens of thousands of endpoints, product selection is critical. Solicit feedback from incident response firms with first-hand knowledge of what works, and what doesn’t.
#2 DOMAIN NAME SYSTEM
In just about any context, what can be extracted from DNS records is extraordinarily helpful for those seeking to breach an organization. When these types of records are managed by technical firms, distinct and predictable patterns can emerge. Typical findings may include commonality amongst unique A-records that are present across managed clients. For example, you may register the A-record of firewall.clientdomain.com, or rdp.clientdomain.com for a given organization. While helpful, this can allow threats to very quickly isolate attack surfaces. Conversely, IP blocks can now be harvested and scanned directly. With regard to your own organization’s domain name, routine DNS dictionary attacks may reveal predictable targets, such as remote.mspdomain.com, citrix.mspdomain.com, or perhapsvpn.mspdomain.com. Another tendency is that the same “registrant name” is used for the majority of domain names procured or managed by the MSP. A quick reverse-registrant search can allow threats to develop a nice list of your clients and, in turn, begin harvesting DNS records to isolate attack surfaces.
Recommendation: Your team should ditch the obvious and predictable DNS names. Moreover, don’t register domain records utilizing the public-facing domain name(s). Instead, purchase an obscure domain name for login portals and remote tools. Lastly, private register all domain names, use a name that is never associated with your organization, or both.
#3 PASSWORD MANAGEMENT
With the evolution of password management tools and services such as IT Glue, password management vulnerability has notably improved. With that being said, there are unique exposures that MSPs should be aware of. For example, even though generated passwords may be long and complex, using the same long and complex password across multiple clients is damaging. In 65 percent of all penetration testing performed by Breadcrumb, whereas we conducted testing on multiple clients of the same MSP, it was found that the same hash value for the Domain Administrator account was present for all clients. Meaning, while the password potentially remained unsolved, we knew it was the same. This is detrimental to security. Leveraging tactics from the DNS section described above, a threat could then extrapolate a fairly accurate client list and thus begin to exploit this knowledge. If by chance the password is solved, the consequences will be far worse.
Recommendation: Make every admin password, for every client, unique. Never repurpose passwords, or increment them. This includes passwords for domain accounts, local administrators, firewalls, switches, routers, and anything else you manage. Bottom line, there should never be crossover between client accounts.
#4 HELP DESK PROTOCOL
Help desk impersonation? Yes! This old-school tactic is reliable. When calling a client and posing as the MSP, it can be difficult for the client to determine who is really on the phone. With caller-ID spoofing, it can become nearly impossible. Likewise, if a client calls into the MSP help desk asking for assistance, it can be challenging for the technician to validate the party. Compounding the situation is the “new guy” effect. Breadcrumb will typically seek out new hires on LinkedIn or Facebook and leverage their lack of client history to exploit the process. This year alone, we’ve assisted two firms whose customers were breached, with the external threat posing as the MSP. The inherent phone/email relationship between MSP and client exacerbates this risk.
Recommendation: Develop an authentication protocol for call-in procedures. This should hold true for both the client validating the technician and the technician validating the client. Many support tools are incorporating a challenge/response feature. Whatever procedure is decided upon, this will be a vulnerability your firm will want to address.
#5 RMM / SUPPORT PORTAL
Most MSP software solutions give technical organizations integrated platforms for remote support portals, agent downloads, or the ability for clients to log in and check their ticket status or account balances. In some cases, backup and recovery options can be managed by the client, too. While this is great for automating workflows, it creates alluring targets. When freely distributing your agents, you’re giving external threats the ability to understand your scripting and automation logic or, in some cases, your local-admin password strength. The fact that these services are public allows them to be discovered and directly attacked. It’s important to understand such risks and weigh them against the benefits. This year, our team assisted an MSP who was breached, with the source being a compromised PSA/RMM portal. The compromise resulted in 50+ clients subsequently being breached.
Recommendation: There’s no easy answer here. The first step is to obscure your attack surface. For example, the most common DNS entry we come across is connectwise.mspdomain.com — while this is logical from an operational lens, it makes for an overly obvious target. This type of DNS entry allows a threat to both understand what RMM platform is in use and what surface to start probing. Instead, we recommend leveraging the DNS section above and utilizing a random domain name, with an inconspicuous record name.
The next time your organization reviews the security controls for a given client or perhaps your internal controls, understand that external threats are actively seeking the specific nuance that exists between an MSP and its clients. In fact, last month the U.S. Computer Emergency Readiness Team, a division of the DHS, issued an alert (TA18-276B) explicitly concerning advanced persistent threat activity exploiting managed services providers. It’s the understanding of these risks and the adopting of reciprocating organizational protocols and processes, that will further aid in defending the data of your clients and the reputation of your firm.
BRIAN HORTON is the CEO of Breadcrumb Cybersecurity, Inc. A serial entrepreneur, Brian has spent his career advancing the security goals of organizations throughout the U.S.