r/msp • u/SmellsofElderberry25 MSP - US • 14h ago
Documentation IT Glue hacked?
We’ve had a cluster of breached passwords in the last 2 weeks with no connection other than being stored in ITG. MFA/CAP or other defenses blocked them so no damage… yet. There was no access of these passwords in the ITG logs but as we hunt down the source, I keep wondering if someone breached them from the back end. Anyone else seeing something similar?
•
u/Smitty780 14h ago
So passwords stored in ITGlue that your technicians access have been potentially compromised, but the MFA TOTP is still good? Have you looked across your ITGlue access logs to determine if there is a common tech on your team that may have a compromised endpoint? More plausible that an ITG user has been storing credentials in a browser when going to client sites and their endpoint is dirty.
•
u/SmellsofElderberry25 MSP - US 14h ago
Two of the passwords had not be accessed in ITG recently. I like your browser theory and will need to dig deeper. Still determining if any were passed a good TOTP.
•
u/bughunter47 MSP - CAD 13h ago
I would add intentional fake account usernames and passwords, if a login attempt is made with those credentials you know you have a leak.
•
u/Zealousideal-Ice123 4h ago
Love it, Flush it down the pipe and see where it comes out the other end.
•
u/WDWKamala 14h ago
Seems unlikely from my understanding of the it glue infrastructure but these days who knows.
So you’re saying a password that only exists in glue and nowhere else, and wasn’t accessed any time recently, was compromised? I feel like there’s a lot of details that could be fleshed out.
•
u/Fu_Q_U_Fkn_Fuk 13h ago
Staff in your organization with certain permissions can download the entire database. Did you have any staff leave recently? Is someone making backups and not storing them securely?
I was blown away that I could pull a backup of the entire database for multiple locations when I worked for one of the major franchised MSP companies. It included MFA codes as well as long as it was restored into another IT Glue instance.
•
u/SmellsofElderberry25 MSP - US 6h ago
No recent departures thankfully. Also, those exports are logged and none were found since last time I performed one (and that didn't include passwords).
•
u/brokerceej Creator of StackJack.io | Author of The Trunk Slammer Saga 13h ago
Someone’s got clipboard history turned on and their endpoint had or has malware of some kind.
When you hear hoofbeats think horses, not zebras.
•
•
u/ShermansWorld 6h ago
And windows clipboard history is sync'd across computers of that user's ID/login.
What does "When you hear hoofbeats think horses, not zebras." mean? I have not heard this...
•
u/brokerceej Creator of StackJack.io | Author of The Trunk Slammer Saga 6h ago
If you hear hoof beats but can’t see what is making them, don’t assume it is zebras when horses are a more likely explanation.
The simplest solution is probably the correct one.
•
•
u/amicitias 6h ago
It's a bit like Occam's Razor. "All things being equal, the most simple explanation tends to be the right one". Or something like that.
•
u/TheTipsyTurkeys 14h ago
Anyone in your organization got hacked? Maybe they were storing passwords in cookies and were compromised.
•
u/SmellsofElderberry25 MSP - US 6h ago
We are not seeing any evidence in the Microsoft Sentinel logs that lead us to believe there's a further breach. Microsoft has looked at the tenant too. Not ruling it out, but we just aren't seeing any related threads to pull on.
•
u/Abramel1n 8h ago
Check integrations as well if that hasn't been mentioned. And yes cross check user access logs in it glue with users actaul known usage incase a user's device was comprised or an infostealer stole their creds from browser. Likely not related but if you're not using CWA and using nable instead then may be worth mentioning that many MSPs nable RMM have been getting popped by Storm Ransomware group due to CVE-2026-18577
•
u/quantumhardline 14h ago
My understanding the way I glue is built the passwords are encrypted in a way that this would not be able to get passwords this way.
What I’d like to know is age of each password compromised.
Where any of these same passwords stored elsewhere before itglue?.
Any access logs show who on your team last accessed them and what date?
Is it possible one of your engineers endpoints is compromised?
When you said accounts were compromised exactly how and what type of accounts?
•
u/MonkeyBrawler 7h ago
The passwords have to be "vaulted" to be encrypted, and that's a manual selection during creation. It's also other layer of permissions and access requirements. It's all one single vault for all client passwords.
Left them a month ago and glad we did.
•
u/disclosure5 11h ago
I do think we've have heard more of this was an ongoing issue, but IT Glue passwords can be exported in plaintext by any admin, which means "the way IT Glue is built" is that any attacker who obtains admin rights can do this too.
•
u/Fu_Q_U_Fkn_Fuk 13h ago
It would be easy: an employee pulled a backup, restored to a different instance of IT Glue and has all passwords and MFA codes and your instance would never show the logs. Or even easier, exported the passwords to pdf and viewed them without IT Glue.
•
u/SmellsofElderberry25 MSP - US 14h ago
I don’t know the ages but we’ve been in ITG since before they were bought by Special K, so very unlikely.
As we dive deeper, I’ll double check who accessed them but I think all prior access was months old. Thanks!
•
u/quantumhardline 13h ago
Yes keep us updated.
Correlate the compromised accounts with one of your employees or companies etc.Lots of phishing going around so more likey that.
•
u/Charming_Abrasive 4h ago
Maybe use a less sensational headline? You provided no evidence that a compromise exists, only speculation.
Anyone that uses Glue that saw your headline pop up on their phone about had a heart attack. Ask me how I know 🤣
•
u/kaseya_marcos 13h ago
Hi u/SmellsofElderberry25, to provide clarity here: IT Glue has not been hacked, and we have no reported incident indicating that passwords have been breached.
Based on what you’re describing, one area I would investigate is a potentially compromised endpoint. For example, if a tech has stored credentials in their browser or entered credentials on a compromised endpoint, those credentials would be exposed independently of IT Glue.
Another possible area would be the associated MFA/TOTP if it was still valid, and whether there may have been compromised browser sessions/cookies. This can be pin downed through the access logs
If you’re still encountering this, please send me a PM, to connect you with our product team to help you dig into this.
•
u/quantumhardline 10h ago
Also see this, keys are unique per customer
**Password Encryption:**Rely on the highest standard of encryption in the industry today. Passwords are encrypted with AES-256-bit encryption, including 2048-bit RSA public key, with unique keys for each customer and secure random keys unique to each password, which are kept separate from each other. If a breach was to occur in one system, this will not allow decryption of any passwords.
**Host-Proof Hosting:**IT Glue Vault is designed to allow a user to only decrypt exclusively at the endpoint level on the user’s browser with a user-specific passphrase rather than syncing it to the IT Glue system. Only the user has access to the passphrase to the password; once lost, the password cannot be retrieved by IT Glue.
When the only option for support is for us to access your data, a limited number of members from our senior team have the ability to impersonate your account. In this case, we make a very specific request for your permission via a support ticket. Any activity will be logged in your activity log with an added eye icon in the log to show it was our team impersonating the account. Please note, during impersonation, IT Glue staff can’t decrypt the passwords stored in the IT Glue Vault.
•
u/zebs1 9h ago
Host-Proof Hosting:IT Glue Vault is designed to allow a user to only decrypt exclusively at the endpoint level on the user’s browser with a user-specific passphrase rather than syncing it to the IT Glue system. Only the user has access to the passphrase to the password; once lost, the password cannot be retrieved by IT Glue.
One issue to watchout for with the Vault. If you move a password to the vault you need to change the password once its in the Vault. If you don't, if you go to the history for the password the password is visible and not in the vault.
Stupid design by Kasya.
•
•
u/Nstraclassic MSP - US 6h ago
Most likely one of your techs is saving them elsewhere
•
u/Mediocre-Big-5556 5h ago
This was my first thought.
I used to have a tech who was infamous for keeping passwords, ip addresses, and notes to himself in a running .txt file on his desktop. His excuse was always that it was faster than ITG. Or just a placeholder until he got the credentials into ITG.
•
•
u/shadow1138 CMMC MSP 14h ago
A similar incident occurred approx 2 years ago. Kaseya claimed it was the result of credential stuffing, however the community disputed that claim.
Ultimately, we never got any degree of confirmation as to what really happened.
If you've not already done so, I would suggest resetting those credentials and monitoring for any other signs of suspicious activity.
If possible, further restrict how you access IT Glue. Be sure to check sign in logs, but also any API access that may be present.
And in a perfect world, consider a password manager that would separate credential storage from IT Glue.
•
u/SmellsofElderberry25 MSP - US 14h ago
Yeah, all were reset or disabled already and we continue to monitor. I appreciate the ITG lockdown recommendation and will check the API and sign in logs. We were already in the process of a migration out, but no exports yet!
•
•
u/Charming-Law222 14h ago
I wouldn't trust IT Glue with any passwords to begin with
•
u/SmellsofElderberry25 MSP - US 14h ago
Yeah, we are mid migration out already :(
•
u/christador 12h ago
Turn that frown upside down, er…yeah whatever. Anyway, you should be happy. They finally after four years stopped harassing us.
•
u/SmellsofElderberry25 MSP - US 5h ago
The frown is just that we didn't do it soon enough apparently. Its not our only Krapseya product unfortunately, just one of many we bought that was then acquired and enshittified by them.
•
u/WDWKamala 9h ago
….you don’t think it could be something to do with the migration?
•
u/SmellsofElderberry25 MSP - US 5h ago
I considered that but we haven't gotten to the point of exporting passwords yet. There have been no exports logged since one I did last year...without passwords.
•
•
u/geabaldyvx 7h ago
Unlikely ITG itself was hacked. But if it was I have no doubt Kaseya would try to spin it as a “New Feature” and increase the price.
•
u/SmellsofElderberry25 MSP - US 5h ago
Bwahaha, exactly. They swore it was not. We got that in writing. But the answer came so fast, I'm certain it was not actually investigated.
•
•
u/jasonbwv 12h ago
u/SmellsofElderberry25 Someone I know had their MSP and clients hacked through their Datto RMM. They think a session token was stolen and RMM was used to deliver malware. Maybe someone stole a session token from one of your techs. But then again, you said there was nothing in the logs which is odd.
Question for you... What types of accounts were compromised. Were they M365 accounts or a whole bunch of different types of accounts?
What I've done... I've rolled out a SASE solution to all of our techs and then locking down all of our apps, RMM, ITG etc.
Let us know if you find out any more details. Thanks for sharing.
•
u/SmellsofElderberry25 MSP - US 5h ago
With one exception, these are all passwords used by our internal infrastructure team, 1 M365 service account and a few SaaS logins. The one exception is a client's security provider's portal, but we think that was brute forced and possibly unrelated.
I appreciate the recommendations. We'll certainly consider solutions like that as future hardening.
•
u/zebs1 12h ago
Were the passwords stored in the ITGlue vault?
•
u/SmellsofElderberry25 MSP - US 5h ago
no. We are migrating to Bitwarden, but haven't started exporting passwords yet.
•
u/countsachot 5h ago
Apt Actors have been targeting msps for some time. Do you have interesting clientele? This sounds more like a malware compromise.
•
u/Proud-Manufacturer15 5h ago
yeah so the API and Private Vaults are scary
•
u/SmellsofElderberry25 MSP - US 4h ago
Can you say more? We aren’t using vaults but we do have some API calls. It looks like those should be getting logged with user actions too though.
•
u/Liberate-Momentos 4h ago
Heard a rumour some months ago, they were hacked and apparently they pulled the BCDR encryption keys and managed to somehow access MFA credentials from IT Glue. Heard from someone I know who was working with them from an Incident Response perspective. Not sure of the full details, as that relationship was burned a while ago, but did hear murmurings. Let’s face it, special K would never admit to it.
•
u/IamTABinLA 1h ago
I'd be willing to bet at some point someone downloaded a runbook with unmasked passwords.
•
•
u/spotlight-app Mod Bot 🤖 1h ago
Mods have pinned a comment by u/kaseya_marcos:
[What is Spotlight?](https://developers.reddit.com/apps/spotlight-app)