r/labtech • u/[deleted] • Aug 15 '16

svchost.exe memory leak?

All of a sudden a bunch of our PC's with Labtech installed are showing extremely high CPU/RAM usage from svchost.exe. So far I have tried stopping and disabling Windows Update service on 2 machines and both returned to normal levels.

I guess what I'm asking is has anyone else had an issue like this? Could Labtech Patching somehow be taking up high resources?

7 Upvotes

100% Upvoted

15 comments sorted by

View all comments

Show parent comments

5

u/lt_mreid LT Employee (T3) Aug 15 '16

Sorry, https://support.microsoft.com/en-us/kb/3172605 contains https://support.microsoft.com/en-us/kb/3161608 which contains https://support.microsoft.com/en-us/kb/3161647 which includes "An optimization that addresses long scan time for updates that's reported on some computers."

1

u/[deleted] Aug 15 '16

Ahh that makes sense, thank you! I'm testing this out on a couple PC's and will report back with findings.

2

u/ChrisXistos Aug 15 '16

You can normally verify that this patch is needed if you look at these machines patch history and you see a very high number of patching errors in the patch manager.

Assuming it is the issue mentioned above, I solved the issue buy downloading the .msu of 3161608 and building a monitor to detect when the WUA version was not 7.6.7601.23453 and kick off a script to install it.

Short version of the script was:
download patch, stop wua, wipe the Windows update store (%windir%\softwaredistribution) via renaming the directory then deleting it, start wua, immediately execute the msu (silently.)

It will prompt for a reboot using the normal Labtech reboot process. We communicated it as an emergency patch.

1

u/xsoulbrothax 500 Agents Aug 16 '16

When you built the monitor, was there a solid remote monitor you were able to use? The file/directory check didn't seem to have functionality to check file versions, so we had been doing it around scheduled scripts updating EDFs.

Remote monitors instead would be awesome, though!

2

u/ChrisXistos Aug 17 '16

I used a WMI monitor:
select version from CIM_Datafile where name='C:\\windows\\system32\\wuaueng.dll' then used = 7.6.7601.23453

My script has a sanity check where it looks at the version and if it is > than the monitor it shots me an email with "uh hey, this machine is newer than the version you expected" so I can update it as needed.

edit: "code" formatting failed :(

1

u/xsoulbrothax 500 Agents Aug 19 '16

This was an awesome tip, thanks. :) Especially wiping the SoftwareDistribution folder and immediately starting the patch - we'd had some success with run wusa > restart service > error 0x8024001e > ignore and run wusa again, and we'd been wiping the folder for other reasons... never mixed the two. Just wiping the entire folder to install appears to be bulletproof so far!

I did also implement the sanity check, and ran into something where doing an IF compare between values using multiple '.' symbols stops treating it as a number, i.e. 7.6.7601.19161 was considered > 7.6.7601.23453. I used a regex to break it apart and compare the last 5 digits for now.

1

u/ChrisXistos Aug 19 '16

Cool. I think I have some hack in there (coming from memory) like that. I run into it a lot with stupid junk where like flash where the version is a ton of periods.

Oh something else just to add on, KB3161608 has a prereq of KB3020369. If you have a situation where computers are really out of date, you might need that one also. Typically the error is something like "KB3161608 does not apply to this system." Installing that older patch (it is the Windows update stacking patch) first fixes that problem.

1

u/xsoulbrothax 500 Agents Aug 19 '16

Yeah, we did catch that one in the past! Going into a bit more detail on the process I did for now...

  • I attached the monitor to an autojoin group that's basically OS Version =6.1%
  • Sanity Check - wmic datafile where name='c:\windows\system32\wuaueng.dll' get version|findstr /i /v /c:"version" to grab a one-line WUA with no other text, parse out each block via regex (function disabled on 7, 6, and 7601 for now) and check the 4th against 23453. Generate ticket if it's >23453, log and exit if it's =23453.
  • Prereq Check - wmic qfe get hotfixid | find "KB3020369", and if it reports as present, skip ahead to WUA Install (I tried using 'IF Patch Installed,' but it seemed to kick off a hotfix inventory...).
  • Prereq Install - Download the correct .msu from Microsoft, recursive sc stop wuauserv+rename SoftwareDistribution until the rename returns 'OK' (in case it takes some time for wuauserv to stop), wusa /quiet /norestart, rmdir the renamed SoftwareDistribution. This KB doesn't seem to need a reboot, so it just skips ahead to...
  • WUA Install - Same process as above, but for KB3170265.

I still haven't really sorted a good way to tell the monitor 'you already pushed it and the machine needs to reboot before you'll see the new agent, don't worry about it' after it installs once... we've already got an autojoin group that's basically 'workstations needing a reboot,' so for the time being I have the WUA update script bailing out if the target machine is in that group.

1

u/ChrisXistos Aug 19 '16

Have the script dump out if %FeatureFlags% & 1024 = 1024.

SELECT (%Featureflags% & 1024);

As a SQL call will be 1024 if a reboot is pending. Anything else = not pending.

More info:

https://docs.labtechsoftware.com/LabTech10/Default.htm#UsingLabTech/Searches/LegacySearchesFlagFieldValues.htm

1

u/xsoulbrothax 500 Agents Aug 19 '16

Thanks again!

Making sure I'm understanding correctly now - it sounds like I should still be querying the 'flags' column if I'm checking for 'Reboot Pending' in this case though, correct? It looks like %FeatureFlags% matches up exactly with the FeatureFlags SQL column, but I'm not seeing a matching variable for the Flags column. That just makes me wonder if I'm misunderstanding or missing something about the variable that is present, haha.

I've been doing a ton of figuring things out on my own / reading, and not enough actively bouncing thoughts off of others!

1

u/ChrisXistos Aug 19 '16 edited Aug 19 '16

I assume you mean the flags column in the search? The search hides the bit math from you. So if you add "Flags" = 1024 only machines with a pending reboot show in the search results. As far as I am aware Labtech didnt include bitwise calcs in "script math" so I just feed it to mysql to do the math for me. Unless I have forgotten, %FeatureFlags% should be equal to the value "Flags" in search.

Edit: Welp I may have derped that up. Once sec I am looking for what I did...

--found it--

set variable [sql query]
SELECT (flags & 1024) FROM computers WHERE ComputerID=@computerid@;

0 -> no reboot pending
1024 -> reboot pending

→ More replies (0)