SCCM Configmgr 2012 Troubleshoot Client software update issues

 

One of most important and critically used feature in configuration manager 2012 is  Software updates .It is always challenging and import task for any sccm administrator to achieve good patch compliance success rate within the given SLA(Service level agreement).Patch compliance success rate is depends mainly on heath of your SCCM clients and some times things may go wrong even though sccm client is healthy (able to receive applications/packages and performing inventory except patches).

I have created lot of SSRS reports on software update compliance out of many,one of the widely used report is get the patch compliance status of software update group for specific collection with linked report to get the computers with unknown and required status for troubleshooting (to check when was the last hardware,last software scan,last user ,OS etc).

Coming to the subject line, I have been seeing many questions on the configuration manager forums and social networking sites on software update patching issues .couple of questions on the subject line are like

1) Client getting packages ,applications but not software updates

2) Most of the clients receiving deployed software updates but still few do not get.

3)Clients not detecting software updates

4) clients log says ,patches required but sccm reports says,updates not required( means complaint)

5) Client log says patches not required but sccm report says ,updates required.

6)Software update failing to install ,how to fix 

7) I have added patches to the existing software update group/deployment and these newly added patches not deploying successful and many more ….

The solution for the most of the above issues can be identified and solved by analyzing the the client logs before we do in-depth troubleshooting.

In this blog post (SCCM 2012 Troubleshoot software update client issues),I will explain you the basic troubleshooting steps (only on client side ) which will help you to resolve issues on your own by analyzing the logs and take it further afterwards.

Before we jump into the troubleshooting,I would like to illustrate the main components which are involved in deploying software updates.

When you enable software update agent setting in client agent settings,a policy will be created with this setting and stored in SQL Database.So when client initiate machine policy,it communicate with management point which includes the software update client feature installation instructions to be installed or applied on the client. In this process, Client will create local GPO with WSUS Settings by leaving automatic updates .

If you do not  disable automatic updates (Via GPO) leaving the door open for the WUA to do things on its own outside the control of ConfigMgr including installing any updates approved directly in WSUS (including new versions of the agent itself which are automatically approved) and rebooting systems which have a pending reboot. Neither of these is desirable in a ConfigMgr managed environment and thus the recommendation for disabling automatic updates. As for the rest of the Windows Update GPO settings, they are meaningless in the context of ConfigMgr so it doesn't really matter what you set those to if you disable automatic updates,more from here

If you choose to create a GPO for WUA, you must configure the Windows Update Server option to point to the active software update point server in the site or location. If there is an existing GPO that was intended to manage standalone WSUS prior to implementing Configuration Manager in your environment, the GPO could override the local GPO created by Configuration Manager, which can cause issues when the software update client tries to communicate with the software update point server.

Software update Components involved are:

1.Windows update agent (WUA)

2.Software update client agent (from SCCM)

3.Windows management instrumentation (WMI)

Note: Make sure you disable the automatic updates via GPO,further reading http://blog.configmgrftw.com/software-updates-management-and-group-policy-for-configmgr-cont/

Windows Update agent(WUA): is responsible for scheduling and initializing scan, detection, download, and install of updates on the client machine. WUA Agent is an implanted service in a Windows service (SVCHOST.exe) and is named Windows Update which you can see from services.msc.

If you disable WUA Agent, software update agent will not function correctly. So it always recommended to not disable this service.

Software update client agent (from SCCM): When you enable the software update agent,it will install 2 actions on the client 1) Software update scan cycle 2) software update deployment Evaluation Cycle

Software Update Scan Schedule :This action perform the software update scan (along with WUA) against the Microsoft update catalog, which occurs every 7 days by default.

software update Deployment evaluation:This action Initiate the software update deployment to start download and install the updates.

Note: when you create software update deployment with deadline for ex: at 4.00 PM ,the actual time that software update client start updating the installation is depends on on setting disable deadline randomization ((located in the Computer Agent client settings)

A delay of up to 2 hours will be applied with deadline time to install required software updates . This randomization prevents all software update clients from starting update installations at the same time (This setting is disabled by default). More info,read https://technet.microsoft.com/en-in/library/gg682067.aspx?f=255&MSPPError=-2147217396 . If you enable this setting,then the deployed software updates will be installed with deadline what you set i.e at 4.00PM (based on Client local time or UTC).

It is also good to know the patch compliance states which are sent as state messages by client to site server .Patch compliance is calculated based on these 4 states.

Installed :This means the software update is applicable and the client already has the update installed. 

Not Required: This means the software update is not applicable to the client .

Required: This means the software update is applicable but is not yet installed.Alternatively, it may mean that the software update was installed but the state message has not yet been sent to to the site server.

Unknown :This means either that the client system did not complete the software scan or the site server did not receive the scan status from the client system.

Enough theory , Lets have a look at client troubleshooting steps. (Note: Client logs can be found at %windir%\ccm\logs\ ,if you have not changed the default path).

There are many logs on the client which help you to troubleshoot client issues,but we only look at important logs what is required for software updates.

1. First log to check is locationservices.log—>This log is used to check the correct software update point has been detected by the client.You can also see the management point and distribution point entries from this log.

image

2. 2nd log to check is wuahander.log –> when the software update scan cycle initiated, Windows update agent (windows update service) will contact WSUS (SUP) for scanning and if is successful,a state message will be sent to site server confirming that,software update scan is completed successfully which can be seen from this log. Get the report to know the software update scan results from here

For some reason,if you don’t see the successfully completed scan message,you should start troubleshooting from this log based on the error .

You can get the error description from CMTrace.exe tool. Copy the error code and use ctrl+L (Error lookup) from your cmtrace.exe ,get the error description  .

If WSUS entries are not set correctly or having any issues locating the correct WSUS,you can set WSUS entry manually or script.Further troubleshooting is required .

The registry location for the WSUS entries as follows:

HKLM\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate

HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate\AU with UseWUSserver =1

 

image

3. 3rd log is windowsupdate.log –>If software update scan is successful from wuahandler.log ,you can ignore this log file and directly move to next log (updatesdeployment.log) .If Software update scan is not successful then,you should look at this log for more information.  This log Provides information about when the Windows Update Agent connects to the WSUS server and retrieves the software updates for compliance assessment and whether there are updates to the agent components.

Using these 2 logs (wuahandler.log and windowsupdate.log) ,try to fix the errors and make sure ,you see the scanning successful from wuahandler.log

image

4.4th log to check is UpdatesDeployment.log—> Provides information about the deployment on the client, including software update activation, evaluation, and enforcement. Verbose logging shows additional information about the interaction with the client user interface.

This log shows the number of updates and deployments being targeted to a machine.

image

From above log snippet ,you see that,the total actionable updates = 0 means ,client do not require any additional updates that you targeted to this PC.For some reason,if the client says non-compliant from your sccm reports,try to refresh compliance state using https://msdn.microsoft.com/en-us/library/cc146437.aspx ,and monitor updatestore.log to see if the state messages (like Successfully raised Resync state message)has been sent to the site server (MP) or not.

you can alternatively use the below PowerShell script ,deploy to your clients monthly twice or once as per the business needs.

$SCCMUpdatesStore = New-Object -ComObject Microsoft.CCM.UpdatesStore
$SCCMUpdatesStore.RefreshServerComplianceState()
New-EventLog -LogName Application -Source SyncStateScript -ErrorAction SilentlyContinue
Write-EventLog -LogName Application -Source SyncStateScript -EventId 555 -EntryType Information -Message "Sync State ran successfully"

updatedeployment.log also tell you that,what assignments (Update deployments) made with count of updates in each deployment. From above log, Assignment {C37C45D8-E722-4EB7-AC21-014925079560} has total CI = 6 ,means ,the assignment has total 6 patches .

How do you check the deployment name for particular assignment ? well ,you can add Deployment Unique ID column for software update deployment or use below SQL syntax .

SELECT * FROM vSMS_UpdateGroupAssignment
WHERE vSMS_UpdateGroupAssignment.Assignment_UniqueID= '{C37C45D8-E722-4EB7-AC21-014925079560}'

For some reason,if you don’t see the newly added patches installing ( issue no:7) ,you can check updatedeployment.log with particular assignment group and patch count .If the count of patches are less than what it supposed to be,then you may have to refresh the machine policy ,initiate software update scan and wait for a while before client start downloading the policies.

If you see some updates are pending for action (total actionable updates <>0)  but not installing,look at CAS.log if your client is able to locate the content on the Distribution point or not.

UpdatesDeployment.log will also tell you ,if enough maintenance window (ServiceWindowManager.log) time available to install the updates.Read the following blogs to know the maintenance window calculation for software update installation.

http://blogs.technet.com/b/configurationmgr/archive/2013/06/11/configmgr-2012-support-tip-clients-never-reboot-even-when-required-in-a-deployment.aspx

http://blogs.technet.com/b/csloyan/archive/2010/10/24/maintenance-window-calculations-explained.aspx

5.5th log to check check is UpdatesStore.log—>Provides information about the compliance status for the software updates that were assessed during the compliance scan cycle (Status like Missing/Installed).

If you see all things working good, the final log to refer is RebootCoordinator.log—>Provides information about the process for coordinating system restarts on client computers after software update installations.

Below diagram shows the configuration manager Client side software update deployment flowchart captured from configuration manager software update management filed experience guide.

 

image

For troubleshooting clients, You can use tools like deployment monitoring tool,configuration manager support center etc.

I normally use the configuration manager support center to troubleshoot the client issues to check if the policy for the deployed software update group received correctly or not based on the PolicyIVersion.

Open the support center (you can download from Microsoft) ,connect to remote machine (need admin rights on remote computer) .

go to policy tab,click on requested  and then Load requested policy .you will see list of wmi instances on the left.

click on settings(root\ccm\policy\machine\requestedconfig) ,click on CCM_updateCIassignment ,click the policyID ,on the right side,you will see information about the software update group.

check the policy version on the client and on the site server .now you know how to take it further troubleshooting. Good luck.

image

 

Couple of common workarounds when troubleshooting software update issues :

1. Stop the windows update service,rename or delete the Software Distribution folder (%windir%\softwareDistribution) and start windows update service. This approach provides a fresh start with a new Windows Update data store if the Datastore.edb file is corrupted.

2. Restart the windows update service ,trigger software update scan cycle and software update deployment evaluation cycle.follow the logs.

4.http://eskonr.com/2014/11/sccm-configmgr-2012-software-update-scan-failed-onsearchcomplete-failed-to-end-search-job-error-0xc8000222/

5.http://eskonr.com/2014/11/sccm-configmgr-2012-patch-report-ou-based-compliance-status-per-update-group/

6.http://eskonr.com/2014/10/sccm-configmgr-2012-software-update-scan-error-group-policy-settings-were-overwritten-by-a-higher-authority-error-code-0x87d00692/

7.Refer software update client issues  https://technet.microsoft.com/en-in/library/bb932189.aspx

This post will be updating with possible solutions frequently.So keep checking this blog post.

Feel free to comment if corrections or anything missed via comment section.

 

26 Responses to "SCCM Configmgr 2012 Troubleshoot Client software update issues"

  1. Hi Eswar

    This is a great information you shared with us.
    I have one query can you please send me all the steps of Software update point and Software distribution in SCCM 2012 with all logs files associated with.

    Reply
  2. In SCCM 2012 auto client deploy is not working in Suprface pro 4 and desktops.Can you please give the solution.
    When i installed through SCCM client is installing,but when i connect new machine is not able to install sccm client default.

    Reply
  3. Hi Eswar,

    In my company environment we are using SCCM 2012 R2 Sp1 CU2 (5.0.8239.1301), recent days I am facing a weird issue, we have disabled software updates deployment (crossed deadline) which has been deployed for a collection, due to an issue we had to rollback the patches from installed server manually, however the members of the collection getting the software updates installed automatically, the behavior looks like a bug? is there any fix available for this issue? anyone facing similar issue? any help much appreciated.

    Reply
    1. Hi Sudarsan,
      can you check the logs as mentioned in the blog post to see what software update group is deploying the patches ? pick one server and check if that server is member of any other collection that has patches deployed ? client logs can help you to find more information.

      Thanks,
      Eswar

      Reply
  4. Hi Eswar,
    Thank you for great article. Hopefully I am not too late to follow up and if possible get a couple of answers.
    1. I am preparing Windows updates deployments with SCCM in medium size company - around 500 computers.
    In my test environment I deployed Windows updates for last 6 months, more than 200 definitions got updated but a couple failed temporarily on some machines. When I mentioned temporarily they are installed after retry in software center.
    Challenge is the notification in status bar which shows that update failed, even though more than 200 updates got installed notification points to couple of failing updates.
    I can hide all notifications when I configure settings for deployment and keep only end user notified for computer restart. This will probably bypass failing updates notification but will not install required (failed) updates.
    If I keep this notification then end users will generate numerous tickets for Help Desk assuming updates are not installing.
    I can't send introductory email to 500 people with something like:
    "If you notice you have a couple of updates which failed please hit retry and they will get installed"

    What are the best practices in the industry?
    2. Second thing would be restart policy with SCCM. We can configure in Administration/Client Settings/Default Client Settings ( or create a custom client settings)/Computer Restart option to reboot computer max 24hours after updates are installed.
    This will make sure updates got installed and computer is compliant. But it's not that simple as it seems.
    VIP people don't want to be forced to reboot their computers after 24 hours if they are in the middle of presentation or on meeting.
    It's really hard to plan and reboot computer prior to important events but there is no option to postpone.
    If we don't make reboot mandatory than computers will be vulnerable.
    There are 3rd party tools on the market, like shutdown tool from deploymentresearch group but not sure if this would be desirable as it may reboot computers which are not even patched by mistake after 7 days. I have to test it.
    Any advice, the best solutions out there?
    Thanks,
    Blaf

    Reply
  5. good and very useful article sir,

    sir I had an issue, as mentioned I renamed the c:\windows\software distribution folder and tried to install the failed update (Adobe activeX update), it installed sucessfully. but on the next day again it is showing the update as failed in software center.

    my WUAhandler.log shows: Failed to download updates to the WUAagent datastore. ERROR: 0x800b0109,

    the error lookup description is: A certificate chain processed, but terminated in a root certificate which is not trusted by the trust provider.

    the same update is sucessful in 70% of machines and is failed in remaining machines,
    all are in the same location.

    please guide me sir...

    Reply
  6. Hello and thanks for the informations.
    Just a little question. How to check the policy version of the sotware update group on the server ?
    thank you

    Reply
        1. ok ,I could not able to find what I said earlier but may I know why are you looking for that info ? for client troubleshooting to know if the particular deployment has updated on the client or not ,you can simply check UpdatesDeployment.log for number of updates as I described in the blog.

          Reply

Leave a Reply