Debugging Layer Integrity Problems in Citrix App Layering 4.x and later

In V4, when you’re ready to shutdown and finalize a layer, you run the Shutdown for Finalize icon on the desktop (As Administrator). It makes a call to uniservice.exe to get the current Layer Integrity state. Uniservice is tracking all the same things it always has for Layer Integrity: NGEN or MSCORLIB is still running, a reboot is pending, a domain operation is still pending, or a RunOnce script is still waiting.

Shutdown for Finalize is checking to see if anything is still pending that should happen in the layer rather than happen in the image later. If something is, it does not shut down, and instead puts up a statement about the pending issue. Fix the issue (for instance, reboot) and try again. It also writes this information into two log files:

C:Program FilesUnideskUniserviceLogLayerIntegrity.txt

C:Program FilesUnideskUniserviceLogUniBilcLogs_X.txt

You can’t know exactly which UniBilcLogs file it’s using, so look for the one with the latest timestamp. That will be for the current boot. Search for “Integrity”.

You might think you could bypass the Layer Integrity check by just shutting down the machine normally and finalizing that. But if you try, you will find the ELM will stop the task and return you to the Packaging Machine, because it knows that the Layer Integrity Checks either failed or never ran. You must successfully run that Shutdown for Finalize script to finalize a layer.

The registry key, noted at the end of this article, to bypass or ignore integrity problems still works, and you should be just as reluctant to use it as ever. But it’s still a valid way to give up and bypass it.

There are 7 Layer Integrity warnings you can see:

“a RunOnce script is outstanding – please check and reboot the Packaging Machine”

“a post-installation reboot is pending – please check and reboot the Packaging Machine”

“a Microsoft NGen operation is in progress in the background {0}”

“an MSI install operation is in progress – please check the Packaging Machine”

“a reboot is pending to update drivers on the boot disk – please check and reboot the Packaging Machine”

“a Microsoft NGen operation is needed”

“Software Center Client is configured to run, but the SMSCFG.INI is still present. See https://social.technet.microsoft.com/wiki/contents/articles/23923.implementing-sccm-in-a-xendesktop-vdi-environment.aspx”

“A RunOnce script is outstanding” is telling you that there is a key in either of these two locations:

HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsCurrentVersionRunOnce

HKEY_LOCAL_MACHINESOFTWAREWow6432NodeMicrosoftWindowsCurrentVersionRunOnce

Windows normally deletes those keys on reboot, but we have seen circumstances (especially with our own script, envsetup.cmd) where that doesn’t happen. You can manually run the referenced script and delete the key, or just delete the key if the script file no longer exists.

“A post-installation reboot is pending” is looking at six different registry keys. Your first course of action should be to reboot, more than once(in some cases it has taken 3+ reboots), just to make sure that it isn’t a real reboot being requested by some software. It may also be helpful, if the problem is NetLogon, to restart the Unidesk Service for Message Management.

First we check for the existence of any of these three:

HKLMSystemCurrentControlSetControlSession ManagerPendingFileRenameOperations

HKLMSOFTWAREMicrosoftWindowsCurrentVersionComponent Based ServicingRebootPending

HKLMSOFTWAREMicrosoftWindowsCurrentVersionWindowsUpdateAuto UpdateRebootRequired

You can manually modify any of these to suit your needs, including just deleting them.

Then we look for changes in the NetLogon key (if the current value is now different from what it was at bootup), and to see if the computer name doesn’t match the active computer name. This is how we determine that a domain-join operation is still waiting for a reboot.

HKLMSYSTEMCURRENTCONTROLSETSERVICESNETLOGONStart

HKLMSYSTEMCURRENTCONTROLSETCONTROLCOMPUTERNAMEACTIVECOMPUTERNAME

HKLMSYSTEMCURRENTCONTROLSETCONTROLCOMPUTERNAMECOMPUTERNAME

Generally you cannot modify these. I’ve seen some software modify the NETLOGONStart key on every reboot, so maybe that’s happening. If, after cleaning out the top three, you still get the prompt on reboot, you will need to use the flag to ignore layer integrity checks.

“A Microsoft NGen operation is in progress in the background” is telling you that a foreground or background NGEN operation (where .Net assemblies are compiled into native images) is still in progress. Generally this is simply true: the ngen rebuild is still running. To watch it in the foreground, run “ngen update /force”. Or you can wait it out, and run “ngen queue status” periodically to see how it’s doing, but that will slow it down because the background process pauses every time you check its status in the foreground. Don’t reboot or you might cause it to have to start over.

It’s important to let NGEN finish. If you kill the process or reboot in the middle, you might wind up with partially written .Net assemblies that crash programs when they show up in an image. You should be patient. However, sometimes we have seen background MSCORSVW.EXE processes, clearly doing nothing, that just don’t finish even after hours. A reboot might help those.

We are looking for the following services in the running process: ngen.exe, ngentask.exe, mscorsvw.exe.

“An MSI install operation is in progress” is very specific: it is saying that a system mutex (mutual exclusion object) named precisely Global_MSIExecute exists. The MSI installer uses that to ensure that only one installer can run at a time. I don’t know of anything you can do about this manually, if you are certain that no MSI installations are happening.

(Note, there was in App Layering 4.2 a bug with upgrading an existing Windows 10 layer from 1611 to 1703 where this flag could be set and not cleared.)

“A reboot is pending to update drivers on the boot disk” is telling you that a service or driver that is set to start at system boot time (the START= value in the registry is 0) was modified or installed, and App Layering wants to make sure the modified driver can boot successfully. Normally you just need to reboot once, and the driver will work fine. We have on some occasions seen software (like Microsoft Defender) attempt to modify its driver file on every single boot, triggering this integrity check every time, so no number of reboots is sufficient to clear it.

“a Microsoft NGen operation is needed” is telling you that an application was installed on the packaging machine and that it scheduled items to be updated at a priority level of 3. That means that the ngen will run when idle and that it is simply waiting until there is no more activity. We are blocking because the ngen needs to create the binaries now instead of on every machine that the application will be deployed to in order to ensure that the application will run in the most optimal way. You should run an ngen eqi 3 in both the c:windowsmicrosoft.netframeworkv4.0.30319 directory and the c:windowsmicrosoft.netframework64v4.0.30319 dirctory to have the ngen complete the operations that are needed. You can also wait, as the ngen will typically pick up and run on its own after 15 minutes of idle time.

The values that is being examined are HKLMSOFTWAREMicrosoft.NETFrameworkv2.0.50727NGenServiceRootsWorkPending and HKLMSOFTWAREWOW6432NodeMicrosoft.NETFrameworkv2.0.50727NGenServiceRootsWorkPending. A value of 1 means that there are work items queued up to be processed.

“Software Center Client is configured to run, but the SMSCFG.INI is still present….” is telling you that we have seen that this machine has ccmexec.exe configured as a service and that it is not configured as disabled. Since we know that any layers created on a packaging machine need to be sealed properly in order to deploy correctly in a VDI environment, we are checking to make sure the SMSCFG.ini is not present. See the web page indicated to get an understanding of why the software center client needs to be sealed. We have provided the commands to run in a batch command file that you can use to seal the layer (run c:windowssetupscriptsSEALSCCMCLIENT.cmd for an administrator command window).

If you have a layer that simply won’t ever get to finalize, for whatever reason (like it always thinks it still has a reboot pending, or you don’t care about corrupted .NET assemblies and don’t want to wait for NGEN to finish), you can tell that single layer to ignore its layer integrity checks and allow you tin shutdown to finalize, using a registry key.

Run regedit.exe and create this key

[HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesUniservice:]

“BypassLayerCheck”=DWORD 1

The value doesn’t matter, all that matters is that the value exists. This will block all layer integrity checks and allow a layer to be finalized regardless as to how bad we think it might be.

Please do not use this key except as a last resort. We block you from finalizing in these 4 circumstances specifically because we believe allowing you to finalize will irreparably harm the layer and/or the image you publish with it. Always try to solve the problem within Windows first.

Related:

Leave a Reply