Citrix PVS Target Device Trust Relationship Failed

Issue

Attempting to log into a Citrix Provisioning Services client device a user or domain administrator receives an error that the "Trust Relationship between this workstation and the primary domain has failed." This error is not seen when logging into the client as the local administrator.

Backstory

This issue is often seen when a "Golden Image" is booted to the HDD sometime after the XenConvert process has completed. It is also seen with server and client class system Target Devices deployed via Provisioning Services, often after upgrades involving Private Image mode or when machine account password management has not been properly configured. The cause is a mismatched machine account password between the device and Microsoft Active Directory.

References

TechNet: Netdom
TechNet: Reference Point: "The trust relationship between this workstation and the primary domain failed."
KB162797
CTX134340
CTX132289

Step-By-Step Guide

For a PVS Target Booting to vDisk

1.      Shut down the target device
2.      In the Provisioning Services, navigate to the device's site collection. It should show down.

3.      Right-click on the target device and in the context menu select Active Directory>Resent Machine Account Password.
4.      In the pop-up window, select the Domain and Organizational Unit of the Target Device.

5.      Finally, select Reset Account.

6.      Assuming this is successful, boot the device and log in with your Active Directory credentials.

For a PVS Target Booting to Hard Disk

Method 1 (Preferred)

1.      Log into the server as the local administrator

2.      Verify NetDom is available on your target device (it may have to be downloaded for a desktop OS)

3.      Verify the computer account is in the desired Active Directory Organizational Unit

4.      Launch cmd.exe as an administrator

5.      Run:

netdom.exe resetpwd /server:<server> /userd:<user> /passwordd:*
<Server> Specifies the domain controller to use to set the computer account password.
<User> Specifies the user account to use to make the secure connection with the domain that you specify in the /server parameter. You must specify the user account in the Domain\User format.
* Specifies the password of the user account that you specify in the /userd parameter. If you specify the value of this parameter as a wildcard character (*), this parameter prompts you for the password.

6.      Reboot

Method 2

1.      Log into the server as the local administrator

2.      Remove the server from the domain using your desired method and reboot

3.      Verify the computer account has been removed from Active Directory

4.      Pre-create the computer account in the desired Active Directory Organizational Unit

5.      Re-join the target device to the domain

6.      Reboot

Method 3 (if the local administrator account is unknown and passwords are cached on the client)

1.      Unplug (or disconnect if a VM) the NIC. This should allow you to log in with cached credentials to a domain account.

Verification of Configuration

It is important that management of computer object passwords is configured properly for Citrix PVS deployments to work properly. This involves configuring the password management in both Active Directory and Provisioning Services.

PVS & Active Directory Settings

1.      In the Citrix PVS Console, navigate to the desired site and Servers node.

2.      Right-Click on each server and select Properties.

3.      In the options tab make sure your server is configured to manage computer accounts. The days between updates is usually left at 7 but may be changed if desired.

4.      Next verify the vDisk in use is managed properly by navigating the vDisk Pool node and right-clicking on the desired disk. Select Properties from the context menu.

5.      On the General tab, verify Enable Active Directory machine account password management is checked (the screen shot is greyed out because this disk is online and cannot be changed).

6.      Now move from PVS to Microsoft's Group Policy Management MMC.

7.      It is important that machine account password changes be disabled for the Organizational Unit that hosts Target Devices and the maximum machine account password age must be set. This value must be greater than the number of days specified above in the PVS console.

Edgesight Unrecoverable, fatal DB error

One error to watch out for on Citrix EdgeSight deployments is the one below.

Error:  An unrecoverable, fatal database error has occurred.  Shutting down the Citrix System Monitoring Agent. 

This error is generally a sign of local FireBird database corruption. This could happen for a variety of reasons: system power failure, hard drive corruption, or improper anti-virus exclusions to name just a few. 

If this error were to experience frequent re-occurrence, be sure to take a look at the EdgeSight logs (SYS_EVENT_TXT) to see if a cause can be found. 

Additionally, keep an eye on the local DB size. If it regularly exceeds the size specified in the management infrastructure I have found corruption is much more likely.

Finally, to fix this error when corruption is the cause (the System Monitoring service will often not restart if corruption is the cause) follow the steps below to clean the local database.

1. Stop the Citrix System Monitoring service on the problem server
2. Rename the FireBird database file for EdgeSight (RSDATR.FDB)
3. Restart the Citrix System Monitoring service.

The service should restart and the FDB database recreate. Yes, you will loose any data that was not previously uploaded to the primary infrastructure database.

Not sure where the FireBird database is? (Common for PVS deployed devices with a re-directed DB.) Check out the DataPath value in this registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Citrix\System Monitoring\Agent\Core\4.00

Creating a Microsoft Mandatory or Citrix Template Profile

I have been asked several times lately by colleagues how to create a profile that could be used with Citrix User Profile Manager as a template or a Citrix Mandatory Profile. The process for this basically the same as creating one for use with Microsoft Mandatory Profiles. An outline of the required steps can be found below.


Reference Articles:

• User Profile Best Practices for Citrix XenApp
• Customize the Default Local User Profile
• Creating a Mandatory User Profile
• The Cache Option for Offline Files Must Be Disabled on Roaming User Profile Shares
• The "Set roaming profile path for all users logging onto this computer" Group Policy setting also applies to local user accounts in Windows Server 2008
• Deploy Roaming User Profiles

Goal:
Create a profile that can be utilized for all users of a system to either speed logon or set base configurations void of personalization settings.

Prerequisites:

1. A "like" operating system (Server class 2008 or newer, Windows desktop OS 7 or newer).
   See this article for details.
2. Network storage*
3. Administrative privileges

*Network storage can be avoided by copying the profile locally prior to user access which eliminates the need to have the profile centrally available. 

Step-by-Step:

1. Select a "clean" machine for creating the user profile. This machine should be ready to deploy as if it were going to production, but should should not have been in use. This eliminates as many divergences from the ideal build as possible.
2. Log into the server and create a new local user "ManUser1." (I am utilizing Microsoft Server 2008 R2 in this case.)
3. Add the new user to the local administrators group on the server
4. Log off of the current session and into the server with the new use
5. Customize the user environment as needed
• Alternatively the reference article above for customizing the default profile could be done ahead of time 
6. Log off of this account and back in as your original administrator account
7. Create a network share for central access
• Example: \\MyLab_NAS\MandatoryUser
• Share Permissions: Everyone Full Control
• NTFS Permissions: Follow this TechNet article
• Disable Caching 
8. Copy the entire folder C:\Users\ManUser1 to the new share
9. Rename the folder ManUser1.V2
10. Clean the profile:
1. Delete the AppData Local and AppData Local Low folders
2. Open RegEdit and mount the ntuser.dat hive:
• Search for any instances of "ManUser1" and clear those instances
• Check the various software run and policy locations (Run, Run Once, Policies, etc.) and remove any unwanted software launches / settings
11. Set permissions:
1. Return to RegEdit with the already mounted the ntuser.dat hive:
• Right-click on "ManUser1" and select permissions
• Delete "ManUser1"
• Add Authenticated Users Full Control
• Unmount ntuser.dat from RegEdit
12. In the root of ManUser1.V2, delete files other than NTUser.dat and ntuser.ini (i.e. Log, BLF, etc.)
    *For use with Citrix Profile Manager, skip to the next section
13. Rename ntuser.dat to ntuser.man
14. Specify the profile \\MyLab_NAS\MandatoryUser\ManUser1 in the group policy to set mandatory profiles with Server 2008 RDS (XenApp) or newer. For Windows desktop OS, reference this.
• Computer Configuration > Policies > Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Session Host > Profiles
• Enable: Use mandatory profiles on RD Session Host server
• Enable: Set path for Remote Desktop Services Roaming User Profile
  *Remember NOT to include .V2 in your path!
15. Not run gpudate /force on any clients targeted by the deployed group policy to have the new mandatory profile applied

Citrix User Profile Manager

There a a couple differences when utilizing Citrix UPM 5.x for managing profiles. Your profile created above can still be used (though there is no need to rename it to *.man nor apply the Microsoft group policy). To check out the steps for configuring UPM to for use, check out the Citrix eDocs page here. It is rather straightforward and you have already taken care of all the prerequisites!

NetScaler VPX on Hyper-V 2012 R2

Unfortunately the current Citrix download for their NetScaler VPX virtual appliance was exported from Hyper-V 2008. This means that the appliance can not natively be imported into Hyper-V 2012 R2 due to a deprecated feature (reference).

I found that I could get around this little hiccup in my lab environment by using VMware Workstation to stand up a Hyper-V 2008 R2 instance. Once the legacy instance is up, import the VPX appliance. You can now copy and paste the VM folders from that appliance directly into your Hyper-V 2012 R2 infrastructure and add the VM that way - without using the export feature on the legacy Hyper-V system.

You could also stand up 2012 original release and use it as an intermediary host - though I did not test this method.