XenApp & XenDesktop 7.x – Logon delay of 20 seconds at ” Please wait for Citrix User Profile Manager”


If you ever have a situation where your Citrix logons sits on “Please wait for Citrix User Profile Manager” for around 20 – 25 sec, you have come to the right post. I have had this strange one where users started to see their logons taking longer than usual. Below is my setup and your results may vary depending on the UPM version that you are running

  • XenApp 7.5 site
  • Server 2012 R2 VDAs
  • Citrix UPM 5.1
  • McAfee AV 8.8

Issue Manifestation – Delay of 20-25 seconds on ” Please wait for Citrix User Profile Manager” and this was consistent.

Remediation – Turn ON Citrix Profile Streaming

I am partially to blame for this issue as I had turned OFF Profile Streaming when I had my users complaining that the logoff was taking a little while. Now i need to figure out what is causing the logoffs to take roughly 10 seconds to complete. Already looked at the AV side of things and i have the required Exclusions. Hunt continues and will update with the findings..

So that’s a quick one there for you guys and hopefully someone will find it useful. If you find it working/not working for you, let me know

Using XPERF to troubleshoot slow logons


Installing XPERF to capture a slow boot or logon trace

  1. Install XPERF from the Windows SDK for Windows 7 and .NET Framework on the slow boot or logon computer.
    Hint 1: It is possible to install only the Windows Performance Toolkit from the Windows SDK.
    Hint 2: I suggest installing the WPT in an X:\XPERF directory rather than the default directory recommended by setup. It’s easier to access and copy files in and out of, and change paths, to the short-labeled directory.
    Hint 3: Once installed on a computer, the XPERF installation directory can be copied to other computers that you want to capture ETL traces from or view ETL traces on. There are no external files, DLL registration or registry changes required to make or view a capture. Make a copy of the X:\XPERF directory and copy at will.
  2. If taking a network trace on a 64-bit computer, enable the following registry key and reboot before capturing ETL data. This prevents kernel mode data from being paged out of memory.
Registry Path HKLM\System\CurrentControlSet\Control\Session Manager\Memory Management
Setting DisablePagingExecutive
Data Type: REG_DWORD
Value: 1

Using XBOOTMGR to capture slow boots, or slow logons caused by slow boots

  1. Logon as an Administrator of the computer you want to trace (either a local Administrator or Domain Admin account that is a member of the local machine’s Administrators group).
  2. Open an elevated command prompt.
  3. Run the following command in the WPT directory (default path is C:\Program Files\Microsoft Windows Performance Toolkit). This syntax is useful to capture slow boots as well as slow logons thought to be caused by a delay in OS startup:
    xbootmgr -trace boot -traceflags base+latency+dispatcher -stackwalk profile+cswitch+readythread -notraceflagsinfilename -postbootdelay 10       

    This command will:

  • Reboot the local computer
  • Capture ETL tracing during the boot and logon operation (you provide user name, domain name, and password for the slow logon account)
  • Stop tracing at 10 seconds after disk and CPU utilization fall below a certain threshold after user logon. Increase the value for “-postbootdelay” as required to troubleshoot user desktops that are unresponsive to mouse and keyboard input post boot.

Using XPERF to capture slow logons

  1. Logon as an Administrator of the computer you want to trace (either a local Administrator or Domain Admin account that is a member of the local machine’s Administrators group).
  2. Open an elevated command prompt and run this command from WPT Install directory (default path is C:\Program Files\Microsoft Windows Performance Toolkit.                                                                                                                                                                                                                                                                                                  xperf -on base+latency+dispatcher+NetworkTrace+Registry+FileIO -stackWalk CSwitch+ReadyThread+ThreadCreate+Profile -BufferSize 128 -start UserTrace -on “Microsoft-Windows-Shell-Core+Microsoft-Windows-Wininit+Microsoft-Windows-Folder Redirection+Microsoft-Windows-User Profiles Service+Microsoft-Windows-GroupPolicy+Microsoft-Windows-Winlogon+Microsoft-Windows-Security-Kerberos+Microsoft-Windows-User Profiles General+e5ba83f6-07d0-46b1-8bc7-7e669a1d31dc+63b530f8-29c9-4880-a5b4-b8179096e7b8+2f07e2ee-15db-40f1-90ef-9d7ba282188a”  -BufferSize 1024 -MinBuffers 64 -MaxBuffers 128 -MaxFile 1024
  3. Note: This syntax works on Windows Vista (Windows Server 2008) and Windows 7 (Windows Server 2008 R2) computers
  4. Press CTRL+ALT+DEL and then Switch User.
  5. Logon with the user account experiencing the slow user logon to reproduce the issue.
  6. Stop the trace. While logged on with the slow user account, open an elevated CMD prompt and type:
xperf -stop -stop UserTrace -d merged.etl

Close the slow logon user session and the admin logon session opened in step 2 as required.

IMPORTANT: The double “-stop” call in step 5 is not a typo but is required. The first “-stop” terminates kernel tracing. The second “-stop” terminates user mode tracing.

Note: You can also stop the trace by using Switch User to return to the admin user logon established in step #2 and running the same XPERF stop command in the elevated command prompt used to start the trace. This results in a larger, longer trace and requires that you discern the different logons encapsulated in the trace.

  1. Send the MERGED.ETL file to Microsoft or an Independent Solution Vendor (ISV) for analysis, or review it yourself.By default, the MERGED.ETL file will exist in the XPERF installation directory, which by default is %systemdrive%\program files\microsoft windows performance toolkit directory or if you followed our recommendations early in the doc, the c:\XPERF directory (that is, the XPERF installation directory).

Double Click the merged.etl file to open it in Windows Performance Analyzer ( Ensure that you have the Windows Performance Toolkit Installed on the computer that you are opening the file from)

I am now going to talk about one of our customer environments where that had a Citrix XenApp 6.5 deployment and the user logons used to be around 90 sec. I agree that this isn’t such a bad situation to be as I have seen even worser logon times.

The customer environment consists of the below

  • Citrix XenApp 6.5 with HRP 02
  • EdgeSight Server with agent version 5.4 loaded on the servers
  • StoreFront 1.2
  • Group Policies applied with login scripts and policy preferences
  • ESET NOD 32 Antivirus

The users complain that the screen will sit at “Welcome” screen for approx 30 sec after which the logon moves faster and towards the end , there is a further delay after “Preparing the Desktop”

First things first, during these situations AV will be the first to be blamed. Decided to stop the AV service and turned off Real Time scanning to rule that out. Tried to login with a test account and it is the same 90 +secs

Tools at my disposal ( the customer has got a platinum XenApp License, so they own a lot of good stuff) 🙂

  • EdgeSight Reports – showed no problems whatsoever with the Group Policy processing, logon scripts or printer mapping scripts. The issue seems to be well before the GP processing and scripts are run. What else is there to look at?
  • Citrix UPM logs – Citrix User Profile Manager has logs of its own and i decided to take a look at it, there was nothing obvious with it other than the little hint it could give towards the end of the logon process
  • Autoruns – This gave me a hint at what could be going wrong towards the end of the logon process. It was an edocs print pro process. Turned that off and tried a logon. Bingo, the final 15 seconds of desktop preparation has gone away.
  • XPerf – Analyzing the XPerf logs also pointed out issue with the “edocs Print Pro” software printer. I just can’t find the screenshots of my XPerf analysis anymore as this happened a while ago and I just couldn’t write about it on time.

 

XPerf is a super cool tool to troubleshoot slow logon/boot up times alongside Citrix UPM logs. Autoruns does help to selectively stop all or some logon processes and thus should not be overlooked when troubleshooting long logon times. I will try to include more XPerf tracing and analysis in my future blog posts. I will talk to you in my next post.

Automatically Load/Unload Excel Addins for specific users – XenApp/XenDesktop


Hello There,

Time for another quick write-up. I was asked by one of my customers to enable Excel add ins for a selected bunch of users. These users need access to TM1 and SUN Financial plugins which are a bit slow to load so it doesn’t make sense to have this plugin available to all the users.

The environment is based on  XenApp 7.5 site with Windows Server 2008 R2 Hosted Shared desktops and apps. Microsoft Excel 2010 is the where the add in needs to be integrated.

 

  • Step 1 is to find out the location of the Excel Add in – they are of 2 types, COM Add ins and Excel Add ins
  • Create a user policy via preferences to add a registry entry to the user’s HKCU hive
HKCU\SOFTWARE\Microsoft\Office\14.0\Excel\Options

create a REG_SZ value and name it OPEN. The value is the path of the Excel Add in in quotes. if you have multiple add ins, call it OPEN1, OPEN2 and so on

Capture3

Please note that if the Office software is 64 bit, the above must be done on the WOW6432Node  and also the version # of Excel/Office changes with a different Ms Office suite. I used Excel 2010 which is 14

There might also be cases where the plugins need to be selectively removed from automatic launch in Excel and for that, the relevant registry keys could be removed via group policy preferences.

This methodology can also be applied to non-Citrix environments and VDI solutions