Storefront Load balancing using NetScaler


It’s been a while since I wrote on my blog so let’s get straight into the post without much mucking around. This time we will discuss how to go about setting up Storefront load balancing using NetScalers. This can be configured on a standalone NetScaler or a NetScaler pair in HA. The recommendation is obviously to get this setup on a HA NetScaler pair so that NetScaler outage wouldn’t result in Storefront also being unavailable.

My Storefront version is 3.11 and have a cluster with 2 Storefront servers. NetScaler version is 11.1 but the NS version shouldn’t matter much as the steps would be more or less the same for other NetScaler firmware versions – newer or older. (unless you are too far behind)

Pre-Requisites

To configure Storefront load balancing we need the following –

  • 2 or more Storefront servers
  • an IP address for the virtual server that hosts the LB configuration
  • SSL certificate that points to the intended load balanced URL of Storefront – the certificate can be a wild card or a named certificate

First Things First

Logon to your NetScaler and navigate to System — Settings — Configure Basic Features. Ensure that Load Balancing is selected, if not select it and click OK

1

NetScaler Configuration

Create Servers

Now, navigate to Traffic Management — Load Balancing — Servers. Click Add

2

Give the Storefront server a name and enter the IP address of the server. Ensure that “Enable after creating” is selected. Click Create

Add the second Storefront server following the above steps. If you have more than 2  servers, add all of them.

3

Create Monitors

New NetScaler version come with a built-in Storefront monitor so we are going to make use of it here. Go to Traffic Management –Load Balancing — Monitors and click Add

Here I am only going to create a single monitor to probe all my Storefront servers. You can choose to create multiple monitors depending upon the number of Storefront servers that you have. In my case, i will create just one.

Give a name to the monitor and select the type as STOREFRONT

5

Now select Special Parameters tab and provide the name of the Store that you have created in Storefront. Check the 2 entries – Storefront Account Service and Check Back End Services. 

4

Click on the Standard Parameters tab. Ensure that Secure is selected as below. Click Create

6

Create Service Groups

Go to Traffic Management –Load Balancing — Service Groups

Give a name to the service group and select the protocol as SSL. Check the entries below

  • State
  • Health Monitoring
  • AppFlow Logging (only if you have NetScaler MAS in your environment)

Click OK

7

Under Service Group Members, add the server entities that we created earlier. Once done, they will look like the below

8

Under Settings, type the Header as X-Forwarded-For

9

Under Monitors, bind the monitor that we created before

10

Under SSL Parameters, setup the settings as below

11

Under Ciphers, setup the ciphers based on your company security policy.

12

Once done, Service Group for Storefront should look like this

13

Now, it’s time to create the Virtual Server

Virtual Server

As mentioned in the pre-requisites section , we need an IP address for this. If the NetScalers are sitting in the DMZ, a DMZ IP address is required. In my case, NetScalers are hosted internally so i will use an internal unused IP address.

We will also need the SSL certificate here.

Go to Traffic Management –Load Balancing — Virtual Servers

Click Add

Give a Name to the virtual server and select the protocol as SSL

Specify the IP address under IP Address field and specify the port # as 443

14

Click More and specify the settings as below (note, that AppFlow logging only needs to be enabled if you have a NetScaler MAS setup or other monitoring solutions that could make use of AppFlow logs)

15

 

Under Services and Service Groups, click on Load Balancing Virtual Server ServiceGroup Binding

Click Add Binding and select the Service Group that you created in the previous step. Click OK

Once completed, the page should look like the below. Click Close and click Done

16

It’s time to attach the certificate. Go to Traffic Management — SSL — Manage Certificates / Keys / CSRs

17

 

Click on Upload button and upload your certificate file to NetScaler

Go to Traffic Management — SSL — Certificates — Server certificates

Under Certificate, click on Server Certificate and then Install

Give a certificate key-pair name and choose the certificate that was just uploaded in the previous step. Click Install

Now, go back to Traffic Management –Load Balancing — Virtual Servers

Select the Virtual server created for Storefront and click Edit. Under Certificates, select Server Certificate and then Click Add Binding

Under SSL Ciphers, select the ciphers that you would like to be in place. I am going with the default one. This is not the most secure for a production setup so go with something that’s secure enough for your organization.

Under SSL Parameters, configure the settings as below. Click OK

18

Under Method, Select LEASTRESPONSETIME for the Load Balancing Method. Configure a Backup LB Method, I choose LEAST CONNECTION

You can read more about the LB Methods here

19

Click OK

Under Persistence, select COOKIEINSERT for Persistence with a time-out value of 0. You can also read why I selected the timeout value of 0 here

Under Backup persistence, select SOURCEIP with a timeout of 60. Fill in the Netmask as in the picture

20

Click OK and then Done

We have now completed almost 90% of the config. There are a couple of things left so hold on tight.

The configuration so far will ensure that load balancing will be performed between the Storefront servers ( I know, i know I haven’t setup the DNS entries for the load balanced VIP)

If someone type in the http URL of LB Storefront in their browser, it will not go anywhere. It will show them the IIS page instead. So how do we ensure that the users are redirected to the correct Storefront page (https version) every single time? We will setup another virtual server on port 80 with a redirect URL configured.

Let’s do that now.

Under Traffic Management –Load Balancing — Virtual Servers, Click Add

Under Basic Settings, give the virtual server a Name and select protocol as HTTP

Specify the same IP address as for the Storefront LB VIP and provide 80 for the Port #

Click OK/Create

Under Persistence, select SOURCEIP with a timeout of 2 mins

21

Click OK

Under Protection, type in the correct HTTPS URL that you would want the users to be redirected to under Redirect URL field

22

Click OK. Then click Done

You will notice that the virtual server will be marked as down

23

DNS Changes

Now head over to the DNS server and open the DNS Console

Create an A record pointing to the Storefront LB name with the IP address configured on the vServer for LB configuration.

Storefront Changes

This is the last step, I promise. Head over to the Storefront servers and it’s time now to run some Powershell commands

Now, the monitors that we created earlier will be marked as Down if we didn’t perform this step prior to creating them on the NetScaler. That’s because the monitor created was based on HTTPS and by default, Storefront monitoring is done on HTTP

To change this to HTTPS. We need to configure the monitor service to use HTTPS instead. On all the StoreFront 3.0 servers perform the following steps.

Run PowerShell as an administrator.

Navigate to the Scripts (C:\Program Files\Citrix\Receiver StoreFront\Scripts) folder via the Powershell on the Storefront server,

Run ImportModules.ps1

24

Run the below command

 Get-DSServiceMonitorFeature

25

Now, type the below to setup the Storefront Monitor on HTTPS

Set-DSServiceMonitorFeature -ServiceURL https://localhost:443/StoreFrontMonitor

 

Repeat the above steps on all the Storefront servers.

Now, head back to the NetScaler and you can see that the monitor will be in GREEN and showing a status of UP

That’s all we need to do to setup Storefront load balancing using NetScalers.

 

 

 

 

Advertisements

The curious case of NetScaler access with error message ” The Connection to “Desktop” failed with status (Unknown client error 1110)”


I was pulled into to look at a problem for one of our customers with their Netscalers which stopped the user connections intermittently throwing a very “helpful” error message ” the connection to the desktop failed with status (unknown client error 1110).

The customer description was “it only started to happen a few weeks ago and these days its quite impossible to land a successful connection from the outside of our corporate network”

I managed to get a couple of screenshots of error messages from the users and they appeared like below. When queried, the internal access via Storefront is working fine.

image001

Looking at the error message, there are a multitude of reasons why you would get that and i am outlining the common areas to look in such cases.

  • Check if the Root certificates and intermediate certificates are available on the client devices. If frequently patched, the client will most probably have the latest and update Root CA’s from various public CAs. Check the IE’s / Other browsers’ certificate store to verify the Root and Intermediate CA SSL certs
  • If using non-IE browsers for connectivity, switch over to IE to see if it connects. IE is the safest bet when it comes to connectivity to Citrix environments.
  • Check for SSL ciphers attached to the NetScaler Gateway vServer. If high security ciphers are used, this issue may occur. relax the cipher suites to see if that makes a difference. Again, if cipher suites are an issue, the issue will occur every single time when you connect and not sporadically.
  • Check the STAs on the NetScaler and ensure that it matches with the STAs configured on the  WI/Storefront. This is one of the most important setting to check and probably the first one to check if the issue occurs only sporadically. There is a high possibility of an STA mismatch as it turned out to be in my case.
  • Check the FW from the NetScaler to the VDA – As the title says ensure that the Citrix ports to the VDA are open from the Netscaler

Citrix XenApp – Long logon times and potential fixes


Long login times are something that we have been hearing from time to time working with Citrix XenApp /XenDesktop environments. I have had a similar issue recently for one of my deployments with XenApp 7.5 on Windows Server 2012 R2 workloads. My logon times averaged around 30 seconds which is not bad at all. I still wanted to make it better and my target was to bring it under 15 seconds 🙂

Below are a few things that you can do to reduce the logon times. Please note that this is not a comprehensive list so feel free to comment below with your findings on this post so that i can update it and make the list better.

If you have Citrix Director in your environment, that would be the first place to look. It gives you in-depth details on where the profile load takes longer so that you can focus on those areas first.

I had Citrix Director in the environment and looking at it, Interactive Session seems to take a major chunk of the overall login time.

  1. Anti-Virus – This is one item that is overlooked often so ensure that you have set the required exclusions for your AV product. I would even go head and recommend turning OFF Real-time scanning for MCS/PVS created images as they are only read only. Please ensure that you run Real-time scanning on the network shares that hosts the profiles/home folders and also on the Write Cache location in case of PVS images.
  2. Enable Legacy Graphics Mode –This is a Citrix policy and enabling this is found to increase the logon speed. This is Adaptive Display First Generation which is good on older operating systems like Windows Server 2008 R2. Not recommended to be enabled in Windows Server 2012 R2 as it is found to cause some/all applications to fail consistently or randomly. In short, apply this setting with caution if your workloads are Windows Server 2012 R2 / Windows 8.x
  3. Remove CD-ROM drives from your virtual Citrix servers – May sound silly but having a CD-Drive on the server is found to increase the logon time.
  4. Active Setup – My suggestion is to check the Active Setup on the Citrix servers. I use SysInternals Autoruns tool to disable (not delete) the unwanted Active Setup keys under Installed components for HKLM\SOFTWARE\Microsoft\Active Setup\Installed Components and HKLM\SOFTWARE\Wow6432Node\Microsoft\Active Setup\Installed Components  as well.
  5. Autoruns This is a brilliant tool from SysInternals and throws a lot of light into what runs when a user logs in to a Windows Server. Run this and disable all that is not required for your environment.
  6. Internet Explorer – Uncheck In Internet Explorer Options Advanced -> Security,  disable “Check for publisher´s certificate revocation” and “Check for Server Certificate Revocation
  7. DisableStatus registry – Again apply this fix with caution as this is found to introduce the blue login screen(Windows GINA) when accessing applications which is not ideal. http://support.citrix.com/article/CTX135782 . Some have reported to have reduce the login times by doing this.
  8. Citrix UPM Profile Streaming – Profile Streaming is sometimes found to adversely affect the logon times especially when McAfee is used. Turn OFF UPM Profile Streaming completely to see if it makes any difference.
  9. Themes Key in Active Setup – Remove the key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Active Setup\Installed Components\{2C7339CF-2B09-4501-B3F3-F3508C9228ED}. Make sure that the key is removed for the user profile as well under HKCU
  10. Receiver version – Use the latest stable version of Receiver on the client devices. Running the latest version on the server side will help in launching published application quickly.
  11. Drive Mappings – It could either be via logon scripts or via GPPs. Citrix Director can easily show you if this is the case so that you can focus on the right area from the beginning itself.
  12. Printer Mappings – Same as drive mappings. the GPPs should be set to move on if it errors and not wait for it and time out.
  13. Group Policy Processing errors – Look in the EventViewer for any potential policy processing errors and fix them.
  14. Default delay of 5-10 seconds for VDAs based on Windows 8.x and Server 2012 – Microsoft introduced a delay of 5-10 secs for operating systems starting from Windows 8 and hence this does apply to Server 2012 OSes as well. To remove the delay, add the registry value StartupDelayInMSec (REG_DWORD) to 0 in HKEY_CURRENT_USER\Software\Microsoft\Windows \CurrentVersion\Explorer\Serialize   (You can add the key “Serialize” if not present already). This will greatly reduce the “interactive logon” delays
  15. Last but not the least (this should have been higher up in the order), check the size of user profiles and find out what is causing the profile bloat. In most cases, publishing Google Chrome and Firefox is one of the most common causes of large profile sizes. It is recommended to exclude the whole of \AppData\Local\Google\Chrome and just have the per-requisite files/ folder synchronized using UPM policy. I would start with the below synchronization list for Chrome
AppData\Local\Google\Chrome\User Data\First Run
AppData\Local\Google\Chrome\User Data\Local State
AppData\Local\Google\Chrome\User Data\Default\Bookmarks
AppData\Local\Google\Chrome\User Data\Default\Favicons
AppData\Local\Google\Chrome\User Data\Default\History
AppData\Local\Google\Chrome\User Data\Default\Preferences

You can find more about Chrome and Firefox exclusion and synchronization policies here

16. Enable the Microsoft policy “Set maximum wait time for the network if a user has a roaming user profile or remote home directory” and set the value to 0. The policy could be found under Computer Configuration – Policies – Administrative Templates – System – User Profiles

17. Check the Citrix KB here – http://support.citrix.com/article/CTX133595/

In my case it turned out to be the Active setup key for Themes and the CD-ROM – made a difference of ~ 12 seconds

There is another fantastic article out there on XenAppBlog

That’s an interesting read as well. i will continue to update the post as I find out more and please feel free to post your comments below.

Issue with Microsoft Outlook 2013 reconfiguring on every launch – XenDesktop / XenApp


In one of the recent XenDesktop 7 deployments, I was told that Outlook 2013 doesn’t launch well on Server 2012 Hosted Shared Catalogs by one of the end users.  Every time Outlook is started, it tries to reconfigure itself as if the Outlook profile isn’t there or it cant find the profile. This caused the Outlook plugins or add ins to fail giving errors. My customer setup was below

  • Outlook 2013 Professional Plus
  • XenDesktop 7.0
  • Server 2012 Hosted Shared Catalogs with Desktops and Apps

Resolution – I came across a Microsoft blog which was talking about how important “Windows Search” Service is for the full functionality of Outlook. I decided to give that a try and started the service and viola, the issue is gone and Outlook launches way faster after that.

 

This might be the same case for older versions of Outlook  and other XenApp versions. i haven’t tested them yet so cant comment.

 

Open File Security Warning – Enable or Disable


Have you seen the below message appearing on your Citrix servers and just wished you could turned that OFF. I did, not once but quite a few times that I have decided to document it so that I can quickly revisit the page when I deploy Citrix next time.

The annoying error looks like the below and it is plain ugly to say the least.

Capture27

 

I tried a few things that Internet had to offer but nothing worked for me until I tried a combination of the below 2 GPOs

Capture28

Capture29

 

The GPOs could be found here

User Configuration - Administrative Templates - Windows Components - Internet Explorer - Internet Control Panel - Security Page
  • Intranet Sites: Include all local (intranet) sites not listed in other zones
  • Intranet Sites: Include all network paths (UNCs)

 

The popup should now go away!