A customer was attempting to configure ICA Proxy mode on Citrix Access Gateway Enterprise Edition with XenApp 5.0 and Web Interface. The customer reported that when configuring the same, the 401 – Unauthorized Access is denied due to invalid credentials error message is displayed on the Web browser after a successful authentication to the Citrix Access Gateway Enterprise Edition Login page, as shown in the following screenshot:
The customer had installed the following hardware and software components on the network:
- Windows Server 2008
- Internet Information Server 7
- NetScaler appliance
- Web Interface 5.0
- XenApp 5.0
To troubleshoot this issue, the Technical Support Engineers investigated the Windows event logs of the XenApp Server and observed an error message in the Citrix Web Interface event log, as shown in the following screenshot:
This prompted the engineers to shift the focus of the investigation towards the XenApp Server. The engineers recorded network packet traces on the XenApp server during a login attempt. Each time, the engineers killed the Access Gateway Enterprise Edition session to ensure that a new session starts. The Web Interface makes the outbound https request to the Access Gateway Enterprise appliance to retrieve the SmartAccess settings, such as VServer and Session Policy Name.
When analyzing the packet traces, the engineers observed that when the XenApp Server communicates to the URL in the preceding screenshot, /CitrixAuthService/AuthService.asmx, the XenApp Server sends a FIN-ACK packet during the Secure Socket Layer (SSL) handshake negotiation, as shown in the following screenshot:
When attempting to open the /Citrix/XenApp1/auth/agesso.aspx URL, the Web Interface sends the 401 response code because the XenApp server could not complete the SSL handshake.
After further investigating the event logs, the engineers noticed that there was an issue with the SSL certificates.