Clicky

Have an issue where our citrix server will hang and become unresponsive. Attempting to remote into the server hangs at the Welcome screen when it is in this state. The event viewer starts displaying a bunch of services that are unresponsive but I do not think they are related to the problem only caused by it.
One event error that seems to show near the start of the problem is:
The server {AAC1009F-AB33-48F9-9A21-7F5B88426A2E} did not register with DCOM within the required timeout.

I have also had times where the only error was from me forcing the shutdown. No other errors were logged.

After restarting the server, everything will function as it should. This problem occurs as often as twice a day and as little as once every few days. The same box is hosting another Citrix server running the same versions of Windows and Citrix without the issue. Another server is also on this VM host running BES 5 without any issues. Because of this, I dont think it is an issue with the VMware setup or compatibility.
I am not sure what to try to get this resolved. Its so random. Any suggestions would be greatly appreciated.

Is there any logging I can enable to help with the issue?

asked 06/20/2011 04:28

EJC9999's gravatar image

EJC9999 ♦♦


13 Answers:
Let's have some more information?

How much memory is in the host ESX server?

How much memory is in the Guest XenApp server?

You could check teh vmware*.log file in the VM folder, for any issues.

How many users accessing the service?

Do you have any backups, snapshots, DRS, vMotion running on a ESXi cluster?
link

answered

hanccocka's gravatar image

hanccocka

36GB of memory on host
That guest has 20GB
The regular VM log shows that the server as running as normal until I power it down.
The server only has 12 users max at a time. Most of the time there are only 5.
Not running any backups or anything besides base OS and apps. No Vmotion.

Apps are:
Meridian Prolog 2008
MS Office 2003 spk3
Adobe Acrobat Pro X
Oracle Primavera 6.7
Internet Explorer 8
link

answered 2011-06-20 at 12:43:21

EJC9999's gravatar image

EJC9999

All of these apps are up and running in a citrix environment on Server 2003 not Virtualized.

Thanks so much for your help.
link

answered 2011-06-20 at 13:22:58

EJC9999's gravatar image

EJC9999

...Sytmantec Endpoint Protection is the only thing running consistantly.
link

answered 2011-06-20 at 13:24:16

EJC9999's gravatar image

EJC9999

Can I just query one statement

"All of these apps are up and running in a citrix environment on Server 2003 not Virtualized."

You have a Virtual Server and Physical Server running side by side with same apps, is that what this means?

When the server hangs?

Can you ping it, RDP it it? Access it at the console?

What do you do to re-gain control , power off?
link

answered 2011-06-20 at 13:26:00

hanccocka's gravatar image

hanccocka

Yes, I have these apps running on another server too, that is not virutalized and the OS is server 2003 instead of 2008. The problem does not exist there.
I can RDP to WELCOME, it hangs there forever. The server will not answer to powershell commands either once it "hangs".
Yes, you can ping it.
I need to power off to regain contorl.

I have been on the server (remote desktop) when it hung up. The mouse remains active, I can view the start menu items, right click on desktop menu will show but nothing will execute. Task manager will not open, Explorer will not open, nothing will run.

Thank You,
link

answered 2011-06-20 at 13:29:51

EJC9999's gravatar image

EJC9999

Okay, I suspected as much, fyi, we don't virtualize terminal services or Citrix servers because performance versus Physical Servers is very poor.

Background, 70 concurrent users, DL360G5, 2003, 4G RAM, with 200+ published applications including Office 2003 Suite, versus 15-20 users on a Virtual Server of same spec 2 vCPU, 4GB ram. The number of concurrent users just do not stack up.

Anyway back to your issue, what's the performance like CPU, Disk, Memory, Networking of the VM, and does this occur when you get to 15 concurrent users? or is it random event?
link

answered 2011-06-20 at 14:39:14

hanccocka's gravatar image

hanccocka

I have not turned it over to more than 12 users and of those 12, there are only about 5 on at a time, average.

The environment for this server is small, 40 users max. I thought this was a good solution to split conflicting applications up without having to buy another server but thats another story.

There are dual CPUs E5520 4 core 2.27ghz. Networking is straightforward. I do not suspect this to be a resource issue but maybe you have a better understanding. The OS and apps are all on C drive 200gb. raid 5.

I have followed numerous citrix / vmware best practices without any luck.


...tomorrow I am going to rebuild the same environment on an older server with half the resources, same apps, citrix version and OS but it will not be virtualized. This should confirm is all apps can behave together on server 2008 and if VMware was causing the issue. I will post results but if you have any suggestions for my current configuration I would appreciate them.

link

answered 2011-06-20 at 14:48:30

EJC9999's gravatar image

EJC9999

:::CORRECTION:::

Sorry, the C is hosting the OS w 50gb and an E drive is hosting everything else w 120GB....
link

answered 2011-06-20 at 15:15:55

EJC9999's gravatar image

EJC9999

I certainly would be interested to see if your results confirm our findings.
link

answered 2011-06-20 at 15:17:36

hanccocka's gravatar image

hanccocka

I would suggest that you monitor the hypervisor, VM and the applications (Citrix XenApp). Look for a solution that can monitor all of these, correlate between metrics and give you reports to see what happened when the problem occurs again. There are many products in the market that can give you this. For instance, look at the eG Enterprise solution:

http://www.eginnovations.com/web/egcitrix.htm

http://www.eginnovations.com/web/vmware.htm

link

answered 2011-06-20 at 15:19:34

solvedata's gravatar image

solvedata

I noticed this isnt solved.

I  dont think the VM is the issue. I have the exact same error. AAC1009F-AB33-48F9-9A21-7F5B88426A2E DCOM register, then followed by a bunch of 7011 timeout errors. Happens randomly about once a week.

Several other users have a similar issue. I seen this http://forums.citrix.com/thread.jspa?threadID=264425&start=15&tstart=0

Somewhere I saw the issue might be registry permission on the HKCR/CLSID key.

We have 2 servers in a Citrix farm and both have had the same hanging issue.

However their CLSID permissions are quite different. (08 server x86 sp2)

Trying to find the correct permission to give the CLSID key but not finding it. I would think system FC and users read only would suffice.  One of my servers has allow RESTRICTED read only access.
The other one doesn't.

OP. May be worth looking at.

RR
link

answered 2011-11-13 at 08:17:42

rrosa3's gravatar image

rrosa3

I am having this exact same issue. Does anyone have a fix? I cannot get more than 8 users on a 2008 32bit Citrix server without the server starting to produce strange errors and eventually stops responding. Only a power recycle fill fix. Would appreciate any guidance. Desperate.
link

answered 2011-11-22 at 15:43:42

iteknique's gravatar image

iteknique

Your answer
[hide preview]

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Tags:

×30
×1
×4

Asked: 06/20/2011 04:28

Seen: 917 times

Last updated: 12/16/2011 02:47