Windows Player Out of Memory Errors about xibo HOT 35 CLOSED

Thank you for your first bug report! I've edited your original post so that the code you included is shown correctly (just a bit of "markdown" syntax).

I suspect you are struggling with a long running bug whereby the Internet Explorer process leaks memory when viewing certain pages - causing the XiboClient.exe to balloon and eventually crash due to memory pressure.

We included the Chrome browser to try and solve that problem (although we may have introduced other issues!).

What would be interesting is:

• The Crash Report
• The Memory Usage at various points in the day (to assess if it is increasing)

Thanks,
Dan

from xibo.

I ran into the same issue with Internet Explorer absolutely ballooning. How do I switch to Chrome?

from xibo.

Create a display profile in the CMS for windows, edit it and select Use CEF Browser - either mark it as default or edit your display and assign it in the advanced tab.

Disclaimer - The Chrome Engine is experimental at the moment, some things it does better, others it does worse.

from xibo.

For reference, I think this is your prior question @AllieKat - https://answers.launchpad.net/xibo/+question/261224

If so, then I am not sure we are looking at the same issue - if you have only Xibo modules - no embedded or webpage media, then you shouldn't experience a memory leak.

I'll wait for you to confirm if going to CEF solved it, but in any case an export of your layout would be appreciated so I can run it here and try to reproduce the leak (also interested if it still leaks in 1.7.0)

from xibo.

Yes, I was only using built-in Xibo modules. I have an exported layout but Github doesn't allow me to attach a ZIP.

from xibo.

Oh, that's a shame.... :(

You could wetransfer it to [email protected]

from xibo.

Here it is I also posted it in the other bug: https://www.dropbox.com/s/tkqpt2wtejts7t3/export_default-layout.zip?dl=0

from xibo.

Google Chrome looks TERRIBLE on refresh as it creates background-coloured blocks and slowly loads region by region instead of IE where everything is transparent until it loads and loads much faster. So I'm going to hope it just doesn't crash again... I'll report back if/when it does!

from xibo.

I'll run with it for a while and see what happens to my memory usage then.

from xibo.

I'll try to capture a crash report. For the memory snapshots, do you have a recommendation for a script or repeating task I could run to take a snapshot? My schedule is too random to be able to give a consistent periodic view. I can have somebody figure out something, but it would be better if I could do it the same way everyone else does to make it easier to consume the data.

from xibo.

On windows it is easiest to use performance monitor. There is an article for collecting CPU statistics here - we want RAM, but the same applies - just select the memory metric instead.

from xibo.

I emailed to [email protected] a zip of the windows crash information. I'll work on the memory. Anything else that would help?

from xibo.

I have attached a photo of mine (the screenshot was just black in Xibo...). Is this what you are seeing?

from xibo.

I don't get the same message. I get the more generic, "Xibo client
stopped "
On Jan 27, 2015 10:11 AM, "AllieKat" [email protected] wrote:

[image: image]
https://cloud.githubusercontent.com/assets/10714310/5920721/eb11ec66-a5fb-11e4-9a24-66efae418f20.jpg

I have attached a photo of mine (the screenshot was just black in
Xibo...). Is this what you are seeing?

—
Reply to this email directly or view it on GitHub
#36 (comment).

from xibo.

Just a FYI - I've been running with the example layout for 3 hours before I had to switch to something else. In that time memory usage started out at about 60MB and then alternated between 60 and 100 and back again.

The crash information sent points to KERNALBASE as the faulting module

from xibo.

As I think it appears we have different issues, I'll go ahead and post another bug report with mine... his error message is completely different from my out of memory one.

P.S. actually, no, I won't - I just realised one thing that changed from when I did all my testing and Xibo seemed stable until now. I installed LogMeIn for remote access to the box. I wonder...

from xibo.

Is there anything suggested for addressing the KERNALBASE piece? I assume
by the name that this is in the OS, but am just guessing. I haven't
updated to the official 1.7.0 release, as I don't think there were any
specific fixes for this. If it would help, I can.

I'm leaving the log running until it crashes again on that TV so the trend
can show where it broke. My laptop test bed crashes more often, but it is
an old machine without a lot of memory, so I don't want to add chaffe to
the mix. If you think that would be informative, I can start running that
too.

On Wed, Jan 28, 2015 at 4:16 AM, Dan Garner [email protected]
wrote:

Just a FYI - I've been running with the example layout for 3 hours before
I had to switch to something else. In that time memory usage started out at
about 60MB and then alternated between 60 and 100 and back again.

The crash information sent points to KERNALBASE as the faulting module

—
Reply to this email directly or view it on GitHub
#36 (comment).

from xibo.

Actually, it would be worthwhile to take a look in the Windows Application Event Log (in Event Viewer) and search for exceptions from XiboClient.exe.

It might be that the exception stack trace will give us more information than the minidump - FYI the minidump was difficult to use because the error originated from the framework and not within Xibo.

We have had KERNALBASE errors before and have always suggested checking the latest .NET framework is installed and that all your drivers are up to the latest stable. It is particularly difficult to debug or locate.

Are you still getting the memory usage expanding?

from xibo.

Here's the memlog and an eventlog for the most recent crashes.

On Wed, Jan 28, 2015 at 9:55 AM, Dan Garner [email protected]
wrote:

Actually, it would be worthwhile to take a look in the Windows Application
Event Log (in Event Viewer) and search for exceptions from XiboClient.exe.

It might be that the exception stack trace will give us more information
than the minidump - FYI the minidump was difficult to use because the error
originated from the framework and not within Xibo.

We have had KERNALBASE errors before and have always suggested checking
the latest .NET framework is installed and that all your drivers are up to
the latest stable. It is particularly difficult to debug or locate.

Are you still getting the memory usage expanding?

—
Reply to this email directly or view it on GitHub
#36 (comment).

from xibo.

Sorry, GitHub doesnt support file attachments - if you can put these in a dropbox or something and link them that would be great.

from xibo.

https://www.dropbox.com/sh/zja5geafuqkwzor/AAC6tBlO-d8yXadGBYObfnO7a?dl=0

Hopefully this works.

On Fri, Jan 30, 2015 at 9:06 AM, Dan Garner [email protected]
wrote:

Sorry, GitHub doesnt support file attachments - if you can put these in a
dropbox or something and link them that would be great.

—
Reply to this email directly or view it on GitHub
#36 (comment).

from xibo.

This is interesting:

Application: XiboClient.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.ComponentModel.Win32Exception
Stack:
   at System.Windows.Forms.NativeWindow.CreateHandle(System.Windows.Forms.CreateParams)
   at System.Windows.Forms.Control.CreateHandle()
   at System.Windows.Forms.Control.get_Handle()
   at System.Windows.Forms.Control.MarshaledInvoke(System.Windows.Forms.Control, System.Delegate, System.Object[], Boolean)
   at System.Windows.Forms.Control.Invoke(System.Delegate, System.Object[])
   at Xilium.CefGlue.WindowsForms.CefWebBrowser.InvokeIfRequired(System.Action)
   at Xilium.CefGlue.WindowsForms.CefWebLoadHandler.OnLoadingStateChange(Xilium.CefGlue.CefBrowser, Boolean, Boolean, Boolean)
   at Xilium.CefGlue.CefLoadHandler.on_loading_state_change(Xilium.CefGlue.Interop.cef_load_handler_t*, Xilium.CefGlue.Interop.cef_browser_t*, Int32, Int32, Int32)

from xibo.

Anything you need from me to help further isolate it? When I do go to
restart the client, I often see what appears to be multiple instances
running - presumably one crashes, and then another starts up. Or, it's
just Windows weirdness.

On Sat, Jan 31, 2015 at 7:06 AM, Dan Garner [email protected]
wrote:

This is interesting:

Application: XiboClient.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.ComponentModel.Win32Exception
Stack:
at System.Windows.Forms.NativeWindow.CreateHandle(System.Windows.Forms.CreateParams)
at System.Windows.Forms.Control.CreateHandle()
at System.Windows.Forms.Control.get_Handle()
at System.Windows.Forms.Control.MarshaledInvoke(System.Windows.Forms.Control, System.Delegate, System.Object[], Boolean)
at System.Windows.Forms.Control.Invoke(System.Delegate, System.Object[])
at Xilium.CefGlue.WindowsForms.CefWebBrowser.InvokeIfRequired(System.Action)
at Xilium.CefGlue.WindowsForms.CefWebLoadHandler.OnLoadingStateChange(Xilium.CefGlue.CefBrowser, Boolean, Boolean, Boolean)
at Xilium.CefGlue.CefLoadHandler.on_loading_state_change(Xilium.CefGlue.Interop.cef_load_handler_t_, Xilium.CefGlue.Interop.cef_browser_t_, Int32, Int32, Int32)

—
Reply to this email directly or view it on GitHub
#36 (comment).

from xibo.

Sorry for the delay - i've been picking the low hanging fruit just to try and keep ahead.

There is actually something you could do for me - I have just linked a commit which completely disables CEF from initializing when it has been disabled in the settings on the CMS.

Can you try the XiboClient.exe from that commit?

It will need a reboot of the player software after the changed settings have been transferred (if the player previously had CEF set to ON).

from xibo.

If I understand, you are wanting to turn off the CEF (Chrome browser?) and
keep it from initializing, which I assume means it will default to IE?
Which version of IE should I run? In the past, it has been pretty unstable
and the reason I had high hopes for Chrome.

On Tue, Feb 10, 2015 at 8:44 AM, Dan Garner [email protected]
wrote:

Sorry for the delay - i've been picking the low hanging fruit just to try
and keep ahead.

There is actually something you could do for me - I have just linked a
commit which completely disables CEF from initializing when it has been
disabled in the settings on the CMS.

Can you try the XiboClient.exe from that commit?

It will need a reboot of the player software after the changed settings
have been transferred (if the player previously had CEF set to ON).

—
Reply to this email directly or view it on GitHub
#36 (comment).

from xibo.

Another thing I had considered was running a script every few hours to
shutdown and restart Xibo to clean up memory. Ideally, you wouldn't need
that. However, is there a cleaner way to do that - like somehow holding an
image or black screen while shutting down the browser and then restarting
it? Call it the "Auto-Cleanup" feature. :-)

On Tue, Feb 10, 2015 at 9:09 AM, Ron Garnett [email protected] wrote:

If I understand, you are wanting to turn off the CEF (Chrome browser?) and
keep it from initializing, which I assume means it will default to IE?
Which version of IE should I run? In the past, it has been pretty unstable
and the reason I had high hopes for Chrome.

On Tue, Feb 10, 2015 at 8:44 AM, Dan Garner [email protected]
wrote:

Sorry for the delay - i've been picking the low hanging fruit just to try
and keep ahead.

There is actually something you could do for me - I have just linked a
commit which completely disables CEF from initializing when it has been
disabled in the settings on the CMS.

Can you try the XiboClient.exe from that commit?

It will need a reboot of the player software after the changed settings
have been transferred (if the player previously had CEF set to ON).

—
Reply to this email directly or view it on GitHub
#36 (comment).

from xibo.

Another thing I noted today. Previously, I saw multiple Xibo windows on
the Win7 taskbar and killed them. I didn't bother checking the task
manager, but did today and saw many copies of xiboclient running, all
chewing up memory. That might explain why it gets less stable as time goes
on. I created a batch file to kill all of those and restart, and I will
probably schedule it to run several times a day to see if that helps.

On Tue, Feb 10, 2015 at 9:12 AM, Ron Garnett [email protected] wrote:

Another thing I had considered was running a script every few hours to
shutdown and restart Xibo to clean up memory. Ideally, you wouldn't need
that. However, is there a cleaner way to do that - like somehow holding an
image or black screen while shutting down the browser and then restarting
it? Call it the "Auto-Cleanup" feature. :-)

On Tue, Feb 10, 2015 at 9:09 AM, Ron Garnett [email protected]
wrote:

If I understand, you are wanting to turn off the CEF (Chrome browser?)
and keep it from initializing, which I assume means it will default to IE?
Which version of IE should I run? In the past, it has been pretty unstable
and the reason I had high hopes for Chrome.

On Tue, Feb 10, 2015 at 8:44 AM, Dan Garner [email protected]
wrote:

Sorry for the delay - i've been picking the low hanging fruit just to
try and keep ahead.

There is actually something you could do for me - I have just linked a
commit which completely disables CEF from initializing when it has been
disabled in the settings on the CMS.

Can you try the XiboClient.exe from that commit?

It will need a reboot of the player software after the changed settings
have been transferred (if the player previously had CEF set to ON).

—
Reply to this email directly or view it on GitHub
#36 (comment).

from xibo.

I appreciate all of those ideas - and I am sure they will help... however I still don't know what is causing the multiple processes... the idea of disabling CEF completely is to check and be 100% sure that CEF is responsible for creating those windows - and then find out why and stop it doing so.

If you could help isolate that fact to start with, that would be great!

Chrome has proved terrible so far - which might be down to the implementation as it was a real pain to get working.

from xibo.

I have your request running on my test laptop. It was crashing too, so
hopefully it will help.

On Mon, Feb 16, 2015 at 2:13 PM, Dan Garner [email protected]
wrote:

I appreciate all of those ideas - and I am sure they will help... however
I still don't know what is causing the multiple processes... the idea of
disabling CEF completely is to check and be 100% sure that CEF is
responsible for creating those windows - and then find out why and stop it
doing so.

If you could help isolate that fact to start with, that would be great!

Chrome has proved terrible so far - which might be down to the
implementation as it was a real pain to get working.

—
Reply to this email directly or view it on GitHub
#36 (comment).

from xibo.

First crash with the client per your directions. I didn't see a Chrome
reference in it. Just a .net and xibo.

Log Name: Application
Source: .NET Runtime
Date: 2/18/2015 4:43:04 AM
Event ID: 1026
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Description:
Application: XiboClient.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.OutOfMemoryException
Stack:
at
System.Threading.Thread.StartInternal(System.Security.Principal.IPrincipal,
System.Threading.StackCrawlMark ByRef)
at System.Threading.Thread.Start(System.Threading.StackCrawlMark ByRef)
at System.Threading.Thread.Start()
at XiboClient.XiboTraceListener.Flush()
at XiboClient.XiboTraceListener.AddToCollection(System.String,
System.String)
at XiboClient.XiboTraceListener.Write(System.String, System.String)
at XiboClient.XiboTraceListener.WriteLine(System.Object, System.String)
at System.Diagnostics.TraceInternal.WriteLine(System.Object,
System.String)
at System.Diagnostics.Trace.WriteLine(System.Object, System.String)
at XiboClient.XmdsAgents.RequiredFilesAgent.Run()
at System.Threading.ThreadHelper.ThreadStart_Context(System.Object)
at
System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext,
System.Threading.ContextCallback, System.Object, Boolean)
at
System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext,
System.Threading.ContextCallback, System.Object, Boolean)
at
System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext,
System.Threading.ContextCallback, System.Object)
at System.Threading.ThreadHelper.ThreadStart()

Event Xml:



1026
2
0
0x80000000000000

10503
Application



Application: XiboClient.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.OutOfMemoryException
Stack:
at
System.Threading.Thread.StartInternal(System.Security.Principal.IPrincipal,
System.Threading.StackCrawlMark ByRef)
at System.Threading.Thread.Start(System.Threading.StackCrawlMark ByRef)
at System.Threading.Thread.Start()
at XiboClient.XiboTraceListener.Flush()
at XiboClient.XiboTraceListener.AddToCollection(System.String,
System.String)
at XiboClient.XiboTraceListener.Write(System.String, System.String)
at XiboClient.XiboTraceListener.WriteLine(System.Object, System.String)
at System.Diagnostics.TraceInternal.WriteLine(System.Object,
System.String)
at System.Diagnostics.Trace.WriteLine(System.Object, System.String)
at XiboClient.XmdsAgents.RequiredFilesAgent.Run()
at System.Threading.ThreadHelper.ThreadStart_Context(System.Object)
at
System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext,
System.Threading.ContextCallback, System.Object, Boolean)
at
System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext,
System.Threading.ContextCallback, System.Object, Boolean)
at
System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext,
System.Threading.ContextCallback, System.Object)
at System.Threading.ThreadHelper.ThreadStart()

Log Name: Application
Source: Application Error
Date: 2/18/2015 4:43:07 AM
Event ID: 1000
Task Category: (100)
Level: Error
Keywords: Classic
User: N/A
Description:
Faulting application name: XiboClient.exe, version: 2.0.0.0, time stamp:
0x54da0a68
Faulting module name: KERNELBASE.dll, version: 6.1.7601.17651, time stamp:
0x4e211319
Exception code: 0xe0434352
Fault offset: 0x0000b9bc
Faulting process id: 0x1f8c
Faulting application start time: 0x01d04af2e5dffcd6
Faulting application path: C:\Program Files (x86)\Xibo Player\XiboClient.exe
Faulting module path: C:\windows\syswow64\KERNELBASE.dll
Report Id: 8a26fa5a-b752-11e4-ba8d-001e3700e6c1
Event Xml:



1000
2
100
0x80000000000000

10504
Application



XiboClient.exe
2.0.0.0
54da0a68
KERNELBASE.dll
6.1.7601.17651
4e211319
e0434352
0000b9bc
1f8c
01d04af2e5dffcd6
C:\Program Files (x86)\Xibo Player\XiboClient.exe
C:\windows\syswow64\KERNELBASE.dll
8a26fa5a-b752-11e4-ba8d-001e3700e6c1

On Tue, Feb 17, 2015 at 2:55 PM, Ron Garnett [email protected] wrote:

I have your request running on my test laptop. It was crashing too, so
hopefully it will help.

On Mon, Feb 16, 2015 at 2:13 PM, Dan Garner [email protected]
wrote:

I appreciate all of those ideas - and I am sure they will help... however
I still don't know what is causing the multiple processes... the idea of
disabling CEF completely is to check and be 100% sure that CEF is
responsible for creating those windows - and then find out why and stop it
doing so.

If you could help isolate that fact to start with, that would be great!

Chrome has proved terrible so far - which might be down to the
implementation as it was a real pain to get working.

—
Reply to this email directly or view it on GitHub
#36 (comment).

from xibo.

So that looks like our "old favourite" out of memory error, which we have been battling for a long time - basically the Internet Explorer control is leaking memory.

So we know two things

• With CEF completely disabled, the libCEF error disappears (this is great)
• The Web Browser memory leak is still there (not great, but expected)

Aside from ensuring your .NET framework is up to date we don't currently have any solutions to this.

from xibo.

My kill all and restart script that runs daily at 6 AM and on error seems
to have really helped the up time. I don't think either Internet Exporer or
Chrome was designed to be a hardened, run-all-the-time program. If they
were intended as such, their creators need to rethink it a bit and add in
better garbage collection, leak monitoring, etc.

I'm still running Chrome on the production TVs.

On Thu, Feb 19, 2015 at 5:18 AM, Dan Garner [email protected]
wrote:

So that looks like our "old favourite" out of memory error, which we have
been battling for a long time - basically the Internet Explorer control is
leaking memory.

So we know two things

• With CEF completely disabled, the libCEF error disappears (this is
  great)
• The Web Browser memory leak is still there (not great, but expected)

Aside from ensuring your .NET framework is up to date we don't currently
have any solutions to this.

—
Reply to this email directly or view it on GitHub
#36 (comment).

from xibo.

Well, yes I think you are correct... but I think it is actually the way they are marshalled into Windows Forms applications that is causing a problem. Xibo doesn't run the same instance all the time, it gets recycled with each media item.

What is odd is that not everyone experiences the issue, which makes it even harder to work out why.

from xibo.

I just wanted to let everyone know where I have been for the last several weeks. This error just... stopped occurring. Nothing changed other than some pictures being added to the default layout. I've been hesitant to update, but I will to 1.7.2 and resume testing of Xibo...

from xibo.

That is curious!

I think it would be good to "reset" these topics from 1.7.2 onwards and see if people still have issues. We did spend what felt like days with memory profilers and such to produce 1.7.2, do hopefully it has some improvements.

Perhaps if you still have a problem you could repost on our new community forum.

from xibo.