Crashes on OSX with Nvidia graphics

[omarev-git]

Hello,
Im was working fine with OS X 10.9 (Mavericks) but by requirements of our office I made the upgrade to Yosemite (10.10.3) everything compile good but when I tray to run Protonect I have the following error:
[Freenect2Impl] enumerating devices...
[Freenect2Impl] 12 usb devices connected
[Freenect2Impl] found valid Kinect v2 @21:15 with serial 500086241942
[Freenect2Impl] found 1 devices
[Freenect2DeviceImpl] opening...
[Freenect2DeviceImpl] opened
[Freenect2DeviceImpl] starting...
[Freenect2DeviceImpl] ReadData0x14 response
92 bytes of raw data
0x0000: 00 00 12 00 00 00 00 00 01 00 00 00 43 c1 1f 41 2e2e2e2e2e2e2e2e2e2e2e2e432e2e41
0x0010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e
0x0020: 0a 21 33 55 c2 00 17 20 00 08 00 00 10 00 00 00 2e2133552e2e2e202e2e2e2e2e2e2e2e
0x0030: 00 01 00 00 00 10 00 00 00 00 80 00 01 00 00 00 2e2e2e2e2e2e2e2e2e2e802e2e2e2e2e
0x0040: 31 33 00 00 00 01 07 04 47 4d 58 36 31 30 2e 31 31332e2e2e2e2e2e474d583631302e31
0x0050: 58 00 00 00 00 00 00 00 00 00 00 00 582e2e2e2e2e2e2e2e2e2e2e

[Freenect2DeviceImpl] ReadStatus0x090000 response
4 bytes of raw data
0x0000: 5d 26 00 00 5d262e2e

[Freenect2DeviceImpl] ReadStatus0x090000 response
4 bytes of raw data
0x0000: 5f 26 00 00 5f262e2e

[Freenect2DeviceImpl] enabling usb transfer submission...
[Freenect2DeviceImpl] submitting usb transfers...
[Freenect2DeviceImpl] started
device serial: 500086241942
device firmware: 4.3.3912.0.7
[DepthPacketStreamParser::handleNewData] not all subsequences received 512
[TurboJpegRgbPacketProcessor] avg. time: 19.9244ms -> ~50.1898Hz
footer->length: 671449296
wb.length: 215233
[DepthPacketStreamParser::handleNewData] image data too short!
footer->length: 298496
wb.length: 83111
[DepthPacketStreamParser::handleNewData] image data too short!
[DepthPacketStreamParser::handleNewData] not all subsequences received 1015
[TurboJpegRgbPacketProcessor] avg. time: 19.8961ms -> ~50.261Hz
Abort trap: 6

As you can see footer->length > w.length in DepthPacketStreamParser, also all the computer freeze by some seconds after the line "device firmware: 4.3.3912.0.7" is printed.

Thanks in advance by your time.

[stewdio]

I’m experiencing the same issue 😢

[heisters]

same here. I'm trying to debug the Protonect code, but am totally unfamiliar with it. Any suggestions of where to start looking?

[heisters]

not sure I'm doing this right, but compiled Protonect with debug symbols using cmake -DCMAKE_BUILD_TYPE=Debug, ran Protonect under gdb, and saw this:

[Freenect2DeviceImpl] enabling usb transfer submission...
[Freenect2DeviceImpl] submitting usb transfers...
[Freenect2DeviceImpl] started
device serial: 024885140647
device firmware: 1.1.3323.0.7
[DepthPacketStreamParser::handleNewData] not all subsequences received 512
[TurboJpegRgbPacketProcessor::doProcess] Failed to decompress rgb image! TurboJPEG error: 'Invalid JPEG file structure: two SOI markers'

here's the backtrace:

(gdb) bt
#0  0x00007fff90d53286 in __pthread_kill ()
#1  0x00007fff9a35742f in pthread_kill ()
#2  0x00007fff8db42b53 in abort ()
#3  0x00007fff8fa62b81 in gpusGenerateCrashLog ()
#4  0x000012344022728a in gldCreateDevice ()
#5  0x00007fff8fa64068 in gpusSubmitDataBuffers ()
#6  0x000012344031a49b in gldUpdateDispatch ()
#7  0x000012344031a301 in gldUpdateDispatch ()
#8  0x000012344032fbe4 in gldBlitFramebufferData ()
#9  0x00007fff96f90bf8 in glReadPixels_Exec ()
#10 0x00007fff91c58bd5 in glReadPixels ()
#11 0x00000001000326bf in libfreenect2::Texture<libfreenect2::ImageFormat<4ul, 33326u, 6403u, 5126u> >::flipYBuffer () at /Users/ian/sw/src/libfreenect2/examples/protonect/src/opengl_depth_packet_processor.cpp:300
#12 0x00000001000326bf in libfreenect2::Texture<libfreenect2::ImageFormat<4ul, 33326u, 6403u, 5126u> >::downloadToNewFrame (this=0x106079518) at /Users/ian/sw/src/libfreenect2/examples/protonect/src/opengl_depth_packet_processor.cpp:330
#13 0x0000000100030c1c in libfreenect2::OpenGLDepthPacketProcessorImpl::run (this=0x106079200, ir=0x10e702e78, depth=0x10e702e70) at /Users/ian/sw/src/libfreenect2/examples/protonect/src/opengl_depth_packet_processor.cpp:653
#14 0x000000010002f3b8 in libfreenect2::OpenGLDepthPacketProcessor::process (this=0x1016013f0, packet=<value temporarily unavailable, due to optimizations>) at /Users/ian/sw/src/libfreenect2/examples/protonect/src/opengl_depth_packet_processor.cpp:935
#15 0x000000010001bdd1 in libfreenect2::AsyncPacketProcessor<libfreenect2::DepthPacket>::execute (this=0x1015d1940) at async_packet_processor.h:105
#16 0x000000010002bce3 in tthread::thread::wrapper_function (aArg=0x1015d1a40) at /Users/ian/sw/src/libfreenect2/examples/protonect/src/tinythread/tinythread.cpp:169
#17 0x00007fff9a355268 in _pthread_body ()
#18 0x00007fff9a3551e5 in _pthread_start ()
#19 0x00007fff9a35341d in thread_start ()

That seems to point here: https://github.com/OpenKinect/libfreenect2/blob/master/examples/protonect/src/turbo_jpeg_rgb_packet_processor.cpp#L115

[xlz]

[DepthPacketStreamParser::handleNewData] not all subsequences received 512 - depth packets lost some parts in the middle.
[TurboJpegRgbPacketProcessor::doProcess] Failed to decompress rgb image! TurboJPEG error: 'Invalid JPEG file structure: two SOI markers' - JPEG packet lost some parts in the middle.

The data loss WAs probably caused by changes in USB3 performance. Stream parsers did not check the packets very well. Corrupted packets were given to TurboJpegRgbPacketProcessor and OpenGLDepthPacketProcessor. OpenGLDepthPacketProcessor did not handle corrupted packets quite well - it crashed.

To solve the problem, there are a few things to try:

• Apply these two commits ad79c2f 1f55c9f Merge PR 221 to clean up stream parsers ; PR Rgb packet handling #73 A go at making the depth_packet_stream_parser better. #75 are also in the work to solve the problems in stream parsers.
• Investigate what changed USB3 performance.
• Use OpenCL depth processor.

[astarasikov]

I've written a hackish OpenGL-based renderer for the protonect demo on mac (because I like GLSL and OpenCV one does not work the majority of time. Thought once in a dozen runs it outputs a single frame), you can try taking the code from here and see if it works astarasikov@511f3d3

I've noticed some issues which apply to both the original protonect and the GL one:

• Sometimes it looks like the computer is frozen, but I think it's just because there are too many events. If you unplug the kinect, the computer will unfreeze eventually
• I've had bad luck with the NVIDIA GPU, but if I use the "gfxCardStatus" app to force the intel card, it works fine. Even with OpenCV version, switching to intel at least prevents freezing.

HTH

[andrewadare]

I noted that several potentially relevant commits were made since this issue (and #214) were filed. I tried pulling master and rebuilding tonight. This same runtime error persists. Since the suggestions made by @xlz are now out of date (PR #73 and #75 are both closed), I wonder what I steps I could try to fix this issue? Would very much like to get libfreenect2 working on OSX, but not very familiar with this codebase....

[xlz]

@andrewadare
PR 221 supersedes 73 and 75. You can merge commits from PR 221 and test.

[andrewadare]

I checked out PR #221 and it seems to get a little farther than before, but still ends with abort trap 6:

[protonect]$ ./bin/Protonect
[Freenect2Impl] enumerating devices...
[Freenect2Impl] 8 usb devices connected
[Freenect2Impl] found valid Kinect v2 @21:32 with serial 013274450547
[Freenect2Impl] found 1 devices
[Freenect2DeviceImpl] opening...
[Freenect2DeviceImpl] opened
[Freenect2DeviceImpl] starting...
[Freenect2DeviceImpl] ReadData0x14 response
92 bytes of raw data
0x0000:  00 00 15 00 00 00 00 00 01 00 00 00 43 c1 1f 41    2e2e2e2e2e2e2e2e2e2e2e2e432e2e41
0x0010:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e
0x0020:  0a 21 33 55 ef 00 17 40 00 08 00 00 10 00 00 00    2e2133552e2e2e402e2e2e2e2e2e2e2e
0x0030:  00 01 00 00 00 10 00 00 00 00 80 00 01 00 00 00    2e2e2e2e2e2e2e2e2e2e802e2e2e2e2e
0x0040:  31 33 00 00 00 09 13 14 47 4d 46 31 35 32 2e 31    31332e2e2e2e2e2e474d463135322e31
0x0050:  58 00 00 00 00 00 00 00 07 00 00 00                582e2e2e2e2e2e2e2e2e2e2e

[Freenect2DeviceImpl] ReadStatus0x090000 response
4 bytes of raw data
0x0000:  01 26 00 00                                        2e262e2e

[Freenect2DeviceImpl] ReadStatus0x090000 response
4 bytes of raw data
0x0000:  03 26 00 00                                        2e262e2e

[Freenect2DeviceImpl] enabling usb transfer submission...
[Freenect2DeviceImpl] submitting usb transfers...
[Freenect2DeviceImpl] started
device serial: 013274450547
device firmware: 2.3.3913.0.7
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] not all subsequences received 512
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] skipping depth packet
[DepthPacketStreamParser::onDataReceived] not all subsequences received 768
Abort trap: 6
[protonect]$

[andrewadare]

Here's the tail of running in lldb (after saying "skipping depth packet" a few hundred times):

Process 9581 stopped
* thread #9: tid = 0x3d7389, 0x00007fff9b564286 libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT
    frame #0: 0x00007fff9b564286 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill:
->  0x7fff9b564286 <+10>: jae    0x7fff9b564290            ; <+20>
    0x7fff9b564288 <+12>: movq   %rax, %rdi
    0x7fff9b56428b <+15>: jmp    0x7fff9b55fc53            ; cerror_nocancel
    0x7fff9b564290 <+20>: retq

And here's a backtrace:

Process 9581 stopped
* thread #9: tid = 0x3d7389, 0x00007fff9b564286 libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT
    frame #0: 0x00007fff9b564286 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill:
->  0x7fff9b564286 <+10>: jae    0x7fff9b564290            ; <+20>
    0x7fff9b564288 <+12>: movq   %rax, %rdi
    0x7fff9b56428b <+15>: jmp    0x7fff9b55fc53            ; cerror_nocancel
    0x7fff9b564290 <+20>: retq
(lldb) bt
* thread #9: tid = 0x3d7389, 0x00007fff9b564286 libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT
  * frame #0: 0x00007fff9b564286 libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fff8e8cc42f libsystem_pthread.dylib`pthread_kill + 90
    frame #2: 0x00007fff9a63eb53 libsystem_c.dylib`abort + 129
    frame #3: 0x00007fff8ceeab81 libGPUSupportMercury.dylib`gpusGenerateCrashLog + 173
    frame #4: 0x000012344022728a GeForceGLDriver`___lldb_unnamed_function6431$$GeForceGLDriver + 9
    frame #5: 0x00007fff8ceec068 libGPUSupportMercury.dylib`gpusSubmitDataBuffers + 500
    frame #6: 0x000012344031a49b GeForceGLDriver`___lldb_unnamed_function11146$$GeForceGLDriver + 349
    frame #7: 0x000012344031a301 GeForceGLDriver`___lldb_unnamed_function11145$$GeForceGLDriver + 232
    frame #8: 0x000012344032fbe4 GeForceGLDriver`___lldb_unnamed_function11281$$GeForceGLDriver + 4774
    frame #9: 0x000000010ca91bf8 GLEngine`glReadPixels_Exec + 1390
    frame #10: 0x00007fff9b53cbd5 libGL.dylib`glReadPixels + 57
    frame #11: 0x0000000100032570 libfreenect2.dylib`libfreenect2::Texture<libfreenect2::ImageFormat<4ul, 33326u, 6403u, 5126u> >::downloadToNewFrame() [inlined] libfreenect2::Texture<libfreenect2::ImageFormat<4ul, 33326u, 6403u, 5126u> >::downloadToBuffer(data=<unavailable>) + 112 at opengl_depth_packet_processor.cpp:300
    frame #12: 0x0000000100032551 libfreenect2.dylib`libfreenect2::Texture<libfreenect2::ImageFormat<4ul, 33326u, 6403u, 5126u> >::downloadToNewFrame(this=0x000000010683d318) + 81 at opengl_depth_packet_processor.cpp:329
    frame #13: 0x0000000100030acc libfreenect2.dylib`libfreenect2::OpenGLDepthPacketProcessorImpl::run(this=0x000000010683d000, ir=0x0000000111554e78, depth=0x0000000111554e70) + 972 at opengl_depth_packet_processor.cpp:653
    frame #14: 0x000000010002f24c libfreenect2.dylib`libfreenect2::OpenGLDepthPacketProcessor::process(this=0x0000000102900410, packet=0x0000000102d3fc38) + 236 at opengl_depth_packet_processor.cpp:935
    frame #15: 0x000000010001b711 libfreenect2.dylib`libfreenect2::AsyncPacketProcessor<libfreenect2::DepthPacket>::execute(this=0x0000000102d3fc20) + 81 at async_packet_processor.h:105
    frame #16: 0x000000010002bb73 libfreenect2.dylib`tthread::thread::wrapper_function(aArg=0x0000000102d00550) + 19 at tinythread.cpp:169
    frame #17: 0x00007fff8e8ca268 libsystem_pthread.dylib`_pthread_body + 131
    frame #18: 0x00007fff8e8ca1e5 libsystem_pthread.dylib`_pthread_start + 176
    frame #19: 0x00007fff8e8c841d libsystem_pthread.dylib`thread_start + 13

[xlz]

The log shows it's not an issue with USB. If you really want to debug this, there are two issues to debug:

• OpenGL processor being slow - insert timing log points and find out where is the bottleneck
• The crash - did new Frame(width, height, bytes_per_pixel); returns NULL? Otherwise it's some driver issues.

It seems yours is Nvidia GPU. Perhaps you can try out the CUDA processor in PR 222. Merge PR 221 and 222, and redo cmake. (Install CUDA SDK first)