593608 - drawArrays() crash [@vm3dgl]

Status: RESOLVED FIXED

(Core :: Graphics: CanvasWebGL, defect)

Description

[Christoph Diehl [:posidron]]

eax=00000000

vm3dgl+0x600b:
1200600b 80781c00        cmp     byte ptr [eax+1Ch],0       ds:002b:0000001c=??


gl.drawArrays(6, 32766, 255);


This was tested inside VMWare Fusion.

Comment 1

[Benoit Jacob [:bjacob] (mostly away)]

Can you attach a backtrace?

My suspiscion is that is is caused by the patch from bug 571054 which landed on Thursday:
    http://hg.mozilla.org/mozilla-central/rev/8d80930e10f3

Can you try to see if the crash already happens with a build from Wednesday (September 1) or before ?

Comment 2

[Benoit Jacob [:bjacob] (mostly away)]

(In reply to comment #0)
>
> gl.drawArrays(6, 32766, 255);

ah! we have something else to suspect here. 32766 is just before 32767 which is the max value of 16 bit signed ints. Normally that shouldn't be a problem as the arguments of glDrawArrays are of type GLint which the spec says is always 32 bits wide. However, I can see how this could be a place where buggy OpenGL implementations could get things wrong: since the indices in Element Arrays used by glDrawElements are at most 16 bit wide, it's realistic to expect that buggy OpenGL implementations make the mistake of letting glDrawArrays too use 16 bit values. If the OpenGL implementation also fails to check for integer overflow, that could explain a crash.

If you confirm that crashes only occur when the 2^15=32768 boundary is crossed, that would certainly prove that this is a bug in your OpenGL setup, as 32768 plays no special role in our implementation. We use 32 bit values here, and moreover we do check for integer overflow (at 2^31).

Does VMWare fusion provide its own implementation of OpenGL or is it just using the host system's OpenGL setup? In the latter case, what is your graphics card and driver?

Comment 3

[Christoph Diehl [:posidron]]

Attachment: callstack

Hi, drawArrays(6, 0xf, 0xf) for example crashes at the same place. When using Gecko/20100901 Firefox/4.0b6pre none of the two variants produce a crash. I believe it's not a VMWare Fusion problem? Currently I am having trouble to get a callstack for this specific thread.

Comment 4

[Christoph Diehl [:posidron]]

Attachment: testcase

Comment 5

[Benoit Jacob [:bjacob] (mostly away)]

Thanks a lot of the testcase, I can reproduce here with NVidia driver on linux x86-64.

Relevant part of backtrace:

#0  0x00000000400bff48 in ?? ()
#1  0x000000004005f000 in ?? ()
#2  0x00007fffe5533438 in ?? ()
#3  0x000000000000007f in ?? ()
#4  0x0000000000000080 in ?? ()
#5  0x000000386efa3460 in ?? () from /usr/lib64/nvidia/libGLcore.so.1
#6  0x000000386efa64ae in ?? () from /usr/lib64/nvidia/libGLcore.so.1
#7  0x00007ffff588a0ff in mozilla::WebGLContext::DrawArrays (this=0x26f3ae0, mode=6, first=32766, count=255)
    at /home/bjacob/mozilla-central/content/canvas/src/WebGLContextGL.cpp:1037
#8  0x00007ffff60c771a in nsICanvasRenderingContextWebGL_DrawArrays (cx=0x1303780, argc=3, vp=0x7fffec267270)
    at dom_quickstubs.cpp:27952
#9  0x00007ffff6bb31a4 in js::Interpret (cx=0x1303780) at /home/bjacob/mozilla-central/js/src/jsinterp.cpp:4696


As expected, WebGLContextGL.cpp:1037 is:

    gl->fDrawArrays(mode, first, count);


So this confirms the suspiscion from comment 1.

Comment 6

[Benoit Jacob [:bjacob] (mostly away)]

Indeed, I confirm that gl.drawArrays(6,1,1) already crashes. _This_ is really what confirms comment 1.

Comment 7

[Benoit Jacob [:bjacob] (mostly away)]

Ah, some news.

In the error console I see this error message:

Error: WebGLFloatArray is not defined
Source File: https://bug593608.bugzilla.mozilla.org/attachment.cgi?id=472210
Line: 49

This is because the old deprecated name for typed arrays, WebGLFloatArray, is no longer supported. Fixing this (using Float32Array instead) makes it no longer crash.

Still, we shouldn't crash on any kind of ill-formed JS code. Investigating.

Comment 8

[Benoit Jacob [:bjacob] (mostly away)]

Attachment: reduced testcase

Here's a reduced test case. Crashes here.

Comment 9

[Benoit Jacob [:bjacob] (mostly away)]

... but if I comment out the bindBuffer call, it doesn't crash anymore.

Comment 10

[Benoit Jacob [:bjacob] (mostly away)]

Attachment: Fix crash: take care of VBOs in attrib 0 simulation

Here's the fix! Makes neither testcase crash anymore. The issue was that when doing the attrib 0 simulation on OpenGL (non-ES) I forgot to take care of bound VBOs.

Note that if you want your WebGL code to do what you want it to, you really need to update it to no longer use the deprecated WebGL arrays types! See above comment. Do watch the error console!

Comment 11

[Benoit Jacob [:bjacob] (mostly away)]

http://hg.mozilla.org/mozilla-central/rev/b40c3f4138d9