Skip to content

De-ancient-ify OpenGL shaders#3109

Merged
j9ac9k merged 17 commits intopyqtgraph:masterfrom
pijyoi:mesh-shaders
Aug 19, 2024
Merged

De-ancient-ify OpenGL shaders#3109
j9ac9k merged 17 commits intopyqtgraph:masterfrom
pijyoi:mesh-shaders

Conversation

@pijyoi
Copy link
Copy Markdown
Contributor

@pijyoi pijyoi commented Jul 28, 2024

This PR updates all the shaders used by GLMeshItem and GLScatterPlotItem to OpenGL ES 2.0 syntax, i.e.

  1. No fixed pipeline, both vertex and fragment shaders have to be defined
  2. Use generic attributes instead of built-ins gl_Vertex, gl_Color, gl_Normal
  3. Compute own transformation matrices instead of relying on ftransform() and gl_NormalMatrix
  4. Use VBOs instead of client arrays

Some fixes:

  1. GLMeshItem example creates the wrong number of colors for the cylinders.
  2. On the RPI5, the animated surface plot has a different coloration compared to regular Desktops. This was tracked down to use of undefined behavior.
    rpi5_surface_plot_discolored

@pijyoi pijyoi force-pushed the mesh-shaders branch 2 times, most recently from bc51609 to 582e8f2 Compare July 28, 2024 13:44
@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Jul 28, 2024

As a bonus of switching from client side arrays to VBOs, the following script gets a nice performance boost.

import importlib

import numpy as np

import pyqtgraph as pg
import pyqtgraph.opengl as gl
from pyqtgraph.Qt import QtCore, QtGui
from pyqtgraph.examples.utils import FrameCounter

pngfile = importlib.resources.files("pyqtgraph.icons.peegee") / "peegee_512px@2x.png"
qimg = QtGui.QImage.fromData(pngfile.read_bytes()).mirrored(True, True)
qimg.convertTo(QtGui.QImage.Format.Format_RGBA8888)
data = pg.functions.ndarray_from_qimage(qimg).astype(np.float32) / 255
colors = data.reshape((-1, 4))

pg.mkQApp()
win = gl.GLViewWidget()
win.show()

rows, cols = data.shape[:2]
length = 5.0
radius = length**0.5
md = gl.MeshData.cylinder(rows=rows-1, cols=cols, radius=[radius, radius], length=length)
md.setVertexColors(colors)
item = gl.GLMeshItem(meshdata=md, smooth=False, shader='shaded')
item.translate(0, 0, -length/2)
win.addItem(item)

def update():
    win.orbit(-1, 0)
    framecnt.update()

timer = QtCore.QTimer()
timer.timeout.connect(update)
timer.start(0)

framecnt = FrameCounter()
framecnt.sigFpsUpdate.connect(lambda fps: win.setWindowTitle(f'{fps:.1f} fps'))

pg.exec()

@pijyoi pijyoi marked this pull request as draft July 29, 2024 05:26
@pijyoi pijyoi force-pushed the mesh-shaders branch 3 times, most recently from 20f8928 to f4ec023 Compare July 29, 2024 14:29
@pijyoi pijyoi marked this pull request as ready for review July 29, 2024 14:40
@pijyoi pijyoi marked this pull request as draft July 30, 2024 01:03
@pijyoi pijyoi force-pushed the mesh-shaders branch 2 times, most recently from 42b88a8 to fbe831f Compare July 30, 2024 13:29
@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Jul 30, 2024

All the GLGraphicsItems have been updated to use shaders and to use the programmable graphics pipeline.
The only known remnants of legacy code is the fetching of the ModelView and Projection matrices, but only as a stop-gap.

To fully de-ancient-ify the code would require removing functionality from GLViewWidget, specifically the itemsAt() method that uses GL_SELECT.

@pijyoi pijyoi marked this pull request as ready for review July 30, 2024 13:43
@pijyoi pijyoi force-pushed the mesh-shaders branch 9 times, most recently from 3262904 to 65b038f Compare August 5, 2024 12:24
@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Aug 9, 2024

As GL_SELECT is a legacy feature, whether it works with modern-ish OpenGL is highly dependent on your system platform and its gpu drivers.

Attached is a pyqtgraph-free script to test out whether GL_SELECT works on your system.
In order of decreasing likelihood that item picking will work:

  1. python script.py
    • use legacy immediate mode
  2. python script.py --mode client
    • use legacy client side arrays
  3. python script.py --mode server
    • use VBOs
  4. python script.py --mode server --use-shader
    • use VBOs and shaders

If the last one does not work, then after this PR, item picking of built-in GLGraphicsItems will no longer work (if it even used to work, that is)

Note: even if GL_SELECT does work on your system, it is likely going through a non-accelerated legacy path of your gpu drivers.

import argparse
from PySide6 import QtGui, QtOpenGL
import numpy as np
from OpenGL import GL
from OpenGL import GLU
from OpenGL.GL import shaders

def debug_callback(source, type_, id_, severity, length, message, userParam):
    debug_enums = [x for x in dir(GL) if x.startswith("GL_DEBUG_") and not x.endswith("_KHR")]
    lut = { getattr(GL, name) : name[9:] for name in debug_enums }
        
    print(lut[source], lut[type_], hex(id_), lut[severity], message)

# these are extremely old-style shaders
VERT_SRC = """
void main() {
    gl_Position = gl_ProjectionMatrix * gl_Vertex;
    gl_FrontColor = gl_Color;
}
"""
FRAG_SRC = """
void main() {
    gl_FragColor = gl_Color;
}
"""

class GLWindow(QtOpenGL.QOpenGLWindow):
    def __init__(self, mode, useShader):
        super().__init__()

        self.mode = mode
        self.useShader = useShader

        sfmt = QtGui.QSurfaceFormat()
        sfmt.setOption(QtGui.QSurfaceFormat.FormatOption.DebugContext)
        self.setFormat(sfmt)

    def initializeGL(self):
        context = self.context()
        format = context.format()

        if format.version() >= (4, 3):
            GL.glEnable(GL.GL_DEBUG_OUTPUT)
            callback = GL.GLDEBUGPROC(debug_callback)
            GL.glDebugMessageCallback(callback, None)
            self.callback = callback    # segfault on Linux if we don't hold a reference

        triangle = np.array([
            [ -0.25, -0.25, 0], [ 0.25, -0.25, 0], [ 0.0, 0.25, 0],
        ], dtype=np.float32)

        centers = [[-0.5, -0.5, 0], [0.5, 0, 0], [-0.5, 0.5, 0]]
        self.vtx = np.vstack([triangle + center for center in centers]).astype(np.float32)
        self.colors = np.ones((3, 3), dtype=np.float32)

        if self.mode == 'client':
            GL.glVertexPointer(3, GL.GL_FLOAT, 0, self.vtx)
            GL.glEnableClientState(GL.GL_VERTEX_ARRAY)

        elif self.mode == 'server':
            self.vbo = GL.glGenBuffers(1)
            GL.glBindBuffer(GL.GL_ARRAY_BUFFER, self.vbo)
            GL.glBufferData(GL.GL_ARRAY_BUFFER, self.vtx.nbytes, self.vtx, GL.GL_STATIC_DRAW)
            GL.glVertexPointer(3, GL.GL_FLOAT, 0, None)
            GL.glEnableClientState(GL.GL_VERTEX_ARRAY)

        if self.useShader:
            self.program = shaders.compileProgram(
                shaders.compileShader(VERT_SRC, GL.GL_VERTEX_SHADER),
                shaders.compileShader(FRAG_SRC, GL.GL_FRAGMENT_SHADER),
            )
            GL.glUseProgram(self.program)

    def drawTriangle(self, index):
        GL.glColor3f(*self.colors[index])

        if self.mode in ['client', 'server']:
            GL.glDrawArrays(GL.GL_TRIANGLES, 3*index, 3)
        else:
            GL.glBegin(GL.GL_TRIANGLES)
            GL.glVertex3f(*self.vtx[3*index+0])
            GL.glVertex3f(*self.vtx[3*index+1])
            GL.glVertex3f(*self.vtx[3*index+2])
            GL.glEnd()

    def paintGL(self):
        GL.glClearColor(0, 0, 0, 1)
        GL.glClear(GL.GL_COLOR_BUFFER_BIT)

        GL.glInitNames()
        GL.glPushName(0)

        for idx in range(3):
            GL.glLoadName(idx)
            self.drawTriangle(idx)

    def mouseReleaseEvent(self, ev):
        self.makeCurrent()

        lpos = ev.position() if hasattr(ev, 'position') else ev.localPos()

        GL.glSelectBuffer(100)
        GL.glRenderMode(GL.GL_SELECT)

        vp = (0, 0, self.width(), self.height())
        GL.glMatrixMode(GL.GL_PROJECTION)
        GL.glLoadIdentity()
        GLU.gluPickMatrix(lpos.x(), vp[3] - lpos.y(), 5, 5, vp)

        self.paintGL()

        GL.glMatrixMode(GL.GL_PROJECTION)
        GL.glLoadIdentity()

        hits = GL.glRenderMode(GL.GL_RENDER)
        for hit in hits:
            print(hit.names)
            for idx in hit.names:
                self.colors[idx][idx] = 1.0 - self.colors[idx][idx]
            self.update()

parser = argparse.ArgumentParser()
parser.add_argument('--mode', choices=['immediate', 'client', 'server'], default='immediate')
parser.add_argument('--use-shader', action='store_true')
ARGS = parser.parse_args()

app = QtGui.QGuiApplication([])
win = GLWindow(ARGS.mode, ARGS.use_shader)
win.resize(400, 300)
win.show()
app.exec()

pijyoi added 5 commits August 18, 2024 09:24
in the original code, we are generating faceCount colors (400) which i
more than the number of vertices (220).

the number of colors used by the OpenGL code is exactly the number of
vertices. (or in other words, each vertex has its own color)

the code still runs, just that only slightly more than half the generated
colors are used. it seems like this was not intended.
1) replace built-in matrices with uniforms
2) replace built-in variables with attributes and varyings
3) replace client side arrays with VBOs
draw points using shader w/o using texture
faces and edges indices are at most uint32.

np.uint size is platform and numpy version dependent.
prior to numpy 2, it was equivalent to "unsigned long".
after numpy 2, it is 64-bits on 64-bit platforms.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants