Overview of the PlayStation GPU
GPU Rasterizer
The GPU uses 1 megabyte of video RAM organized as a framebuffer of 512 lines of 2048 bytes. The CPU can upload textures to this buffer using GPU commands (it's not directly memory mapped in the CPU address space), it can also read back portions of the framebuffer using other commands.
The GPU also contains a relatively simple 2D rasterizer capable of drawing lines and triangles (and "quads" which are really just two triangles side-by-side). It supports solid colors, textures (truecolor and paletted), several transparency modes and gouraud shading. It can also apply a dithering pattern before outputing the 16bit color. The GPU has a small texture cache to speed up rendering of textured primitives.
The rasterizer always outputs 16bits per pixel (1555 RGB, where the MSB is the "mask" bit) so as far as it's concerned the VRAM is a framebuffer of 1024x512 pixels. Therefore all the draw commands use this system of coordinates.
Note that the GPU is fully 2D, it's not capable of 3D projection and therefore has no notion of depth (so no depth buffer or anything like that). The PlayStation does 3D projection on the CPU using the Geometry Transform Engine (GTE) coprocessor. That means for instance that the GPU cannot do perspective-correct texture mapping which is the source of some easily recognizeable PlayStation graphical artifacts.
The coordinate system used by the GPU is simply the 16bit per pixel coordinate in the video RAM, so (0, 0) is the top-left of the framebuffer while (1023, 511) is the bottom right. You can see a list of the GPU draw commands in the No$ specs.
GPU video output
Once a scene has been rendered/uploaded to the framebuffer it needs to be displayed on the TV through the NTSC/PAL analog video output. In order to do this the GPU video output can be configured to select a rectangle in the framebuffer and stream it to the TV.
The size of this window depends on the video timings used. For NTSC it ranges from roughly 256x240 to 640x480 while for PAL it's from 256x288 to 640x576. I say roughly because since it's an analog output you can tweak the timings in many ways so you can actually "overscan" the output to increase the resolution or crop it furthermore depending on what you're trying to do.
Interestingly even though the rasterizer only outputs 16bits per pixel the video output can be configured to use 24bits per pixel. That's of course mostly useless for graphics generated by the rasterizer but it can be used to display pre-rendered 24bit images, for instance videos decoded with the console's MDEC and uploaded on the GPU VRAM. An other application could be 24bit images dumped directly from the disc and used as static load screens.
Design of the emulated OpenGL renderer
Features
First and foremost I think accuracy should be the main focus. Of course if it was the only objective a software renderer would be better suited but I think with modern OpenGL and its programable pipeline it should be possible to reach a decent level of accuracy (except maybe for the GPU cache, see below).
OpenGL would also make it easier to implement certain enhancements to the game's graphics compared to the original console, for instance increased internal resolution, texture replacement, normal maps etc...
Later on we could even attempt to salvage the raw 3D coordinates from the GTE and use them to render the 3D scene directly with OpenGL. That would allow us to have higher precision for our vertex coordinates, perspective correct mapping and many other things only possible with a fully 3D scene.
I think it's important to keep those features in mind when designed the basic renderer architecture so that we don't end up breaking everything when we try to implement one of them.
Potential difficulties
As you can see from the previous sections the PlayStation GPU is extremely simple compared to a modern graphic card, however it features some quirks and "exotic" modes which don't fit the OpenGL pipeline very well as far as I can tell.
Textures and palettes
At first I thought the most obvious approach to emulate the GPU video memory would be to use a single 1024x512 texture/FBO (or bigger if we increase the internal resolution) and use it as our Video RAM. There are a few potential issues with that approach however.
Upscaling and filtering
When a game wants to upload textures and palettes to the video RAM it must use one of the "Copy Rectangle" commands saying where the data should end up in the framebuffer (always using the same 1024x512 coordinate system) and then send the data 32bits at a time.
At this point there's no easy way to know what the data contains, it can be a 24bit RGB image from a video, it could be a 16bit "truecolor" texture, it can be a paletted texture, it can be a palette or several of those things at once. We'll only know how to interpret the data when the GPU actually uses it, either through a textured draw command or by the video output configuration if it's just meant to be dumped on the TV screen without further processing.
This seriously limits what we can do with the raw framebuffer data if we don't want to break anything.
For instance if we have a single big FBO representing the entire framebuffer at an increased resolution (say, 2048x1024). When the CPU attempts to upload data to the GPU we could upscale and optionally filter it and store it in our big buffer. Easy.
Except of course upscaling and filtering palettes and paletted textures won't work as intended, the intermediate pixel values will be meaningless since we basically have a non-linear color space. We cannot risk destroying palettes, therefore we can't really mess with the uploaded data until we know what it's going to be used for. Or at least whatever we do must be reversible if we need to go back to the original value later on.
I'm not sure what's the best way to deal with this. Maybe we could have two framebuffers instead: one at the native 1024x512 resolution containing the raw framebuffer data with no fancy enhancements that would be used for paletted textures and a bigger framebuffer containing the rasterizer's output at increased resolution. Keeping the two coherent will be a challenge however and I don't know where 24bit images fit in there. Maybe we could use a completely different rendering mode when the video output is set to 24bit mode so that we can ignore it the rest of the time.
If we want to implement texture replacement we also need to figure out when it should take place. That's a complex subject however, maybe we can leave it for later.
OpenGL texture sampling
An other potential issue if we use a single texture/FBO for the entire video RAM is that we need to be able to render into it while we sample a texture in an other location in the same buffer. So we would be rendering to an FBO while it's also bound as a texture.
As far as I know this kind of configuration is not well supported by OpenGL and can quickly lead us into undefined behavirour territory.
I believe that this should be achievable using the GL_ARB_texture_barrier extension which is part of OpenGL 4.5 but maybe we can work around it.
Otherwise we could maybe use two framebuffers and "Ping pong" between the two between each frame instead, this way we would write to the current FBO while we use the previous one for input. That could be innacurate if a game decide to use a polygon rendered during the current frame to texture a subsequent one, I know some games use similar features to create some fancy visual effects.
Semi-transparency
The PlayStation GPU rasterizer has several pixel blending modes used for semi-transparent primitives (copied from the No$ specs):
B=Back (the old pixel read from the image in the frame buffer)
F=Front (the new halftransparent pixel)
* 0.5 x B + 0.5 x F ;aka B/2+F/2
* 1.0 x B + 1.0 x F ;aka B+F
* 1.0 x B - 1.0 x F ;aka B-F
* 1.0 x B +0.25 x F ;aka B+F/4
Unfortunately I don't think the OpenGL blending fixed function is flexible enough to accomadate all these modes without a significant number of hacks. Besides for accuracy's sake we might want to handle the blending calculations in our own shader code to make sure we don't have weird rounding and saturation discrepancies (if we want to be bit accurate with the real hardware).
For this reason I think it would be better to handle the blending in the fragment shader. Once again this is not generally how things are done in OpenGL as far as I know, but it should be possible at least using the same OpenGL 4.5 GL_ARB_texture_barrier extension mentioned before or by "ping ponging" the buffers.
Masking
The MSB of the 16bit pixel is used to store a "mask" field. When the GPU renders a primitive it can be configured to set this bit to either zero or one. An other configuration flag can tell the GPU to treat framebuffer pixels with the mask bit set as "read only" and refuse to overwrite them. It effectively works like a simplified stencil test in OpenGL.
The problem if we decide to use a stencil buffer to emulate this masking feature is that we'd potentially need to update the stencil buffer after each primitive, since it's possible for any primitive draw to set the mask bit if the GPU is configured to do so. I don't know if it's possible to meaningfully modify the stencil buffer in an OpenGL fragment shader. Barring that we won't be able to use the stencil test to accurately emulate the masking.
Alternatively we could use the same trick I proposed to handle the semitransparency modes above: we fetch the target FBO pixel in the fragment shader and if the masking is enabled and its MSB is set we don't change its value. If we already handle transparency that way it might be relatively straightforward to add, theoretically.
Video output
So far I only talked about rending things inside the framebuffer, we also have to implement the video output to display the relevant part of the framebuffer on the screen.
The PlayStation video output streams the pixels from the framebuffer directly on the screen without any kind of intermediate buffer. In other words the display is "continuous", if a game wants to implement double buffering it does it by rendering to two different parts of the framebuffer and swapping the video output source position and GPU rasterizer draw offset when it wants to "flip" the buffers.
In order to emulate this properly we'd have to basically render one pixel at a time, keeping track of the output video pixel clock. I don't think that can be done very efficiently in OpenGL, nor does it sound necessary to emulate correctly the vast majority of games.
Instead we could display the entire visible portion of the framebuffer at once at the end of every frame or the beginning of the next one maybe. If the game uses double buffering we want to it right after it swaps its buffers to reduce latency. I'm guessing the swapping would take generally take place during the vertical blanking to reduce tearing artifacts, so maybe we could do the frame rendering at the end of the vertical blanking. I need to do more tests to make sure that's really how it works in practice though.
24bit mode
As explained before while the PlayStation rasterizer can only output 16bit RGB to the framebuffer the video output is capable of treating it as 24bits per pixel to display pre-rendered graphics.
When displaying in 24bit mode we'll have to find a way to take our 16bit RGB framebuffer and display it as a 24bit image. I don't know how difficult that is with OpenGL. I guess at worse we could sample two 16bit pixels in the fragment shader and reconstitute the correct 24bit value there. We could also implement 24bit image upscaling and filtering there to avoid having to handle it in the 16bit code. After all 24bit mode is a very limited special case on the original hardware.
Texture cache
The PlayStation GPU has a small texture cache used to speed up the rendering of textured primitives. Normally it only affects the speed at which a primitive is rendered, however if the cache becomes "dirty" (for instance by overwriting a texture while some of it is in the cache) it could potentially change the aspect of the resulting triangle.
I have no idea how to emulate this particular feature in OpenGL. As far as I can tell the only way to emulate it accurately would be to draw each primitive pixel-by-pixel, updating the cache state in the process but that goes against the spirit of the massively parallel GPUs we have today.
Fortunately I believe that for the vast majority of the games we can completely ignore this cache, so maybe we can ignore it or at least put it very low in our priority list.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent
