Name and Contact Information

  • name: Tarek Yasser
  • website: github, linkedin
  • gitlab username: knockerpulsar
  • timezone: UTC+3

Title

Replace OpenGL by a multi-API rendering library

Short description of work done

  • Converted Splash to run on the Raspberry Pi 4 by using OpenGL ES, This consisted of a few phases:

    1. Converting OpenGL 4.5 calls to their corresponding OpenGL ES 3.2 counterparts,
    2. Converting shaders to work with OpenGL ES 3.2 (version and floating point precision directives, various minor fixes),
    3. Removing uniform initializations from shaders as it isn’t supported by OpenGL ES 3.2,
    4. Merging the existing OpenGL 4.5 and OpenGL ES 3.2 code paths,
    5. Refactoring out graphics code to separate classes to allow further extension,
    6. Lots of code cleanups and bug fixes

  • Extended Splash’s unit testing code to allow for image comparisons: I thought this might be a good idea to start with to familiarize myself with Splash’s testing code. It’s always a good idea to start a refactor with some tests to ensure your refactor maintains the same output. This is not yet used significantly, but can be used as a basis for more graphics testing.

  • Researched various rendering libraries (magnum, Granite, bgfx, among others): One of our first ideas was to outright replace existing rendering code with something that might handle all the complexity for us. And while all the mentioned libraries are more than enough for Splash, they also introduced more complexity. For example, some would introduce complexity with shaders. We then decided to explore other avenues such as Vulkan and OpenGL ES.

  • Researched the viability of using Vulkan (for a rewrite) and OpenGL ES (as a drop-in replacement) with existing rendering code: The Raspberry Pi (Rpi) supports both. Vulkan is better for low-end devices like the Rpi as it has lower CPU overhead, while OpenGL ES was mainly considered as it was close to existing code and compatible with the Rpi. It also wouldn’t be as complex to implement and maintain as Vulkan.

  • Tested most graphics related functionality to catch any bugs not immediately apparent.

  • Researched ways to profile the Pi’s GPU performance, and did some rudamentary profiling runs.

What code got merged

  • MR #595: My first contribution to the project, a simple fix in the docs where a link didn’t point to the proper location.
  • MR #600: Extracted out duplicated error fetching code into separate functions. It also used newer C++ features instead of the older C style code.
  • MR #602: Second attempt at making Splash work on the Pi through OpenGL ES, cleaner with better commit separation and more bugs fixed vs MR #598 (the first version).
  • MR #618
    • Merges the new OpenGL ES code path with the existing OpenGL code path
    • Tries to automatically create a renderer if not given via the CLI.
    • Provides an API to create and use OpenGL objects depending on the selected renderer.
    • Fixes many newly discovered bugs.
  • MR #621: Fixes a bug where Image objects didn’t use the passed specification, resulting in an incorrectly sized image and out of bounds accesses.
  • MR #623: Fixes a bug where OpenGL debug callbacks would crash due to casting and passing in an object of the incorrect type.
  • MR #628: Pulls graphics code out of existing classes to make it easier to implement other APIs if needed, as well as refactoring some previously untouched graphics code.

What code didn’t get merged

  • MR #596: Turned out that testing Splash manually wasn’t that time consuming, so this fell to the wayside while I was busy porting Splash for onto the Rpi. However, the MR still has some prototype code for graphics testing, so it might be merged or used in the future.
  • MR #598: First attempt at using OpenGL ES instead of OpenGL to make Splash run on the Rpi. Overall a success, but the branch was a bit messy, and was superceeded by MR #602.
  • MR #601: Just a basic test to get comfortable with Splash’s unit testing framework, for similar reasons to MR #596, this wasn’t given much attention.
  • MR #622: Already included in MR #628. replaces duplicated and lower level code to read a texture off the GPU with a higher level RAII implementation.

What’s left to do

Some objectives of this project weren’t fully realized due to (mostly) technical limitations. An example includes:

  • Profiling GPU performance on the Rpi: We settled on using OpenGL ES, but the spec doesn’t require any profiling functionality. Thus, the Rpi doesn’t have any native/portable profiling options. Another option we could’ve gone with was using the extension GL_AMD_PERFORMANCE_MONITOR, but it required some work to set up and even more work to use properly. It also wasn’t guaranteed to be portable across different GPUs. Ultimately, we decided it would be too much work for one platform and decided to skip profiling GPU performance altogether.
  • Graphics optimization: Due to not being able to pinpoint where exactly our bottlenecks were, the only option to optimize graphics code was to blindly optimize parts that we thought would improve performance on the pi. This of course would’ve been too time consuming, so it was also skipped for the time being.

Further reading

Blog post detailing more parts of the journey.