Stage Lighting Replication
An Unreal Engine Plugin that replicates the lighting conditions of an image.
Skills: Computer Graphics, Computer Vision, 3D Math, Optimization Algorithms, Artificial Intelligence, Research
Languages: C++, UE5 Blueprints
For my master’s project, I attempted to create an Unreal Engine 5 C++ plugin that would allow developers to recreate stage lighting for digital applications. This was primarily aimed at AR-enhanced stage plays, like "Singring and the Glass Guitar," which I helped develop in 2023. For additional reading, please check out my paper on this project.
The first part of this project was creating a system that generates an approximate mapping of the light present in images of the stage. This system required two images captured from the same position and angle: one fully illuminated, and one of the target illumination. Assuming that the fully illuminated stage was lit uniformly, the difference in color levels between each pixel represents the amount of light present in each pixel. By projecting the image onto the geometry, each pixel can be mapped to a location on the stage.
This mapping acts as a “ground truth” for a genetic algorithm. This algorithm optimizes the angles, colors, and sizes of spotlights in the scene. Each proposed solution assigns each of these variables to have the levels of light at each point in the mapping be as close to the ground truth as possible. Overall, the success of this project was limited - the algorithm produced results that had correct ambient levels and spotlight colors, but overfocused on the brightest parts of illuminated regions. Additionally, this method required a significant amount of time, and was far too slow to be of practical use.