Work Product of Haokun Song - Development of Generalizable SATIE Mappers that Optimized Audio Spatialization Flexibility

Name and Contact Information

name: Haokun Song
gitlab username: @kuk1song
timezone: UTC+02:00

Title

Development of Generalizable SATIE Mappers that Optimized Audio Spatialization Flexibility

Short description of work done

During my participation in the Google Summer of Code (GSoC) 2023 program, my work primarily focused on researching the mappers of SATIE, a plugin designed to link the virtual sound source and the spatializer, and develop the innovative generalizable mappers using SuperCollider to enhance the flexibility and scalability of SATIE, allowing for more immersive audio spatilization experiences.

After gaining a solid understanding of the core principles and code structure, I successfully implemented a new mapper cartesianControl1. This mapper can manipulate the position of the virtual source through the specification of 3D points in the Cartesian coordinate system, and introduced the geometric mathematical model to realize the physical phenomenon where the gain attenuation as the distance between the sound source and the user increases.

Building upon the foundation of cartesianControl1, I developed two more mappers: cartesianControl2 and xDirAdjacent. These mappers were designed to enable multi-spatializer control through calculations between diverse geometric relationships so that users can dynamically and flexibly customize their desired audio spatialization effects.

What code got merged

The new mapper:

cartesianControl1: This mapper can manipulate the position of the spatializer through the specification of 3D points in the Cartesian coordinate system.

The posted blog:

The found bug(fixed):

Conflict problem in sound source.

What code didn’t get merged

The new mapper(wait for review):

cartesianControl2: when there are two spatializers, this mapper can be employed to place them independently in two irrelevant cartesian coordinate systems, enabling separate control over the distinct spatializers without any mutual interference.
xDirAdjacent: Using this mapper, two spatializers can be defined as adjacent mode, and the direction is consistent with the x-axis.

The found bug(solution has been identified and is pending review):

There are distortions in the negative regions of x and z in the cartesianControl1 mapper, solved in the merge request !533 of the SATIE project.

What’s left to do

Develop yDirAdjacent and zDirAdjacent counterparts based on xDirAdjacent to introduce the new layer for multi-spatializer control.
Enhance multi-spatializer control by either expanding the existing mapper or creating new ones. This involves introducing new parameters and discovering additional geometric relationships to enrich the functionality.
The current multi-spatializer control is limited to two spatializers. Once this mode is stabilized, it can be further expanded to accommodate a higher number of spatializers, enabling comprehensive multi-spatializer control.
By employing UGens like Changed and LastValue to realize automation functions, the objective is to enable concurrent control of the position of spatializer in both cartesian and spherical coordinates within a single mapper, ensuring a seamless integration without any conflicts.
Establish a connection between parameters on external devices and mappers through OSC messages. For instance, linking the gyroscope and accelerometer parameters from a mobile phone to the position control variable of the mapper, allowing spatializer position control through the mobile phone. This enhances the interactivity and enjoyment of SATIE, adding to its overall appeal.

Overall, my GSoC journey has been a rewarding experience of hands-on coding, problem-solving, and collaboration with my mentors Edu and Michał, I am very honored to be guided by both. I hope this final implementation will pave the way for more possibilities within mappers even SATIE in the future.