Explore the world of NOTESPACE BEAT, a science fiction musical comic activity book! A storybook adventure that is packed with games about TIMING, INTONATION, SYNTHESIZER FILTERS, BEAT MATCHING, TURNTABLE SCRATCHING AND MORE!
This project was packed full of challenges from all angles. From a visual perspective, we had many multi-layered, high fidelity, full-screen images to handle, our Animation pipeline was through Flash but perhaps the most interesting were the rhythm games.
Rhythm is such an internal fundamental that you immediately know when something is off. With a platform like console, this is mitigated by the fact that the hardware is all unified but on low powered mobile devices, especially back in 2014 there was a lot of audio optimization to be done. After looking at a few Midi plugins, we used an open source c# solution and then rewrote half of it to support SoundFont 1 & 2. We came up with an audio streaming solution where we matched the streaming audio with the MIDI track. Once we had this sorted out, we came up with a predictive algorithm that smoothed out the user's input to still make them feel like rock stars, even if they were a little off. All in all, it is still one of my little triumphs!