Work in pairs. Create a new musical composition. Make it interactive! (Or Don’t…) You can use oscillators, audio samples, the microphone, silence, videos, pictures, pixels!, colors, the mouse, your body, anything else that inspires you. The possibilities are endless!
Simone and I wanted to incorporate body movement in our sketch to step out of our comfort zone. I had the idea of using PoseNet to detect high five motions between two players to create the game. However, we tried the PoseNet to focus solely on the hand movement. It only detects the wrist and it is not as good as we would like it to be.
Therefore, we played around with handtrack.js and it was not working since there was only a few documentation on the handtrack. We spent hours on it and decided not to go down that route.
We still wanted to make the game interactive with body movement. Therefore, we used the nose position of the body with PoseNet.
We created 8 black balls while marking the nose blue at first. Simone got inspired by the motion and suggested us to create bees sniffing flowers. It was such a great idea and we decided to work towards it!
Simone created 8 types of flowers while I created 14 tracks of sound from Ableton Live. It was quite fun to create piano chords (we wanted to make the sound simple yet open for freedom to compose from the users). Simone used Procreate to make the flowers and the bee.
Flip the video
We were having trouble flipping the video. The practice we did were about pixels. I didnt’ think we need to follow the route of using each pixel and flip them. Therefore, we posted the question on Slack and Max helped to fix it with three lines of code. It was quite helpful to know to use translate(), and image().
Each flower is filled with different flower image
We created a class of flower to store all the features we wanted. We were struggling calling the image in the class, while we overlooked to have a variable in the constructor to call it when we create the flowers in setup.
Each flower is assigned with different chord
We were having trouble with the chords and make them play. We initially created the function to play chords when the bee is intersecting with the flowers. We put that in the nested for loops in draw. However, the sound keeps overlapping and it becomes very overwhelming.
Overlapping and Intersecting
We didn’t have time to figure out how to make the flowers created not overlap with each other. Therefore, we hard coded the position. However, we asked Aidan, the resident. He suggested that it would be better to create an update() function in the class. In the update() function, we can put the intersect() function that checks if the flower intersects with each other inside. In this way, we can assign the x and y position for the flower when we create them in setup. I would love to continue working on it to fix the situation rather than do the hard code.
It was quite fun working with Simone. We spent lots of time finalizing our concept. We had to make our idea doable for the scope of one week and our coding ability. Yet, it is hard to grasp what is doable within a week. We wanted to do great things and would love to learn and expand our potential in creative coding. This becomes a little bit of stressful when it is close to the deadline. The aesthetic is good. Yet the execution could be better. The office hour and asking around friends such as Max and Schuyler really helped!