The Flight of the Bumblebee – ICM – Week 11

Assignment

Work in pairs. Create a new musical composition. Make it interactive! (Or Don’t…) You can use oscillators, audio samples, the microphone, silence, videos, pictures, pixels!, colors, the mouse, your body, anything else that inspires you. The possibilities are endless!

Presentation

With the movement of the nose, the flower plays a different chord.

Code here.

Process

Simone and I wanted to incorporate body movement in our sketch to step out of our comfort zone. I had the idea of using PoseNet to detect high five motions between two players to create the game. However, we tried the PoseNet to focus solely on the hand movement. It only detects the wrist and it is not as good as we would like it to be.

Therefore, we played around with handtrack.js and it was not working since there was only a few documentation on the handtrack. We spent hours on it and decided not to go down that route.

We still wanted to make the game interactive with body movement. Therefore, we used the nose position of the body with PoseNet.

We created 8 black balls while marking the nose blue at first. Simone got inspired by the motion and suggested us to create bees sniffing flowers. It was such a great idea and we decided to work towards it!

Simone created 8 types of flowers while I created 14 tracks of sound from Ableton Live. It was quite fun to create piano chords (we wanted to make the sound simple yet open for freedom to compose from the users). Simone used Procreate to make the flowers and the bee.

Problems

Flip the video

We were having trouble flipping the video. The practice we did were about pixels. I didnt’ think we need to follow the route of using each pixel and flip them. Therefore, we posted the question on Slack and Max helped to fix it with three lines of code. It was quite helpful to know to use translate(), and image().

Each flower is filled with different flower image

We created a class of flower to store all the features we wanted. We were struggling calling the image in the class, while we overlooked to have a variable in the constructor to call it when we create the flowers in setup.

Each flower is assigned with different chord

We were having trouble with the chords and make them play. We initially created the function to play chords when the bee is intersecting with the flowers. We put that in the nested for loops in draw. However, the sound keeps overlapping and it becomes very overwhelming.

Overlapping and Intersecting

We didn’t have time to figure out how to make the flowers created not overlap with each other. Therefore, we hard coded the position. However, we asked Aidan, the resident. He suggested that it would be better to create an update() function in the class. In the update() function, we can put the intersect() function that checks if the flower intersects with each other inside. In this way, we can assign the x and y position for the flower when we create them in setup. I would love to continue working on it to fix the situation rather than do the hard code.

Conclusion

It was quite fun working with Simone. We spent lots of time finalizing our concept. We had to make our idea doable for the scope of one week and our coding ability. Yet, it is hard to grasp what is doable within a week. We wanted to do great things and would love to learn and expand our potential in creative coding. This becomes a little bit of stressful when it is close to the deadline. The aesthetic is good. Yet the execution could be better. The office hour and asking around friends such as Max and Schuyler really helped!

Published by Yiting Liu

NYU ITP '21

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Create your website with WordPress.com
Get started
<span>%d</span> bloggers like this: