Inspired by a Biblical Hebrew phrase found in the Genesis creation narrative that describes the condition of the earth immediately before the creation of light, often interpreted as "without form, and void", Tohuwabohu is a nonlinear universal media control app for live performances developed in React.js and Electron.
Performance essays is a multimodal approach to conveying ideas and concepts to an audience by taking advantage of the many avenues of perception by humans. When we see the media we must ask how we believe we can interact with it. How do we connect, modify, and layer different media types? How do these interactions extend into the physical space? How do we generalize an all-media "instrument"?
From the layout to the actual functionality, Tohuwabohu gives a user much flexibility to do whatever they want instead of predefining everything. A bit of chaotic energy at first, maybe you don't know what this button does, or where the window went (as everything is movable); but as you get a hang of it, it becomes a good partner of yours.
Features include:
Preloading performance files
Adding local file and YouTube videos on the fly with thumbnails
Reorderable playlist in 1D/list and 2D/grid layout
Multiple resizable previews
Audio recording of the performance
Text window with basic WYSIWYG
Overlay canvas for live free-hand drawing
Each component is draggable to anywhere with in the app screen, collapsable anytime
MIDI support, fullscreen, hotkey binding, dark mode
Over the years, individually and sometimes collaboratively, I have done artwork and writing concerned with gentrification, the US prison industrial complex, subcultures and countercultures, alternative histories of the Internet, and the Black Atlantic.
My art practice includes music, sculpture, painting, videos, writing, photography, garments, performances, and even software. In our hope to understand a rapidly changing cultural landscape, there is a gap. In my research, I’m embarking on an account of “cyphering” as a social practice and media technique that emerges in the Caribbean and the Americas by Afrodiasporic peoples. Cyphering is originally a term from HipHop referring to MCs rhyming and beatboxing with one another, showcasing new styles, some of which are later recorded on songs. In cryptography, the cipher is an algorithm functioning as a lock or key. Thrust into new social relations, the Black subject learned and created various social codes for survival, and produced her own. I reflect on the Black analog networks springing from the Americas to deepen understandings of the material and ideological function of contemporary digital networks.
The research I’m embarking on is a chance to better understand how cultures of the African Diaspora have influenced the Americas and the World, and how that has influenced digital society. I have engaged with the topic with projects that look at the connections between Houston’s Chopped and Screwed music and Monterrey’s cumbias rebajadas, and the overlaps between jazz, beat poetry, and folk scenes in 20th century NYC. This current project grew from a performance essay I did at the invitation of the Center For Experimental Lectures which focused on Black/American culture before and after telecommunication technologies, looking at Juneteenth, US quilt codes and Gee’s Bend quilting practices, spray can art, Black Twitter, fractals in architecture and hair braiding in West and Central Africa, Bamana sand divination’s relationship to modern computing, and more.
Creativity is a big part of my life, and through it, I reflect on the things that happen in the world around me – the things that have impacted me, and the lives of the people I love most. Through art, I share information in a compelling way. I deeply believe in art and the power of culture to help us understand ourselves, and connect with others. Artists have the ability to make all kinds of research findings accessible to wider audiences and to create new models and ways of thinking, encouraging folks to do the same. These are some of the reasons why I have dedicated my life to it.
In NYC shortly after the tragic paradigm shift of the Covid-19 pandemic, I met with Jingxuan Sun and William Jose Leon II virtually and began working on 2 software with support from Cornell Tech’s \Art initiative which offered some creative responses to ongoing issues in contemporary culture. One project is called tohuwabohu (named after the Hebrew term for the time before God said “let there be light” in the Book of Genesis). Tohuwabohu is a non-linear presentation app for sound, video, and image. I initially planned this software as a tool to facilitate me in making performance essays, but then, with the lockdowns limiting the number of live performances, I shifted perspective and see the program as a way to make surprising new kinds of virtual experiences for sharing ideas, affect, and expressions. What I call performance essays are a multimedia performance format that draws from personal narrative, poetry, lecture, live music, video presentation, and performance art to create a unique experience, traversing various topics. The second application: Otherwards, is going to be released as part of a larger project that includes a multimedia album that I have been working on here in Brooklyn and Queens. Otherwards is an Augmented Reality mobile application for experiencing and interacting with sound in a different way. By spatializing various controls, a person has a chance to physically move around, and interact with their media in real time, whether that be turning components on and off, or using special effects to change the sonic character of the compositions. With the loss of live concerts, and upwards of 50,000 songs uploaded daily to platforms like Spotify, I started thinking of other ways to make sound more active and engaging. With all the hardships, one exciting thing coming out of the pandemic has been a rethinking of what community is and the ways it can be cultivated, physically, or in digital spaces.
> KELLY SUN
Fellow Statement
This year’s effort is significantly different from the former projects: we are actually developing software tools that assist the artists’ performance. Instead of creating a finite, discrete, single piece of art, we shift our perspective onto reimagining and transforming the way to interact with and engage an audience, from both artists’ and audience’s viewpoints. This is correspondingly reflected from our two projects, Tohuwabohu and Otherwards.
I figure the subjects we are exploring have an optimistic view towards technology and they fit perfectly into my personal definition of technology: the purpose of technology is to increase efficiency. Tohuwabohu serves as a one-stop-shop for file manipulation and its use cases can be elaborate to all types of presentation activity. The integration of MIDI opens the possibility of using the software as an instrument too, non-linear launching and playback controls has the potential to bring sensorimotor inputs together like “visual DJ-ing”. (It doesn’t sound like much but I’m secretly thinking this is super cool, for example I would love to try using it to create some glitchy visuals accompanying Death Grips’ jarring exponential soundscapes!! But of course they are very creative and their visuals are superb enough.)
For Otherwards, the inception of the concept cannot be more in time considering the state of the world. We are already mentally and emotionally isolated for years (ironically thanks to technology), and now people start to realize this fact as we are also physically isolated. Otherwards creates a way to reconvene ourselves to engage in music experiences and in the physical world by introducing some “obstacles” (you have to physically move to explore more sounds), very clever. Personally I find this concept fascinating, because as we have so much resources for music (also everything) today, I find myself essentially stopping the exploring process. It’s such a shame that ultimate efficiency breeds inertia and I’m so nostalgic about digging out the gems by myself. By introducing a little bit of inefficiency, I’m surprised that we are able to regain the momentum of discovery.
Contrasting with previous years’ effort, the process of creating these tools are not too different from normal “techy” software development life cycle: we had a comparatively straightforward goal, the steps and milestones are more or less clearly laid out, every now and then we catch up to do a stand-up and some feature iterations. What is different from before though, is we are working with non-technical members and effort is needed to introduce everyone to the product management part of the picture as well as to adapt quickly to the unchanging truth of constant changes regarding needs and use cases. In this sense, I’m really being a product manager and a software engineer at the same time!
I hate to relate any hardship to the pandemic cliche even remotely, but it is the culprit of a smaller project scope and a fairly chaotic workflow. With a ton of plans being disturbed, our team works mostly in async mode, especially as Bil and I are leading two separate projects. This means, again in company terms, we are the on-call person 24/7 for our corresponding project. This can be quite stressful when I learn React or Unity or digging into MIDI controllers for the very first time where I’m the only point-of-contact in researching and solving problems. I’d say going through the project in this fashion gains me more patience and composure when facing challenges, as well as confidence in my problem solving skills. But of course, everyone in the team is very collaborative and offers great ideas, initiatives and feedback all the time. I firmly believe the openness, transparency and empathy is the driver to thrive against all odds.
As a closing note, I’m super proud of the little thing I build and the person I become. Life is all about perspectives, so as technology. Instead of pondering upon and indefinitely reflecting on the negatives of tech like a critique, we choose to create something with a positive and humane touch, and that makes me smile.
Otherwards is a mobile app to reshape music in your surroundings, with 'Cook' -- tools for remixing musical tracks using augmented reality -- and 'Drift', which places songs along derive-style walking paths to engage a viewer/listener/participant. The backbone of the idea is to create individual experiences with sound in space, whether that space be one's bedroom or outside on the street, in a field, a park, or elsewhere. With nearly 100,000 daily uploads to streaming services, artists need to activate listeners with unique, memorable experiences now more than ever, and this would be a provocative first dive. Our ambition is to create a multi-perspective mixed reality experience involving sonic, visual, and physical inputs, using digitized versions of my analog street photography, poetry fragments, field recordings made in Brooklyn and Queens, New York, and songs from an upcoming multi-genre musical album, In Other Words.
