> PILGRIMAGE INTO WALDEN TWELVE \ART 2019

Miao Ying, Ben Hwang, and Michael Barron

Pilgrimage into Walden Twelve is a live simulation software that can be run on computer and smartphone. The simulation has three distinct chapters: The Honor of Shepherds, The People’s Quest, and Battle for Glorious Spells. The project is inspired by AI neural network machine learning, medieval christian indulgences mechanisms, conflicting political ideologies, popular online Chinese S&M romance novels, and the recently emerged “social credit system” in China. The simulation takes place in a medieval magical fantasy land--Walden Twelve. Walden Twelve refers to “Walden Two”, which is a utopian novel written by behavioral psychologist B. F. Skinner, a society where perfect behavior is achieved as a result of reinforced positive conditioning of the desired behaviors while meeting basic needs of citizens. During the Medieval Era, the institution interpreted the religious ideology for the people who were illiterate, which led believers to exchange money for their sins--the indulgence system. The social credit system in China is a digital indulgence system, which records big data from its citizens and rewards or punishes them according to their social credit score. In a structure similar to that of the medieval church, the Chinese government explains the Marxist doctrine to its people and has implemented the technology to enforce citizen behavior. pilgrimageintowaldentwelve.com

> MIAO YING

Artist Statement

In our first meeting, I asked Michael and Ben if they can hack the the Great Fire Wall, they looked at my excited face then they looked at each other and said “No.” Then, they patiently explained to me the difference between computer science and hacking. I often think of that moment and laugh at myself. I have learned so much from them and it was such a privilege to have an experience like that.

I am very proud of our final project Pilgrimage into Walden Twelve, all three of us see how many layers of process and labour there are in the piece. It was a great challenge to all of us, I was frustrated about learning 3D modeling and game engine but I am so glad I pushed myself.

The first chapter, The Honor of Shepherds, shows 6 Artificial Intelligence “social shepherds” who represent different classes: urban migrant worker, outsourced American farmer, kung fu master, Chinese hero, Bourgeois ruling class golfer, and sophisticated intelligentsia. All 6 characters have different AI brains. Each shepherd is trained on millions of examples of human-like movement patterns as a metaphor of Big Data and are scored according to a deviation from a set of ideal expressions. The Shepherds try to minimize the difference between what they believe is a correct movement to express and what is known to be the true movements shown in the data using neural network supervised learning.

In addition, the characters must learn how to act correctly, while also accounting for the natural physics in the game world. Namely, not only must they behave a certain way, they must do so while using their own natural physics. During the simulation, the system samples, and tries to determine which ones are underperforming. Those that are deemed unacceptable by the algorithm, have their neural network replaced, and the associated character model is discarded. 

The second chapter of the adventure, The People’s Quest, represents the inner struggle of the kingdom. It is a self-played survival game/simulation utilizing machine learning takes place in a medieval village. In Christianity, the role of the shepherd is to guide and protect the weak. Cockroaches are despicable and disposable, but are also resilient. In People’s Quest, the shepherd AI hunts the roach AI who are trying to escape. Each round they learn from each others behaviors and will evolve their strategies.

The ultimate goal for the shepherd is to protect the roach from the ultimate danger—-capitalism demons, with the powers of Communist magic. The last chapter, Battle for Glorious Spells, is the outer struggle of the kingdom, a combat game engine between the different ideologies. Each attack has a unique name that is inspired by different political ideology systems and medieval fantasy games. Such as ”Class Struggle Cracking Smash”, “State Capitalism, Enforcing Flush”, “Rugged Individualism Slash”, “Nimble Chain of Social mobility”, “Bourgeois Exploitation Force”, etc. Chaotic magical effects manifest arbitrarily.

All chapters are live simulations, each time you run the software, it will appear differently. The reality of present China could be a simulated plausible future to the rest of the world.

Ying’s photo by Don Stahl.

> BEN HWANG

Fellow Statement

It’s hard to know where to start. \Art has been such an incredible experience, and one that has taken so many turns that I feel like we could write volumes about our journey. Despite this, and the number of times we made those turns and pivoted to new ideas, we eventually made it! And I am so proud of what we were able to accomplish. I learned so much about the artistic process–and subsequently how it interacts and sometimes clashes with the common engineering mindset. I also internalized what it takes to shape a singular concept and a plethora of ideas into a polished piece of art.

In this case that singular concept was the implications of the Chinese Social Credit System and how modern technologies enabled its conception. This in itself is a huge concept to explore and it led us to so many different ideas of how to express it. We explored abstract data visualization techniques to try and represent data that might be used to judge people’s credit score. We also implemented different generative art techniques blending Chinese propaganda with Medieval Catholic Imagery using various Neural Network Architectures. And we even made instagram bots trying to craft internet personas as reflected by the ads served to them. It took many attempts and tries, but eventually we landed on building Machine Learning powered 3D humanoids in the game engine Unity. Even at this point we weren’t all convinced or assured that this would finally bear the clear vision that we sought. But with much determination and trial we made the initial models work and provided a technical foundation for Ying’s design chops to really shine. I’ll always remember the day when Ying finished the first 3D model for us to use in Unity. After months of wondering when we would figure this out, we finally had some semblance of what this piece could be, and how awesome it would become.

Lastly, I like to think that if this pandemic didn’t happen, we may have actually continued on working and finding ways to continue making art + tech projects happen. But we cannot change what happened, and we as a team made the best of what we had. Working with Ying and Michael throughout the year and especially during the start of the pandemic made me appreciate how awesome they are and how much fun working with them has been! They were both excellent mentors in their respective fields and our endless conversations and exchanges were second to none. They really are what made this experience great. And although the project is over–for now–I hope to carry on the spirit of this fellowship in my own creative pursuits in music and wherever else life leads.

> MICHAEL BARRON

Fellow Statement

Creating an artistic piece that was both technically compelling and integral to the actual thematic goals was a constant source of brainstorming, recalibrating, and continuously collaborating. While we knew that the focus was largely around the social credit system (SCS), what that would mean from a technology standpoint, and as an art piece was not. To that end, it was us prototyping, seeing how it fit the themes, realizing the issues, and starting over. Collectively we explored generative images, generative text, reinforcement learning, and finally 3D procedural animation. Beyond the many, many technical challenges, the solution we ultimately arrived at was creating something that used AI in a way that actually mirrored SCS and still could be compelling from an artistic standpoint.

To give some context, each character is a representation of an archetype persona that is ultimately following the master/roach (system). This is expressed via both their character design, and from an ML perspective, how they literally learned to move and react in a physics based environment. Each character is trained on a subset of 3D movement data that is tuned to certain behaviors (running, fighting, dancing, etc). They then try to emulate a primary character that is perfect modeling a specific sequence of actions. Because of the nature of their training, and the physics of the environment, each behavior is continuously different and unique, and can never truly conform to the master/roach, but there’s always an explicit instruction to do so.

In terms of actualization, I think what was probably the ‘whoa’ moment (and I’m sure Ben would agree) was seeing Ying’s artistic characters and models implemented in the 3D environments we had all designed. Having these ML models live in a 3D simulation with physics wasn’t entirely meaningful until it had a ‘face’. It also viscerally reinforced how important it is to have a team, creative or otherwise where everyone has a diverse background and perspective.

Creating art isn’t grounded with an explicit template. It really needs an openness and level of exploration that just wouldn’t have happened if we were working in isolation. Or maybe it would have. But, it certainly wouldn’t have been satisfying. For me, working with Ben and Ying was as important as the project’s result. I’m proud of what we accomplished, but more so that everyone was so willing to engage, collaborate, and genuinely embrace each other’s ideas and perspectives.

> EXHIBITIONS

- The Asia Culture Center, Gwangju, South Korea, 10-15-2020 to 03-14-2021

> BEHIND THE SCENES

> APHASIA \ART 2019 SIDE PROJECT

Miao Ying, Ben Hwang, and Michael Barron

Three channel video. 5’20”. Singing and composition by Bibi Zhou/周笔畅. Lyrics by machine learning text generation neural networks. Voice over by Benjamin Hwang. Arrangement by Dizzla D Music. Commissioned by the UCCA Centre of Contemporary Art and supported by \Art at Cornell Tech.

The music in Aphasia is a mix of Christian Rock and romantic R&B, to worship ideologies instead of the spiritual. We used models trained to simulate the writing styles of humans--which had successfully generated text in the style of Shakespeare in the past--to generate the lyrics of the music of the video. We fed the computer to learn from B F skinner’s behaviorism theory, Chinese online novels (in the popular style/template of “Bossy, boss loves me”), and 45,000 classic romantic R&B lyrics and rapper Kanye West. Each text represented a particular ideology, those different blending of texts and ideologies achieved a unique style of writing, that is completely computer generated.

Aphasia is an inability to comprehend or formulate language because of damage to specific brain regions. The video shows the relationship between the sheep/followers and their shepherds/controllers, the dog and drone. The middle screen represents the sheep, while the screen on the left represents the dog as the the original shepherd, and the screen on the right shows the modern shepherd, which is a herding drone that can mimic the dog barking.

We asked Chinese pop star Bibi Zhou/周笔畅 to sing the the song with a romantic R&B beat according to the lyrics generated by the computer. Bibi Zhou debuted her singing career by winning the second place at the Chinese singing contest Super Girl in 2005, in which viewers are able to participate in the judging process by sending text messages with their mobile phones to vote for their favorite contestants, considered as one of the largest "democratic" voting exercises in mainland China.

Lyrics

Verse 1:
You changed my sympathy
I can’t face the questions
I’m just a behavior
I don’t think about her cause
I believe in love forever
So darling I love you
and I don’t need no conditioning

Chorus:
I want to wake up from me
I think I’m a process
This is the purpose
I will always be your kind contrast
Love is on the selection

Verse 2:
Stay with me
With your practical fire
Contact the culture that we used to be

Bridge:
Shine your autonomy on me
I hear the initiation coming
You made me feel culture

Chorus:
I want to wake up from me
I think I’m a process
This is the purpose
I will always be your kind contrast
Love is the only selection

Exhibited at Castello di Rivoli, Museum of Contemporary Art in Torino, Italy from February 25, 2020 to August 30, 2020, and UCCA Center for Contemporary Art in Beijing, China from November 13, 2019 to December 4, 2019.