Issue #144 Synthetic Exercises

Synthetic Exercises

Andrius Arutiunian

Andrius Arutiunian, Synthetic Exercises, installation, 2023. Courtesy of the artist.

Issue #144
April 2024

In the late 1980s a former classical flute student, Andy Hildebrand, changes his career path and moves on to study advanced digital signal processing. Having become a specialist in stochastic estimation theory (and by now a doctor), Hildebrand starts working for Exxon, one of the world’s largest oil and gas companies. There he develops a very idiosyncratic solution to the one simple question that the corporation asks him—where is the oil? Hildebrand develops a program to process data from reflection seismology, using seismic waves to locate hidden gas and oil sources. By sending sonic signals into the earth and using advanced algorithms to correlate and predict (in other words, attune the data), Dr. Hildebrand could finally locate the oil as if by magic.

Around ten years later a small US-based software company called Silent Talker begins developing an algorithm to detect micro-facial expressions. Through the decades that follow the company becomes the world-leading developer of automated deception recognition systems, or put simply, algorithmic lie detectors. Then a few former employees come up with their own version of the software, which they call “iBorderCtrl.” Developed with the border patrols of Spain, Greece, and the UK, this software was commissioned and funded through the European Union’s Horizon 2020 program. The algorithm scans the facial micro-expressions of migrants and asylum seekers entering the EU and, based on their facial movements, determines their level of veracity—essentially whether they are “lying” or not.

One day over dinner Dr. Hildebrand is jokingly challenged by a friend to “invent something.” The friend then proceeds to suggest that Andy invent something that makes him sing in tune. It dawns on Dr. Hildebrand that the technologies of oil extraction and voice tuning have a lot in common—correlation (statics determination), linear predictive coding (deconvolution), synthesis (forward modeling), and formant analysis (spectral enhancement).

He rushes home and soon after patents a new technology called “Auto-Tune,” a software to automatically tune the human voice in an organic and imperceptible manner. This software becomes the dream tool of music studios and singers, as it eliminates the possibility of notes going out of tune ever again. The algorithm becomes a dirty secret of the music industry, as nobody publicly admits using it until the 1998 pop hit “Believe” by Cher. The song uses the now-ubiquitous auto-tuning sound effect—its first massive recognition and commercial success.

Having no scientifically reasonable way to train their algorithm in the art of lying, the iBorderCtrl team trained it using thirty-two hired actors, who simulated deceptive and truthful situations in a lab setting. This data was then used to train the algorithm to determine whether migrant are being deceitful about their intentions for entering the EU.

Synthetic Exercises uses a sound library I compiled while working on another piece for which I subverted and retrained the iBorderCtrl algorithm. In Synthetic Exercises, this library, which was used to train the algorithm to “vocalize” the data, became my main point of fascination. As I trained the algorithm, it started spitting out early, noisy variants—not quite the human voice, not quite the finished sonority either. It produced a library of its own—artefacts, vocal errors, and sonic by-products, all documenting the birth of a particular digital vocalization. Synthetic Exercises is an arrangement of these sonic events, rendered heavily through auto-tune, exercising speech without words.

Artificial intelligence, Algorithms, Sound Art
Return to Issue #144

Made with the support of ZKM | Center for Art and Media, BALTIC Centre for Contemporary Art, and the Creative Industries Fund NL.

Andrius Arutiunian is an Armenian-Lithuanian artist and composer exploring sonic dissent, aural cosmologies, and vernacular histories. His research experiments with speculative instruments, non-Western knowledges, and alternate methods for world-ordering. Through playful investigation of hypnotic and enigmatic forms, his installations, films, and performances challenge the concepts of musical and political attunement.


e-flux announcements are emailed press releases for art exhibitions from all over the world.

Agenda delivers news from galleries, art spaces, and publications, while Criticism publishes reviews of exhibitions and books.

Architecture announcements cover current architecture and design projects, symposia, exhibitions, and publications from all over the world.

Film announcements are newsletters about screenings, film festivals, and exhibitions of moving image.

Education announces academic employment opportunities, calls for applications, symposia, publications, exhibitions, and educational programs.

Sign up to receive information about events organized by e-flux at e-flux Screening Room, Bar Laika, or elsewhere.

I have read e-flux’s privacy policy and agree that e-flux may send me announcements to the email address entered above and that my data will be processed for this purpose in accordance with e-flux’s privacy policy*

Thank you for your interest in e-flux. Check your inbox to confirm your subscription.