Dancing Delicacies: New meaning to the concept of ‘playing with your food’

Jialin Deng has a background in food design, fine art and interaction design but now the PhD student within Monash’s Creative Technologies research group is exploring all three – with food at the centre

But this isn’t food as we know it; it’s not what we understand food to be. It’s food as a medium, a material, a plaything, and a research tool in psychology and technology.

Deng will finish her doctorate in 2024, and is working on a project called Edible Symphony, where food – in this case, crackers – become audio speakers, using gold leaf and a magnetic field connected to an amplifier – and it works! The crackers can play Spotify! Which is great, but why?

“You might bite into it and consume the sound as well,” says Deng. “But what’s the meaning? That’s what we’re going to find out. How does it make sense to people?

“For us, this is the question. It’s not about the product, it’s about how we use this food material to conduct, to realise, or compute interactivity.”

The Creative Technologies group is within Human-Centred Computing, part of the Faculty of Information Technology.

Among other things, researchers from the group build interactive and augmented technologies – current projects include human-drone interaction, brain-computer integration, digital aquatic play, and a human-bicycle interface.

Deng’s most recently completed project is “Dancing Delicacies: Designing Computational Food for Dynamic Dining Trajectories”, published here in July, and also demonstrated on her own video below.

Moving food around the plate

The idea of it is that food can move around a plate and merge with other foods on its own. Deng’s website explains this as a process to “computationally animate the inanimate for dynamic trajectories of eating where food items can be ‘programmed’ and ‘reconfigured’.”

“Dishes can regulate their own deployment (including arrangements, flavour combinations and visual presentations) in real-time through multiple operations governed by an electrowetting (voltage via a liquid to an electrode) system.”

So droplets of food materials – water, flavourings, coffee – move, apparently independently.

What do we, as consumers of food, think of this? What might a food creator, a chef, think of this? Is it food? Could it ever be food?

Deng has also finished projects on a programmable pudding, or a microchip snack, called a “Logic Bonbon”, published here.

In essence, the fact that food is used in these experiments is key. “It becomes a meta-material,” she says. “The materiality of it realises the computational process. The case studies are exploring designs in which the food is the medium by which computation is realised, so in the end, you can ‘taste’ computation.”

A glimpse into the future

Professor Florian “Floyd” Mueller, an interaction, game and play design expert from the Creative Technologies group, says the research is a glimpse into the future of food and computing.

“Computing, which has already moved into the kitchen, is now moving further ‘into’ the food itself,” he says. “But it also works the other way around. The integration of food and computing will transform how we understand both computing and food as not two very different things, but a new frontier that combines the best of both, opening up new questions, such as: What if you can eat a computer? What does ‘computation’ taste like?

“This will not only change the hospitality industry (where we work with our industry partner, Worksmith, and are supported by the Australian Research Council), who can create much more engaging experiences by being able to tell new and different stories through interactive food, but also computer science education, where students learn about computing by eating food.”

Experimenting with the dining experience

Deng and her fellow researchers are talking to Melbourne’s renowned food community about the possibilities of Dancing Delicacies, positioning it as culinary art intersecting with technology. It was demonstrated at a hospitality community event at Worksmith last year.

“For Dancing Delicacies we designed the plate, and underneath is the electrode array. We can control the flavourings, the condiments, or any other ingredients, and we can play with them and move them around.

“This is just basic operations using this technology, but the point is, how are we going to use the food material’s properties and computational capabilities to achieve different dining experiences?

“For example, a chef can predefine the locations where they want to put the droplets, and they can program it frame by frame, like you do in animation. We can put solid items and watery items together, we can merge two different flavours, we can transport various things towards the plate, we can play with chemical or physical reactions like in molecular gastronomy.

“We want to see unusual qualities that are different from the traditional way of having your food. It might, for example, enhance your sensory experience, or enhance your taste experience. So that's why we're running studies at the moment with food creators and consumers.”

What do they think, especially the chefs?

“Instead of thinking of serving the dish as the last thing a chef needs to do, it’s now only the beginning of telling a story through the dish, enabling a tighter interaction between the chef, the food and the diner, allowing for a richer engagement with what we eat, and what the people behind it want to achieve.”

“It allows the food creators and diners to come closer together!”