A DOCUMENTATION OF EXPERIMENTS EXPLORING HUMAN-PLANT INTERACTION.
Human-plant interaction can be facilitated through the use of capacitive sensors. Capacitive sensing detects changes in the capacitance of an electrical field, which can be caused by the presence of a conductive object, such as a human finger.
Capacitive sensing is commonly used in touchscreens, but it can also be used in a wide range of other applications, such as what we aim to achieve here with Human-Plant Interaction.
Capacitive sensing can be a powerful tool for creating interactive plant experiences. By incorporating sensors into planters or soil, we can create new ways for people to engage with nature in minimally invasive manners.
In this experiment, we look into culturally contextualising explorations and creating interfaces through identified culturally-specific objects of nature.
On the right, we have responses to 'describe an interaction you’ve had involving nature that is common in your culture.'
While it seemed obvious in hindsight, the reoccurrence of leaves in almost every answer was notable. Leaves are extremely diverse in form and function.
In Indian culture, particularly in South India, banana leaves hold significant cultural relevance. They are commonly used to serve food, especially during special occasions like weddings, festivals, and religious ceremonies.
This experiment test capacitive sensing and its performance with banana leaf surfaces.
The proximity-based banana leaf experiment opens up avenues for slider-based interactions, and the wilting banana leaf proved that decaying organic matter can also be used to facilitate interactions.
Through the deliberate placement of copper wire, the banana leaves allowed for some difference in signal based on where the reading was taken.
Playing around with decay. Readings taken from extremely wilted areas were less in magnitude when compared to fresher areas.
Here, we take the concept of human-plant interaction and applying it in an outdoor environment context through an excursion, using the same capacitive sensing circuit we built previously.
Attempting to get discernable readings, but all the sensor could detect was noise.
This set-up with the current capacity and power supply was not enough to sense tactile interactions in such a large area.
Here we investigate the use of decaying leaf litter as a potential interface.
We realised, decaying matter is conductive! However, once leaves are dead they turn into more static entities and simply hold organic material properties, but not life. Is the concept of Living Media Interfaces with dead leaves still applicable?
Cellular automata are mathematical models that simulate the behavior of complex systems, such as physical or biological phenomena. They consist of a grid of cells, each with a certain state, that change over time according to a set of rules. These rules are based on the states of neighboring cells, and they determine how each cell will evolve in the next generation.
In cellular automata, its emergent behavior can give rise to complex and unpredictable patterns, such as fractals and self-replicating structures, which can also be found in patterns created by flora and fauna.
With this experiment, we use biofeedback generated from tangible interactions with the plant to overlay different corresponding cellular automata into one pattern.
The light projection-mapping cellular automata system does not work very well because the light is unidirectional and casts shadows when the participant’s hands get in the way.
Till now, focus has remained on the augmentation of the plant. This intervention focuses on wearable garments and embodied interactions, where the user can easily interact with any botanical object of nature in their surroundings.
We built a starter 'glove' equipped with LED lights that change intensity depending on capacitive input received.
The experiment did not yield the desired outcome and was extremely unrefined. Additionally, it mistakenly interpreted the human body's inherent capacitance as a signal, leading to inaccurate measurements.
Creating real time illustrations in tandem with human-plant interaction.
Using Axidraw's python API to connect our capacitive sensing system to the Axidraw and allowing plant tangible interactions to dictate the drawing of a shape.
We used simple circles as a visual language because we needed to give the Axidraw sufficient time to the draw out its responses to any input.
The aim of this experiment was to differentiate between gestures initiated by the user and allow for more complexity and nuance with tangible interactions.
Additionally, develop a language in response to these gestural differences to set up a conversational turn-taking exchange between participant and user.
Potential Visual Response
Active Visual Response
Previously, in experiment 1, (Capacitive Sensing) we were receiving these byte values in their raw form and analysing them based on whether their numerical value spiked or not. Since there was a rapid influx of values, it was difficult to perceive any nuances for different gestures.
To tackle this, we utilised data visualisation techniques that helped us process this data better. and represented the data using graphs on p5.js. Furthermore, the data was also normalised when interacting with different plant subjects since their sensitivity meandered and was highly reliant upon internal factors like water retention and plant size, surface area and other characteristics.
Tapping resulted in a steep peak, while a longer, more prolonged touch resulted in more of a plateau. Moving forward, in order to set up a basic gesture library, we identified five states or gestures to work with.
We fed the graphs into an Image Classification Model that could automatically detect the gestures in real-time. Google’s Teachable Machine Image model was ideal for this as it allowed the incorporation of the model code (TensorFlow.js) with the p5 library. Having set this up, the code could identify the current gestural interaction with the plant, opening up opportunities for more distinctive inputs to be visualised for the participant to process.
Using‘small-talk’ phrases to simulate a conversation being held between participant and plant. A single tap opens up the conversation, and follow-up gestures include double and triple taps, which keeps the conversation going.
Teachable Machines proved to be quite performance-heavy, so we shifted to using a more basic set of gestures that just included tapping and holding, equivalent to dots and dashes. We also used RiveScript, a primitive chatbot that can be manually written via a text file. This setup was tested and general feedback was that while it was nice to see plants having personality, the conversation seemed repetitive and banal.
We also thought about facilitating conversation by drawing direct parallels with how plants communicate with each other.
The rhizome is a network of interconnected underground stems that allow plants to communicate through chemical signals. This communication helps plants coordinate their growth, respond to environmental changes, and share resources like water and nutrients.
With this in mind, we set up an interactive experience that faciliated root growth (via projections) through tangible interactions with plants. The more one interacts, the more complex the root systems grow.
We explored how a visual language could manifest in a physical form. This prototype aims to discover ways to create meaningful and personal connections with nature through Living Media Interfaces. One way to do so is by humanising or attributing some level of sentience to the plants.
This notion of plant sentience can instead be explored through the physical representation of a visual language, something akin to a human using handwriting to communicate or express themself.
Explorations were conducted with the help of an Axidraw, a drawing machine/pen plotter. Using the Axidraw as a vehicle to illustrate outputs, a participant could understand what it might be like to tangibly interact with and converse with nature.
Axidraw and plant setup, with posca markers used as a drawing pen, thanks to their rich and vibrant colors.
CHANGE OF STATE SPEED OF CHANGE
Hand reaches out to touch plant. Axidraw inactive.
Hand draws away, signal sent. Axidraw active.
Capacitive sensor takes readings every 2 seconds.
Axidraw takes on average 3 seconds to draw out shape.
Slighter touch results in smaller ellipses, as well is touching areas like baby leaves and new growth.
Stroking the plant stem as well as gripping a leaf produces larger elliptical responses.
The visual language of a cactus is manifested in the form of lines, spiky and long.
One sub-experiment was to use dead leaves as a page, and play about with image making.
Experiments with overlaying and overlapping of different user tested outcomes.
Also making sure to record the details of the interaction.
Setup for user testing and documentation. Participants were asked to interact with the plant for about a minute each and very quickly found their rhythm in the turn-taking interplay.
Every participant took different approaches when interacting with the plant. Some tapped lightly while some chose to squish the plant.
Some participants also explored tactile interactions with the soil of the plant and the pot as well.
Outcomes from the user testing session were collated into a rough loose-leaf zine, and served as a documentation or record of every interaction with either the Fittonia plant or the Pennywort plant.
Since the patterns of each cellular automaton can change so drastically yet provide immensely intricate results, we use biofeedback and gestural differentiations instigated by the user to map out different patterns.
The way each cellular automaton is also drawn is akin to typing out sentences, further supporting the narrative of a dialogue or conversation.
Additionally, while cellular automata are based on an algorithm naturally occurring in nature, it also has applications in computer science and engineering. Therefore, it sufficiently represents a shared phenomenon between the entities of nature and technology.
It was also ideal to spatially bring together both the input, in this case, the plant, and the output, in this case, cellular automata. Therefore, TFT screens were utilised and mounted onto the plants via a rod for a minimally invasive augmentation.
Participants are prompted to step closer and interact with the plant via some text on the screen.
Once they tangibly interact with the plant, the cellular automata begins evolving.
Designed by Aditi Neti as part of her Graduation Project for BA (Hons) Design Communication at LASALLE College of the Arts functionditi.xyz @functionditi
