Where to Perch Responsive perch Bench Individual Work Instructor - Yiqing Wang Jul.2024 - Jan.2025 This project is an interactive bench that dynamically responds to users' body language and emotional states.
By leveraging OpenPose for skeletal data analysis and neural networks for emotion classification, the bench translated negative emotions into defensive postures through motorized adjustments of its wooden structure. The bench not only reacts to individual emotions but also incorporates the social behaviors of multiple users, creating dynamic, differentiated responses. This project aimed to challenge the polarity of hostile architecture by prompting users to reflect on their own behaviors and their relationship with public spaces. Through this work, I began to see tangible interfaces as a medium for fostering dialogue between design, users, and the environment.
Human
Disposition/
Attitude
Physical Posture
Sensor
Human Perception
Machinery Dynamic
Human
Bench
Object
Reaction
Individuals and Groups Activities with Benches Happenings Around Benches Dataset Public Surveillance Camera Live Streams
+
Videos taken of various locations
Detection
Transform
Mapping
Human Detection YOLO
Pixel coordinates to world coordinates conversion Tsai calibration method
Processing kernel density maps
Behavior to Interaction Pattern
Happy
Defensive: Valence: Positive
Arousal:
Responsiveness:
High
Sleepy
Defensive: Valence: Positive
Responsiveness:
Arousal:
Depression
Low
Defensive:
Valence: Negative
Responsiveness:
Arousal: Low
Anger
Defensive: Valence: Negative
Arousal: Low
Responsiveness:
Physical Architecture Capture
Classify
Communication Bridge
Embedded Control Board
Motor Driven Board
Camera
Open Pose
Firefly
Arduino
Coded motor drive
Perceive input
Position detection
Distribute digital commands to actuators
Drive the motors and control speed
Mapping real-world actions and pushing motion commands to Arduino
+ DNN Emotion classify
Shaft of rotating body
Bearing
Bench main frame Triangular rotating component
Shaft adapter Worm Gear Reducer
Mounting Backplate Drive motor
Drive motor
DC Encoder Motors Converting electronic signals into physical motion
Manhattan Cube A 3D Printing Construction Combinatorial Methods Based on Voronoi Variants
Individual Work Instructor - Yiqing Wang Jul.2024 - Dec.2023 Traditional construction methods rely on prefabricated components and connectors, which limit the possibilities for integration and adaptability. With the advent of advanced manufacturing techniques, particularly additive manufacturing, previously difficult-to-manufacture specialized structures have become possible, and new methods of construction have emerged. This study explores the application of Voronoi-based partitioning methods, specifically variants measured by the L1 Manhattan distance, as well as previously optimized polyhedral versions for microstructural 3D printing. Through small-scale prototypes using clay 3D printing combined with cement casting, this research investigates how these methods can enhance collaborative construction, interlocking structures, material hardening, and printing efficiency. The L1 Manhattan distance metric offers potential benefits in controlling the basic XY stepping movements in 3D printing, which can aid in achieving consistent build times across different styles.
Construct target model
Collaborative printing construct
Random points
Genetic algorithm
iteratively
Partition cell seeds
Optimize printability distribution
Voronoi tessellation Mesh
Voronoi partition in different distance metric
Printed Components
Iterative process of genetic algorithm Use a cube as an example to generate printable cell divisions
Gen 0
Overhangs exist
Exaggerated overhang
Exaggerated overhang
Gen 4
Overcrowded edges
Gen 1
Right
Gen 2
Front
Gen 3
Top
Printable units
Lookink A Phone-Based Gaze Interaction Method for Large Language Models
Group Work - 100% Concept, Development, 50% Testing Instructor - Xipei Ren Jan.2024 - Nov.2024 Multimodal large models have opened up numerous possibilities for personal assistant devices in scenarios like smart homes. However, unlike human assistants, these devices lack visual references from the real world and cannot understand gestures like eye movements or the referential use of pronouns such as 'that lamp.' To address this gap, we introduce Lookink, a more versatile method of eye-tracking interaction aimed at real-world environments. The system first maps eye movement coordinates captured by a front-facing camera to the field of view of the rear-facing camera to obtain the content of the user's actual focal point view. Then, the images are segmented based on annotated contours and fed to a large language model. We have developed a prototype program on smartphones and conducted comparative tests on several different gaze information annotation methods for input to large models, experimenting with various small-scale specific application tasks to evaluate its effectiveness and value.
Front and rear capture
Front monocular
with IR Tof gain eye movement Rear camera calibration, correction, calculation of the imaging plane
Pinpoint the intersection of the user's gaze in camera view
Vision comprehend Image segmentation
LLM
recognition
Contextual
feedback or execution
Process The ability of Large Language Models (LLMs) to relate and analyze textual context is becoming stronger and stronger. At this stage, it's crucial to offer visual context input methods to enhance interactions.
Coordinate transformations
map
the content of annotations
Here is a schematic of the ATTENTION mechanism
There is a yellow mug on the table , could you pass it to me ?
There is a yellow mug on the table , could you pass it to me ? Previous sentence generated the weights to explain the pronoun's reference, obviously, we wouldn’t speak this way in reality
Could you pass me that ?
Front Cam
Eye Capture
Rear Cam
Environment Capture
How should the model be understood at this point?
Detect
and
segement book - 30.8%
potted plant - 61.4%
chair - 80.4%
cup - 89.4%
bounding box
contour outlining
point circle
fill and stroke
Labeling method
Scenario An example of a searching using the LLM
Search about that book You mean the black one?
Searching ...
TactiGuide An accessible evacuation guidance system
Group Work - 100% Concept, Development, 50% Testing Instructor - Xipei Ren Jan.2024 - Nov.2024 In the context of evacuations during fire and earthquake incidents, current solutions predominantly rely on visual cues for guidance, overlooking the challenges faced by the visually impaired and situations where visibility is compromised, such as in smoke-filled environments. We present TactiGuide, an innovative non-visual evacuation guidance system that utilizes Ultra-Wideband technology for positioning and provides tactile feedback through vibration for direction. Designed to complement existing illuminated signage systems, TactiGuide employs the smartphone as a terminal to obtain the direction angle through continuous one-dimensional positioning provided by UWB signage, delivering nuanced vibratory interactions to guide users. The system's effectiveness and reliability were assessed through testing with real smartphones and UWB modules, demonstrating the potential of vibratory feedback combined with UWB positioning as a viable solution for enhancing evacuation safety for all individuals, including those with visual impairments.
Bluetooth
Ultra-wideband
Haptic Feedback
User movement
Direction guidance pattern
The LRA variables that constitute the vibrational patterns: intensity, sharpness, and interval time Rough alignment ±15°
Contious guider
Precise alignment ±2.5°
Evaluation 用户测试
Turn reminder START POINT
Escape route setup Door indicator
Angle positioning system mode Test route Records
EXIT
Memories Relive A Generative Transformation and Memory Retention App For Furniture
Individual Work Feb.2024 - Jan.2025
Focusing on sustainability, this project addresses the environmental and emotional impacts of modern consumption habits. To challenge the throwaway culture, I developed an generative-assisted co-creation app designed to guide users in repurposing discarded furniture. By analyzing existing components,
the app generates redesign ideas and offers practical suggestions, such as transforming a table leg into a chair back. Beyond reducing waste, this tool encourages creativity and ownership, enabling users to preserve sentimental value while reimagining functionality. This project advocates for a culture of reinvention, inspiring individuals to uncover potential where others see obsolescence.
Aged Furniture
Capture
Generation
Memory Retention
Renewal and Elimination
3D Scanning
Reformulation Inspirations
Physical Preservation
Difficulties of Discard and Recycle
Preferred Parameters ...
Virtual Memories ...
Digital Existence ...
Environmental Pollution ...
Liftglow
A unique interactive wall lamp Concept Design
Individual Work Instructor - Dehua Yu Feb.2023 - Mar.2023 2.
1.
1. Product usage scenarios 2. Interaction patterns that follow the natural semantic understanding of the user 3. Illustration of technical realization
When we imagine a home environment, we often think of a cozy, warm, and comfortable atmosphere. Liftglow is a wall lamp designed to provide this kind of atmosphere for home environments.
Its design is inspired by the soft light that can be seen by lifting the corner of a curtain, and the lamp is turned on and the brightness adjusted by folding the top corner of the lamp. This interactive design makes Liftglow a unique and eye-catching lamp. 3.
Arc Type Optimization and Evaluation of Tablet Keyboard Layout in the Unsupported Case Prototype Research
2.
Group Work - 100% Concept, Development, 50% Testing Instructor - Yun Chen Sep.2023 - Jan.2024
3.
4.
5.
1.
1. Unicode icon replacements to avoid prior familiarity effects 2. Graphs that illustrate thumb length, gender and layout differences in typing performance, with linear regression 3. Thumb length measurement and grouping reference 4. Early testing scenario 5. Heatmap from the pilot test and the arc layout design process based on it
Tablet PCs are mobile devices that possess a mid-range size and processing speed, situated between that of cell phones and computers. They are highly favored for their easy-to-use and portable features and thus are popularly employed in light office settings due to their availability to a wide user base.
Therefore, we created a customized, ergonomic, and optimized virtual keyboard for touchscreen tablets by combining different existing keyboard layouts. Our goal was to address the common issues faced by users who hold the tablet with both hands and type with their thumbs.
We based the keyboard design on the comfortable typing area of the fingers on a tablet device and developed a test prototype. To minimize interference of existing layout proficiency, we replaced familiar letters of the alphabet with special symbols. An equal number of male and female participants underwent testing comparing the traditional keyboard layout to the new one. Researchers measured the length of the participants' thumbs, administered subjective scale questionnaires, and separately recorded the time and accuracy of each participant's use of both keyboards.
Nevertheless, the virtual keyboards on tablets currently available possess layouts and designs that are derived from the physical keyboards of computers or the full 26-key keyboards of smartphones. These keyboards, merely stretched and scaled to fit the tablet screen, lack ergonomic design considerations for specific tablet usage scenarios. As a result, users engaged in text-based office work may experience inconvenience.
A regression analysis was conducted to examine the relationship between finger length and typing efficiency among subjects, concerning existing studies. The subjects were divided into three groups based on their finger length - short, medium, and long. Notably, the short-finger group comprised mostly female participants. Results demonstrate that the curved arrangement of the split keyboard significantly improved typing efficiency for users in the short-finger group, while showing no significant change for the other groups. However, there were some negative effects on the correctness rate.
It was found that finger length was negatively correlated with the enhancement effect.
LifeWrap
2.
Inflatable Versatile Headscarf Concept Design
Group Work - 100% Concept, 60% Developing, Making Instructor - Chao Gong
3.
Feb.2023 - Feb.2024
4.
2.
1. One of its forms functions as an inflatable life buoy or a portable water carrier 2. The finalized heat-pressing and stitching pattern is shown alongside several other full-scale prototypes tested during the process. 3. Making process
1.
5.
LifeWrap is a multi-functional product designed for refugees and vulnerable populations. Its inspiration from traditional headscarves and triangular Bandage, with modern materials and inflatable structures, enabling it to serve multiple functional forms.
4. The folding method during transportation is designed to optimize space, with compact packaging and planned stacking arrangements to aim for cost efficiency
It addresses several common challenges faced by refugees: as a headscarf for protection from the sun and dust, as well as a water carrier, life buoy, and triangular bandage for first aid purposes. Its multi-functional design replaces the need for multiple items, making it a cost-effective and accessible solution for those in need.Made from eco-friendly cotton and silicone, it has a low environmental impact and is accessible to refugees.
We explored design iterations and optimized details through prototyping. Different base shapes and sizes were tested. To simulate the need for costeffective production, we replaced digital heat press engraving with a straight-line heat sealer typically used for food packaging. Various inflatable structures were explored using Kangaroo simulations in Grasshopper and small-scale physical models to experiment with different configurations. Final decisions were made based on full-scale prototypes tested with real users. Due to the processing limitations of dipped canvas, we selected PSPE for the inner lining after experimenting with different fabrics, with waterproof and fire-resistant PVC canvas stitched as the outer layer.
5. Several usage forms examples: fastening and adjusting tension using Velcro, functioning as a tourniquet for emergency bleeding control through tying compression, and carrying water with a rear-shifted center of gravity for an alternative load distribution option