SEP 2023 - APR 2024 (6 MONTHS)
SenseScape
Designing a gesture-controlled interactive experience for healing and occupational therapy at pediatric hospitals.
Hospitalization is known to be an uncomfortable, frustrating, and painful experience, especially for young children who are more susceptible to long-term trauma. SenseScape explores transforming that same experience into playfulness and healing for families and their little ones.
ROLE
Experience Designer, Developer
TYPE OF PROJECT
Senior Capstone project at California College of the Arts
CONTRIBUTION AND IMPACT
3 rounds of concept testing with insights from user interviews and observational research.
1 month of desk research to compose a thorough literature review of the topic area.
Achieved Dean’s Spotlight for exceptional work in the BFA Interaction Design program.
THE STORY BEHIND SENSESCAPE
This was me as a child ↓
and as a child, I got terribly sick 🤒
Some of my earliest memories are those of white walls in hospitals, a sterile environment that turned into bitter memories.
🏫
Skipping out on most of my pre-school years.
MY HOSPITAL STAYS RESULTED IN
💉
Developing a deep-rooted fear of needles.
👪
Unintentionally causing a great deal of emotional and mental stress to my parents.
FURTHERMORE, INSIGHTS FROM A LITERATURE REVIEW REVEAL THAT
“Estimates of the incidence of emotional problems resulting from hospital experience have been reported to be as much as 90% for slight emotional upset in hospitalized children.” (Yap JN, 1988)
Multiple studies reveal that the long-term effects of hospitalization can manifest as
Aggravated symptoms of PTSS
Psychological distress and anxiety
A turbulent spectrum of emotions such as anger, fear, sadness, and confusion.
So, I wanted to turn those bitter memories into moments of delight, minimizing the effects of healthcare trauma and turning them into uplifting spaces for children and families.
INTRODUCING SENSESCAPE
SenseScape is an interactive experience that uses gesture controls to bring playful sea creatures to life amongst the background of a cityscape.
SOME OF ITS KEY FEATURES INCLUDE
1
Vibrant and whimsical character illustrations that keep children engaged and entertained without overwhelming them.
2
Intuitive gesture-driven game mechanics that allow users of all ages and abilities to easily participate and take delight in the experience.
and while most existing solutions provide a positive distraction,
what makes SenseScape unique is
3
A two-fold functionality with interactions that seamlessly turn occupational therapy exercises into an extension of children’s imagination.
ADDRESSING PAIN-POINTS IN THE HOSPITAL EXPERIENCE
SenseScape serves as a spatial intervention that counters pain-points in both the inpatient and outpatient hospital experience.
A study by Sumaya Almaazmi et al., 2023, revealed that the main drivers for patient stress and frustration was the time spent for registration and waiting. My initial solution focused on providing comfort to families during that waiting time.
However, an ethnographic field study of the UCSF Children’s Hospital revealed insights that helped me extend my solution to fit within the inpatient hospitalization experience as well.
PROCESS OVERVIEW
But how did it all begin?
SenseScape was a result of multiple rounds of pivoting, jumping from one potential idea to another. Below is a quick glimpse into 6 months of work.
LITERATURE REVIEW INSIGHTS
A thorough scope of my problem space first began with a deep dive into three major topic areas that overlapped and would eventually shape the approach to my final solution.
PEDIATRIC SPATIAL DESIGN
Designing modifications in pediatric spaces that alleviate emotional stress responses
↓
FAMILY-CENTERED CARE PLANNING
Integrating a FCC approach to planning for shared care-delivery experiences
↓
HOSPITALIZATION TRAUMA
Understanding the adverse effects of hospitalization and the subsequent development of post-traumatic stress syndrome in children
IMPLEMENTING A FAMILY-CENTERED CARE APPROACH
Parents and children as patients have complex psychosocial needs and challenges that form the core of a family-centered solution.
A diversion from everyday routine can cause stress and feelings of isolation. A solution, research introduces, is the importance of intentional spatial design interventions.
WHAT IS GOOD PEDIATRIC SPATIAL DESIGN?
1
Introduce elements of fun and playfulness through playrooms and entertainment systems.
“Spatial modifications that do not resemble “typical hospital associations” have the ability to negate adverse health consequences.” (Shahin Payam et al., 2023)
Multiple studies emphasize the importance of setting architectural intentions that:
2
Afford social interaction between other children and families of patients.
3
Instill an increased sense of personal autonomy and control in a seemingly unfamiliar environment.
From my analysis, I learned that existing pediatric interior spaces architecturally support…
Play and exploration for children.
Familiar routines of living for families, and,
Healing structures for socialization.
BRAINSTORMING POTENTIAL CONCEPTS
Sense Garden was the result of my initial research findings; a gesture-responsive panoramic wall which allows users to control the movement of blooming flowers across changing seasons.
Prototyping preliminary ideas for the interactive experience: The user waves their hand to activate the blooming motion of a flower.
Prototyping preliminary ideas for the interactive experience: The user waves runs across the wall while a bee follows their shadow.
While the concept was mostly well-received, multiple considerations around hardware and the storytelling narrative were overlooked:
Does the concept within the narrative and visual aesthetic of the hospital?
Would a singular sensor-based hardware support hand and body gestures? How does the scale of the experience shape the technology being used?
Can the user represent themselves in this immersive world?
CONTEXTUALIZING THE SOLUTION
A crucial aspect of solidifying the concept was establishing its foundation, its home at the UCSF Benioff Children’s Hospital
An onsite survey of the hospital helped me counter challenges with establishing the concept, its elements, its characters, and its interactions.
A marine motif is prevalent across the interior design of the Pediatric Radiology department on the 1st floor.
Playful characters like whales, colorful fishes, and plant life like seaweed can be seen all throughout the walls.
CONCEPT DEVELOPMENT: PHASE 1
The marine motif became an inspiration for the visual narrative
I collaborated with an illustrator to create basic 2-frame animations with playful sea creatures such as jellyfishes, whales, and starfishes. Preliminary low-fi storyboards were shared to convene on basic interaction flows and ideas were built on from there.
Every character had its own unique animation triggered by an special gestural interaction: a wave, a closed fist, and a pinch.
A conversation with an Occupational Therapist at UCSF Benioff’s provided me with valuable insights into the tools and approaches OT’s utilize when working with children post-treatment,
👨⚕️
“Occupational therapy is geared more towards activities of daily living, called ADLs. A lot of it is hands and fine motor.”
🖐️
My design rationale behind the gestural interactions was based on the nature of fine motor activities and their focus on improving palm and wrist motion.
CONCEPT DEVELOPMENT: PHASE 3
Creating basic interactions with the Leap Motion sensor
I began experimenting with the functionalities of the leap motion sensor, integrating captured sensor data into p5.js to code the visual character interactions.
Initialization of the leap motion sensor and palm detection setup
Exponential increase of the palm detection area after V1 prototype
The constraint of the sensor is its short detection distance.
I modified the code so that the palm detection radius is increased to give users more space for movement and interactivity.
CONCEPT DEVELOPMENT: PHASE 4
Augmentation VS Immersiveness: Insights from concept testing
My design choices were guided by 3 rounds of concept testing moving from photorealistic augmentation from V1 to a fully-illustrated world that allows users to fully immerse themselves with the creatures.
V1: Outlined gesture that traces the shape of the hand
With a stock photo background
V2: Hand signifier that represents the gestural interaction
With an illustrated background
V3: Playful clownfish that follows the hand
With an illustrated background
and after 800 lines of code,
this is what I learned
Design can be fun(ctional)
I struggled to understand the purpose of my project early on because I believed complexity = value. After observing people interact with my prototype did I realize that design that is delightful can also be successful.
Small design choices can have a big impact
Sound played a big role in shaping the immersiveness of the experience. Choosing the right sound FX, even for split-second animations, added to the atmosphere.
But what would I differently?
Experiment with the Kinect motion sensor to create an experience that caters to full-body movement instead of just the hands.
and what’s next for SenseScape?
Add more scenes to build worlds that are based off of different cities. I started with San Francisco but hope to expand the project to the rest of the world.
FINAL THOUGHTS
