Dumbo

-honors academy project-

WhatsApp Image 2019-06-19 at 13.26.11.jp

Description

 

"Dumbo" is a one-semester project I did in addition to my degree. The project is part of the Honors Academy, an excellence program for students to prepare for personal leadership as well as scientific, societal and/or industrial leadership in a society that is affected exponentially by changes and developments. "Dumbo" is part of the Professional Development. In addition, I also did a course on Personal Leadership. This project was optional and is still in development. The project is conducted in collaboration with the University of Utrecht, department of Computer Science, faculty of Artificial Intelligence. The result is a paper which has been submitted to the 2020 CHI conference in the "case study" section and at EICS 2020.  

Dumbo is a an experimental device which labels human activities in the home by analyzing ambient sound. It does this with the help of Machine Learning (namely, feature extraction). What it aims to do is adapt to the different sounds of different households (different acoustics, different cultures, different routines) by retraining with the help of user feedback - the user can tell Dumbo whether the detection is right or not, and can provide new activity labels. It is trained on the SINS dataset to "get a hold" of what home ambient sound is. Then, transfer learning is applied with new samples from different households. The improved detection rate proves the possibility that Dumbo can be adaptable in real households. Dumbo is designed with the IoT in mind aiming at using the labels in improving the user experience in a smart home. 

The interface is divided in two parts: the physical Dumbo, and the mobile application. The physical Dumbo works with a Raspberry Pi 3 and a microphone. The microphone records sounds and the software running on the Pi classifies them. then, the classification is sent to the mobile phone. The user can ask Dumbo what activity it hears at the moment and make a decision whether the labeling is correct. Within the app, the user can playback mislabeled sounds, rename or add labels and review the list of existing labels. Please check the Gallery for more pictures and app screens.

microphone

1½ cups

Butter

sliding handles

LEDs

speaker

power supply (Raspberry Pi + LEDs)

magnetic switch to detect when lid slid back

when open, speaker announces what activity is detected atm

soft buttons for decision making (fabric layered on top of conductive textile connected to capacitive  sensor)

harsh laser engraved velt for "deny"

soft suede for "accept"

breadboard

Raspberry Pi

hover me!

why?

For me,  The Honors Academy program allowed me to make decisions about my own education freely and decide for myself in which area of expertise (design-wise) I want to experiment. It was an opportunity to grow and experience a hands-on approach to design that I would have probably not gotten in my Masters. All the projects in the ID department have a degree of freedom to a certain extent which is limited more and more depending on the people who are coaching you. 
The Honors also opened a new door for a new way of organizing my life and my thoughts through the Personal Leadership course. In a way, it proved that anything is possible but it all depends on myself to make it happen. 
I consider that the satisfaction at the end comes from the fact that have spent extra time on learning things I truly would have liked to learn, did not get the chance to during my Masters, but were only possible in an educational environment.

I wanted step out of my comfort zone by working with technology and paying attention to creativity and aesthetics this time. In my thesis, I had to outsource the software and I have been lacking practice in this area that I would not like to lose contact with. Moreover, because of budget issues and the fact that the product was only digital, I could not focus on aesthetic aspects too much. After attending Dr. Funk’s elective called “Designing Intelligence in Interaction“, I have been interested in working with him on one of his projects which revolves around designing for products which use Artificial Intelligence to the benefit of the user experience. It was a way for me to branch out and work in a different area (UX design for intelligent products) to see whether this would be something I could pursue in a future job

 

I worked with two Artificial Intelligence Master students from the University of Utrecht: Shiqi Bai and Albert Mwanjesa. Since we were experts in our own areas, we helped each other and exchanged knowledge all the way during the project. 

responsibilities

project manager

manage meetings, keep track of progress, document the process, make sure other teammates document their problems and advancements, bring everybody on the same page when things went crazy

designer

design the user interfaces (both physical and digital), design the user experience, the functionalities, how the back-end should be structured in harmony with the front-end, decide what pieces of hardware are needed and purchase them, integrate the hardware according to the concept, test the hardware, put the hardware together inside the physical device, create the blueprints of the physical device and laser cut the pieces, assemble the case and design the inside partitioning of the hardware

learning

This project has contributed the most to my technological and creative skills. However, I also improved on how to work with individuals with a more technical background than me and help them in the research process. At the beginning, I lead the process quite frequently by showing them the way through the ideation and exploration phases, but also showing them how to document their development steps. Since I had more experience in research processes, I showed them how to define their research challenge and helped them with their academic writing. As the project went on, I started having difficulties with my own tasks and, therefore, started having more of a collaborative leading method rather than me leading them. 
 

I also realized that it is important to assess the complexity of the task beforehand, in order to perform the best time-management possible. 


All in all, I do feel like the final result, even though not validated in the right manner, is a successful one considering the size of the initial task. I do feel like we managed to contribute to the world of technology and design. Moreover, I am really happy with all the things I learned during this semester. It has definitely not been easy, sometimes spending hours on a problem that seemed like it would never get solved. At the end of the day, I managed to keep it realistic and prioritize the design activities. 

 

creative & aesthetic
skills

The first learning goal consisted of designing  a Graphical User Interface and a Physical User Interface for an autonomous, intelligent product which adapts to the behaviour of the user. This goal has been completed, by integrating “Creativity and Aesthetics” in the design method. I started with an ideation process from functionality to form by first researching the design opportunity and some similar products existing on the market. Afterwards, with the help of content strategy, I divided the concept into the physical one and the digital one based on the functions that the system had. For the design of the PUI (physical UI), I integrated different rich interaction techniques I learned during the course “A Designerly Perspective on IoT” given by Dr. Frens (see UClean), assistant professor of the Industrial design Department, such as making mode-switching physical (creating layers for modes-of-use to make it less confusing), or principles from calm technology. Afterwards, I used surveys and a focus group to narrow down from a pool of design possibilities to one ideal design, touching upon “User and Society”. However, hardware limitations came into the picture and I managed to think on my feet and compromise with the design accordingly. Moreover, I also used knowledge generated from "Black Textures", my research project regarding all-black soft interfaces by integrating smart black textiles into the prototype. For the GUI, I used simple design principles I “carried” from the human technology interaction courses I learned during my Bachelors. 

photo gallery

Dumbo app different screens
Dumbo Front sketch
Dumbo front physical prototype
Dumbo front physical prototype
Dumbo back sketch
Dumbo back physical prototype
Dumbo side sketch
Dumbo side physical prototype
Dumbo side open physical prototype
Dumbo side open sketch
Dumbo side open physical prototype
Dumbo soft buttons sketch
Dumbo soft buttons physical