Dumbo
-honors academy project-
Description
"Dumbo" is a one-semester project I did in addition to my degree. The project is part of the Honors Academy, an excellence program for students to prepare for personal leadership as well as scientific, societal and/or industrial leadership in a society that is affected exponentially by changes and developments. "Dumbo" is part of the Professional Development. In addition, I also did a course on Personal Leadership. This project was optional and is still in development. The project is conducted in collaboration with the University of Utrecht, department of Computer Science, faculty of Artificial Intelligence. The result is a paper which has been submitted to the 2020 CHI conference in the "case study" section and at EICS 2020.
Dumbo is a an experimental device which labels human activities in the home by analyzing ambient sound. It does this with the help of Machine Learning (namely, feature extraction). What it aims to do is adapt to the different sounds of different households (different acoustics, different cultures, different routines) by retraining with the help of user feedback - the user can tell Dumbo whether the detection is right or not, and can provide new activity labels. It is trained on the SINS dataset to "get a hold" of what home ambient sound is. Then, transfer learning is applied with new samples from different households. The improved detection rate proves the possibility that Dumbo can be adaptable in real households. Dumbo is designed with the IoT in mind aiming at using the labels in improving the user experience in a smart home.
The interface is divided in two parts: the physical Dumbo, and the mobile application. The physical Dumbo works with a Raspberry Pi 3 and a microphone. The microphone records sounds and the software running on the Pi classifies them. then, the classification is sent to the mobile phone. The user can ask Dumbo what activity it hears at the moment and make a decision whether the labeling is correct. Within the app, the user can playback mislabeled sounds, rename or add labels and review the list of existing labels. Please check the Gallery for more pictures and app screens.
microphone
1½ cups
Butter
sliding handles
LEDs
speaker
power supply (Raspberry Pi + LEDs)
magnetic switch to detect when lid slid back
when open, speaker announces what activity is detected atm
soft buttons for decision making (fabric layered on top of conductive textile connected to capacitive sensor)
harsh laser engraved velt for "deny"
soft suede for "accept"
breadboard
Raspberry Pi
hover me!
why?
For me, The Honors Academy program allowed me to make decisions about my own education freely and decide for myself in which area of expertise (design-wise) I want to experiment. It was an opportunity to grow and experience a hands-on approach to design that I would have probably not gotten in my Masters. All the projects in the ID department have a degree of freedom to a certain extent which is limited more and more depending on the people who are coaching you.
The Honors also opened a new door for a new way of organizing my life and my thoughts through the Personal Leadership course. In a way, it proved that anything is possible but it all depends on myself to make it happen.
I consider that the satisfaction at the end comes from the fact that have spent extra time on learning things I truly would have liked to learn, did not get the chance to during my Masters, but were only possible in an educational environment.
I wanted step out of my comfort zone by working with technology and paying attention to creativity and aesthetics this time. In my thesis, I had to outsource the software and I have been lacking practice in this area that I would not like to lose contact with. Moreover, because of budget issues and the fact that the product was only digital, I could not focus on aesthetic aspects too much. After attending Dr. Funk’s elective called “Designing Intelligence in Interaction“, I have been interested in working with him on one of his projects which revolves around designing for products which use Artificial Intelligence to the benefit of the user experience. It was a way for me to branch out and work in a different area (UX design for intelligent products) to see whether this would be something I could pursue in a future job
I worked with two Artificial Intelligence Master students from the University of Utrecht: Shiqi Bai and Albert Mwanjesa. Since we were experts in our own areas, we helped each other and exchanged knowledge all the way during the project.
responsibilities
project manager
manage meetings, keep track of progress, document the process, make sure other teammates document their problems and advancements, bring everybody on the same page when things went crazy
designer
design the user interfaces (both physical and digital), design the user experience, the functionalities, how the back-end should be structured in harmony with the front-end, decide what pieces of hardware are needed and purchase them, integrate the hardware according to the concept, test the hardware, put the hardware together inside the physical device, create the blueprints of the physical device and laser cut the pieces, assemble the case and design the inside partitioning of the hardware
learning
This project has contributed the most to my technological and creative skills. However, I also improved on how to work with individuals with a more technical background than me and help them in the research process. At the beginning, I lead the process quite frequently by showing them the way through the ideation and exploration phases, but also showing them how to document their development steps. Since I had more experience in research processes, I showed them how to define their research challenge and helped them with their academic writing. As the project went on, I started having difficulties with my own tasks and, therefore, started having more of a collaborative leading method rather than me leading them.
I also realized that it is important to assess the complexity of the task beforehand, in order to perform the best time-management possible.
All in all, I do feel like the final result, even though not validated in the right manner, is a successful one considering the size of the initial task. I do feel like we managed to contribute to the world of technology and design. Moreover, I am really happy with all the things I learned during this semester. It has definitely not been easy, sometimes spending hours on a problem that seemed like it would never get solved. At the end of the day, I managed to keep it realistic and prioritize the design activities.
photo gallery