Choreoassist
Description
"ChoreoAssist" was created during the course "Designing Intelligence in Interaction" during my second year of Masters in Industrial Design at the TU/e.
Intelligent systems are becoming increasingly important in the area of sports and can support athletes in all kinds of disciplines. ChoreoAssist is developed to support dance teachers and choreographers in their work. To prevent them from being distracted when developing their dances, the software helps them document their progress.
ChoreoAssist uses a Kinect device to be able to detect the dancers poses and identify their names. The identified poses are then placed on a timeline by ChoreoAssist. Additionally, these recordings and the documented poses can later be reviewed on the interface and used for later adjustments. When the choreography is completed, it can be uploaded on a database to be reviewed by others. By introducing a universal way of documentation and combining a video with a timeline, it can be easily understood and used by third parties. The detection is done by analyzing joint coordinates and classifying them with supervised learning. Four classification algorithms were used in training and testing the system: Neural Networks (Multilayered Perceptron), Decision Trees (Random Forest and Gradient Tree Boost) and Support Vector Machines.
ChoreoAssist was a team project with Friederike Ruemelin, Magdalena Breitinger, Guo-Ling Tsai. My responsibilities and implications in the project were:
responsibilities
programmer
data scientist
designer
writing all the code: from gathering the training set, to the code for the training and the classification
figuring out what kind of data needed to be gathered, being involved in the data collection process, tweaking with the data sets to decide which data is "good" and which is not
included in the ideation process, finding a concept that does need intelligence in order to be useful to the user, integrating the sensor into the design
learning
I have previously worked with Kinect twice (once with body detection - POZE, once with face detection - RowBot). However, in both those projects I focused on translating the data from the Kinect into an aesthetical user interface with good feedback and creating a fun and engaging user experience. With ChoreoAssist, I learned how to use machine learning not only in theory, but also in practice on the data Kinect produces.
ChoreoAssist was an exciting learning experience for me. This is the first time I have implemented supervised learning algorithms onto real data sets coming from real sensors. I enjoyed coding, and gathering the data, and figuring out why the first data set failed to be a good one, and trying different software to train and test the algorithms. Most of all, I was really excited to see that (after many tries and fails) the algorithms actually produced very good results and that body detection with Kinect is pretty accurate for the kind of sensor it is.
If you wish to read more about the data collection, the classification and the design process, please check this document:
photo gallery