What is ml5?
ml5.js, spearheaded by New York University’s Associate Professor, Daniel Shiffman, stands as a collaborative effort within the NYU Interactive Telecommunications Program (ITP) community. It emerged from a collective vision to democratize machine learning, making it more accessible and inclusive for artists, creative coders, students, and beyond.
Built on TensorFlow.js, ml5.js simplifies the utilization of machine learning models in web browsers, aiming at reducing the learning curve often faced by beginner programmers and artists. The ml5 library streamlines the process, eliminating the need for boilerplate code and intricate configurations. It also seamlessly integrates with Processing Foundations JavaScript graphics library, p5.js, enabling a spectrum of creative possibilities.
My Contributions
My journey with ml5.js started in the spring of 2023. When I joined the team, the library was undermaintained for some time, and many of its features have become outdated amidst the rapid evolution of machine learning technology. Our team embarked on a journey to rebuild the library from the ground up, introducing new and improved features.
Within our small team, I wore multiple hats — designing the API, writing documentation, implementing new features, addressing bugs, and optimizing performance. Notably, I adapted pretrained models such as TensorFlow's Hand Pose Detection and Image Segmentation, enhancing the library's functionality.
The Future
Since my involvement with ml5, we made an alpha release of the revamped library. The updated machine learning models deliver faster and more accurate inferences. Our sights are set on achieving a full public release of the updated library. Beyond coding, I am committed to making machine learning more accessible and approachable for everyone.