Organic waste management is a crucial component of a circular economy, which prioritizes reducing waste through the reuse and recycling of products and materials. It is a tedious and complicated task, largely accomplished through manual labor. We introduce a novel ‘in-the-wild’ multimodal image dataset of 15-band NIR multi-spectral and single band thermal images of bulk food waste in an industrial setting. The dataset showcases a number of complex computer vision problems that are unavoidable constraints in this setting. Benchmarking against different computer vision algorithms is performed to highlight these challenges. The key issues and their place in robotic waste processing for industrial applications, and grand challenge objectives are discussed.
Meet
James Ireland
Research Student
Dr James Ireland is a recent PhD graduate researching the fields of Computer Vision, Computer Graphics, and Artificial Intelligence; specifically focusing on Small Object Instance Counting, Synthetic Data Generation, Multiple View Geometry and Domain Adaptation.
His thesis, entitled ‘Computer-vision based instance counting of surgical instruments’ focus on human error prevention during surgical operations by assisting in ensuring all medical devices and materials are accounted for post-procedure. Highly motivated by research that has applications in the real-world and for the betterment of either mental or physical health outcomes. Further information and links to datasets can be found on his website (https://james-ireland.github.io/).
James's Publications
Counting is a common preventative measure taken to ensure surgical instruments are not retained during surgery, which could cause serious detrimental effects including chronic pain and sepsis. A hybrid human-AI system could support or partially automate this manual counting of instruments. An important element to evaluate the viability of using deep learning computer vision-based counting is a suitable large-scale dataset of surgical instruments. Other domains, such as crowd analysis and instance counting, have leveraged synthetic datasets to evaluate and augment different approaches. We present a synthetic dataset (SORT), which is complemented by a smaller real-world dataset of surgical instruments (MSMI), to assess the hypothesis of whether synthetic training data can improve the performance of multi-class multi-instance counting models when applied to real-world data. In this preliminary study, we provide comparative baselines for various popular counting techniques on synthetic data, such as direct regression, segmentation, localisation, and density estimation. These experiments are repeated at different resolutions – full high-definition (1080×1920 pixels), half (690×540 pixels), and a quarter (480×270 pixels) – to measure the robustness of different supervision methods to varying image scales. The results indicate that neither the degree of supervision nor the image resolution during model training impact performance significantly on the synthetic data. However, when testing on the real-world instrument dataset, the models trained on synthetic data were significantly less accurate. These results indicate a need for further work in either the refinement of the synthetic depictions or fine-tuning upon real-world data to achieve similar performance in domain adaptation scenarios compared to training and testing solely on the synthetic data.
Researching human-robot interaction “in the wild” can sometimes require insight from different fields. Experiments that involve collaborative tasks are valuable opportunities for studying HRI and developing new tools. The following describes a framework for an “in the wild” experiment situated in a public museum that involved a Wizard of OZ (WOZ) controlled robot. The UR10 is a non-humanoid collaborative robot arm and was programmed to engage in a collaborative drawing task. The purpose of this study was to evaluate how movement by a non-humanoid robot could affect participant experience. While the current framework is designed for this particular task, the control architecture could be built upon to provide a base for various collaborative studies.