Food Depth Estimation Using Low-Cost Mobile-Based System for Real-Time Dietary Assessment
##plugins.themes.bootstrap3.article.sidebar##
##plugins.themes.bootstrap3.article.main##
Abstract
Real time estimation of nutrition intake from regular food items using mobile-based applications could be a break-through in creating public awareness of threats in overeating or faulty food choices. The bottleneck in implementing such systems is to effectively estimate the depths of the food items which is essential to calculate the volumes of foods. Volumes and density of food items can be used to estimate the weights of food eaten and their corresponding nutrition contents. Without specific depth sensors, it is very difficult to estimate the depth of any object from a single camera. Such sensors are equipped only in very advanced and expensive mobile devices. This work investi gates the possibilities of using regular cameras to calculate the same using a specific frame structure . We proposed a controlled camera setup to acquire overlapping images of the food from different positions already calibrated to estimate the depths. The results were compared with the Kinect device's depth measures to show the efficiency of the proposed method. We further investigated the optimum number of camera positions, their corresponding angl es, and distances from the object to propose the best configuration for suchn a controlled system of image acquisition with regular mobile cameras. Overall the proposed method presents a low-cost solution to the depth estimation problem and opens up the possibilities for mobile-based apps for dietary assess ment for various health-related problem-solving.
##plugins.themes.bootstrap3.article.details##
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
Xiaoyong Zhang, Hans Dagevos, Yuna He, Ivo Van der Lans, and Fengying Zhai. Consumption and corpulence in china: a consumer segmentation study based on the food perspective . Food Policy, 33(1):37-47, 2008.
Chang Xu, Ye He, Nitin Khannan, Albert Parra, Carol Boushey, and Edward Delp. Image-based food volume estimation . In Proceedings of the 5th international work shop on Multimedia.for cooking & eating activities, pages 75-80, 2013.
Kaylen J Pfisterer, Robert Amelard, Audrey G Chung, Braeden Symyk, Alexander MacLean, and Alexander Wong. Fully-automatic semantic segmentation for food intake tracking in long-te1m care homes. arXiv preprint arXiv:1910.11250, 2019.
Parisa Pouladzadeh, Abdulsalam Yassine, and Shervin Shirmohammadi. Foodd: food detection dataset for calorie measurement using food images. In International Confer ence on Im age Analysis and Processing, pages 441-448. Springer, 2015.
Shaobo Fang, Chang Liu, Khalid Tahboub, Fengqing Zhu, Edward J Delp, and Carol J Bous hey. ctada: The design of a crowdsourcing tool for online food image identification and segmentation. In 20I8 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI), pages 25-28. IEEE, 2018.
Yu Wang, Ye He, Carol J Boushey, Fengqing Zhu, and Edward J Delp. Context based image analysis with appli cation in dietary assessment and evaluation. Multimedia tools and applications, 77(15):19769-19794, 2018.
Joachim Dehais, Marios Anthimopoulos, and Stavroula Mougiakakou. Food image segmentation for dietary assessment. In Proceedin gs of the 2nd International Workshop on Multimedia Assisted Dietary Management, pages 23-28, 2016.
Rahul Garg, Neal Wadhwa, Sameer Ansari, and Jonathan T Barron. Learning single camera depth esti mation using dual-pixels. In Proceeding s of the IEEE In ternational Conference on Computer Vision, pages 7628- 7637, 2019.
Guoshen Yu and Jean-Michel Morel. A fully affine invariant image comparison method. In 2009 IEEE International Conference on Acoustics, Speech and Signal Processin g, pages 1597-1600. IEEE, 2009.
Rui Hu and John Collomosse. A performance evaluation
of gradient field hog descriptor for sketch based image retrieval. Computer Vision and Imag e Understanding, 117 (7):790-806, 2013.
Shahram Izadi, David Molyneaux, Otmar Hilliges, David Kim, Jamie Daniel Joseph Shotton, Stephen Edward Hodges, David Alexander Butler, Andrew Fitzgibbon, and Pushmeet Kohli. Reducing interferenc e betwe en multiple infra-red depth cameras, January 26 2016. US Patent 9,247,238.
Miles Hansard, Seungkyu Lee, Ouk Choi, and Radu Patrice Horaud. Time-of-flight cameras: principles, methods and applications. Springer Science & Business Media, 2012.
Johannes L Schonberger and Jan-Michael Frahm. Structure-from-motion revisited. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog nition, pages 4104-4113, 2016.
Ravi Garg, Vijay Kumar BG, Gustavo Carneiro, and Ian Reid. Unsupervised cnn for single view depth estimation: Geometry to the rescue. In European Conferen ce on Computer Vision , pages 740-756. Springer, 2016.
Fayao Liu, Chunhua Shen, and Guosheng Lin. Deep convolutional neural fields for depth estimation from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 5162- 5170, 2015.
Anirban Roy and Sinisa Todorovic. Monocular depth estimation using neural regression forest. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 5506-5514, 2016.
Christopher J.C. Burges. A tutorial on support vector ma chines for pattern recognition. 2009. doi: 10.l.1.18.1083.
Gianluigi Ciocca, Paolo Napoletano, and Raimondo Schettini. Food recognition: a new dataset, experiments and results. IEEE Journal of Biomedical and Health Informati cs, 2017. doi: 10.1109/JBHI.2016.2636441.
Yining Deng and B. S. Manjunath. Unsupervised seg mentation of color-texture regions in images and video. Ieee Transactions on Pattern Analysis and Ma chine Intel ligence, 23(8):800-810, 2001. ISSN 19393539, 01628828, 21609292. doi: 10.1109/34.946985.
Fengqing Zhu, Marc Bosch, Tusa Rebecca Schap, Nitin Khanna, David S. Ebert, Carol J. Boushey, and Edward J.
Green University Press 09
Delp. Segmentation assisted food classification for dietary assessment. Proceedings of Spie - the Inte rnational Society for Optical Engineering, 7873(1):78730B, 2011. ISSN 1996756x, 0277786x. doi: 10.1117/12.877036.
Austin Myers, Nick Johnston, Vivek Rathod, Anoop Ko rattikara, Alex Gorban, Nathan Silberman, Sergio Guadar rama, George Papandreou, Jonathan Huang, and Kevin Murphy. Im2calories: Towards an automated mobile vision food diary. Proceedings of the leee International Conference on Computer Vision, 2015:7410503, 1233- 1241, 2015. ISSN 15505499, 23807504. doi: 10.1109/ ICCV.2015.146.
Oscar Beijbom, Neel Joshi, Dan Morris, Scott Saponas, and Siddharth Khullar. Menu-match: Restaurant-specific food logging from images. Proceedings - 2015 leee Winter Conference on Applications of Computer Vision, Wacv 2015, pages 7045971, 844-851, 2015. ISSN 24726737. doi: 10.1109/WACV.2015.117.
Johan AK Suykens and Joos Vandewalle. Least squares support vector machine classifiers. Neural processing letters, 9(3):293-300, 1999.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Syste ms, 2:1097- 1105, 2012. ISSN 10495258.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Ser manet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. Proceedings of the Ieee Com puter Society Conference on Computer Vision and Pattern Recognition, 07-12-:7298594, 1-9, 2015. ISSN 2332564x, 10636919. doi: 10.1109/CVPR.2015.7298594.
Liang-Chieh Chen, George Papandreou, Iasonas Kokki nos, Kevin Murphy, and Alan L. Yuille. Semantic image segmentation with deep convolutional nets and fully con nected crfs. page 14, 2016.
Dario Allegra, Marios Anthimopoulos, Joachim Dehais, Ya Lu, Filippo Stanco, Giovanni Maria Farinella, and Stavroula Mougiakakou. A multimedia database for automatic meal assessment systems. In International Conference on Image Analysis and Processing, pages 471-478. Springer, 2017.
Yang Yu, Kailiang Zhang, Li Yang, and Dongxing Zhang. Fruit detection for strawberry harvesting robot in non structural environment based on mask-rcnn. Computers and Electronics in Agriculture, 163:104846, 2019.
David Eigen and Rob Fergus. Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. 2015.
Giovanni Maria Farinella, Dario Allegra, and Filippo Stanco. A benchmark dataset to study the representation of food images. Lecture Notes in Computer Science (including Subseries Lecture Notes in Artificial Intelli gence and Lecture Notes in Bioinformatics), 8927:584- 599, 2015. ISSN 16113349, 03029743. doi: 10.1007/ 978-3-319-16199-0_41.
Giovanni Maria Farinella, Dario Allegra, Marco Molti santi, Filippo Stanco, and Sebastiano Battiato. Retrieval
and classification of food images. Computers in Biol ogy and Medicine, 77:23-39, 2016. ISSN 18790534,
doi: 10.1016/j.compbiomed.2016.07.006.
Gianluigi Ciocca, Paolo Napoletano, and Raimondo Schettini. Leaming cnn-based features for retrieval of food images. Lecture Notes in Computer Science ( including Subseries Lecture Notes in Artificial Intelli gence and Lecture Notes in Bioinformatics), 10590:426- 434, 2017. ISSN 16113349, 03029743. doi: 10.1007/ 978-3-319- 70742-6_41.
Jhacson Meza, Andres G Marrugo, Enrique Sierra, Mil ton Guerrero, Jaime Meneses, and Lenny A Romero. A structure-from-motion pipeline for topographic recon structions using unmanned aerial vehicles and open source software. In Colombian Conference on Computing, pages 213-225. Springer, 2018.
