{"id":1400,"date":"2019-06-07T12:03:35","date_gmt":"2019-06-07T10:03:35","guid":{"rendered":"https:\/\/genr.interpunct.dev\/?p=1400"},"modified":"2021-02-09T18:39:00","modified_gmt":"2021-02-09T16:39:00","slug":"interview-with-flora-incognita-innovation-in-citizen-science-using-machine-learning","status":"publish","type":"post","link":"https:\/\/genr.interpunct.dev\/interview-with-flora-incognita-innovation-in-citizen-science-using-machine-learning\/","title":{"rendered":"Interview with Flora Incognita: Innovation in Citizen Science Using Machine Learning"},"content":{"rendered":"\n
Images: All images courtesy Flora Incognita https:\/\/floraincognita.com\/de\/pressemappe\/<\/a> <\/p>\n\n\n\n Authors<\/strong> DOI <\/p>10.25815\/xp2t-w456<\/a> <\/summary> Citation format: The Chicago Manual of Style, 17th Edition<\/em> An interdisciplinary team has come up with a mobile app for identifying plants based on users taking a photo of the plant on their mobile. The Flora Incognita app applies machine learning to identify plant species in near real-time \u2014 flowers, plants, and trees. Simplicity and Innovation are both hard to accomplish but this is where Flora Incognita has excelled and to achieve both deserves a mention. Currently the app suite works with flora in the German Central European region, based on 4,800 species, using 1.7 million images, with a 100,000 images coming from users in 2018 alone. For Citizen Science the enthusiastic engagement of the public with Flora Incognita shows a clear path forward for more widespread uses of machine learning in public participation with science and scholarship, and in knowledge creation. <\/strong><\/p>\n\n\n\n\n\n\n\n Image: Screenshots of Flora Incognita app<\/p>\n\n\n\n Download<\/a> Download<\/a> GenR:<\/em> Would you like to give a short introduction to yourselves and your roles in the Flora Incognita research?<\/strong><\/p>\n\n\n\n Jana W\u00e4ldchen<\/strong>: I am leading the biological\npart of the Flora Incognita project at the Max Planck Institute for Biogeochemistry,\nJena<\/a> (BGC). I studied Landscape Management and Nature Conservation and did\nmy PhD in ecology. Since my studies, I have been aware that accurate species identification is an important component of\nworkflows in ecological research. Many activities, such as studying the\nbiodiversity richness of a region, monitoring populations of endangered\nspecies, determining the impact of climate change on species distribution, and\nweed control actions depend on accurate identification skills. These activities\nare a necessity for: farmers, foresters, taxonomists, conservation biologists,\ntechnical personnel of environmental agencies, or just fun for laypersons.\nAutomating the task and making it feasible\nfor non-experts is highly desirable, especially considering the continuous loss\nof biodiversity \u2014 and ironically of taxonomists to monitor biodiversity.<\/p>\n\n\n\n Patrick\nM\u00e4der:<\/strong> I\u2019m a professor in\nsoftware engineering and data-intensive systems at the Technical University of\nIlmenau<\/a>. Automating and simplifying processes has always been a major objective\nof my work, initially as consultant and developer in industrial projects, later\nin developing methods for safer software development, and today in developing\nmethods for interactive classification processes. Recent boosts in data\navailability, new and improved computer hardware, accompanied by substantial\nprogress in machine learning algorithms pushed automated, image-based species\nidentification into reality. Jana and I had the idea together in 2011. We wrote\na project proposal and started the Flora Incognita project, which I coordinate at\nthe TU Ilmenau, in 2014. Right from the start, this was a very\ninterdisciplinary project in which computer scientists and biologists worked\ntogether.<\/p>\n\n\n\n What can people do with the Flora-APPS suite of mobile apps?<\/strong><\/p>\n\n\n\n Jana: <\/strong>The goal of the Flora\nIncognita project is developing a semi-automated plant identification tool for\nmobile devices. To achieve this goal we have developed three different apps (two\nare publicly available), which people can freely download from the Google Play\nStore and the iOS AppStore. With the Flora Capture\nApp<\/a> (Flora Incognita n.d.) users can capture observations of wild plants containing\nimages from different perspectives. Botanists at the MPI-BGC<\/a> are validating and identifying\nthese observations. The user\nreceives feedback about the identified plant species directly on their device. This\nmeans, the user gets a species identification from a human expert botanist in\nexchange of high-quality images depicting this plant. A steadily growing number of interested people support our\nresearch by capturing such observations.<\/p>\n\n\n\n Patrick: <\/strong>In return, we as computer\nscientists get a lot of structured plant images through the Flora Capture App,\nwhich are necessary for training the neuronal networks used for the automatic\nspecies identification in our second app, the Flora Incognita App<\/a>\n(Flora Incognita n.d.). The Flora Incognita App allows\nfor identifying plants automatically. The identification process adapts to the\nsituation meaning that depending on the unknown plant\u2019s life form (herb, tree,\ngrass, or fern) and the current date, we ask the user to take an initial image\nof a certain organ of the plant. If this image allows for an accurate\nidentification, the process completes otherwise we follow up and ask for an\nadditional image of the plant and so on. Underneath, we utilize a sophisticated cascade of latest deep neural\nnetworks and many additional data science technologies.<\/p>\n\n\n\n How are Flora-APPS being used in Citizen Science projects? For example do\npeople just use it as individuals or as part of an organized project like with\na club, or nature project?<\/strong><\/p>\n\n\n\n Jana: <\/strong>At the moment,people use our\napps mostly for their individual purposes. The Flora Incognita App is currently\nused by a wide range of people, e.g., pupils, students, parents teaching their\nchildren, laypersons, and even expert ecologists and botanists. We found that Flora Capture user\nare often people who want to combine their hobby of photography with getting a\nbetter understanding of nature. However, there are also a growing number of\ninitiatives and organizations that use or suggest our apps for student camps,\nconservation initiatives aiming to protect certain areas, as part of guided\nwalks through nature and so on.<\/p>\n\n\n\n What was the germ of the Flora Incognita research and how did your two\ninstitutions come to work together on the project?<\/strong><\/p>\n\n\n\n Jana: <\/strong>Actually it was really simple. In 2011, a biologist asked a computer\nscientist if it could be possible to automate plant identification by\ndeveloping a smartphone app.<\/p>\n\n\n\n Patrick: <\/strong>I was immediately enthusiastic\nabout the idea and we worked together on a project proposal. With the joint\nfunding program of the German Federal Ministry of Education and Research (BMBF<\/a>) and German Federal Ministry for\nthe Environment, Nature Conservation and Nuclear Safety (BMU<\/a>) in 2012 for the implementation of the National\nStrategy on Biodiversity<\/a> (BMU\n2007) we had a\nsuitable opportunity to submit our research idea. Especially, the combination\nof science and implementation makes this project so successful.<\/p>\n\n\n\n What has been the timeline of the development of the research, significant\nsteps, and how has the uptake of the app suite by the public developed?<\/strong><\/p>\n\n\n\n Patrick:<\/strong> At the beginning we spent a lot of time developing and filling a data\nrepository. In fact, we collected more than one million images from expert\ncollections, individuals and through Flora Capture to train the neural networks\nand developed methods to predict which species can occur at a certain position\nand at a certain time. And last but not least a considerable part of the work went\ninto the multi-platform app development. <\/p>\n\n\n\n Can you tell us about the technology being used in the research, how you\nbrought the different parts together, and what insights and innovations have\nyou made over the course of the development?<\/strong><\/p>\n\n\n\n Patrick: <\/strong>Latest machine learning methods are\nused for analysis of multimodal data. Images are classified using deep neural\nnetworks. For this purpose, a machine-optimized network architecture consisting\nof more than ninety million parameters is used, which was trained by \u201cdeep\nlearning\u201d algorithms on a very rich dataset of already more than 1.7 million\nplant images. The location of each observation is analyzed in terms of\nenvironmental, geographic and climatological characteristics and also used for\nthe identification purpose. Similarly, the current date is used for predicting regionally\nspecific observation periods and weather-dependent flowering seasons. Using date and location factors, such\nas soil type, slope, and average temperature, a probability model allows us to\ninfer plant species that are likely to occur naturally at the current location.\nThe results of the species prediction is then fused with the result of the image\nanalysis. On a SINGLE image without any of the other information sources, like\nlocation and time, we currently reach an average identification accuracy of\n86.5% for the 4,800 supported species including all 2,770 naturally occurring German plant\nspecies. In 95.7% of the cases, the correct species is among the first five\nrecognized plants. Using all available data and multiple images, our actual accuracy\nin the app is substantially higher.<\/p>\n\n\n\n How have you designed your citizen science programme of research: how did\nyou arrive at deciding on a app suite to bring the public into contact with\nyour subject area; what type of UX, UI, or design research methods and\nprocesses have been put together?<\/strong><\/p>\n\n\n\n Jana:<\/strong> Originally, the computer scientists in Ilmenau designed and implemented\nthe Flora Capture for the field recordings of the biologists in Jena. After\nusing the Flora Capture App 2016 exclusively for the project team, we decided\nthat we would get many more pictures if we would make it publicly available. The\nmore images the better the recognition that can later be provided by the Flora\nIncognita App. To be honest we were surprised about so many observations. In\n2018, citizen scientists collected more than one hundred thousand images\nthrough the Flora Capture App. In 2019, the community of helpers further\nincreased and we receive many Flora Capture observations every day.<\/p>\n\n\n\n Patrick<\/strong>: Our interdisciplinary team has among others a substantial background in\nsoftware engineering and machine learning. From the beginning, we employ latest\ndesign principles like agile development, continuous integration (CI),\ntest-driven-development. We use a CI setup where each major code change results\nautomatically in a new build of the changed apps, which are then automatically\ndistributed to a closed group of testers continuously evaluating the latest app\nversions and reporting issues. UX and UI are very important to us. We conducted\nseveral studies with users to identify requested features and especially on how\nthey interact with the app. We try to continuously improve UX, but we also\nlearned that different users sometimes completely disagree what is a good or\nbad UX and that it is almost impossible to satisfy everybody.<\/p>\n\n\n\n You are combining existing taxonomies and methods from your area of the\nstudy of flora with new technologies of machine learning and I assume a complex\ntechnology stack. Can you tell us about the experiences of marrying these two\nknowledge areas and skill sets?<\/strong><\/p>\n\n\n\n Patrick:<\/strong> Automated plant species identification is a topic mostly driven by\nacademics specialized in computer vision, machine learning, and multimedia\ninformation retrieval so far. Only a few studies have been conducted by\ninterdisciplinary groups of biologist and computer scientists during the last\ntwo decades. Increasingly, research is moving towards more interdisciplinary\nendeavors. As our project shows, effective collaboration between people from\ndifferent disciplines and backgrounds is necessary to gain the benefits of\njoined research activities and to develop widely accepted approaches.<\/p>\n\n\n\n What are your views on the value and uses of Citizen Science as researchers\nwho have embraced its use and as practitioners of Citizen Science? And what do\nyou think other researchers can learn from you in developing their Citizen\nScience projects in other disciplines?<\/strong><\/p>\n\n\n\n Jana<\/strong>: For me as a scientist it is\na new and inspiring experience to work in a research project with a large\nnumber of volunteers willing to support our work. I have learned that there are\nmany citizens who want to participate actively and who see themselves as\ncitizen scientist also as part of our project. We invest a lot of resources\ninto communication with these people through social media, via e-mails but also\nreceive many phone calls of interested people. I think it is important to be\napproachable and also to share the scientific knowledge that results from the\nproject with the people. Furthermore, it is very rewarding how people use and\noften love technology that we invented; or to give a presentation and have an\naudience that is prepared with many concrete questions and is interested in\nevery new development step.<\/p>\n\n\n\n What are your future plans for Flora Incognita?<\/strong><\/p>\n\n\n\n We are continuously working on\nimproving the apps and especially the underlying machine intelligence services.\nIn the near future, this will include features for organizing and working with\nobservations, adding more species and making the whole process even more\nintelligent and intuitive. We would like Flora Incognita to become a standard\ntool for the identification of plants. This will not only need more technical\nadvances but also a stable community of users that support our work with Flora\nCapture observations and contributions on species information.<\/p>\n\n\n\n Flora Incognita. Flora Capture App<\/em>. Accessed 6 June 2019. https:\/\/floraincognita.com\/de\/apps\/plant-image-capture\/<\/a>.<\/p>\n\n\n\n Flora Incognita<\/em>. Accessed 6 June 2019. https:\/\/floraincognita.com\/apps\/flora-incognita-app\/<\/a>.<\/p>\n\n\n\n
Jana W\u00e4ldchen ORCiD 0000-0002-2631-1531<\/a>
Patrick M\u00e4der ORCiD 0000-0001-6871-2707<\/a><\/p>\n\n\n\n Cite as:
W\u00e4ldchen, Jana & M\u00e4der, Patrick. \u2018Interview with Flora Incognita: Innovation in Citizen Science Using Machine Learning\u2019, 2019. https:\/\/doi.org\/10.25815\/xp2t-w456<\/a>.<\/p>\n<\/div><\/details><\/div>\n\n\n\n
\n\n\n\n
<\/figure><\/li>
<\/figure><\/li>
<\/figure><\/li><\/ul><\/figure>\n\n\n\n
Flora Incognita
for iOS and Android<\/p>\n<\/div>\n\n\n\n
Flora Capture
for iOS and Android <\/p>\n<\/div>\n<\/div>\n\n\n\n
<\/figure><\/li>
<\/figure><\/li>
<\/figure><\/li>
<\/figure><\/li><\/ul><\/figure>\n\n\n\nReferences<\/h2>\n\n\n\n