# Fileset

[Uslu_2024_Mach._Learn.-_Sci._Technol._5_015027.pdf](https://mdr.nims.go.jp/filesets/f41b74e7-cc68-4ce3-843f-5b6e00631c3a/download)

## Creator

Jan-Lucas Uslu, Taoufiq Ouaj, David Tebbe, Alexey Nekrasov, Jo Henri Bertram, Marc Schütte, [Kenji Watanabe](https://orcid.org/0000-0003-3701-8119), [Takashi Taniguchi](https://orcid.org/0000-0002-1467-3105), Bernd Beschoten, Lutz Waldecker, Christoph Stampfer

## Rights

[Creative Commons BY Attribution 4.0 International](https://creativecommons.org/licenses/by/4.0/)

## Other metadata

[An open-source robust machine learning platform for real-time detection and classification of 2D material flakes](https://mdr.nims.go.jp/datasets/5d818258-3ecf-46ee-855a-44db72ab22da)

## Fulltext

An open-source robust machine learning platform for real-time detection and classification of 2D material flakesMachine Learning: Science andTechnology     PAPER • OPEN ACCESSAn open-source robust machine learning platformfor real-time detection and classification of 2Dmaterial flakesTo cite this article: Jan-Lucas Uslu et al 2024 Mach. Learn.: Sci. Technol. 5 015027 View the article online for updates and enhancements.You may also likeGeographic variation in projected USforest aboveground carbon responses toclimate change and atmosphericdepositionAspen Reese, Christopher M. Clark,Jennifer Phelan et al.-Satellite data for environmental justice: ascoping review of the literature in theUnited StatesTanya Kreutzer Sayyed, UfuomaOvienmhada, Mitra Kashani et al.-Survey of X-Rays from Massive StarsObserved at High Spectral Resolution withChandraPragati Pradhan, David P. Huenemoerder,Richard Ignace et al.-This content was downloaded from IP address 144.213.253.16 on 17/02/2024 at 07:42https://doi.org/10.1088/2632-2153/ad2287/article/10.1088/1748-9326/ad2739/article/10.1088/1748-9326/ad2739/article/10.1088/1748-9326/ad2739/article/10.1088/1748-9326/ad2739/article/10.1088/1748-9326/ad1fa4/article/10.1088/1748-9326/ad1fa4/article/10.1088/1748-9326/ad1fa4/article/10.3847/1538-4357/ace9d6/article/10.3847/1538-4357/ace9d6/article/10.3847/1538-4357/ace9d6Mach. Learn.: Sci. Technol. 5 (2024) 015027 https://doi.org/10.1088/2632-2153/ad2287OPEN ACCESSRECEIVED15 August 2023REVISED29 December 2023ACCEPTED FOR PUBLICATION25 January 2024PUBLISHED13 February 2024Original Content fromthis work may be usedunder the terms of theCreative CommonsAttribution 4.0 licence.Any further distributionof this work mustmaintain attribution tothe author(s) and the titleof the work, journalcitation and DOI.PAPERAn open-source robust machine learning platform for real-timedetection and classification of 2D material flakesJan-Lucas Uslu1,∗, Taoufiq Ouaj1, David Tebbe1, Alexey Nekrasov2, Jo Henri Bertram1,Marc Schütte1, Kenji Watanabe3, Takashi Taniguchi4, Bernd Beschoten1,5, Lutz Waldecker1,∗and Christoph Stampfer1,61 2nd Institute of Physics and JARA-FIT, RWTH Aachen University, 52074 Aachen, Germany2 Visual Computing Institute, RWTH Aachen University, 52074 Aachen, Germany3 Research Center for Electronic and Optical Materials, National Institute for Materials Science, 1-1 Namiki, Tsukuba 305-0044, Japan4 Research Center for Materials Nanoarchitectonics, National Institute for Materials Science, 1-1 Namiki, Tsukuba 305-0044, Japan5 JARA-FIT Institute for Quantum Information, Forschungszentrum Jülich GmbH and RWTH Aachen University, 52074 Aachen,Germany6 Peter Grünberg Institute (PGI-9) Forschungszentrum Jülich, 52425 Jülich, Germany∗ Authors to whom any correspondence should be addressed.E-mail: jan-lucas.uslu@rwth-aachen.de andwaldecker@physik.rwth-aachen.deKeywords: 2D materials, machine learning, open source, condensed matter physics, Gaussian mixture modelAbstractThe most widely used method for obtaining high-quality two-dimensional (2D) materials isthrough mechanical exfoliation of bulk crystals. Manual identification of suitable flakes from theresulting random distribution of crystal thicknesses and sizes on a substrate is a time-consuming,tedious task. Here, we present a platform for fully automated scanning, detection, and classificationof 2D materials, the source code of which we make openly available. Our platform is designed to beaccurate, reliable, fast, and versatile in integrating new materials, making it suitable for everydaylaboratory work. The implementation allows fully automated scanning and analysis of wafers withan average inference time of 100 ms for images of 2.3 Mpixels. The developed detection algorithmis based on a combination of the flakes’ optical contrast toward the substrate and their geometricshape. We demonstrate that it is able to detect the majority of exfoliated flakes of various materials,with an average recall (AR50) between 67% and 89%. We also show that the algorithm can betrained with as few as five flakes of a given material, which we demonstrate for the examples offew-layer graphene, WSe2, MoSe2, CrI3, 1T-TaS2 and hexagonal BN. Our platform has been testedover a two-year period, during which more than 106 images of multiple different materials wereacquired by over 30 individual researchers.1. IntroductionTwo-dimensional (2D) materials offer an unprecedented opportunity to artificially engineer van der Waals(vdW) heterostructures for a wide range of applications, as well as to study fundamental material properties[1]. These vdW heterostructures can range from monolayers encapsulated in hexagonal boron nitride (hBN)for improved electronic and optical properties and spatial homogeneity [2–6], to stacks containing up to tenlayers, such as various semiconductors, graphitic gates, and hBN dielectrics and spacer layers [7–9]. Despiteprogress in the growth of 2D materials [10, 11] and their subsequent fabrication of vdW heterostructures ona wafer scale [12, 13], the exfoliation and mechanical stacking of 2D crystals remains the method of choicefor basic research and proof-of-principle devices [14].Currently, most researchers manually identify suitable flakes by scanning the exfoliated crystals on thesubstrate under a microscope [15], which is a time-consuming and tedious task. This is particularly true forair-sensitive materials, which need to be exfoliated and identified in an inert environment. Therefore,automating the identification of flakes has the potential to drastically improve the efficiency of samplepreparation.© 2024 The Author(s). Published by IOP Publishing Ltdhttps://doi.org/10.1088/2632-2153/ad2287https://crossmark.crossref.org/dialog/?doi=10.1088/2632-2153/ad2287&domain=pdf&date_stamp=2024-2-13https://creativecommons.org/licenses/by/4.0/https://creativecommons.org/licenses/by/4.0/https://orcid.org/0009-0009-1069-5778https://orcid.org/0009-0003-8316-523Xhttps://orcid.org/0009-0003-1691-0340https://orcid.org/0000-0002-7230-0294https://orcid.org/0009-0009-9467-9096https://orcid.org/0000-0003-3701-8119https://orcid.org/0000-0002-1467-3105https://orcid.org/0000-0003-2359-2718https://orcid.org/0000-0002-0898-3860https://orcid.org/0000-0002-4958-7362mailto:jan-lucas.uslu@rwth-aachen.demailto:waldecker@physik.rwth-aachen.deMach. Learn.: Sci. Technol. 5 (2024) 015027 J-L Uslu et alIn general, the automatic detection of flakes can be achieved by recording optical microscope images witha motorized microscope and a digital camera and subsequent image analysis by a computer algorithm [16].Several groups have reported demonstrations of such detection algorithms, e.g. using classicalmachine-learning tools such as support vector machines or K-means [17–19]. These methods rely on the factthat 2D materials exist in integer numbers of layers and their optical contrast values with respect to asubstrate are therefore discrete [20]. Other groups have employed neural networks to achieve the task of flakedetection [21, 22]. This approach provides greater versatility, as the algorithm can learn to identify flakes bydifferent types of features and established networks can be extended to the detection of new materials.However, neural networks tend to need large amounts of labeled data, a requirement which is impractical,especially when dealing with materials with a low yield of mono to few-layer flakes upon exfoliation. Trainingthe detection algorithm on a small number of manually identified flakes is therefore highly desirable for theintegration of new materials on a reasonable timescale as well as for the implementation of the algorithm ondifferent optical setups. For neural networks, this has been attempted using transfer learning [22, 23].However, the challenge of requiring large quantities of labeled data can also be alleviated by directly usinginductive biases based on physical features, such as the optical contrast of the flakes, to reliably classifyinstances.Next to an efficient detection algorithm, an automated flake search system needs to be fast, reliable, andcompatible with the workflow of the other fabrication steps in order to fully replace manual search. In thiswork, we present an automated workflow for the scanning, detection, and classification of exfoliated 2Dmaterials on SiO2 using classical machine learning models. We address the full task of flake detection startingwith the exfoliation of crystal flakes on optimized substrates, over real-time detection and classification, tothe storage of detected flakes in a database. The main focus is on the development of a robust and fastpipeline suitable for daily use in the laboratory as a truly competitive alternative to manual searching by aresearcher; an overview of the process workflow is given in section 1. At the center of the implementation isthe detection algorithm, which utilizes a Gaussian mixture model (GMM) to fit clusters in the opticalcontrast space, as well as a simple logistic classifier to assign a confidence value based on the geometricfeatures of the detected flakes. The details of the implementation are given in section 2, while the code ismade publicly available at [25]. The training of our algorithm is performed with approximately five exampleimages of a given material and number of layers, as described in section 3. While the contrast values of many2D materials, including few-layer graphene, WSe2, MoSe2, CrI3, 1T-TaS2 as well as hBN form discreteclusters and can therefore naturally be implemented with our method, practically continuous distributions,such as those of hBN between 10–40 nm thickness, can also be fitted. The biggest challenge to the stability ofthe detection algorithm is variations in the oxide thickness of the substrates, which typically vary betweenwafer batches from different growth runs, but can be compensated for in the training process. To quantifythe performance of the detection algorithm and to allow a fair comparison to other implementations, wepresent metrics of the detection in section 4. We identify the average recall (AR50) as particularly relevant tothe problem, as it describes the fraction of exfoliated flakes which have been properly detected. It variesbetween materials and flake sizes from 63% for small WSe2 up to 100% for flakes larger than 400 µm2,demonstrating that large flakes are practically always detected by our algorithm.2. Workflow overviewA comprehensive overview of the workflow is presented in figure 1. We start with mechanical exfoliation of2D crystals using tape onto Si/SiO2 substrates [26], resulting in flakes of arbitrary size and shape. Bychoosing a particular oxide thickness, thin-film interference leads to a large optical contrast in the visiblerange [20, 27]. The choice of oxide thickness depends on the material and will be discussed in section 3. Thewafer pieces are placed on the motorized scanning table (Märzhäuser SCAN 100× 100) of the microscopesetup (Nikon Eclipse). The microscope is equipped with a motorized revolver holding five microscopeobjectives between 2.5× and 100×magnification, a digital color camera (The Imaging Source DFK33UX174), and an infrared autofocus module (Nikon LV-DAF), which works independently of the camera tostabilize the focal plane. The entire setup is placed in a glovebox with an inert nitrogen atmosphere.Our program starts by scanning the entire area of the scanning table in the smallest magnificationinstalled (CFI T Plan EPI 2.5×), see figure 1(a). The recorded images are downsampled and stitched togetherto create an overview image of all wafer pieces on the scanning table. The downsampling is not strictlynecessary but reduces the filesize of the resulting overview image. From the overview image, the position andorientation of the wafer pieces is deduced by thresholding the image using Otsu’s method [28], yielding abinary mask of wafer and background. The overview mask is later used to define the positions which will bescanned in higher magnification for the detection of the flakes, and thus allows using wafer pieces ofarbitrary shape while minimizing the overall scan time.2Mach. Learn.: Sci. Technol. 5 (2024) 015027 J-L Uslu et alFigure 1.Workflow overview for the example of few-layer graphene. (a) Si/SiO2 wafer pieces with exfoliated graphite are placedon a microscope stage. An infrared (IR) autofocus module allows to keep the images in all magnifications in focus in realtime.The full range of the scanning table is scanned in the lowest magnification (2.5×) from which the positions of the wafer pieces areinferred. (b) Images are then recorded in higher magnification (here 20×). (c) These images are analyzed by the detectionalgorithm using their optical contrast (see section 2 and figure 2). (d) The detection algorithm classifies flakes by their opticalcontrast and assigns the layer number (1 L, 2 L, etc) together with an instance mask (colored outlines). The sidelength of theboxes are 75 µm. (e) After the scan has finished, images of each detected flake are taken in all possible magnifications. (f) Allimages as well as their metadata (i.e. area, number of layers etc) are uploaded to an interactive website for inspection and selectionby the user. An example website can be accessed at [24].For the following flake search, we use a magnification of 20× (CFI TU Plan Fluor EPI 20×), which is atrade-off between the time needed and the minimal size of detectable flakes (figure 1(b)). The entire surfaceof all wafer pieces is then scanned, and each image is analyzed at run-time. We detect flakes of interest byapplying an image detection algorithm (figure 1(c)) based on the combination of a GMM and a shapediscriminator, which is the main focus of this paper and will be described in detail in section 2.For each detected flake (figure 1(d)), the image, together with an instance mask, metadata, and the globalposition, are saved, and a marker is placed at its position in the overview image. Images not containingdetected flakes are discarded.At the end of the scan, every detected flake is revisited in all magnifications (2.5×, 5×, 20×, 50×, 100×)to record images with them in the center of the field of view, see figure 1(e). Images recorded in largermagnifications help for later validation of the quality of the detected flakes by the user, e.g. the cleanliness ofthe surface, whereas images in smaller magnifications are essential in order to locate the flakes for furtherprocessing, i.e. for stacking of flakes into vdW heterostructures.The collection of images, positions, and metadata, such as their thickness, size, or aspect ratio, areuploaded to a database on a server; see figure 1(f). The data is accessible via a web interface, which alsoallows for filtering and sorting. The interface enables for a fast and efficient selection of flakes for a particulardevice structure. It also encourages sharing of materials between different users or projects, as often differentmaterial thicknesses or shapes are needed (e.g. few-layer graphene for back gates and contacts, mono- orbilayer graphene for heterostructure). Thus, easy access to the gathered data is an integral part of aproductive workflow; a demo version can be accessed at [24].3. Detection algorithmThe detection algorithm introduced here is based on the extraction of two general and robust features ofexfoliated flakes of 2D materials. First, the optical contrast of the flakes towards their substrate [19, 20, 29] iscalculated. We then compare the contrast of individual pixels to pre-determined average contrasts of eachnumber of layers, which were extracted beforehand using the GMM. This enables effective segmentation andclassification of various areas within the image. Although the optical contrast discriminates different flakethicknesses, it is not sufficient for a reliable flake detection, as other objects, such as shadows and taperesidues, can have a similar contrast and might thus falsely be classified as flakes. This is most notable whensearching for materials with small contrasts, such as graphene or, in particular, few-layer hBN. Therefore, in s3Mach. Learn.: Sci. Technol. 5 (2024) 015027 J-L Uslu et alFigure 2. Detection algorithm. (a) Workflow of the detection algorithm (see text for details). (b) Color histograms of a raw image(top) and the preprocessed image (bottom), in which the vignette has been removed. The main features (arrows) originate fromthe bare Si/SiO2 wafer piece and therefore correspond to the background color. The respective images are shown in the inset; thecolors have been exaggerated for better visibility of the effect. (c) Image containing different graphene flakes (scale bar: 20 µm).(d) Sketch of the working principle of the GMM for layer identification. The contrast distributions of each layer are given asmeans µ⃗i (crosses) and covariance matricesΣ (ellipses). A measured color contrast of a pixel C⃗ has a distance d⃗i to the mean µ⃗i ofeach Gaussian (measured in standard deviations). (e) Final segmentation mask of the image shown in (c). (f) Extraction ofmetadata from detected instances, including bounding box and side lengths. Scale bar: 10 µm.second step, we employ a logistic classifier, which is based on the geometric shape of the detected flakes. Thisadditional classifier assigns a false positive probability to each flake previously detected by the GMM.A flowchart of all individual computational steps of the detection algorithm is shown in figure 2(a). Thedetection algorithm starts with the image taken by the microscope during the scan in higher magnification(here 20×). Three corrections are applied to the image. First, the dark counts of the camera are subtractedfrom each RGB channel. Second, we account for the effect of vignetting, that is, an inhomogeneousillumination of the image with decreased brightness toward the edges (compare the images in the inset offigure 2(b). This is corrected by dividing the image by a flatfield image (pixel-wise) and multiplying it by themean pixel value of the respective color channel, obtaining a preprocessed image I⃗p(x,y) whereI⃗(x,y) = (IR, IG, IB)(x,y) is the pixel value for each RGB color channel at position (x, y) [30]. The flatfieldimage is taken once for each substrate, containing only the bare Si/SiO2 substrate. To demonstrate the effectof the vignette correction, the color histograms of an example image before and after its removal are shownin figure 2(b). Each color channel exhibits a prominent mode (see black arrows) which is at least two ordersof magnitude stronger than others. These correspond to the color of the background, i.e. the intensity of thebare Si/SiO2 substrate, which takes up the largest area of the image. Note that saturated areas of the image,e.g. from bulk crystals, can take up large areas, too, but are excluded by only using color values within therange of 20–230. The effect of the image correction can be clearly seen in the reduction of the widths of thehistograms. After removal of the vignette, a global background color I⃗bg is extracted as the mode of thedistribution, indicated by the arrows in figure 2(b). Third, we apply a median blurring (5× 5 kernel) toreduce camera noise while preserving edges to a good degree. An example is shown in figure 2(c).From the pre-processed images, the contrast C⃗ of each pixel with coordinates (x, y) is calculated:C⃗(x,y) =(⃗Ip (x,y)− I⃗bg)⊘ I⃗bg, (1)where⊘ denotes component-wise division. Using the contrast instead of raw intensities makes the detectionprocess independent of lighting conditions. The resulting contrast images can now be segmented by thetrained contrast values (see section 3 for details on the training). The trained contrasts are defined by a set ofGaussians S with means µ⃗ and covariance matrices Σ, which describe the variation of contrast values for each4Mach. Learn.: Sci. Technol. 5 (2024) 015027 J-L Uslu et alFigure 3. Shape analysis and assignment of false positive probability. (a) Scatter plot of solidity vs arclength·area−1/2 of the masksof 2508 detected flakes. Real flakes (blue) and others (orange), such as shadows or tape residue, have been annotated by hand andshow clearly different distributions, which can be used to assign a false positive likelihood. The solid and dashed black line givesthe 25, 50 and 75% probability boundaries. (b) Examples of six detected instances and their classification using the logisticregression (see text for details). Their positions in panel (a) is indicated by capital letters. The sidelength of the boxes is 115 µm.number of layers nL (see sketch in figure 2(c)). The likelihood of a pixel belonging to a certain layer is thenproportional to its Mahalanobis distance (i.e. the distance in standard deviations) to the mean values µ⃗. Foreach pixel, the Mahalanobis distance dS to each set S is calculated by:dS(C⃗, µ⃗,Σ)=√(C⃗− µ⃗)Σ−1(C⃗− µ⃗). (2)The layer number with the smallest Mahalanobis distance is then assigned to each respective pixel in thesegmented image.Pixels, for which none of the distances are within a certain range, dS > dmax for all S, are classified asbackground. dmax can be adjusted to detect more or fewer pixels, but is typically taken as five standarddeviations.The resulting segmentation mask contains all detected flakes, but also wrongly assigned pixels due toshadows, tape residue or single pixels from camera noise. Single stray pixels are removed by applyingmorphological operations, i.e. through erosion followed by dilation.In the next step, the semantic mask is converted to instances. As flakes do not overlap, we are able toextract instances by finding the connected components on the semantic mask. This is done by using theSpaghetti labeling algorithm from [31]. implemented in OpenCV. Each component is post-processed byfinding the outer contour and redrawing the component, which fills remaining holes. The resulting masksare instances of the contrast-based flake detection; an example image and its corresponding segmentationmask are shown in figures 2(d) and (e). Metadata are extracted for each component, which include the size,aspect ratio, maximum and minimum side-lengths as well as the mask itself (figure 2(f)). These metadata aresaved together with all images in the database.To filter out detected instances that do not correspond to true flakes, we implement a discriminator,which analyzes the shape of the detected instances. It is based on the observation that falsely assignedinstances tend to have curved and irregular outlines (see figure 3(b)). We quantify this observation using thesolidity, i.e. the ratio of the area of the mask to the area enclosed by the convex hull, and the length of theperimeter (arclength) divided by the square root of the area.Figure 3(a) shows a scatter plot of both properties of 2508 instances detected as graphene, which havebeen labeled manually as ‘flakes’ (blue) and ‘others’ (orange). A separation of true and false positives isobserved, with false positives generally having lower solidity and a larger arc length per square root area.Therefore, a logistic regression can be used to separate the two classes by assigning a probability to eachinstance and interpreting this as a false positive probability. We use an L2 logistic classifier to fit the data; thedecision boundary (50% false positive probability), as well as the 25% and 75% probability boundaries, areshown as solid and dashed lines in figure 3(a). The false positive probability for each instance is saved in themetadata without deleting any instances and can be used to sort and filter out flakes.5Mach. Learn.: Sci. Technol. 5 (2024) 015027 J-L Uslu et alNote that the two geometric feature dimensions used here are not exclusive. An additional featuredimension which has been used is the Shannon entropy of the area [17]. The entropy threshold for filtering,however, is less general, as it depends on parameters such as the lighting conditions, the white balance, or thegain of the camera. In contrast, our classifier is material and microscope agnostic, as no direct informationabout the image is used, and it is directly interpretable. More complex models, such as neural networks,might perform even better than our method; however, they tend to either overfit the data, require largecomputational overhead, or introduce unnecessary complexity (millions of parameters versus twoparameters in our case). The performance of the detection scheme for different materials will be discussed insection 4.4. Training of the detection algorithmThe algorithm training process starts by manually identifying approximately 5 flakes for each number oflayers of interest, with a size of at least 200 µm2 each, corresponding to 800 pixels for a magnification of 20×.These images are then annotated semi-automatically using the watershed algorithm (implemented inOpenCV). At this point, it is not needed to define the number of layers of the flakes, as this is inferred later.The vignette of these images is removed, and the color contrast of each pixel within the flakes is calculated asdescribed in equation (1).The contrast distribution of one layer to four layers of graphene is shown in figure 4(a) (four flakes eachhave been used). The contrast values group into four distinct and well separated clusters, as described inprevious publications [19, 32, 33]. For easier comparability, we only show two of the three color channels asheat maps in the following. For the case of graphene on 90 nm SiO2, the heat maps of the red and greenchannels are shown in figure 5(b) (see the discussion on oxide thickness variations below for the choice ofcolor axes).The three-dimensional color contrasts are then fit with three-dimensional Gaussians. After defining thenumber of clusters, the expectation–maximization (EM) algorithm [34] is run and returns a set S ofparameters {π, µ⃗,Σ}. While µ⃗ and Σ are the parameters defining the position and shape of the Gaussian, theπ parameter is a measure for the class weight. It counteracts class imbalance by automatically scaling theGaussians according to the estimated weight of each class. After training the model, the weights π arediscarded since we are only interested in the shape of the Gaussians, as the shape encodes information aboutthe contrast distribution of each class. This leads to a model that is balanced with regard to the classdistribution. To increase the robustness of the fit, more components of the GMM can be added to fit thebackground noise distribution (we typically use a ratio of about 2 to 1). Finally, the layer numbers areassigned to the clusters, which is done by the order of their contrasts. Given enough context (flakes ofdifferent thicknesses), this is usually straightforward, as in the example of few-layer graphene, but it couldalso be supported by simulations [27]. The 2D fits and labels are overlaid on the graphene training data infigure 4.4.1. Implementation of different materialsThe training of our detection algorithm can be performed, as demonstrated above for few-layer graphene,with any 2D material, which has a sufficiently large color contrast on a particular substrate and with asufficiently large contrast difference between individual layers.Figures 5(a)–(e) shows example images and contrast density maps (red and green contrast) obtainedfrom exfoliated few-layer WSe2, MoSe2, 1T-TaS2, CrI3 and few-layer hBN flakes on optimized Si/SiO2substrates. Note that for CrI3, a 550 nm longpass filter was inserted into the microscope, as the material candegrade under blue-light illumination even in an inert nitrogen atmosphere. For all five materials, theclustering of the contrasts is apparent, which makes it possible to employ a GMM to fit their distributions(see the white ellipses in the images). Note that for few-layer hBN, the absolute contrasts as well as thedifferences between neighboring layers are much smaller. Even moderate noise in the images can thereforeblur the differences between layers. In order to observe clustering of the contrasts, which can be fitted withthe GMM, we reduced the camera noise by averaging two consecutive exposures, and took images at 50×magnification, which decreases the influence of the flakes’ edges. The contrast distribution, taken from threedifferent flakes of hBN, is shown in figure 5(e). The measured contrast values appear to have a small offsetwith respect to the background color, i.e. the first contrast step is comparatively bigger than all consecutivesteps. This could be due to a small air gap between the flakes and the substrate. As we observe the offset tovary between flakes, the interpretation of the flake thickness needs to be done with care.Lastly, we have applied the GMM to quasi-continuous contrast distributions, which appear if the contrastdifference between adjacent layer numbers is well below the contrast deviation of a single layer. This isparticularly relevant for the detection of thicker flakes of hBN, which are ubiquitously used to encapsulate6Mach. Learn.: Sci. Technol. 5 (2024) 015027 J-L Uslu et alFigure 4. Training of the detection algorithm for the example of graphene on 90 nm SiO2/Si. (a) The contrast values of few-layergraphene in RGB space show well-separated clusters. (b) Corresponding color contrast density map in the red and green channel.The white lines depict the contours of the first, second and third standard deviation of a multi-dimensional Gaussian fit to thedata. Four flakes of each number of layers have been used.and seal other 2D materials, as well as for gate dielectrics in 2D material devices. In these circumstances, theexact thickness is usually irrelevant as long as it falls into a particular thickness range. Figure 5(f) shows thecolor contrast distribution of hBN flakes of thicknesses between few nm and approximately 40 nm. Thedistribution can be fit with the GMM, though more manual fine-tuning of the fit parameters, such as thenumber of components used and the type of covariance matrix, is needed. Here, the distribution was fit with10 Gaussians with the constraint of a single covariance matrix shared across all Gaussians, corresponding tothickness intervals of approximately 4 nm.4.2. Choice of silicon oxide thicknessThe color contrast of 2D material flakes with respect to a Si/SiO2 substrate arises from thin-film interferenceand therefore depends sensitively on the oxide thickness [19, 29]. More specifically, the oxide thicknessdetermines the spectral reflectivity R(λ), where λ denotes the wavelength, both of the substrate Rsub and ofthe sample on the substrate Rsam. The color contrast C⃗ furthermore depends on the spectral irradiance of thelight source I(λ) and the spectral sensitivity of the camera s⃗(λ) [27]:C⃗=[(ˆdλI(λ) ·Rsam (λ, tox) · s⃗(λ))⊘(ˆdλI(λ) ·Rsub (λ, tox) · s⃗(λ))]− 1. (3)To simulate the color contrasts for different materials on Si/SiO2 substrates, the reflectivities R(λ) havebeen calculated using the transfer matrix method [35]. Inputs to the calculation are the refractive indices ofSi, SiO2 and the 2D materials (here graphene and WSe2 as examples) [36–39], the camera sensitivity curveand an approximate spectrum of the illumination.7Mach. Learn.: Sci. Technol. 5 (2024) 015027 J-L Uslu et alFigure 5. Heat maps of the color contrasts of four different 2D materials and their respective fits with the GMM (left panels) andexample images of flakes of each material, which are used for the training (right panels). The side lengths of the boxes are 45 µm(a)–(e) and 90 µm (f). (a)–(d) Well-separated clusters of the contrasts of few-layer WSe2, MoSe2, 1T-TaS2 and CrI3 are observed.(e) For thin flakes of the large band-gap insulator hBN, the contrast differences of neighboring layers are small. In order toobserve separated clusters, the images underlying the heatmap have been taken in higher magnification (50×) and two exposureshave been averaged to reduce camera noise. (f) For thicker hBN flakes, the distribution becomes quasi-continuous in the rangeshown (up to≈40 nm). We fit this distribution with ten Gaussians, corresponding to intervals of approximately 4 nm.Figure 6. Influence of the silicon oxide thickness. (a) Calculated optical contrast of monolayer graphene and WSe2 on Si/SiO2 inthe RGB channels of a color camera. A high contrast value corresponds to good visibility and an easier detection. A largederivative (positive or negative) in a particular channel, however, renders the rule-based approach more unreliable in terms ofthickness variations. (b)–(d) The contrast distribution of few-layer graphene using 3600 Flakes from 11 different wafers ofnominally 90 nm SiO2 thickness. The observed contrast distribution originates from variations in oxide thickness. For graphene,it is largest in the blue channel, in agreement with the simulations shown in (a).Figure 6(a) shows the simulated color contrast of monolayer graphene and WSe2, respectively, on Si/SiO2wafers vs oxide thicknesses. The maximum contrast of each color channel is reached at a different oxidethickness and also depends on the material. Although a large contrast is a necessary requirement to be able todetect 2D material flakes, a large derivative of the contrast represents a challenge for the reliability of thecontrast-based detection against variations of the oxide thickness. Such variations are common for the wafersproduced by dry oxidation in our facilities and for those of most commercial suppliers.Figures 6(b)–(d) show heatmaps of the color contrast of few-layer graphene on (nominally) 90 nm oxide,obtained from 18 different exfoliations. Instead of well-defined clusters, the color contrasts now vary along acharacteristic curve. The largest changes of contrast are observed along the blue axis, as expected from thesimulation shown in figure 6(a).For a stable operation of the contrast-based algorithm, we, therefore, choose an oxide thickness for whichthe color contrast varies as little as possible. For most materials, this condition cannot be fulfilled in all colorchannels simultaneously. For graphene, we use wafers of 90 nm oxide thickness, for which at least twochannels (red and green) are stable. For WSe2, the contrasts as well as their derivatives are more favorable atan oxide thickness of 70 nm. Of these two options, we find 90 nm oxide better suited for few layer 1T-TaS2and CrI3, and 70 nm oxide for the transition metal dichalcogneides (WSe2, MoSe2) as well as few-layer hBN.8Mach. Learn.: Sci. Technol. 5 (2024) 015027 J-L Uslu et alIn addition, to make the algorithm more stable, some variations can on purpose be included in thetraining set by exfoliating onto different batches of wafers. While this works well to compensate for oxidevariations of a few nanometers, wafers which are outside this range need to be sorted out, e.g. by performingellipsometry, before use.5. EvaluationIn order to quantify the performance and establish a comparison between different approaches for thedetection algorithm, meaningful metrics need to be used. Several recent publications used accuracy tocompare the fidelity of the generated semantic masks [22, 33, 40]. This is not ideal, as the datasets exhibit astrong class imbalance between the background and the flakes of interest. This imbalance can be on the orderof 104, i.e. flakes only occupy less than 0.01% of the image. Using accuracy in this context will yield metricsin the realm of 99.99%, even if only a small fraction of flakes (or none) is classified as such. To mitigate this,we evaluate our algorithm using metrics more suited for imbalanced datasets, in particular precision, recalland the Jaccard index (IoU). In addition to these metrics, which are evaluated pixel-wise, we also use theinstance-based average precision at 50% (AP50) and average recall at 50% (AR50) [41], which can beinterpreted as the ratio of detected real flakes to false positives and the fraction of all real flakes detected bythe algorithm. The latter two are determined using a modified MS COCO evaluator from the Detectron2framework [42].5.1. DatasetWe evaluated the algorithm on two materials, few-layer graphene and WSe2 The few-layer graphene datasetconsists of 1787 images collected over a period of one year from 12 different exfoliation runs, while the WSe2dataset contains 512 images collected from 14 different exfoliation runs over the same period. Each image inthe dataset is labeled with the number of layers present in the material, ranging from 1 to 4 layers forfew-layer graphene and 1 to 3 layers for WSe2. The images were collected using our automated setup,ensuring consistency, avoiding centering bias and have a resolution of 1920× 1200 pixels.The used images were collected by multiple researchers in our lab, resulting in varying conditions such asLED light aging, white balances, and substrate thickness. To ensure reliable training and testing of themodels, we split the datasets into training and testing sets. Specifically, we used two exfoliation runs as thetraining set, and the rest of the runs were used for testing to better probe the generalization capabilities of thealgorithm. We did not employ any data augmentation techniques during the training process. The WSe2dataset comprises 92 training images and 420 test images, while the few-layer graphene dataset includes 425training images and 1362 test images.5.2. MetricsThe shape analysis introduced in section 2 assigns a false positive probability to each detected instance, whichcan be used for filtering out those above a certain threshold. As the value of this threshold affects the metricsof the algorithm, we first demonstrate its effect for different number of layers.The precision, recall and IoU for the examples of monolayer and trilayer graphene are shown in figure 7as a function of the threshold. For both numbers of layers, the recall has its maximum for a threshold of100%, i.e. when no flakes are filtered at all. This is expected, as the false positive (FP) filtering cannot increasethe number of true positives. The precision, on the other hand, can increase as more FPs are filtered. Thedegree to which this happens markedly depends on the material: for monolayer graphene, which is opticallythin, the precision increases sharply, whereas for trilayer graphene, only a small increase is observed. Thereason is that contaminations tend to have a relatively low optical contrast and are therefore mostly classifiedas optically thin materials. The filtering, therefore, has its main use case for monolayer and bilayer grapheneas well as for other materials of low contrast, such as few-layer hBN. A point of good compromise can bedefined by the maximum of the IoU. We therefore provide the metrics at the threshold which maximizes therespective IoU in table 1.We stress that, from a practical standpoint, it is important that as many instances as possible are detectedcorrectly, even if the detection mask is not perfect on a pixel level. We therefore calculate the instance-basedprecision and recall (AP50 and AR50). The results are given for different materials and numbers of layers intable 1.It is apparent that the metrics for all thicknesses of few-layer graphene are better than for WSe2, both interms of IoU and AR50. This is largely due to the significantly smaller average size of WSe2 flakes: thedetection mask and the labeled mask often differ on the very edges of the instances, leading to a lowerprecision and recall for smaller flakes. The size-dependence of the metrics is given in table 2. The table showsthat both, precision and recall, are significantly higher for flakes of sizes>100 µm2. We note that we9Mach. Learn.: Sci. Technol. 5 (2024) 015027 J-L Uslu et alFigure 7.Metrics of the detection algorithm calculated for different levels of filtering based on the shape analysis. Flakes with afalse-positive probability above the threshold are removed, which increases the precision (less false positives), but generally lowersthe recall (found flakes). The gray shaded areas denote the level of false positive probability leading to the highest IoU for the twodifferent numbers of layers. All scores have been evaluated pixel-wise.Table 1.Metrics of the detection algorithm. Top rows: the threshold at which the pixel-wise determined IoU is maximized and therespective precision, recall and IoU at this threshold for graphene (1–4 layers) and WSe2 (1–3 layers). Bottom two rows: theinstance-wise determined average precision (AP50) and average recall (AR50). All quantities are given in %.Few-layer graphene WSe21 L 2 L 3 L 4 L 1 L 2 L 3 LThreshold 33 84 95 93 79 70 22Precision 72.2 83.5 82.8 83.5 76.6 90.3 68.1Recall 83.8 83.3 86.9 87.5 59.9 60.5 61.2IoU 63.4 71.6 73.6 74.6 50.6 56.8 47.6AP50 48.5 55.0 58.1 58.0 62.1 57.1 52.8AR50 89.2 80.3 84.6 88.4 70.5 66.9 80.7Table 2. Size-dependence of the average precision (AP50) and the average recall (AR50). The detection becomes less reliable for smallerflakes as edges in the image make up for a larger fraction of the flake area. All quantities are in %.size (µm2) 50–100 100–200 200–400 >400AP50Gr 1 L 31.9 85.4 88.7 96.4WSe2 1 L 52.7 78.5 77.8 100.0AR50Gr 1 L 82.7 93.3 93.3 100.0WSe2 1 L 62.3 81.9 81.0 100.0currently do not have enough labeled data for MoSe2, TaS2, and CrI3 to report statistically significant metricsfor those materials. Yet, we expect the metrics to be very similar as their optical contrast distributions arecomparable to graphene and WSe2. The reliable detection of few-layer hBN remains challenging due to itslow optical contrast, which is often similar to the optical contrast of tape residue.5.3. RuntimeAnother important characteristic property of the algorithm is the computational time needed to process eachimage. The runtime was evaluated on the CPU (AMD Ryzen 5 3600) of a standard desktop computer.Averaged over 1000 images and searching for four different thicknesses, the runtime was 100 ms per imagewith resolution 1920× 1200. This is significantly faster than other contrast-based implementations, whichhave reported inference times of about 3 seconds per image of size 1920× 1200 pixels [43]. It is also fasterthan implementations based on neural networks, which have reported runtimes of similar magnitude (whenrun on a GPU) but were evaluated on images of 512× 512 and 224× 224 pixels, respectively [22, 44]. Theshort inference time means that the evaluation can be performed on-the-fly, i.e. during the time that thewafer pieces are scanned by the hardware.Lastly, we discuss the time needed to fully complete a scan of wafer pieces to the final upload of imagesonto the server (see figure 1), which is the relevant time for a user. The creation of the overview image takesapproximately 9 minutes, as the entire area of the scanning table is scanned at low magnification. Thescanning at higher magnification reaches a rate of approximately 2 images s−1, corresponding to an area ofabout 25 cm2 h−1. This rate is limited by the hardware, in particular the camera integration time of 120 msand the time for the mechanical stage and autofocus to complete a move of about 280 ms. The revisiting of10Mach. Learn.: Sci. Technol. 5 (2024) 015027 J-L Uslu et althe flakes in multiple magnifications takes the least amount of time and is negligible when compared to thetime of the previous steps.6. DiscussionWe have presented and evaluated a newly developed setup for the automatic detection and classification of2D material flakes. The focus of the implementation has been on its reliability and speed, which allow its easyadaption to different laboratory settings.The performance of our model has been evaluated in terms of the IoU, precision, and recall on a pixeland an instance level for various materials and thicknesses. The (pixel-wise determined) recall has beenshown to be as high or surpassing those reported for implementations using neural networks [22, 44, 45].While the high average recall demonstrates that essentially all flakes are found, the detection algorithm stillwrongly detects contamination or shadows, in particular for materials of low optical contrast. In particular,the detection of monolayers of hBN so far remains unreliable as a result of the overwhelming number of falsepositives from tape residue. However, it might be possible by using different tapes which leave less residue, bydifferent types of substrates, such as silicon nitride [46], which increases the contrast. Furthermore, theproposed pipeline allows for easy extension of the current model by either adding different post-processingsteps, such as Markov random fields, or adding models that are specialized for different materials, such asmodels that utilize PL imaging for TMDs, to improve the effectiveness even further.We have furthermore demonstrated that the algorithm can be trained on a few example images only,which allows one to add new materials relatively quickly. As the detection is based on a physical property ofthe materials, i.e. their color contrast, training could in the future even be fully replaced by simulations(zero-shot learning).The archiving of large amounts of standardized images of detected flakes using our approach (currentlymore than 65 000 flakes) furthermore has the potential to allow some degree of macro-analysis. It could, forexample, be used to analyze properties of the respective crystals, such as preferred crystal-axis breaking [22].In addition, it could be used to improve the yield or cleanliness of common exfoliation methods through astatistical analysis of the data.In our laboratory, the method has been widely adapted, with more than 106 images taken within 24months by over 30 individual researchers and supporting several projects [47–49]. Our implementation notonly speeds up sample fabrication for individuals, but has proven to have synergies in a group working on 2Dmaterials, as exfoliated materials can be easily shared between individuals.Data availability statementThe code for the detection algorithm is accessible on Github [25]. A demo website showcasing the databaseand web interface can be accessed at [24]. The data that support the findings of this study will be openlyavailable at the following URL/DOI: https://doi.org/10.5281/zenodo.8042835.AcknowledgmentsThis project has received funding from the European Union’s Horizon 2020 research and innovationprogram under Grant Agreement No. 881603 (Graphene Flagship) and from the European Research Council(ERC) under Grant Agreement No. 820254, the Deutsche Forschungsgemeinschaft (DFG, German ResearchFoundation) under Germany’s Excellence Strategy—Cluster of Excellence Matter and Light for QuantumComputing (ML4Q) EXC 2004/1—390534769, the FLAG-ERA grant TATTOOS, by the DeutscheForschungsgemeinschaft (DFG, German Research Foundation)—437214324 and by the FLAG-ERA grantPhotoTBG, by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – 471733165. KW and T T acknowledge support from the JSPS KAKENHI (Grant Numbers 20H00354, 21H05233 and23H02052) and World Premier International Research Center Initiative (WPI), MEXT, Japan.ORCID iDsJan-Lucas Uslu https://orcid.org/0009-0009-1069-5778Taoufiq Ouaj https://orcid.org/0009-0003-8316-523XDavid Tebbe https://orcid.org/0009-0003-1691-0340Alexey Nekrasov https://orcid.org/0000-0002-7230-0294Marc Schütte https://orcid.org/0009-0009-9467-9096Kenji Watanabe https://orcid.org/0000-0003-3701-811911https://doi.org/10.5281/zenodo.8042835https://orcid.org/0009-0009-1069-5778https://orcid.org/0009-0009-1069-5778https://orcid.org/0009-0003-8316-523Xhttps://orcid.org/0009-0003-8316-523Xhttps://orcid.org/0009-0003-1691-0340https://orcid.org/0009-0003-1691-0340https://orcid.org/0000-0002-7230-0294https://orcid.org/0000-0002-7230-0294https://orcid.org/0009-0009-9467-9096https://orcid.org/0009-0009-9467-9096https://orcid.org/0000-0003-3701-8119https://orcid.org/0000-0003-3701-8119Mach. Learn.: Sci. Technol. 5 (2024) 015027 J-L Uslu et alTakashi Taniguchi https://orcid.org/0000-0002-1467-3105Bernd Beschoten https://orcid.org/0000-0003-2359-2718Lutz Waldecker https://orcid.org/0000-0002-0898-3860Christoph Stampfer https://orcid.org/0000-0002-4958-7362References[1] Geim A K and Grigorieva I V 2013 Van der Waals heterostructures Nature 499 419[2] Dean C R et al 2010 Boron nitride substrates for high-quality graphene electronics Nat. Nanotechnol. 5 722[3] Neumann C et al 2015 Raman spectroscopy as probe of nanometre-scale strain variations in graphene Nat. Commun. 6 8429[4] Banszerus L, Schmitz M, Engels S, Dauber J, Oellers M, Haupt F, Watanabe K, Taniguchi T, Beschoten B and Stampfer C 2015Ultrahigh-mobility graphene devices from chemical vapor deposition on reusable copper Sci. Adv. 1 e1500222[5] Cadiz F et al 2017 Excitonic linewidth approaching the homogeneous limit in MoS2-based van der Waals heterostructures Phys.Rev. X 7 021026[6] Raja A et al 2019 Dielectric disorder in two-dimensional materials Nat. Nanotechnol. 14 832[7] Xu Y, Liu S, Rhodes D A, Watanabe K, Taniguchi T, Hone J, Elser V, Mak K F and Shan J 2020 Correlated insulating states atfractional fillings of moiré superlattices Nature 587 214[8] Park J M, Cao Y, Watanabe K, Taniguchi T and Jarillo-Herrero P 2021 Flavour Hund’s coupling, Chern gaps and charge diffusivityin moiré graphene Nature 592 43[9] Liu X, Wang Z, Watanabe K, Taniguchi T, Vafek O and Li J I A 2021 Tuning electron correlation in magic-angle twisted bilayergraphene using Coulomb screening Science 371 1261[10] Backes C et al 2020 Production and processing of graphene and related materials 2D Mater. 7 022001[11] Burton O J, Winter Z, Watanabe K, Taniguchi T, Beschoten B, Stampfer C and Hofmann S 2023 Putting high-index Cu on the mapfor high-yield, dry-transferred CVD graphene ACS Nano 17 1229[12] Liu L and Zhai T 2021 Wafer-scale vertical van der Waals heterostructures InfoMat 3 3[13] Xu X, Guo T, Kim H, Hota M K, Alsaadi R S, Lanza M, Zhang X and Alshareef H N 2022 Growth of 2D materials at the wafer scaleAdv. Mater. 34 2108258[14] Liu Y, Weiss N O, Duan X, Cheng H C, Huang Y and Duan X 2016 Van der Waals heterostructures and devices Nat. Rev. Mater.1 16042[15] Novoselov K S, Jiang D, Schedin F, Booth T J, Khotkevich V V, Morozov S V and Geim A K 2005 Two-dimensional atomic crystalsProc. Natl Acad. Sci. USA 102 10451[16] Ryu B, Wang L, Pu H, Chan M K Y and Chen J 2022 Understanding, discovery and synthesis of 2D materials enabled by machinelearning Chem. Soc. Rev. 51 1899[17] Masubuchi S, Morimoto M, Morikawa S, Onodera M, Asakawa Y, Watanabe K, Taniguchi T and Machida T 2018 Autonomousrobotic searching and assembly of two-dimensional crystals to build van der Waals superlattices Nat. Commun. 9 1413[18] Lin X et al 2018 Intelligent identification of two-dimensional nanostructures by machine-learning optical microscopy Nano Res.11 6316[19] Li Y, Kong Y, Peng J, Yu C, Li Z, Li P, Liu Y, Gao C-F and Wu R 2019 Rapid identification of two-dimensional materials via machinelearning assisted optic microscopy J. Mater. 5 413[20] Blake P, Hill E h, Castro Neto A H, Novoselov K S, Jiang D, Yang R, Booth T J and Geim A K 2007 Making graphene visible Appl.Phys. Lett. 91 063124[21] Saito Y, Shin K, Terayama K, Desai S, Onga M, Nakagawa Y, Itahashi Y M, Iwasa Y, Yamada M and Tsuda K 2019Deep-learning-based quality filtering of mechanically exfoliated 2D crystals npj Comput. Mater. 5 124[22] Han B, Lin Y, Yang Y, Mao N and Palacios T 2020 Deep-learning-enabled fast optical identification and characterization of 2Dmaterials Adv. Mater. 32 2000953[23] Masubuchi S, Watanabe E, Seo Y, Okazaki S, Sasagawa T, Watanabe K, Taniguchi T andMachida T 2020 Deep-learning-based imagesegmentation integrated with optical microscopy for automatically searching for two-dimensional materials npj 2DMater. Appl. 4 3[24] Demonstration website of the interface for sorting and filtering of detected flakes (available at: https://flakedemo.uslu.tech/)[25] Python code of the setup control software, detection algorithm and training routines (available at: https://github.com/Jaluus/2DMatGMM)[26] Huang Y, Sutter E, Shi N N, Zheng J, Yang T, Englund D, Gao H J and Sutter P 2015 Reliable exfoliation of large-area high-qualityflakes of graphene and other two-dimensional materials ACS Nano 9 10612[27] Müller M R, Gumprich A, Ecik E, Kallis K T, Winkler F, Kardynal B, Petrov I, Kunze U and Knoch J 2015 Visibility oftwo-dimensional layered materials on various substrates J. Appl. Phys. 118 145305[28] Otsu N 1979 A threshold selection method from gray-level histograms IEEE Trans. Syst. Man Cybern. 9 62[29] Gao L, Ren W, Li F and Cheng H M 2008 Total color difference for rapid and accurate identification of graphene ACS Nano 2 1625[30] Seibert J A, Boone J M and Lindfors K K 1998 Flat-field correction technique for digital detectors Proc. SPIE 3336 348–54[31] Bolelli F, Allegretti S, Baraldi L and Grana C 2019 Spaghetti labeling: directed acyclic graphs for block-based connectedcomponents labeling IEEE Trans. Image Process. 29 1999–2012[32] Masubuchi S and Machida T 2019 Classifying optical microscope images of exfoliated graphene flakes by data-driven machinelearning npj 2D Mater. Appl. 3 4[33] Sterbentz R M, Haley K L and Island J O 2021 Universal image segmentation for optical identification of 2D materials Sci. Rep.11 5808[34] Pedregosa F et al 2011 Scikit-learn: machine learning in python J. Mach. Learn. Res. 12 2825[35] Alonso-Álvarez D, Wilson T, Pearce P, Führer M, Farrell D and Ekins-Daukes N 2018 Solcore: a multi-scale, Python-based libraryfor modelling solar cells and semiconductor materials J. Comput. Electron. 17 1099[36] Weber J W, Calado V E and Van De Sanden M C 2010 Optical constants of graphene measured by spectroscopic ellipsometry Appl.Phys. Lett. 97 091904[37] Green M A and Keevers M J 1995 Optical properties of intrinsic silicon at 300 K Prog. Photovolt., Res. Appl. 3 189[38] Malitson I H 1965 Interspecimen comparison of the refractive index of fused silica J. Opt. Soc. Am. 55 1205[39] Li Y and Heinz T F 2018 Two-dimensional models for the optical response of thin films 2D Mater. 5 02502112https://orcid.org/0000-0002-1467-3105https://orcid.org/0000-0002-1467-3105https://orcid.org/0000-0003-2359-2718https://orcid.org/0000-0003-2359-2718https://orcid.org/0000-0002-0898-3860https://orcid.org/0000-0002-0898-3860https://orcid.org/0000-0002-4958-7362https://orcid.org/0000-0002-4958-7362https://doi.org/10.1038/nature12385https://doi.org/10.1038/nature12385https://doi.org/10.1038/nnano.2010.172https://doi.org/10.1038/nnano.2010.172https://doi.org/10.1038/ncomms9429https://doi.org/10.1038/ncomms9429https://doi.org/10.1126/sciadv.1500222https://doi.org/10.1126/sciadv.1500222https://doi.org/10.1103/PhysRevX.7.021026https://doi.org/10.1103/PhysRevX.7.021026https://doi.org/10.1038/s41565-019-0520-0https://doi.org/10.1038/s41565-019-0520-0https://doi.org/10.1038/s41586-020-2868-6https://doi.org/10.1038/s41586-020-2868-6https://doi.org/10.1038/s41586-021-03366-whttps://doi.org/10.1038/s41586-021-03366-whttps://doi.org/10.1126/science.abb8754https://doi.org/10.1126/science.abb8754https://doi.org/10.1088/2053-1583/ab1e0ahttps://doi.org/10.1088/2053-1583/ab1e0ahttps://doi.org/10.1021/acsnano.2c09253https://doi.org/10.1021/acsnano.2c09253https://doi.org/10.1002/inf2.12164https://doi.org/10.1002/inf2.12164https://doi.org/10.1002/adma.202108258https://doi.org/10.1002/adma.202108258https://doi.org/10.1038/natrevmats.2016.42https://doi.org/10.1038/natrevmats.2016.42https://doi.org/10.1073/pnas.0502848102https://doi.org/10.1073/pnas.0502848102https://doi.org/10.1039/D1CS00503Khttps://doi.org/10.1039/D1CS00503Khttps://doi.org/10.1038/s41467-018-03723-whttps://doi.org/10.1038/s41467-018-03723-whttps://doi.org/10.1007/s12274-018-2155-0https://doi.org/10.1007/s12274-018-2155-0https://doi.org/10.1016/j.jmat.2019.03.003https://doi.org/10.1016/j.jmat.2019.03.003https://doi.org/10.1063/1.2768624https://doi.org/10.1063/1.2768624https://doi.org/10.1038/s41524-019-0262-4https://doi.org/10.1038/s41524-019-0262-4https://doi.org/10.1002/adma.202000953https://doi.org/10.1002/adma.202000953https://doi.org/10.1038/s41699-020-0137-zhttps://doi.org/10.1038/s41699-020-0137-zhttps://flakedemo.uslu.tech/https://github.com/Jaluus/2DMatGMMhttps://github.com/Jaluus/2DMatGMMhttps://doi.org/10.1021/acsnano.5b04258https://doi.org/10.1021/acsnano.5b04258https://doi.org/10.1063/1.4930574https://doi.org/10.1063/1.4930574https://doi.org/10.1109/TSMC.1979.4310076https://doi.org/10.1109/TSMC.1979.4310076https://doi.org/10.1021/nn800307shttps://doi.org/10.1021/nn800307shttps://doi.org/10.1117/12.317034https://doi.org/10.1117/12.317034https://doi.org/10.1109/TIP.2019.2946979https://doi.org/10.1109/TIP.2019.2946979https://doi.org/10.1038/s41699-018-0084-0https://doi.org/10.1038/s41699-018-0084-0https://doi.org/10.1038/s41598-021-85159-9https://doi.org/10.1038/s41598-021-85159-9https://doi.org/10.1007/s10825-018-1171-3https://doi.org/10.1007/s10825-018-1171-3https://doi.org/10.1063/1.3475393https://doi.org/10.1063/1.3475393https://doi.org/10.1002/pip.4670030303https://doi.org/10.1002/pip.4670030303https://doi.org/10.1364/JOSA.55.001205https://doi.org/10.1364/JOSA.55.001205https://doi.org/10.1088/2053-1583/aab0cfhttps://doi.org/10.1088/2053-1583/aab0cfMach. Learn.: Sci. Technol. 5 (2024) 015027 J-L Uslu et al[40] Greplova E, Gold C, Kratochwil B, Davatz T, Pisoni R, Kurzmann A, Rickhaus P, Fischer M H, Ihn T and Huber S D 2020 Fullyautomated identification of two-dimensional material samples Phys. Rev. Appl. 13 064017[41] Lin T Y, Maire M, Belongie S, Bourdev L, Girshick R, Hays J, Perona P, Ramanan D, Zitnick C L and Dollár P 2014 MicrosoftCOCO: common objects in context Computer Vision – ECCV 2014 Lecture Notes in Computer Science vol 8693 pp 740[42] Wu Y, Kirillov A, Massa F, Lo W-Y and Girshick R 2019 Detectron2 (available at: https://github.com/facebookresearch/detectron2)[43] Yang J and Yao H 2020 Automated identification and characterization of two-dimensional materials via machine learning-basedprocessing of optical microscope images Extrem. Mech. Lett. 39 100771[44] Yang B, Wu M and Teizer W 2022 Modified UNet++ with attention gate for graphene identification by optical microscopy Carbon195 246[45] Siao H-Y, Qi S, Ding Z, Lin C-Y, Hsieh Y-C and Chen T-M 2021 Machine learning-based automatic graphene detection with colorcorrection for optical microscope images (arXiv:2103.13495)[46] Hattori Y, Taniguchi T, Watanabe K and Kitamura M 2022 Enhancement of the contrast for a hexagonal boron nitride monolayerplaced on a silicon nitride/silicon substrate Appl. Phys. Express 15 086502[47] Wirth K G et al 2022 Experimental observation of ABCB stacked tetralayer graphene ACS Nano 16 16617[48] Tebbe D, Schütte M, Watanabe K, Taniguchi T, Stampfer C, Beschoten B and Waldecker L 2023 Tailoring the dielectric screening inWS2-graphene heterostructures npj 2D Mater. Appl. 7 29[49] Ouaj T, Kramme L, Metzelaars M, Li J, Watanabe K, Taniguchi T, Edgar J H, Beschoten B, Kögerler P and Stampfer C 2023Chemically detaching hbn crystals grown at atmospheric pressure and high temperature for high-performance graphene devicesNanotechnology 34 47570313https://doi.org/10.1103/PhysRevApplied.13.064017https://doi.org/10.1103/PhysRevApplied.13.064017https://doi.org/10.1007/978-3-319-10602-1_48https://github.com/facebookresearch/detectron2https://doi.org/10.1016/j.eml.2020.100771https://doi.org/10.1016/j.eml.2020.100771https://doi.org/10.1016/j.carbon.2022.03.035https://doi.org/10.1016/j.carbon.2022.03.035https://arxiv.org/abs/2103.13495https://doi.org/10.35848/1882-0786/ac8270https://doi.org/10.35848/1882-0786/ac8270https://doi.org/10.1021/acsnano.2c06053https://doi.org/10.1021/acsnano.2c06053https://doi.org/10.1038/s41699-023-00394-0https://doi.org/10.1038/s41699-023-00394-0https://doi.org/10.1088/1361-6528/acf2a0https://doi.org/10.1088/1361-6528/acf2a0 An open-source robust machine learning platform for real-time detection and classification of 2D material flakes 1. Introduction 2. Workflow overview 3. Detection algorithm 4. Training of the detection algorithm 4.1. Implementation of different materials 4.2. Choice of silicon oxide thickness 5. Evaluation 5.1. Dataset 5.2. Metrics 5.3. Runtime 6. Discussion References