# Fileset [main.pdf](https://mdr.nims.go.jp/filesets/010da261-ffe7-49e9-ad13-2e630457f85f/download) ## Creator [FOPPIANO, Luca](https://orcid.org/0000-0002-6114-6164), [ROMARY, Laurent](https://orcid.org/0000-0002-0756-0508), [ISHII, Masashi](https://orcid.org/0000-0003-0357-2832), [TANIFUJI, Mikiko](https://orcid.org/0000-0001-5284-6364) ## Rights In Copyright[In Copyright](http://rightsstatements.org/vocab/InC/1.0/) ## Other metadata [Automatic Identification and Normalisation of Physical Measurements in Scientific Literature ](https://mdr.nims.go.jp/datasets/c861221b-e03e-4fc9-8cf6-8a4165174f34) ## Fulltext Automatic Identification and Normalisation of Physical Measurements in Scientific LiteratureAutomatic Identification and Normalisation of PhysicalMeasurements in Scientific LiteratureLuca FoppianoFOPPIANO.Luca@nims.go.jpNational Institute for Materials Science (NIMS)Tsukuba, JapanLaurent Romarylaurent.romary@inria.frInriaParis, FranceMasashi IshiiISHII.Masashi@nims.go.jpNational Institute for Materials Science (NIMS)Tsukuba, JapanMikiko TanifujiTANIFUJI.Mikiko@nims.go.jpNational Institute for Materials Science (NIMS)Tsukuba, JapanABSTRACTWe present Grobid-quantities, an open source application for ex-tracting and normalising measurements from scientific and patentliterature. Tools of this kind, aiming to understand and make un-structured information accessible, represent the building blocks forlarge-scale Text and Data Mining (TDM) systems. Grobid-quantitiesis a module built on top of Grobid [5], a machine learning frame-work for parsing and structuring PDF documents. Designed to pro-cess large quantities of data, it provides a robust implementationaccessible in batch mode or via a REST API. The machine learn-ing engine architecture follows the cascade approach, where eachmodel is specialised in the resolution of a specific task. The modelsare trained using CRF (Conditional Random Field) algorithm [11]for extracting quantities (atomic values, intervals and lists), units(such as length, weight) and different value representations (nu-meric, alphabetic or scientific notation). Identified measurementsare normalised according to the International System of Units (SI).Thanks to its stable recall and reliable precision, Grobid-quantitieshas been integrated as the measurement-extraction engine in vari-ous TDM projects, such as Marve (Measurement Context Extractionfrom Text), for extracting semantic measurements and meaning inEarth Science [9]. At the National Institute for Materials Sciencein Japan (NIMS), it is used in an ongoing project to discover newsuperconducting materials. Normalised materials characteristics(such as critical temperature, pressure) extracted from scientificliterature are a key resource for materials informatics (MI) [8].CCS CONCEPTS• Applied computing → Document analysis; Document meta-data; Format and notation.Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.DocEng’19, September 23–26, 2019, Berlin, DE© 2019 Association for Computing Machinery.ACM ISBN 978-x-xxxx-xxxx-x/YY/MM. . . $15.00https://doi.org/10.1145/1122445.1122456KEYWORDSMachine Learning, TDM, Measurements, Physical quantities, Unitsof measurementsACM Reference Format:Luca Foppiano, Laurent Romary, Masashi Ishii, and Mikiko Tanifuji. 2019.Automatic Identification and Normalisation of Physical Measurementsin Scientific Literature. In Proceedings of DocEng’19: ACM Symposium onDocument Engineering (DocEng’19). ACM, New York, NY, USA, 4 pages.https://doi.org/10.1145/1122445.11224561 INTRODUCTIONThe data overflow in scientific publications makes rapid access torelevant information a challenging issue, for both researchers andreaders. One of the essential element found in scientific literature isthe physical quantity or measurement, which combine quantifica-tion of units (such as grams or micrometres) and quantified objector substances. The automatic extraction of measurements has beenstudied for many years. Nowadays, although the technology hasbeen evolved, there are still several challenges to overcome: (1) nat-ural language and writing style have varieties of expressions (forexample length can be expressed as m, meter, metre). (2) Overlapsbetween the different units of measurement (pico Henry inductanceand acidity share the same notation, pH ). (3) The physical quantitiesor measurements are scalable by accompanying units (e.g., 1 pl. =453.6 g), meaning that value and unit combination and its normalisa-tion are necessary for semantic recognition. The need for a preciseautomatic generation of databases from physical measurements iscommon to a wide range of domains.In this paper, we present Grobid-quantities, an Open Sourceapplication for identifying, parsing and normalising measurementsfrom scientific and patent literature. Using Conditional RandomField (CRF) [11], it provides a machine learning framework forextracting information in a robust manner, and then normalisethem toward the International System of Units (SI). This article isorganised as follow. In Section 2 we introduce related work. Then,we describe the system in Section 3 and report its evaluation resultsin Sections 4. Use cases and future scopes are described in Section5. Section 6 concludes this paper.2 RELATEDWORKAttempts to extract measurements from text have been made usingrule-based (formal grammars engines, look-ups in terminologicalhttps://doi.org/10.1145/1122445.1122456https://doi.org/10.1145/1122445.1122456DocEng’19, September 23–26, 2019, Berlin, DE Foppiano, et al.databases) and ML approaches. A known commercial tool, Quan-talyze1, was reported by [9] showing weak recall and supportingonly a limited subset of units [3]. Another approach [1], usingGATE (General Architecture for Text Engineering), addressed theidentification of numeric properties from patents. [2] investigatedissues applied to Russian-derived languages. These approaches lackeither the generalisation to an extensive corpus or deal mainly withspecific languages. [4] described an attempt to recognise units bylooking up terms from an ontology, using ML in combination withpattern matching and string metrics. Other ML-based approachesexist, although limited to specific domains: [10] and [7] describemeasurements extraction from experimental results in biology andnanocrystal device development, respectively. Our work is not re-stricted to a specific domain or subset of measurements and includesa normalisation process.3 SYSTEM DESCRIPTIONGrobid-quantities is a Java application, based on Grobid (GeneR-ation Of BIbliographic Data) [5], a machine learning frameworkfor parsing and structuring raw documents such as PDF or plaintext. Grobid-quantities is designed for large-scale processing tasksin batch or via a web REST API. Results information are standard-ised and can be stored in databases or visualised on PDF, via theobtained GROBID build-in positional coordinates.3.1 Data modelFigure 1: Schema of the data model.The data model (Figure 1) lay its foundation on the concept ofMeasurement, which links an object or a substance with one ormore quantities. We defined four Measurements types: (a) atomic,in case of a single measurement (e.g., 10 grams). (b) interval (from 3to 5 km) and (c) range (100 ± 4 mm) for continuous values, and, (d)a list of discrete values. A Quantity links the quantitative value andthe unit. At this stage we do not support probability distribution ofranges and intervals. Since data extracted from PDFs unavoidablypresent irregular tokens from wrong UTF-8 encoding or missingfonts, we designed this model to allow partial results. The Valueand Unit entities allow three different representations (Figure 1):1https://www.quantalyze.com/Raw as appear in input, Parsed unifies the value into the numericalexpression, and the unit with its properties (system, type). Finally,Normalised contains the transformed unit and values to the SI sys-tem. Value object supports four types of representations: numeric(2, 1000), alphabetic (two, thousand), scientific notation (3 ·105), andtime, which is also expression of measurements. Units objects areorganised following the SI, which allows representing units as prod-ucts of simpler compounds (e.g. m/s tom · s-1) further decomposedas triples (prefix, base and power).3.2 ArchitectureThe system takes in input text or PDF and performs three steps:(a) tokenisation, (b) measurement extraction and parsing and (c)quantity normalisation. The details of each step are summarised asfollows.3.2.1 Tokenisation. This process splits input data into tokens. Grobid-quantities uses a two-phase tokenisation: (1) first it splits by punctu-ationmarks, then (2) each resulting token is re-tokenised to separateadjacent digits and alphanumeric characters. Given the example25mˆ2, first returns a list [25m, ˆ, 2] and then recursively divides25m as [25, m] resulting in [25, m, ˆ, 2].3.2.2 Extraction. The tokenised data is labelled by three models,applied in cascade: the Quantities CRF model determines appro-priate unit and value tags. Results are processed in cascade by therespective Units and Values CRF models as illustrated in Figure 2.Figure 2: The cascade approach in applied CRF models.The Quantities model recognises value and units which arepassed, respectively, toValues andUnits CRF sub-models forfurther extraction.As illustrated in Table 1, Quantities CRF model uses additionallabels (such as , for units) to correctly re-construct complex objects from the flat structure obtained from thesequence labelling output.Previous work (Section 2) presented extensive use of databasesor ontologies. In our solution, we used a similar approach. Wecreated a list of units (in English, French and German) with theircharacteristics: system (SI base, SI derived, imperial, ...) and type(volume, length, ...), and their representations: notations (m3, mˆ3),lemmas (cubic meter, cubic metre) and inflections (cubic meters,cubic metres). We made this list available through the Unit Lexicon,Automatic Identification and Normalisation of Physical Measurements in Scientific LiteratureDocEng’19, September 23–26, 2019, Berlin, DETable 1: Labels description for the CRFmodel for Quantities.In bold the token the label refers to.Label Description Example value of an atomic quantity 2 m least value in an interval from 2 m max value in an interval up to 7 m base value in a range 20 ± 7 m range value in a range 20 ± 7 m list of quantities 2, 3 and 10 m left-attached unit pH 2 right-attached unit 2 m everything else -which offers unit lookups by properties (such as notation, lemma,inflexion). A second gazetteer was created to allow the transforma-tion of alphabetic values in numeric ones (for example, twenty-oneto 21).Features in the Quantities CRF model are generated using stan-dard information (preceding and following tokens, presence ofcapital, digits). Orthogonal features are obtained through the UnitLexicon, like a Boolean indicating whether a token is a knownunit or not. Typographical information (such as format, fonts, sub-script/superscript) are ignored.The Units CRF model works at character level and uses the UnitLexicon to highlight known units or prefixes. The input tokens areparsed and transformed to a product of triples (prefix, base, power)as shown in Table 2. For example Kg/mm2, corresponds toKд ·mm-2and becomes [(K, g, 1), (m, m, -2)] as product of triples.Table 2: Labels description for the CRF Unitsmodel. The ex-ample shows in bold the part referred by the label.Label Description Example prefix of the unit km2 unit base km2 unit power km2 everything else -We then use the structured triples to fetch the corresponding(system, type) information from the Unit Lexicon and attach them tothe resulting object. At the moment we do not exploit any contex-tual information related to the paper domain to resolve ambiguousunits. In parallel, the CRF Values model unifies the format of identi-fied values into numerical formats. It supports four types: numeric,alphabetic, scientific notation, and time expression (see Table 3).Each type is treated with different parsers, namely alphabetic ex-pressions are looked up in the word-to-number gazetteer, scientificnotations are parsed and calculated mathematically. Time expres-sions are processed using the built-in Date Grobid model [5].3.2.3 Normalisation. Themeasurements extracted are transformedto the base SI unit (grams to kg, Celsius to Kelvin, and so on). Weused an external Java library called Units of Measurement whichprovides a set of standard interfaces and implementations for safelyTable 3: Labels description for the CRF model for Values. Inbold an example of tokens for the specific label recognise.Label Description Example numeric value / coefficient 2.5 · 105 alphabetic value twenty