# Fileset

[Relation extraction with weakly supervised learning based on process structure property performance reciprocity.pdf](https://mdr.nims.go.jp/filesets/5c8e1798-4a54-4d44-b384-fa9ddd26788e/download)

## Creator

Takeshi Onishi, [Takuya Kadohira](https://orcid.org/0000-0003-0569-1309), [Ikumu Watanabe](https://orcid.org/0000-0002-7693-1675)

## Rights

Creative Commons BY Attribution 4.0 International[Creative Commons BY Attribution 4.0 International](https://creativecommons.org/licenses/by/4.0/)

## Other metadata

[Relationship Extraction with Weakly Supervised Learning Based on Process-Structure-Property-Performance Reciprocity](https://mdr.nims.go.jp/datasets/bbe7863a-df16-4e86-a85a-508fbb4e473f)

## Fulltext

Relation extraction with weakly supervised learning based on process-structure-property-performance reciprocityFull Terms & Conditions of access and use can be found athttps://www.tandfonline.com/action/journalInformation?journalCode=tsta20Science and Technology of Advanced MaterialsISSN: 1468-6996 (Print) 1878-5514 (Online) Journal homepage: https://www.tandfonline.com/loi/tsta20Relation extraction with weakly supervisedlearning based on process-structure-property-performance reciprocityTakeshi Onishi, Takuya Kadohira & Ikumu WatanabeTo cite this article: Takeshi Onishi, Takuya Kadohira & Ikumu Watanabe (2018) Relationextraction with weakly supervised learning based on process-structure-property-performance reciprocity, Science and Technology of Advanced Materials, 19:1, 649-659, DOI:10.1080/14686996.2018.1500852To link to this article:  https://doi.org/10.1080/14686996.2018.1500852© 2018 The Author(s). Published by InformaUK Limited, trading as Taylor & FrancisGroup.Published online: 19 Sep 2018.Submit your article to this journal Article views: 5867View related articles View Crossmark dataCiting articles: 8 View citing articles https://www.tandfonline.com/action/journalInformation?journalCode=tsta20https://www.tandfonline.com/loi/tsta20https://www.tandfonline.com/action/showCitFormats?doi=10.1080/14686996.2018.1500852https://doi.org/10.1080/14686996.2018.1500852https://www.tandfonline.com/action/authorSubmission?journalCode=tsta20&show=instructionshttps://www.tandfonline.com/action/authorSubmission?journalCode=tsta20&show=instructionshttps://www.tandfonline.com/doi/mlt/10.1080/14686996.2018.1500852https://www.tandfonline.com/doi/mlt/10.1080/14686996.2018.1500852http://crossmark.crossref.org/dialog/?doi=10.1080/14686996.2018.1500852&domain=pdf&date_stamp=2018-09-19http://crossmark.crossref.org/dialog/?doi=10.1080/14686996.2018.1500852&domain=pdf&date_stamp=2018-09-19https://www.tandfonline.com/doi/citedby/10.1080/14686996.2018.1500852#tabModulehttps://www.tandfonline.com/doi/citedby/10.1080/14686996.2018.1500852#tabModuleRelation extraction with weakly supervised learning based onprocess-structure-property-performance reciprocityTakeshi Onishia, Takuya Kadohirab and Ikumu Watanabe caToyota Technological Institute at Chicago, Chicago, IL, USA;bResearch and Services Division of Materials Data and Integrated System, National Institute for Materials Science, Tsukuba, Ibaraki,Japan;cResearch Center for Structural Materials, National Institute for Materials Science, Ibaraki, Tsukuba, JapanABSTRACTIn this study, we develop a computer-aided material design system to represent and extractknowledge related to material design from natural language texts. A machine learning modelis trained on a text corpus weakly labeled by minimal annotated relationship data (~100labeled relationships) to extract knowledge from scientific articles. The knowledge is repre-sented by relationships between scientific concepts, such as {annealing, grain size, strength}.The extracted relationships are represented as a knowledge graph formatted according todesign charts, inspired by the process-structure-property-performance (PSPP) reciprocity. Thedesign chart provides an intuitive effect of processes on properties and prospective processesto achieve the certain desired properties. Our system semantically searches the scientificliterature and provides knowledge in the form of a design chart, and we hope it contributesmore efficient developments of new materials.ARTICLE HISTORYReceived 14 March 2018Revised 12 July 2018Accepted 12 July 2018KEYWORDSNatural languageprocessing; knowledgeextraction; relationextraction; weaklysupervised learning;materials informaticsCLASSIFICATION60 New topics/Others; 404Materials informatics /Genomics1. IntroductionMachine learning and data science for knowledgeextraction are studied in a wide variety of field.Knowledge extraction is to find desired knowledgefrom text. For example, relationships among scientificknowledge are extracted from scientific literature inScienceIE [1], and a knowledge base is extracted fromWeb text in TAC.1 The impressive performance ofmachine learning appears promising for knowledgeextraction in material design as well.Material design is a process of developing newmaterials with specific properties. In most practicalcases, the desired process cannot be envisioned, andinstead it is constructed by trial and error. In thisapproach, a trial is a time-consuming experiment.Minimizing the number of trials is critical for anefficient development. On the contrary, such efficientdevelopment is challenging because 1) The relationbetween a process and a property is unclear andindirect; and 2) The search space (the set of possibleprocesses) is too large to look up.In practice, researchers find these processes relyingon their end-to-end knowledge including effects ofprocesses to the properties. Such knowledge is tech-nical and might not be well formalized, so they spendlong time to obtain such knowledge. We believe it isbeneficial to provide the end-to-end knowledge foraccelerating material developments.1.1. Knowledge representationThe processing-structure-property-performance (PSPP)reciprocity [2] explains effect of processes on propertiesin three stages. The first stage is ‘process’ that can becontrolled to develop a newmaterial. The second stage is‘structure’ of the material that the processes build. Thethird stage is ‘property’ that the structure gives. Theproperties in the third stage give the total performanceof the new material.CONTACT Ikumu Watanabe WATANABE.Ikumu@nims.go.jp Research Center for Structural Materials, National Institute for Materials Science, 1–2–1Sengen 305–0047, Tsukuba, Ibaraki, JapanSCIENCE AND TECHNOLOGY OF ADVANCED MATERIALS2018, VOL. 19, NO. 1, 649–659https://doi.org/10.1080/14686996.2018.1500852© 2018 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permitsunrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.http://orcid.org/0000-0002-7693-1675http://www.tandfonline.comhttp://crossmark.crossref.org/dialog/?doi=10.1080/14686996.2018.1500852&domain=pdfThe PSPP design chart [3] represents end-to-endknowledge in the form of relationships among fac-tors, as shown in Figure 1 [4]. A factor represented bya node is an important phenomenon or concept formaterial design, such as annealing, grain size, andstrength. A factor is classified into one of the threestages, ‘process’, ‘structure’, and ‘property’, where thefactor performs. For example, annealing is an impor-tant concept in ‘process’. Following the PSPP reci-procity, a factor is influenced by connected factors toits left, and influences factors connected to its right(i.e. a process builds structures, and a structure influ-ences properties of the material). These relations arerepresented by their connectivities. The chart intui-tively represents end-to-end knowledge in the formof relationships between processes and propertiesmediated by structures [5].1.2. Data limitationDespite major developments in machine learning, thetechnology suffers from limited data availability formodel training in practical cases. For example,AtomWork [6], one of the largest databases availablefor material design, contains records of more than55,000 properties of materials. However, the knowledgein the database unlikely covers all knowledge needed formaterial design. For example, AtomWorks covers crys-tal structures and related properties such as lattice con-stant and space group but microstructures of materialsare unlisted. Such limited data likely lead to over-fittingduring training, and thus to poor performance.In this study, we aim to overcome the problem oflimited data availability using Natural LanguageProcessing (NLP), an application of machine learningtechnologies to natural language resources, such asscientific articles and Web texts. Natural languageresources are widely available and are machine read-able, for a variety of fields including material design.For example, Elsevier’s API2 provides access to over250,000 fully digitized scientific articles. More impor-tantly, natural language is the most popular means ofrepresenting knowledge, and our desired knowledgeis thus likely to be present.1.3. Weakly supervised learningWe leverage weakly supervised learning to identifyrelationships in the PSPP chart. Weakly supervisedlearning is used to train a model with a minimalnumber of annotations for relation identification[7]. In a typical supervised setting, the trainingdata for relation identification is a sentence labeledwith entities in the sentence and the relationsamong them. However, labeling sentences is expen-sive because an annotator must read a sentence,and label the entities and relations describedtherein. This renders each label clean and strong.On the contrary, in a weakly supervised setting, aknowledge base produces a pair of entities andtheir relations and all sentences containing theseentities are weakly labeled with the given relation.For instance, for the given entities, ElevationPartners and Roger McNamee, and their relationFigure 1. The process-structure-property-performance reciprocity.Sci. Technol. Adv. Mater. 19 (2018) 650 T. ONISHI et al.founded_by, both of the following sentences arelabeled with founded_by,Elevation Partners, the $1.9 billion private equitygroup that was founded by Roger McNamee;Roger McNamee, a managing director at ElevationPartners,. . .;where the first sentence describes the relation foun-ded_by but the second sentence does not. Weak labelsdo not require any annotations but the knowledgebase, however; they are noisy, and a model needs toovercome the noisy labels.In recent years, convolutional neural network(CNN) models have surpassed feature-based models[7–11]. CNNs are a class of neural networks with con-voluted neural units. Residual learning is used to helpthe deep CNN network [12]. Zeng et al. [13] split asentence into three parts, and then applied max poolingto each part of the sentence over a CNN layer.Sentence-level attention is introduced for selectinga key sentence. In this approach, a network takes a setof sentences for a relation between two entities. Eachsentence contains both entities. An attentionmechanism over a CNN allows the network to auto-matically select a key sentence, which is likelydescribing the desired relation. It seems helpful toovercome the problem of noisy labels [14–16].1.4. Our contributionWe develop a computer-aided material design(CAMaD) system with the aim of generating a PSPPdesign chart for desired properties from text. ThePSPP chart suggests prospective processes to achievegiven arbitrarily desired properties. The system isbased on a machine learning model for relationextraction from text. The model is efficiently trainedwith weakly supervised learning, which minimizes theannotation cost of the training data. We believe thePSPP chart helps more efficient material develop-ments by suggesting a prospective process.Our contribution in this study is twofold. First, weproposed a novel knowledge graph based on PSPPcharts and developed a system to build the knowledgegraph from text using NLP technologies. Second, weexperimentally verified that such technical knowledgecan be extracted from text using machine learningmodels. Our target knowledge is relations in PSPPdesign charts. These relations appear rather technicaland significantly different from typical relations inNLP such as ‘has_a’ and ‘is_a’. Extraction of theserelations from text only is difficult and might needother knowledge resource such as equations andproperties of materials. We, however, experimentallyverified that a state-of-the-art machine learningmodel can extract these relations from text.In the following sections, we formalize our task withthree subtasks in Section 2 and describe our pipelinesystem for each subtask in Section 3. We are especiallyinterested in the second subtask; we evaluated oursystem for the subtask in Section 4, and present theresults in Section 5. We also briefly describe the end-to-end implementation in Section 6, and future works inSection 7.2. Our approach and task definitionOur approach is knowledge graph population andgraph search. The knowledge graph representsknowledge concerning material design. It consists offactors and their relations following PSPP reciprocity,and a PSPP chart is considered as a part of the graphwith factors related to desired properties. Thus in thisapproach, we first extract the structure of the graphfrom text (knowledge graph population) and thenfind a PSPP chart from the graph for a desired mate-rial (graph search).The PSPP knowledge graph consists of nodes andedges between them. Each node represents a factor, animportant concept in material design. A factor is clas-sified into one of ‘process’, ‘structure’, and ‘property’.Each edge is a relation between factors represented bythe nodes. The nodes are connected by the edge if andonly if the corresponding factors are related in thePSPP chart. Therefore, there is no edge between pro-cessing factors and structural factors.A PSPP chart is considered part of the PSPPknowledge graph for a developing material. Forinstance, considering a typical material developmentscenario, where a new material is desired with specificproperties, the PSPP chart is composed of factorsrelated to the desired properties. The PSPP chart ispart of the PSPP knowledge graph around the nodesof the desired property factors.Following this approach, the task is decomposedinto three subtasks: factor collection, relation identi-fication, and branching.The first subtask is to collect factors for nodes inthe PSPP knowledge graph, and these factors areclassified into each PSPP class. A factor is an impor-tant scientific concept for material design and is clas-sified into ‘process’, ‘structure’, or ‘property’following PSPP reciprocity. For example, process fac-tors include ‘tempering’ and ‘hot working’; structuralfactors include ‘grain refining’ and ‘austenite disper-sion’; and property factors include ‘strength’ and‘cost’. Each factor is represented by a node in thePSPP knowledge graph, i.e. a node represents a pro-cessing, structural, or property factor.The second subtask is relation identification,where relations among nodes in the PSPP knowledgegraph are identified by reading text. In this subtask,for two given nodes and sentences mentioning theSci. Technol. Adv. Mater. 19 (2018) 651 T. ONISHI et al.factor represented by these nodes, their relation isidentified. Following PSPP charts, the relation islabeled in binary manner, i.e. positive or negative. Apositive relation between factors A and B indicatesthat ‘factor A affects factor B’, and a negative relationbetween factors A and C indicates that ‘factor Aoccurs independently of factor C’. In a chart, twonodes are connected if their factors of the nodeshave a positive relation, and are otherwise uncon-nected. Thus, denoting a pair of factors by ðf1; f2Þ,the desired relation ishðf1; f2Þ ¼ pos=neg: (1)The third subtask is to obtain a PSPP chart bybranching the PSPP knowledge graph. We assume ascenario where a scientist is developing a new mate-rial with certain desired properties and looking forfactors related to the properties in a PSPP chart. Inthis scenario, the PSPP chart is part of the PSPPknowledge graph, with certain factors around thedesired properties. Thus, the subtask is to find partof the PSPP knowledge graph given a set ofproperties.3. System descriptionOur system is a pipeline system consisting of threecomponents for each subtask. The first componentcollects factors from a keyword list (Section 3.1). Inthe second component, a relation between two nodesis identified by reading sentences containing the fac-tors represented by the nodes in the CNN model(Section 3.2). In the third component, part of thePSPP knowledge graph is extracted for the givendesired properties by a simplified maximum flowalgorithm (Section 3.3).3.1. Factor collectionFirst, factors are collected from the keyword list ofthe journal Scripta Materialia.3 Such keywords helpidentify the topics of each article. The keyword list isdivided into five sections: 1) Synthesis andProcessing; 2) Characterization; 3) Material Type; 4)Properties and Phenomena; and 5) Theory,Computer Simulations, and Modeling. In thisapproach, keywords in synthesis and processing arecollected as processing factors, those in material typeare collected as structural factors, and keywords inproperties and phenomena as property factors.Second, structural factors are collected from textusing linguistic rules. From a material science stand-point, the number of structural factors is significantlygreater than those of processing and property factors,and the keyword list is not long enough to cover struc-tural factors. Candidate phrases, noun phrases consist-ing of multiple NNs (singular nouns, or mass nouns),are collected from a corpus described in Section 4.2using Stanford CoreNLP [17]. Each candidate phraseis classified into structural factors if it does not containany words in the keyword list. The phrase containing akeyword is classified as the class of the keyword. Forinstance, Figure 2 lists two sentences with noun phrases.Here ‘phrase_transition’ is classified as a structural fac-tor, but ‘hardness_distribution’ is classified as a prop-erty factor, as ‘hardness’ is in the keyword list.All keywords and the n most frequent candidatephrases are collected, and each word/phrase isassigned a node in the PSPP knowledge graph. Thetotal number of factors was 500, 500, and 1000 forprocess, property, and structural factors, respectively.Table 1 lists samples of the n most frequent phrases.3.2. Relation identificationIn this section, we describe our CNN model for identi-fying the relation of a factor pair by weakly supervisedlearning. We also describe the linguistic resource forthe factor pair where the model was trained.The linguistic resource of a factor pair is a set ofsentences mentioning both factors. A mention is apart of a sentence referring to a factor. A factor ismapped to the mention in the sentence by max-spanstring matching, i.e. a factor is mapped to the men-tion if the mention is the factor name, and no othermention overlaps the given mention. For instance,• Within each phase, the properties are . . .• When a substance undergoes a phase transi-tion . . .The phase in the first sentence is mapped to a factor,‘phase’, but phrase transition is mapped to ‘phase_-transition’ instead of ‘phase’ in the second sentence.Table 1. Samples of factors obtained by the linguistic rules.Process Structure Propertywater quenching carbon dioxide creep behaviorelement modeling grain distribution fatigue behaviorpeak temperature particle sizedistributionmisorientation anglerolling texture matrix phase shock resistancedeformation mode β titanium alloy fracture strainmicrowavesinteringβ grain size tensile ductilityplasma sintering solution strength fracture behaviordischargemachiningpore size vacuum inductionmelting(1)(2)Figure 2. Sentences containing noun phrases.Sci. Technol. Adv. Mater. 19 (2018) 652 T. ONISHI et al.A sentence in the linguistic resource of a factor pair isa sentence mentioning both factors.The CNN model proposed by Huang et al. [12] is astate-of-the-art deep neural network model forweakly supervised relation extraction. For each sen-tence in a linguistic resource, the network takes wordembeddings and the relative position embeddingstoward factors in the sentence. Convolutional unitswith a deep residual learning framework then embedthe sentence into a vector representation. The sig-moid layer over the vector representation producesthe probability distribution of the binary relation.Figure 3 shows the overall structure of the network.The input to the model is a sentence and the outputis a relation r 2 pos; negf g. Let ðf1; f2Þ be factors of therelation and s 2 Sf1;f2 be the sentence, i.e. the linguisticresource, Sf1;f2 is a set of sentences mentioning thefactors. Each sentence s is padded to a fixed length L.A token embedding is a vector representation of atoken in a sentence, denoting the sentences ¼ t0; :::; ti; :::f g, where ti is the ith token. Thetoken representation is xi, which is a concatenationof a word embedding and two position embeddings.A word embedding Wð�Þ is a vector representation ofthe word of the token whereas a position embeddingPð�Þ gives a vector representation of the relative posi-tion of each factor:xi ¼ ½WðtiÞ; P1ðk1 � iÞ;P2ðk2 � iÞ� (2)where Wð�Þ 2 Rdw , Pð�Þ 2 Rdp and k1; k2 are theposition indices of each factor. Note that any relativedistance greater than Dmax is treated as Dmax.A convolution layer takes embeddings aroundposition i, and maps them into ci 2 Rdc :ci ¼ gðwxi:iþh þ bÞ (3)where xi:iþh ¼ ½xi; xiþ1; :::; xiþh�1�, w 2 Rdc�hðdwþ2dpÞand b 2 Rdc is a bias. g is an element-wise non-linearfunction, ReLU.Following the first convolutional layer, the otherlayers are stacked with residual learning connectionsthat directly transmit a signal from a lower to a higherlayer while skipping the middle layers. Thus, the kthresidual CNN block consists of two CNN layers, withone taking signals from the two lower layers:ĉki ¼ gðŵkð~ck�1i:iþh þ ~ck�2i:iþhÞ þ b̂kÞ (4)~cki ¼ gðewkĉki:iþh þ ~bkÞ (5)where ~c0 ¼ c. The first CNN layer ĉki takes a signalfrom the immediately lower layer ~ck�1i:iþh and anothersignal from the lower block ~ck�2i:iþh.Max pooling is performed over the output of thelast CNN units, ~cK 2 RL�hþ1�dc .z ¼ maxpooli~cKi (6)Then, two fully connected layers and a sigmoid func-tion yield the probability distribution of the desiredrelation given in the sentence Pðr ¼ pos=negjsÞ:z1 ¼ gðwg1z þ bg1Þ (7)Figure 3. Structure of the CNN model. The convolutional layers embed a sentence, and the max pooling and two fullyconnected layers give a binary probability distribution with a sigmoid function.Sci. Technol. Adv. Mater. 19 (2018) 653 T. ONISHI et al.z2 ¼ gðwg2z1 þ bg2Þ (8)Pðr ¼ posjsÞ ¼ σðvrz2Þ (9)where wg 2 Rdc�dc and bg 2 Rdc .The desired probability Pðr ¼ posj f1; f2Þ is themaximum of the probabilities over sentences. This isPðr ¼ posj f1; f2Þ ¼ maxs2Sf1 ;f2Pðr ¼ posjsÞ (10)On the contrary, the model is trained on a weaklysupervised approach, where the objective function ismaximized for each sentence.maxΦXðf1;f2;rÞ2DtrainXs2Sf1 ;f2logPðrjsÞ (11)where Dtrain is the training data, a set of tuples offactors f1; f2 and relation r 2 pos; negf g. The para-meters Φ ¼ W; P1; P2;w; bf g3.3. BranchingThe PSPP knowledge graph is branched for the givendesired properties without losing related factors. Weconsider the branching of a max-flow problem, wherethe flow occurs from the given property factors to theprocessing factors. The inlets are all nodes of the pro-cessing factors and the outlets are those of the givenproperties. The capacity of each edge is the score of therelation, i.e. Pðr ¼ posjf1; f2Þ. Wemaximize the amountof flow with a limited number of nodes in the graph.We compute the capacity of a node in the graph,which is the amount of flow that it can accept.Recalling that nodes of structural factors are connectedto property and processing factors, and no processingfactor and property factor are connected, all flow passthrough the nodes of the structural factors. The capa-city of the node of a structural factor fstr isCfstr ¼ minXf2PRCPðr ¼ posj f ; fstrÞ;Xf2PRP0Pðr ¼ posj fstr; f Þ0@1A(12)where PRC represents processing factors and PRP’ isthe desired property factors. Similarly, the capacity ofa node of a processing factor fprc isCfprc ¼Xf2STR0Pðr ¼ posj fprc; f Þ (13)where STR’ represents structural factors that are notbranched.The desired PSPP chart is composed of n processingfactors, m structural factors, and the desired propertyfactors, where n andm are the given hyper-parameters.The nodes of the processing/structural factors are the nandmmost capable nodes. For efficiency, the nodes aregreedily searched such that optimality is not guaranteed.The PSPP chart shows the processing/structural factorsrelated to the desired properties.4. Experiment for relation identificationRelation identification is a challenging subtask in thisresearch. The system performance on the subtask wasevaluated in a weakly supervised relation extractionsetting. In this evaluation, the relation was identifiedas positive/negative for each factor pair.The training data consisted of relationship dataand a corpus. The relationship data consisted of apair of factors and its relation label. The corpus,scientific literature, was a set of sentences describingthe factors. A model was trained on part of therelationship data and the corpus and was tested onheld-out data and the corpus by predicting relation-ships in the held-out data.Our model was trained using stochastic gradientdescent and dropout. Dropout randomly drops somesignals in the network that are thought to help thegeneralization capabilities of the network. Weemployed an Adam optimizer with a learning rateof 0.00005 and randomly dropped signals from maxpooling during training with a probability of 20%.The word embeddings were initialized with GloVevectors [18]. Other hyper-parameters are listed inTable 2.4.1. Relationship dataThe relationship data consisted of tuples of two fac-tors and their binary relation (pos/neg). From fourdesign charts [5], 104 factor pairs were collected asshown in Tables 3 and 4.Table 3. Factors in the relationship data.Category SizeProcess 17Structure 21Property 6Table 2. Hyper-parameters of the CNN model.Parameter ValueL 100Dmax 30K 4h 2dw 50dp 5dc 50L2 regularization 0.0001Table 4. Relations in the relationship data.Relationship type Positive NegativeProcess $ Structure 14 49Structure $ Property 10 31Sci. Technol. Adv. Mater. 19 (2018) 654 T. ONISHI et al.For evaluation, our model was trained on part of therelationship data and tested on the held-out data. Thetraining data consisted of relationships from three arbi-trary charts out of the four, and the test data consisted ofrelationships in the fourth chart. Thus, four pairs oftraining and test data were prepared for the evaluation.For accurate evaluation, the likelihoods of relationshipsin the test data were computed by a model trained usingthe corresponding training data. Precision and recallcurves were then computed for the overall relationshipdata to obtain a smooth curve.4.2. CorpusOur corpus consisted of publicly accessible scientificarticles on ScienceDirect.4 ScienceDirect is an Elsevierplatform providing access to articles in journals in avariety of fields, such as social sciences and engineer-ing. Approximately 3400 articles were collected usingthe keyword search on ScienceDirect. The keywordswere ‘material’ and ‘microstructure’, i.e. each articlewas related to both ‘material’ and ‘microstructure’.The CNNmodels were trained on a pair of factors andsentences. As described in Section 3.2, each sentencementions both factors. For the relationship data, about5000 sentences were founded in the corpus in total,roughly 50 sentences for each pair of factors on average.4.3. Baseline modelsA baseline model is a text-classification-based binaryclassifier where for each factor pair, each classifiertakes a set of sentences mentioning the factors andclassifies the text into a positive or negative relation.The problem setting and the set of sentences wereexactly the same as the one in the CNN model.Logistic regression and SVM with bag-of-words fea-tures were employed for the binary classifier. These arestandard machine learning binary classifiers. Bag-of-words is a feature that indicates whether a word is in aset of sentences. The feature is represented by a sparsebinary vector, where an element is one if the corre-sponding word is in the sentences and zero otherwise.Stop words removal and n-gram features areexplored in Figures 4 and 5; however, the effect waslimited. Note that the radial basis function (RBF)kernel was used in all SVM models.4.4. Evaluation metricThe evaluation metrics were precision and recall,which are the standard metrics for informationextraction tasks. Precision is the ratio of correctlypredicted positive factor pairs to all predicted positivefactor pairs and gives the accuracy of the prediction.Recall is the ratio of correct predictions to all positivefactor pairs in the test data and gives the coverage ofthe prediction. A positive factor pair is a pair whoserelation is positive. We obtain high precision and lowrecall if a system returns only a small number of highconfidence predictions, and low precision and highrecall if a system returns many low confidence pre-dictions. Typically, these are balanced by a hyper-parameter (confidence) of system prediction. Thus,the trajectory of precision and recall pairs is com-puted with various values of the hyper-parameter andis called as a precision-recall curve.In this evaluation, the hyper-parameter was an integert, the number of positive factor pairs in the prediction.For a given t and a set of factor pairs in the test relation-ship data, the system predicts a binary relation, pos/neg,for each pair. It predicts the t most likely positive pairs,and the other pairs are predicted as negative.The factor pairs in the test relationship data werescored by a machine learning model trained on thecorresponding training relationship data, where thescore was Pðr ¼ posjf1; f2Þ. A test data correspondedFigure 4. Precision-recall curve of the logistic regression model.The features are ‘bag of words’, ‘bag of words + stop wordremoval’ and ‘bag of unigram + bigram + trigram’.Figure 5. Precision-recall curve of the SVM model. The fea-tures are ‘bag of words’, ‘bag of words + stop word removal’and ‘bag of unigram + bigram + trigram’.Sci. Technol. Adv. Mater. 19 (2018) 655 T. ONISHI et al.with a training data, unaware of the relationships in thetest data (Section 4.1). A model was trained on thecorresponding training data and scored a pair in thetest data to avoid letting the model know the true rela-tionships during training.Then, a precision and recall pair for a given hyper-parameter t was computed as follows:Precisiont ¼ Rt \Rtestj jt(14)Recallt ¼ Rt \Rtestj jRtestj j (15)where Rtest is the set of factor pairs with positiverelations in all test relationship data, and Rt repre-sents the t most likely positive factor pairs. The like-lihood was a score given by the model.5. Results of relation identificationPrecision-recall curves for the baselinemodels are shownin Figures 4 and 5. These figures show various featurerepresentation schemes, such as stop words and n-grams(Section 4.3) on the logistic and SVMmodels. The logisticmodel performed well on low recall space, i.e. most con-fidently predicted positive factor pairs were actually posi-tively related. On the contrary, the performance of theSVM model was poorer in the space but better overallthan the logistic model. In both models, the effects of thefeature representation schemes were limited.The performance of the CNN model is shown inFigure 6. The precision was one when the recall wasabout 0.4, i.e. roughly speaking half the positive factorpairs were perfectly identified. The performance wassuperior to that of the baseline models.Table 5 shows some representative sentences scoredby the CNN model. A representative sentence is thehighest scored sentence in a sentence set Sf1;f2 foreach factor pair, i.e. a representative sentence iss0 ¼ argmaxs2Sf1 ;f2Pðr ¼ posjsÞ. The score is a likelihoodwhere a positive relation is described in the sentence. Thesentence with the highest score in each sentence set mostlikely represents the positive relation of each factor pair,according to the CNN model.Representative highly scoring sentences seem todescribe the desired relations (sentences 4 and 8) and,interestingly, relations described in the equation werealso discovered by the model (sentences 2, 3, and 6).This implies that some important relations tend to bedescribed in an equation. This result also indicates thatthe relations in which we are interested are significantlydifferent from typical relations in other NLP tasks like‘has_a’, ‘is_a’.6. End-to-end systemAn end-to-end demo system was developed to test ourCAMaD system on Apache Tomcat5 as Figure 7. Thedemo systemworked in a typical scenario, where a scien-tist was looking for factors related to certain desiredproperties. The demo system provides a PSPP designchart for the desired properties that the scientistprovided.The system input consisted of the desired propertiesalong with a base material. The desired properties wereselected from a list of properties collected as in Section3.1. The base material was the target material, such asFigure 6. Precision-recall curve over the relationship data ofthe CNN model.Table 5. Sample representative sentences scored by the CNNmodel. Label P indicates that the factors are positively relatedin the test relationship data and label N indicates a negativerelation. Factors in each sentence are underlined. The score isthe vrz2 of each sentence. See Appendix for the sourcearticles.Score/Label Sentence1 36.5/P . . . the following matrix form: [11] k,u ¼ λu . . .2 34.8/P . . . δc ¼ rσc=τ is the characteristic or critical whiskerlength, f and r . . . τ is the matrix shear strength . . .3 34.2/P . . . toughness (δkcb) and grain . . . dvpwhere, d is thematrix . . .4 31.0/P . . . cast iron has a pearlite matrix and . . .5 28.6/P after solution treatment, the increase of grain size wasnot obvious because of the heat resistanceintroduced by . . . .2) after aging . . . .3) grain refining,size reduction of . . .6 26.0/N solution strengthening and precipitation strengtheningrespectively, . . ., δh� p was the yield strength . . .7 24.7/N . . .dislocation density in lath martensite matrix due tothe high content of element . . . 100 steel delayed therecovery process during tempering . . .8...23.8/P lath martensite, which benefited the impact toughness. . .9 −13.1/P . . . the effect of ingot grain refinement on themechanical properties of al profiles which aremanufactured through hot working . . .10 −14.1/N . . . refining the prior austenitic grain size . . . LONGCONTEXT . . . the mechanical strength and cleavageresistance . . .11 −16.4/N . . . enhanced solid solution strengthening andcomposition homogenization is larger than . . .12 −18.7/N . . . as the solution treatment temperature increases to. . ., the transformation . . . and the formation of rim ophase . . .13 −23.4/N . . . during the aging treatment, the rim o phase at themargin of α2 grains become . . .Sci. Technol. Adv. Mater. 19 (2018) 656 T. ONISHI et al.aluminum or titanium. It was important to obtain thedesired knowledge. For example, the relationshipbetween strength and matrix in titanium alloys mighthave been different from this relationship in aluminumalloys.Thus, relationships were extracted from the scientificliterature describing the base material. A set of studieswas specified by the base material to find the desiredknowledge. As in Section 4.2, the literature was col-lected by keyword search in ScienceDirect. All relationsamong the factors collected in Section 3.1 were scoredby the CNN model as in Section 3.2 and some relation-ships were branched as described in Section 3.3.The system output was a PSPP design chart sug-gesting the required structures and processes. Thechart formed by three columns – process, structure,and property – suggested relations from the pro-cesses to the desired properties. Moreover, for eachrelation, a representative sentence for each factorpair was provided to justify the relation and aidthe researcher’s understanding.7. Conclusions and future workIn this study, we developed and tested a CAMaD system,a progressive knowledge extraction and representationsystem intended to supportmaterial design, by represent-ing knowledge as relationships. Knowledge was repre-sented as relationships in PSPP design charts. Weleveraged weakly supervised learning for relation extrac-tion. The end-to-end system proved our concept, and itsrelation identification performance was superior to thatof other baseline models.Further evaluation is the major feature of ourwork. In spite of the impressive results of relationextraction and brightness of the end-to-end system,a natural evaluation metric for the end-to-end taskremains unclear. Additionally, system performanceson other subtasks, factor collection and branching,are not evaluated in this study. Evaluations of thesesubtasks are not trivial. In factor collection, thereare infinite number of factors in material design,and it is difficult to see the coverage of factors asystem collected. In branching, there is no naturalmetric to compare or evaluate PSPP design charts.Thus, end-to-end evaluations are even morechallenging.We consider factor collection and mapping as thebottleneck of our system. The factor collectiondescribed in Section 3.1 and these factors weremapped into sentences that refer each factor inSection 3.2. Unlike in previous works [7,11], theFigure 7. The end-to-end demo system. a) Desired properties and a base material were selected. b) A sample of the generatedPSPP design chart. The desired properties were toughness and creep strength, and ‘steel’ was selected as base material. c) Therepresentative sentence describing the relation between toughness and carbon content.Sci. Technol. Adv. Mater. 19 (2018) 657 T. ONISHI et al.factors were not named entities. Any noun phrase canbe a factor, and factors were not predefined. At pre-sent, our system recognizes factors in a sentenceusing string matching. The obtained factors appearto be noisy and are not correctly categorized in somecases.A natural extension of this task is multi-labeling.The relation we use at present is binary (pos/neg),and only identifies whether two factors are related.This label might be too abstract. In multi-labeling, arelation between factors is described with a label suchas produce, depress, and independent. We believemulti-labeling renders the task more natural andinformative.Notes1. https://tac.nist.gov.2. https://dev.elsevier.com.3. https://www.journals.elsevier.com/scripta-materialia.4. https://www.sciencedirect.com.5. http://tomcat.apache.org.AcknowledgmentsWe are grateful to Dr. Yutaka Sasaki and Dr. Makoto Miwafrom Toyota Technological Institute for insightful comments.Disclosure statementNo potential conflict of interest was reported by theauthors. The underlying research materials for this articlecan be accessed at https://bitbucket.org/0024takeshi/pspp_relation.ORCIDIkumu Watanabe http://orcid.org/0000-0002-7693-1675References[1] Augenstein I, Das M, Riedel S, et al. SemEval 2017 Task10: ScienceIE -ExtractingKeyphrases andRelations fromScientific Publications. In: Proceedings of the 11thInternational Workshop on Semantic Evaluations(SemEval-2017); Vancouver (Canada); 2017. p. 546–555.[2] Cohen M. Unknowables in the essence of materialsscience and engineering. Mater Sci Eng. 1976;25:3–4.[3] Olson GB. Genomic materials design: the ferrousfrontier. Acta Mater. 2013;61(3):771–781.[4] Olson GB. Designing a new material world. Science.2000;288(5468):993–998.[5] Xiong W, Olson GB. Cybermaterials: Materials bydesign and accelerated insertion of materials. npjComputational Materials. 2016;2:15009 .[6] Xu Y, Yamazaki M, Villars P. Inorganic materialsdatabase for exploring the nature of material. Jpn JAppl Phys. 2011;50(11S):11RH02.[7] Mintz M, Bills S, Snow R, et al. Distant supervision forrelation extraction without labeled data. In: Proceedingsof the Joint Conference of the 47th Annual Meeting ofthe ACL and the 4th International Joint Conference onNatural Language Processing of the AFNLP; Suntec(Singapore); 2009. p. 1003–1011.[8] Hoffmann R, Zhang C, Ling X, et al. Knowledge-basedweak supervision for information extraction of overlap-ping relations. In: Proceedings of 49th Annual Meetingof the Association for Computational Linguistics;Portland, OR; 2011. p. 541–550.[9] Surdeanu M, Tibshirani J, Nallapati R, et al. Multi-instance multi-label learning for relation extraction.In: Proceedings of the 2012 Joint Conference onEmpirical Methods in Natural Language Processingand Computational Natural Language Learning; JejuIsland (Korea); 2011. p. 455–465.[10] Fan M, Zhao D, Zhou Q, et al. Distant supervision forrelation extraction with matrix completion. In:Proceedings of 52nd Annual Meeting of theAssociation for Computational Linguistics; Baltimore,MD; 2014. p. 839–849.[11] Riedel S, Yao L, McCallum A. Modeling relationsand their mentions without labeled text. In:Proceedings of the 2010 European Conference onMachine Learning and Principles and Practice ofKnowledge Discovery in Databases; Catalonia(Spain); 2010. p. 148–163.[12] Huang YY, Wang WY. Deep residual learning forweakly-supervised relation extraction. In: Proceedingsof the 2017 Conference on Empirical Methods inNatural Language Processing; Copenhagen (Denmark);2017. p. 1803–1807.[13] Zeng D, Liu K, Chen Y, et al. Distant supervision forrelation extraction via piecewise convolutional neuralnetworks. In: Proceedings of the 2015 Conference onEmpirical Methods in Natural Language Processing;Lisbon (Portugal); 2015. p. 1753–1762.[14] Lin Y, Shen S, Liu Z, et al. Neural relation extractionwith selective attention over instances. In:Proceedings of the 54th Annual Meeting of theAssociation for Computational Linguistics; Berlin(Germany); 2016. p. 2124–2133.[15] Ji G, Liu K, He S, et al. Distant supervision forrelation extraction with sentence-level attention andentity descriptions. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence;San Francisco (CA); 2017.[16] Liu T, Wang K, Chang B, et al. A soft-label method fornoise-tolerant distantly supervised relation extraction.In: Proceedings of the 2017 Conference on EmpiricalMethods in Natural Language Processing; Copenhagen(Denmark); 2017. p. 1791–1796.[17] Manning CD, Surdeanu M, Bauer J, et al. TheStanford CoreNLP Natural Language ProcessingToolkit. In: Proceedings of the 52nd AnnualMeeting of the Association for ComputationalLinguistics: System Demonstrations; Baltimore(MD); 2014. p. 55–60.[18] Pennington J, Socher R, Manning CD. GloVe: GlobalVectors for Word Representation. In: Proceedings ofthe 2014 Conference on Empirical Methods inNatural Language Processing; Doha (Qatar); 2014.p. 1532–1543.Sci. Technol. Adv. Mater. 19 (2018) 658 T. ONISHI et al.https://tac.nist.govhttps://dev.elsevier.comhttps://www.journals.elsevier.com/scripta-materialiahttps://www.sciencedirect.comhttp://tomcat.apache.orghttps://bitbucket.org/0024takeshi/pspp_relationhttps://bitbucket.org/0024takeshi/pspp_relationAppendix.Source articles.1 B. Wen and N. Zabaras. Investigating variability of fatigueindicator parameters of two-phase nickel-based superalloymicrostructures. DOI: https://doi.org/10.1016/j.commatsci.2011.07.0552 L. Huang and Y. Chen. A study on the microstructures andmechanical properties of Ti-B20–0.1B alloys of direct rollingin the α+β phase region. DOI: https://doi.org/10.1016/j.jallcom.2015.05.2443 Z. Yin, J. Yuan, Z. Wang, H. Hu, Y. Cheng and X. Hu.Preparation and properties of an Al2O3/Ti(C,N) micro-nano-composite ceramic tool material by microwave sintering.DOI: https://doi.org/10.1016/j.ceramint.2015.11.0824 O. Oloyede, T. D. Bigg, R. F. Cochrane and A. M. Mullis.Microstructure evolution and mechanical properties ofdrop-tube processed, rapidly solidified grey cast iron. DOI:https://doi.org/10.1016/j.msea.2015.12.0205 C. Shi, L. Zhang, G. Wu, X. Zhang, A. Chen and J. Tao. Effectsof Sc addition on the microstructure and mechanicalproperties of cast Al-3Li-1.5Cu-0.15Zr alloy. DOI: https://doi.org/10.1016/j.msea.2016.10.0636 C. Wang, C. Zhang, Z. Yang, J. Su and Y. Weng. Microstructureanalysis and yield strength simulation in high Co-Nisecondary hardening steel. DOI: https://doi.org/10.1016/j.msea.2016.05.0697 X. Shi, W. Zeng, Q. Zhao, W. Peng and C. Kang. Study on themicrostructure and mechanical properties of Aermet 100steel at the tempering temperature around 482°C. DOI:https://doi.org/10.1016/j.jallcom.2016.04.0878 H. Xie, L.-X. Du, J. Hu, G.-S. Sun, H.-Y. Wu and R.D.K. Misra.Effect of thermo-mechanical cycling on the microstructureand toughness in the weld CGHAZ of a novel high strengthlow carbon steel. DOI: https://doi.org/10.1016/j.msea.2015.05.0339 W. Haigen, X. Fuzhong and W. Mingpu. Effect of ingot grainrefinement on the tensile properties of 2024 Al alloy sheets.DOI: https://doi.org/10.1016/j.msea.2016.11.01610 A. Di Schino and C. Guarnaschelli. Effect of microstructure oncleavage resistance of high-strength quenched andtempered steels. DOI: https://doi.org/10.1016/j.matlet.2009.06.03211 F.L. Cheng, T.J. Chen, Y.S. Qi, S.Q. Zhang and P. Yao. Effects ofsolution treatment on microstructure and mechanicalproperties of thixoformed Mg2Sip/AM60B composite. DOI:https://doi.org/10.1016/j.jallcom.2015.02.14712, 13 X. Chen, F.Q. Xie, T.J. Ma, W.Y. Li and X.Q. Wu. Microstructuralevolution and mechanical properties of linear frictionwelded Ti2AlNb joint during solution and aging treatment.DOI: https://doi.org/10.1016/j.msea.2016.05.030Sci. Technol. Adv. Mater. 19 (2018) 659 T. ONISHI et al.https://doi.org/10.1016/j.commatsci.2011.07.055https://doi.org/10.1016/j.commatsci.2011.07.055https://doi.org/10.1016/j.jallcom.2015.05.244https://doi.org/10.1016/j.jallcom.2015.05.244https://doi.org/10.1016/j.ceramint.2015.11.082https://doi.org/10.1016/j.msea.2015.12.020https://doi.org/10.1016/j.msea.2016.10.063https://doi.org/10.1016/j.msea.2016.10.063https://doi.org/10.1016/j.msea.2016.05.069https://doi.org/10.1016/j.msea.2016.05.069https://doi.org/10.1016/j.jallcom.2016.04.087https://doi.org/10.1016/j.msea.2015.05.033https://doi.org/10.1016/j.msea.2015.05.033https://doi.org/10.1016/j.msea.2016.11.016https://doi.org/10.1016/j.matlet.2009.06.032https://doi.org/10.1016/j.matlet.2009.06.032https://doi.org/10.1016/j.jallcom.2015.02.147https://doi.org/10.1016/j.msea.2016.05.030 Abstract 1. Introduction 1.1. Knowledge representation 1.2. Data limitation 1.3. Weakly supervised learning 1.4. Our contribution 2. Our approach and task definition 3. System description 3.1. Factor collection 3.2. Relation identification 3.3. Branching 4. Experiment for relation identification 4.1. Relationship data 4.2. Corpus 4.3. Baseline models 4.4. Evaluation metric 5. Results of relation identification 6. End-to-end system 7. Conclusions and future work Notes Acknowledgments Disclosure statement ORCID References