Download all files (2.14GB)

コレクション

引用

説明:

A BERT (Bidirectional Encoder Representations from Transformers) model, which we named “MaterialBERT,” has been generated using scientific papers in wide area of material science as a corpus. A new vocabulary list for tokenizer was generated using material science corpus. Two BERT models with different vocabulary lists for the tokenizer, one with the original one made by Google and the other newly made by the authors, were generated. Word vectors embedded during the pre-training with the two MaterialBERT models reasonably reflect the meanings of materials names in material-class clustering and in the relationship between base materials and their compounds or derivatives for not only inorganic materials but also organic materials and organometallic compounds. Fine-tuning with CoLA (The Corpus of Linguistic Acceptability) using the pre-trained MaterialBERT showed ahigher score than the original BERT.
MaterialBERT could be used as a starting point for generating a narrower domain-specific BERT model in materials science field by transfer learning.

権利情報:

• Creative Commons BY-ND Attribution-NoDerivatives 4.0 International
  

キーワード: word embedding, pre-training, BERT, literal information

刊行年月日:

出版者: National Institute for Materials Science

掲載誌:

研究助成金:

原稿種別: 論文以外のデータ

MDR DOI:

公開URL: https://doi.org/10.51094/jxiv.119

関連資料:

その他の識別子:

連絡先:

更新時刻: 2023-01-25 00:11:19 +0900

MDRでの公開時刻: 2025-04-14 17:02:37 +0900