# Fileset

[Zheng_Small2024.pdf](https://mdr.nims.go.jp/filesets/5b38c09b-7a9d-49c2-97d7-e1e5a7bb4a86/download)

## Creator

[Xiaoyang Zheng](https://orcid.org/0000-0003-1452-5855), [Ikumu Watanabe](https://orcid.org/0000-0002-7693-1675), [Jamie Paik](https://orcid.org/0000-0003-3869-213X), [Jingjing Li](https://orcid.org/0000-0002-6524-3105), [Xiaofeng Guo](https://orcid.org/0000-0003-1971-7442), [Masanobu Naito](https://orcid.org/0000-0001-7198-819X)

## Rights

[Creative Commons BY Attribution 4.0 International](https://creativecommons.org/licenses/by/4.0/)

## Other metadata

[Text‐to‐Microstructure Generation Using Generative Deep Learning](https://mdr.nims.go.jp/datasets/15bfc8f2-a582-4bcf-a1b5-6871e31e414e)

## Fulltext

Text‐to‐Microstructure Generation Using Generative Deep LearningRESEARCH ARTICLEwww.small-journal.comText-to-Microstructure Generation Using Generative DeepLearningXiaoyang Zheng,* Ikumu Watanabe,* Jamie Paik, Jingjing Li, Xiaofeng Guo,and Masanobu NaitoDesigning novel materials is greatly dependent on understanding the designprinciples, physical mechanisms, and modeling methods of materialmicrostructures, requiring experienced designers with expertise and severalrounds of trial and error. Although recent advances in deep generativenetworks have enabled the inverse design of material microstructures, moststudies involve property-conditional generation and focus on a specific type ofstructure, resulting in limited generation diversity and poor human–computerinteraction. In this study, a pioneering text-to-microstructure deep generativenetwork (Txt2Microstruct-Net) is proposed that enables the generation of 3Dmaterial microstructures directly from text prompts without additionaloptimization procedures. The Txt2Microstruct-Net model is trained on a largemicrostructure-caption paired dataset that is extensible using the algorithmsprovided. Moreover, the model is sufficiently flexible to generate differentgeometric representations, such as voxels and point clouds. The model’sperformance is also demonstrated in the inverse design of materialmicrostructures and metamaterials. It has promising potential for interactivemicrostructure design when associated with large language models and couldbe a user-friendly tool for material design and discovery.1. IntroductionDesigning novel materials with improved properties is highlydependent on understanding of material microstructures andhow they affect material properties and performance. SuchX. Zheng, I. WatanabeCenter for Basic Research on MaterialsNational Institute for Materials Science1-2-1 Sengen, Tsukuba 305-0047, JapanE-mail: xyzheng1995@gmail.com; WATANABE.Ikumu@nims.go.jpX. Zheng, J. PaikReconfigurable Robotics LaboratoryÉcole Polytechnique Fédérale de Lausanne (EPFL)Lausanne 1015, SwitzerlandThe ORCID identification number(s) for the author(s) of this articlecan be found under https://doi.org/10.1002/smll.202402685© 2024 The Author(s). Small published by Wiley-VCH GmbH. This is anopen access article under the terms of the Creative Commons AttributionLicense, which permits use, distribution and reproduction in anymedium, provided the original work is properly cited.DOI: 10.1002/smll.202402685microstructural design methods have beenused to fabricate high-entropy alloys withhigh strength and ductility,[1,2] functionalpolymers and composites with enhancedproperties and intelligence,[3,4] and high-entropy ceramics with stability and re-silience against extreme conditions.[5,6]More importantly, tailoring the microstruc-ture arrangements enables the creationof metamaterials with unprecedentedproperties,[7–10] such as nanolattices withultrahigh stiffness-to-density ratios,[11,12]deployable origami structures with multipledegrees of freedom,[13,14] and mechanicalmetamaterials with extraordinary elasticitytensors.[15–17] Materials with delicatelydesigned microstructures can be chem-ically synthesized at the nanoscale andmesoscale,[18,19] physically created usingmicroscale-based laser cutting and engrav-ing methods,[13,20], and additively manufac-tured using a wide range of 3D-printablematerials at multiple scales.[21–26]Deriving inspiration from nature isthe most fundamental approach forunderstanding the intricate 3D microstructures and func-tions of materials.[27–30] Generally, materials can be designedto mimic the architecture of biological systems,[31–34] mech-anisms of animal locomotion,[35,36] and biological responsesof such systems to external stimuli.[37–40] Mathematics,J. LiGraduate School of Comprehensive Human SciencesUniversity of Tsukuba1-1-1 Tennodai, Tsukuba 305-8573, JapanX. GuoSchool of Materials Science and EngineeringSouthwest University of Science and TechnologyMianyang 621010, ChinaM. NaitoResearch Center for Macromolecules and BiomaterialsNational Institute for Materials Science1-2-1 Sengen, Tsukuba 305-0047, JapanSmall 2024, 20, 2402685 © 2024 The Author(s). Small published by Wiley-VCH GmbH2402685 (1 of 12)http://www.small-journal.commailto:xyzheng1995@gmail.commailto:WATANABE.Ikumu@nims.go.jphttps://doi.org/10.1002/smll.202402685http://creativecommons.org/licenses/by/4.0/http://crossmark.crossref.org/dialog/?doi=10.1002%2Fsmll.202402685&domain=pdf&date_stamp=2024-05-21www.advancedsciencenews.com www.small-journal.comarchitecture, arts, and crafts have also inspired the designof material microstructures.[13,14,25,41–43] These microstructurescan be manually modeled using computer-aided design (CAD)modeling tools or automatically generated using computa-tional algorithms such as phase-field modeling,[43] mathemat-ical modeling,[16,33] Voronoi tessellation,[44–46] and topologyoptimization.[47,48] These traditional design strategies followa forward-design approach, in which the design space andoutcome are dependent on experienced designers using severalrounds of trial and error. Moreover, the effective properties ofdesigned microstructures must be investigated using arduousand time-consuming computational simulations or experiments.Recent advancements in deep learning (DL) have revolution-ized material microstructure design processes[49–53]—for exam-ple, material microstructures with optimal properties can nowbe generated by unsupervised learning techniques using gen-erative adversarial networks[54,55] and variational autoencoders(VAEs).[48,56] More importantly, DL frameworks based on su-pervised learning enable the inverse design of microstructures,where microstructures can be autonomously generated with thedesired properties and functions.[45,46,57–69] The inverse designprocess can eliminate designers’ intuition and experience, as wellas the need for inefficient trial and error. The training data forinverse design are labeled geometric representations, where geo-metric representations can be either implicit—such as modelingparameters[59,60,68,70] and numerical representations of geometricelements[57,58,71]—or explicit—such as pixels (images),[46,55,62,63,69]voxels,[45,61,64] point clouds,[72] and meshes. In general, the la-bels are normalized effective properties of the correspondinggeometric representations prepared by computational simula-tions (e.g., finite element method (FEM) simulations). However,such property-conditional microstructure generation can limitthe diversity of microstructures generated and hinder the flex-ibility of human–computer interactions in the microstructuregeneration process. Moreover, most property-conditional genera-tion studies—despite their substantial contributions to mappingstructure–property relationships—have focused only on a spe-cific type of microstructure.[45,46,57–69]With recent advances in text-to-3D generative models, it isnow possible to generate 3D objects using natural languagedescriptions. These text-to-3D generation models can be di-vided into two categories: the first category includes modelstrained directly on paired data (3D objects and their correspond-ing text captions),[73–77] and the other category includes modelsthat are combined with pre-trained text-image models to opti-mize differentiable 3D representations.[78–87] The 3D object gen-eration overhead varies from a number of graphics process-ing unit (GPU) seconds to multiple GPU hours, dependingon the training methods, data formats of 3D representations,and rendering algorithms. Despite the successful implementa-tions of text-conditional models used to generate 3D objects,there has been scant research focused on text-to-microstructuregeneration.[62,63,88] This can be attributed to the lack of large-scalepaired microstructure datasets, lack of a strong 3D prior, andcomplexity of the DL framework compared with text-to-imagegeneration. Consequently, these studies have suffered from diffi-culties in scaling diverse and complex text prompts, expensiveoptimization procedures, and inefficient and non-meaningfulmicrostructure generation. For example, Hsu et al. proposeda DL frame work for the generation of 3D architected mate-rials, which is not direct and requires additional procedure totranslate a neural-network-generated image into a continuous 3Darchitecture.[63]In this study, we proposed a text-to-microstructure deep gener-ative network (Txt2Microstruct-Net) that could generate 3D ma-terial microstructures directly from text prompts without addi-tional optimization procedures. We also created a large, pairedmicrostructure dataset comprising 2,000 diverse 3D microstruc-tures and their corresponding captions. The dataset could be eas-ily expanded using the available modeling algorithms. The diver-sity of the training dataset and rationality of the DL frameworkenabled the generation of diverse and realistic microstructuresusing complex text prompts. Briefly, we created a large numberof 3D microstructures covering metals, alloys, polymers, com-posites, ceramics, architected materials, and metamaterials. Eachmicrostructure was captioned according to its type, geometricfeatures, appearance, modeling method, and effective properties.The Txt2Microstruct-Net model was then trained via a multi-stagetraining method using the prepared dataset. We then showcaseda range of Txt2Microstruct-Net-generated microstructures rep-resented by voxels and cloud points. Finally, we demonstratedthe inverse design of microstructures with target properties us-ing the trained Txt2Microstruct-Net model and experimentallyinvestigated the mechanical properties of a Txt2Microstruct-Net-generated mechanical metamaterial.2. Results and Discussion2.1. Deep Generative ModelTo train the Txt2Microstruct-Net model, we prepared a labeled3D microstructure dataset in the format S = {(Vn, Cn, In)}Nn=1.Each datapoint comprised a microstructure voxel grid (Vn) with64 × 64 × 64 voxels, a caption (Cn) describing the microstruc-ture, and three rendered images (In) of the microstructure inthe x-, y-, and z- directions. To achieve a larger design space andwider diversity, we computationally created a range of materialmicrostructures, including four categories—that is, metals andalloys, polymers and composites, ceramics, and architectedmaterials and metamaterials—as shown in Figures S1–S4(Supporting Information). Each category comprised five classesof material microstructures with different structural features.Moreover, each class comprised a hundred of geometries builtusing different modeling parameters to achieve diversity interms of their volume fractions, geometric differences, elemen-tal variability, and effective properties. The modeling methodsfor these microstructures are detailed in the Supporting Infor-mation. To achieve a higher diversity of captions, these 2,000microstructures were randomly divided into ten subgroups andassigned to ten experts from the fields of mechanical engineer-ing, materials science, computer science, and biology. Severalhints—such as category, class, and modeling methods—wereprovided to help them caption these microstructures. Notethat a bias might be introduced in these created captions eventhough ensuring the diversity. Figure S7 (Supporting Infor-mation) shows the vocabulary used in these captions. Eachmicrostructure was rendered from the front, top, and left views(Figures S1–S4, Supporting Information). The rendering wasSmall 2024, 20, 2402685 © 2024 The Author(s). Small published by Wiley-VCH GmbH2402685 (2 of 12) 16136829, 2024, 37, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/smll.202402685 by National Institute For, Wiley Online Library on [19/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons Licensehttp://www.advancedsciencenews.comhttp://www.small-journal.comwww.advancedsciencenews.com www.small-journal.comFigure 1. Txt2Microstruct-Net framework. a) Training stage 1: a dual encoder is trained to project the representation of images and their captions intothe same embedding space. b) Training stage 2: a VAE is trained to embed the microstructures into a latent space in the form of Gaussian distribution.c) Training state 3: an MLP is trained to predict the shape embeddings of text embeddings generated by the trained text encoder. d) Inference phase:a text prompt is embedded by the trained text encoder, before being converted into the shape embedding; the shape embedding is decoded into 3Dmicrostructure voxels by the trained shape decoder.performed by overlaying the cross-sections of each view withan alpha channel for transparency. Consequently, the trainingdataset composed N = 2, 000 datapoints and could be easilyexpanded using the modeling algorithms provided.The Txt2Microstruct-Net model was trained using a multi-stage training approach, as shown in Figure 1a–c. The multi-stage training approach was used for resource efficiency and en-semble learning: neural networks in the first and second stageswere trained independently, and then their training results werecombined in the third stage to train the other neural network.This approach can help mitigate the risk of overfitting, save com-putational resources, and improve generalization. In the firststage, we trained a dual encoder neural network to build an in-terchangeable text-image latent space, inspired by the contrastivelanguage–image pre-training (CLIP) approach.[89] Image and textencoders were trained simultaneously to project the representa-tion of images (In) and their captions (Cn) into the same embed-ding space. This helped generate similar microstructures withsimilar but different natural language descriptions, because thecaption embeddings were located near the embeddings of the im-ages they described. We used a pre-trained Xception model asthe base for the image encoder and a pre-trained BERT modelas the base for the text encoder.[90,91] The pre-trained Xceptionand BERT models were fine-tuned in this first stage. In the sec-ond stage, we trained a VAE comprising a shape encoder and de-coder. The shape encoder was trained to extract the shape em-bedding (en) with a Gaussian distribution (z ≈  (𝜇x, 𝜎x)) for themicrostructure collection (Vn). The shape decoder learned to de-compress the microstructure representation from the shape em-bedding. We used a voxel-based VAE as the base for the shapeencoder and decoder.[92] In the third stage, we trained a multi-layer perceptron (MLP) to generate a shape embedding (en) con-ditioned on caption embeddings from the text encoder. The MLPhad four fully connected layers as its hidden layer, and output μxand 𝜎x using its output layer. Details of the training process areprovided in the Supporting Information.After the Txt2Microstruct-Net model had been well trained, itwas used to generate a number of microstructures conditionedon the text prompt (Figure 1d). In the inference phase, a textprompt could be converted into a text embedding (etxt) using thetext encoder. The text embedding could then be used as the condi-tion vector to generate the shape embeddings (en) using the MLP.As the text and image encoders were trained to bring the text andimage embeddings into a joint latent space, the shape embed-dings in the near latent space could be generated using similartext prompts. The shape embeddings could then be convertedinto 3D microstructures in the form of voxels using the shapedecoder. The generated voxels could be further post-processed toSmall 2024, 20, 2402685 © 2024 The Author(s). Small published by Wiley-VCH GmbH2402685 (3 of 12) 16136829, 2024, 37, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/smll.202402685 by National Institute For, Wiley Online Library on [19/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons Licensehttp://www.advancedsciencenews.comhttp://www.small-journal.comwww.advancedsciencenews.com www.small-journal.comFigure 2. Voxel-based microstructures generated by the Txt2Microstruct-Net model using different text prompts.clean isolated noisy elements, before being converted into a meshand geometrically optimized for simulation or fabrication.2.2. Microstructure Generation Using Text PromptTo qualitatively evaluate the generative capabilities of the trainedTxt2Microstruct-Net model, we show several voxel-based mi-crostructures generated by the Txt2Microstruct-Net model usingdifferent text prompts in Figure 2. It shows that multiple diversemicrostructures could be generated using a single text prompt,which helped to generate new variations in the material designprocess. Moreover, it is evident that the microstructures based ondifferent categories and classes could be generated using com-mon semantic words, geometric attributes, and technical terms,proving that the Txt2Microstruct-Net model could capture the se-mantic notions of text prompts. Additional generated cases areshown in Figure S8 (Supporting Information) to demonstrate thereliability of the training results. It is evident that the generativequality of these microstructures could be qualitatively comparedwith the ground truth (Figures S1–S4, Supporting Information),and they exhibited good similarity in terms of appearance. How-ever, It should be noted that a few isolated voxel clusters couldbe found in some generated microstructures, which could be at-tributed to the transposed convolution layers in the shape de-coder. These isolated voxel clusters could be removed by filteringthe threshold value of the isolated voxel numbers during post-processing. Moreover, it should be noted that the performanceof the Txt2Microstruct-Net is not good at zero-shot generation.This can be attributed to the weakness of the language model(i.e., BERT model) in the material domain. In fact, BERT modelcan hardly capture the relationship between the textual featuresof qualitative descriptions and the geometrical features of 3Dstructures because BERT model was pre-trained using text fromWikipedia that lacks the specificity of the material domain.To quantitatively evaluate the performance of theTxt2Microstruct-Net model, the intersection over union (IoU),class accuracy, and Fréchet inception distance (FID) wereused to measure the reconstruction quality, generation accu-racy, and generation quality, respectively. The IoU calculatesthe overlap between the ground-truth microstructure voxel(Vn) and the reconstructed microstructure voxel (Vren ) in thereconstruction process of the VAE (Figure 1b), written asIoU = |Vn ∩ Vren |∕|Vn| ∪ |Vren |. The IoU was calculated usingthe 2,000 voxelized microstructures using 64 × 64 × 64 voxelsfrom the dataset. An IoU score of 0.9618 indicated a perfectmatch between Vn and Vren , demonstrating that the microstruc-tures could be perfectly reconstructed. To ensure that theTxt2Microstruct-Net model generated microstructures acrossclasses, we trained a classifier to predict the classes of the givenmicrostructures. The classifier was trained on 2,000 collectedmicrostructures with an assigned label corresponding to one ofthe 20 classes. The classifier shared a similar architecture withthe shape encoder but with 20 dimensions to its output layer.We then created 200 new text prompts (ten prompts for eachclass) that were conditioned to generate microstructures usingthe trained Txt2Microstruct-Net model. The class accuracy of thegenerated microstructures predicted by the classifier was usedSmall 2024, 20, 2402685 © 2024 The Author(s). Small published by Wiley-VCH GmbH2402685 (4 of 12) 16136829, 2024, 37, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/smll.202402685 by National Institute For, Wiley Online Library on [19/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons Licensehttp://www.advancedsciencenews.comhttp://www.small-journal.comwww.advancedsciencenews.com www.small-journal.comTable1.ComparisonoftheTxt2Microstruct-NetmodelwithotherDLmodelsforvoxel-basedmicrostructuregeneration.MetricsOtherinformationDLframeworkIoUFIDClassaccuracyMeansquareerrorInputOutputMicrostructuretypeDatapointsTrainingtimeGPUSliceGAN[61]––––Microstructureimages64×64×64×1or64×64×64×3voxelsAvarietyofmicrostructures,suchaspolycrystallinegrains,ceramic,carbonfiberrods,grainboundary,andbatteryseparator1924hNVIDIATitanXpGPU3D-CGAN[45]–53.91–0.01Materialpropertyvectors64×64×64voxelsVoronoi-basedopen-cellfoams1000032hNVIDIARTXA6000graphicscardTxt2Microstruct-Net0.961872.080.8695–Textprompt64×64×64voxelsAdiversityofmicrostructures,rangingfrommetalsandceramicstocompositesandarchitectedmaterials200049minNVIDIARTXA6000graphicscardTheSliceGANmodeltakesimagesasinputsandoutputs3Dmicrostructures;[61]the3D-CGANmodeltakesmaterialpropertyvectorsasinputsandoutputs3Dmicrostructures;[45]andtheTxt2Microstruct-Netmodeltakestextpromptsasinputsandoutputs3Dmicrostructures.The3D-CGANmodelhasalowerFIDcomparedwiththeTxt2Microstruct-Netmodel,indicatingahighergenerationquality.Thisisbecausethe3D-CGANmodelwastrainedonasingletypeofVoronoi-basedopen-cellfoams.Notethatthecomparisonlacksstrictnessbecauseofthedifferenceininputtypesandthescaleofthetrainingdataset.as a metric to measure the generation accuracy. The resultingclass accuracy of 0.8695 indicated that microstructures could beintentionally generated with the corresponding class. Finally,to evaluate the generation quality, we used the embeddingtaken from the penultimate layer of the pre-trained classifierto calculate the FID of input microstructures (2,000 real andgenerated microstructures), written as FID = ∥μx − μy∥2 − Tr(Cx+ Cy − 2(CxCy)1/2), where x and y denotes the feature vectorsof the real and generated embeddings, respectively, μx and μydenote the magnitudes of the vectors, Tr denotes the trace ofthe matrix, and Cx and Cy denote the covariance matrix of thevectors. The FID score can be used to evaluate the quality ofimages generated by deep generative networks, where lowerscores correspond to higher-quality images.[93] The resulting lowFID score of 72 indicated a higher quality of microstructuresgenerated by the Txt2Microstruct-Net model. We also comparedthese metrics with those of other studies in terms of voxel-basedmicrostructure generation, as shown in Table 1.To investigate the flexibility of the Txt2Microstruct-Net modelfor other geometric representations, we replaced the voxel-basedVAE with a point-cloud-based VAE for the point cloud gener-ation. We retrained the Txt2Microstruct-Net model using thesame procedure, but in training stage 2, we used another VAEnetwork—that is, VG-VAE—for point-cloud-based microstruc-ture reconstruction.[94] The training data for the VG-VAE werepoint cloud representations of 2,000 collected microstructures.Each point cloud was created by randomly sampling 2048 vol-umetric points from a mesh in the collected microstructures.The coordinates of these 2,048 volumetric points were normal-ized in a domain [− 0.5, 0.5] for all the point clouds for theVG-VAE training. Figure 3 shows several point-cloud-based mi-crostructures generated by the Txt2Microstruct-Net model us-ing VG-VAE as the shape encoder and decoder. This showsthat the Txt2Microstruct-Net model could generate microstruc-tures in the point-cloud representations well, although isolatedclusters appeared in some generations. However, compared tothe voxel representations, the point-cloud representations couldhardly preserve the delicate geometric features in complex mi-crostructures, exhibiting variations in microstructures with highvolume fractions (Figure 3). An alternative solution was to usemesh representations; however, the data sizes of these com-plex material microstructures varied individually, making it dif-ficult to normalize them for DL compared to other 3D objectdatasets.[95]A potential application of the Txt2Microstruct-Net model is thefabrication of generated microstructures conditioned using spe-cific text prompts. Figure 4 shows the process used to fabricatemicrostructures with different geometric representations usingadditive manufacturing. For the voxel representations, the iso-lated voxel clusters were cleared before smoothing the surface ofthe generated voxel representations using the non-uniform ratio-nal mesh smooth (NURMS) method (Figure 4a,b). The smoothedvoxel representation was then converted into a mesh representa-tion for either additive manufacturing or FEM simulations. Rep-resentations with separate elements could also be prepared inmultiple phases using multimaterial 3D printers. For the point-cloud representations, a mesh was created by generating spheresusing 2,048 volumetric points as center points. Depending on thevolume fraction, geometric complexity, and geometric type, theSmall 2024, 20, 2402685 © 2024 The Author(s). Small published by Wiley-VCH GmbH2402685 (5 of 12) 16136829, 2024, 37, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/smll.202402685 by National Institute For, Wiley Online Library on [19/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons Licensehttp://www.advancedsciencenews.comhttp://www.small-journal.comwww.advancedsciencenews.com www.small-journal.comFigure 3. Point cloud-based microstructures generated by the Txt2Microstruct-Net model using different text prompts.Figure 4. Fabricating the Txt2Microstruct-Net-generated microstructures using additive manufacturing. a) Generating and fabricating a voxel-basedmicrostructure with multimaterials. b) Generating and fabricating a voxel-based microstructure with a single material. c) Generating and fabricating apoint-cloud-based microstructure.Small 2024, 20, 2402685 © 2024 The Author(s). Small published by Wiley-VCH GmbH2402685 (6 of 12) 16136829, 2024, 37, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/smll.202402685 by National Institute For, Wiley Online Library on [19/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons Licensehttp://www.advancedsciencenews.comhttp://www.small-journal.comwww.advancedsciencenews.com www.small-journal.comFigure 5. Inverse design of microstructures. Comparison between user-input and Txt2Microstruct-Net-output volume fraction (a), Young’s modulus (b),and isotropicity (c).radii of these spheres could be tuned to generate a suitable mesh,which could be further smoothed using the NURMS methodfor additive manufacturing (Figure 4c). Other post-processingmethods—such as mesh editing using CAD tools and topologyoptimization—can also be used to modify the geometry of thegenerated microstructures.2.3. Inverse Design of MicrostructuresAnother promising application of the Txt2Microstruct-Net modelis the inverse design of material microstructures. Unlike qual-itative generation using text prompts, inverse design enablesthe quantitative generation of microstructures to satisfy the in-put target properties. To this end, we calculated additional labelsfor these microstructures—that is, their effective properties—including their volume fraction, effective Young’s modulus, andisotropicity. The effective Young’s modulus was calculated byisotropically approximating the stiffness tensor, and the isotropic-ity was calculated as: isotropicity = min (E𝜃, ϕ)/max (E𝜃, ϕ), whereE𝜃, ϕ denotes the direction-dependent Young’s modulus obtainedby solving the stiffness tensor.[96] The stiffness tensors were cal-culated using a numerical homogenization method.[45,97] Theseadditional labels were appended to the microstructure captionsduring training. Prior to the inverse design, we trained a solverto predict the effective properties of the inversely designed mi-crostructures (Figure S6, Supporting Information). We used theabove-mentioned method to train the Txt2Microstruct-Net modelusing property-appended captions. Figure 5a–c compare the in-put labels (i.e., the target volume fraction, Young’s modulus,and isotropicity) and output labels (i.e., the volume fraction,Young’s modulus, and isotropicity of the generated microstruc-tures) of 2,000 randomly inversely designed microstructures.Here, each coordinate of the scatter plot corresponded to an in-put and output label. The difference between the input and out-put labels could then be evaluated by linearly fitting these scat-ters with a bisection line, and a position closer to the bisec-tion line represents better inverse-design performance. Althoughthe input and output labels were similar in terms of propertydistribution, they were discrete to the bisection line, indicatingpoor inverse-design performance compared to other DL-basedinverse-design frameworks.[45,46,57,59,65,66,68–72] This could be at-tributed to the differences in the training methods—for exam-ple, the Txt2Microstruct-Net model was trained on word embed-dings, whereas the others were generally trained on the embed-dings of numeric property vectors. Compared with vector em-beddings, word embeddings obtained by large language modelscan have difficulty capturing precise target properties, resultingin poor inverse-design performance. However, word embeddingsenable the processing of more information (e.g., microstructuretypes, geometric features, and semantic words). Consequently,the Txt2Microstruct-Net model enables the inverse design ofdiverse microstructures, whereas the other DL-based inverse-design frameworks can only generate a single type of microstruc-tures generally.To demonstrate the design space for mechanical metamate-rials, we generated pairs of auxetic metamaterials using a textprompt, as shown in Figure 6a. An auxetic metamaterial is a me-chanical metamaterial with a negative Poisson’s ratio. Typically, itshrinks dimensionally upon uniaxial compressive loading, whichis highly dependent on its microstructure. Notably, there weresome isolated voxels in these Txt2Microstruct-Net-generated mi-crostructures, which could result in geometric imperfections.An generated auxetic metamaterial was post-processed and com-pared with a reference auxetic metamaterial from the dataset,as shown in Figure 6b. Like the reference auxetic metamaterialwith a perfect cubic symmetry, the generated metamaterial ex-hibited good cubic symmetry, although a negligible imperfectionwas evident in the direction-dependent Young’s modulus curves(Figure 6c). The two auxetic metamaterials with 3 × 3 × 3 unitcells were 3D printed using a rubber-like resin, and they exhib-ited good agreement in terms of appearance (Figure 6d).To investigate the impact of geometric imperfections on themechanical properties of the generated and reference auxeticmetamaterials, we conducted a systematic analysis using FEMsimulations and uniaxial compression tests. Figure 7a shows asequence of progressively deformed shapes of the generated aux-etic metamaterials under four different levels of compressive en-gineering strain obtained from the FEM results. Compared tothe reference auxetic metamaterial that contracted in all direc-tions, the generated metamaterial exhibited no apparent shrink-age along the y-axis, indicating a partially non-negative Poisson’sSmall 2024, 20, 2402685 © 2024 The Author(s). Small published by Wiley-VCH GmbH2402685 (7 of 12) 16136829, 2024, 37, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/smll.202402685 by National Institute For, Wiley Online Library on [19/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons Licensehttp://www.advancedsciencenews.comhttp://www.small-journal.comwww.advancedsciencenews.com www.small-journal.comFigure 6. Inverse design of auxetic metamaterials. a) Four Txt2Microstruct-Net-generated auxetic metamaterials generated using the same text prompt.b) Renderings of a generated and reference auxetic metamaterial. b) Direction-dependent Young’s moduli of the two auxetic metamaterials. d) 3D-printedsamples of the two auxetic metamaterials.ratio. Similar results were evident in the experiments, wherethe 3D-printed generated sample exhibited non-auxetic behav-ior along the y-axis. Figure 7b shows a detailed comparison be-tween the Poisson’s ratio–strain curves obtained from the sim-ulations and experiments. It shows that the Poisson’s ratio de-creases with compressive strain, except for the 𝜈zy of the gen-erated auxetic metamaterial. The partial non-auxeticity can beattributed to the geometric imperfection of the generated aux-etic metamaterial, whose auxetic behavior results from bucklinginstability.[16] Geometric imperfections affect not only the auxeticbehavior but also the stress–strain curves, as shown in 7c, wherefluctuations appear on the stress–strain curve of the 3D-printedgenerated sample. The results indicate that voxel-based gener-ation conditioned on text prompts exhibits a weakness for themetamaterial design, where precise control over the microstruc-ture is required. By contrast, DL frameworks that output implicitgeometric representations perform well in generating delicatelydesigned metamaterials.[57,59,66,70–72]3. ConclusionWe developed a generative DL-based framework,Txt2Microstruct-Net, for text-to-microstructure generation.The Txt2Microstruct-Net model was trained using 2,000 mi-crostructures with captions describing their categories, classes,modeling methods, geometric features, and appearances.This model can rapidly generate multiple diverse voxel-basedmicrostructures without an optimization process using textprompts as inputs. Further, we also showcased the flexibility ofpoint-cloud-based microstructure generation and the inversedesign of microstructures using property-appended labels. How-ever, the Txt2Microstruct-Net model exhibited a weakness inmetamaterial design, where delicate microstructure design wasrequired. It shows that the text description of microstructurescan hardly capture the geometric and physical informationof a microstructures. A universal method to describe a mi-crostructure is expected for addressing this issue. The geometricimperfections resulting from voxel representation and wordembeddings hindered precise control over the microstructuresduring the generation process. This is expected to be improvedusing implicit geometric representations that can preciselycontrol the type, arrangement, and interface of microstructuralelements using additional modeling algorithms.This study demonstrated a pioneering and alternative wayto design and generate material microstructures using textprompts, which was more user-friendly for non-specialists com-pared with other DL frameworks whose inputs were property vec-tors. The proposed framework can be optimized and extendedto the inverse design of more microstructures by extending thetraining dataset (e.g., including more additional details of mi-crostructures in text descriptions), replacing geometric repre-sentations, optimizing neural networks (e.g., using generativeSmall 2024, 20, 2402685 © 2024 The Author(s). Small published by Wiley-VCH GmbH2402685 (8 of 12) 16136829, 2024, 37, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/smll.202402685 by National Institute For, Wiley Online Library on [19/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons Licensehttp://www.advancedsciencenews.comhttp://www.small-journal.comwww.advancedsciencenews.com www.small-journal.comFigure 7. Mechanical property comparison between a generated and reference auxetic metamaterial. Progressively deformed shapes at different com-pressive strains from FEM simulations (a) and uniaxial compression tests (b). Poisson’s ratio–strain curves (c) and stress–strain curves (d) from FEMsimulations and uniaxial compression tests.diffusion models), and using fine-tuning process (e.g., using su-pervised learning and reinforcement learning from human feed-back). Taking advantage of text mining and information extrac-tion using large language models, the Txt2Microstruct-Net modelis a promising tool for material discovery and informatics.[98,99]4. Experimental SectionDataset Preparation: The methods used to create these microstruc-tures are detailed in the Supporting Information. After generating meshfiles (.stl format) of these microstructures, they were converted into 3Dvoxel arrays (Vn) comprising 64 × 64 × 64 voxels using the Python libraryVoxelFuse.[100] The 3D voxel arrays of all microstructures were stored as anumpy array (npy format) using a Boolean data type referring to differentphases. The captions of these microstructures (Cn) were stored in a textfile (.csv format). Images of each microstructure (In) were rendered fromthe front, top, and left views using the voxelized geometry. An increasingvalue of the alpha channel was used to overlay the sliced layers, and theimages were stored as transparent grayscale images (.png format) of 299× 299 pixels. Transparency ensured the preservation of interior microstruc-tural features. The point clouds of these microstructures (Pn) were createdby randomly sampling 2,048 volumetric points from their mesh files usingthe Python library Trimesh.[101] Consequently, a {(Vn, Cn, In)}Nn=1 datasetwas obtained for voxel-based training, and a {(Pn, Cn, In)}Nn=1 dataset wasobtained for point cloud-based training.Implementation of Deep Generative Model: The training process forthe Txt2Microstruct-Net model was conducted using TensorFlow (version2.12.0) on a single NVIDIA RTX A6000 graphics card (48 GB GPU Memory)running on a Linux system. The Linux environment was developed usingPython 3.10 and CUDA 11.8. The neural networks, training process, andtraining results are detailed in the Supporting Information, and the codesare available at https://github.com/xyzheng-ut/Txt2Microstruct-Net.Additive Manufacturing: All models used for additive manufacturingwere post-processed by removing isolated noisy voxels and smoothingtheir surfaces using the NURMS method. Models for different purposeswere fabricated using various 3D printers and materials. The 3D-printedcolored sample (Figure 4a) was fabricated using a multimaterial 3DSmall 2024, 20, 2402685 © 2024 The Author(s). Small published by Wiley-VCH GmbH2402685 (9 of 12) 16136829, 2024, 37, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/smll.202402685 by National Institute For, Wiley Online Library on [19/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons Licensehttp://www.advancedsciencenews.comhttp://www.small-journal.comhttps://github.com/xyzheng-ut/Txt2Microstruct-Netwww.advancedsciencenews.com www.small-journal.comprinter (PolyJet J850 Prime 3D Printer, Stratasys, USA) with vivid translu-cent colored materials (VeroVivid Color Family, Stratasys, USA). The 3D-printed white samples (Figure 4b,c) were fabricated using a stereolithog-raphy (SLA) 3D printer (Form 3, Formlabs, USA) with a photopolymerresin (White resin, Formlabs, USA). The 3D-printed translucent samples(Figure 6d) were fabricated using an SLA 3D printer with a rubber-likephotopolymer resin (Elastic resin, Formlabs, USA). Specifically, 3D-printedauxetic metamaterials were modified using 3 × 3 × 3 unit cells of 40 × 40× 40 mm3 dimensions, which represented the geometric features of peri-odic porous materials for mechanical testing.[33] Moreover, a layer thick-ness of 0.05 mm and operating temperature 33°C were used in the printingprocess. The samples were washed using isopropanol after 3D printing,followed by curing at 60°C for 15 min using Form Cure (Formlabs, USA).Mechanical Testing: The mechanical properties of the 3D-printed aux-etic metamaterials were examined via uniaxial compression tests using amotorized test stand (AGXplus-10kN, Shimadzu, Japan). Static compres-sion tests were conducted at a vertically constant speed of 10 mmmin-1,based on the ASTM D695–15 standard. The stop condition was set to acompressive strain of 0.3 to avoid full contact of the structural elementsof these auxetic metamaterials. The deformation processes were recordedusing two high-speed cameras (front and side views). The stress–straincurves were obtained using the recorded load and displacement data. ThePoisson’s ratios were evaluated by extracting the displacements of thenodes of the deformed geometries from the recorded videos via postpro-cessing in MATLAB (Version R2021b, MathWorks, USA).Finite Element Method Simulation: The mechanical properties of thegenerated and reference auxetic metamaterials were visually examinedusing FEM simulations. A nonlinear FEM simulation (COMSOL Multi-physics version 6.1, COMSOL, Sweden) was performed to examine thelarge-deformation behavior of these auxetic metamaterials. An incom-pressible neo-Hookean material model was assigned to the model us-ing an experimentally measured Young’s modulus of 0.6615 MPa.[16] Asthe auxetic metamaterials can suffer from buckling instability, a linearizedbuckling analysis was first performed to compute the shape of the first-order buckling mode. A post-buckling analysis was performed on the thebuckled geometry using a parametric sweep of the z-axis displacementand a stop condition when adjacent boundaries were in contact. The Pois-son’s ratios and stress–strain curves were calculated via the post-bucklinganalysis. The models were meshed using approximately 3 × 105 second-order tetrahedral solid elements. Periodic boundary conditions were im-plemented using the representative volume element method.[102,103]Numerical Homogenization: The stiffness tensors of the microstruc-tures used for inverse design were calculated using a numerical homoge-nization method. The homogenized 6 × 6 stiffness tensor was obtained bycalculating the element displacements and global displacement field in thecube domain using iterations for six load cases (i.e., three compressionsalong the x, y, and z axes, and three shearing loads), as detailed in previ-ous studies.[45,97] Homogenization was implemented using MATLAB witha 64 × 64 × 64 voxel array consisting of 0s and 1s, where 0 and 1 representdifferent phases. For simplicity, two linear elastic models with differentparameters for the material microstructures with two solid phases wereused. The Young’s modulus and Poisson’s ratio for the two phases wereset to 100 GPa and 0.3, and 1 GPa and 0.3, respectively. For material mi-crostructures with a solid and void phases, a linear elastic model was usedfor the solid phase with a Young’s modulus and Poisson’s ratio of 100 GPaand 0.3, respectively. The volume fraction referred to the proportion of thesolid (stiffer) phase and was calculated as ∑Vn/643.Supporting InformationSupporting Information is available from the Wiley Online Library or fromthe author.AcknowledgementsThis research was supported by a Grant-in-Aid for JSPS Fellows PD (GrantNumber 22KJ0407) and Young Researchers’ Exchange Programme—Special 2023 Call Japan of Japanese-Swiss Science and Technology Pro-gramme (Project no. JP_EG_special_032023_11). The authors acknowl-edged the support from the university club, Research Support, Universityof Tsukuba (RSUT) (https://sites.google.com/view/rs-tsukuba).Conflict of InterestThe authors declare no conflict of interest.Data Availability StatementThe data that support the findings of this study are available on requestfrom the corresponding author. The data are not publicly available due toprivacy or ethical restrictions.Keywordsarchitected material, artificial intelligence, deep generative model, deeplearning, inverse design, metamaterial, microstructureReceived: April 8, 2024Published online: May 21, 2024[1] S. Chen, Z. H. Aitken, S. Pattamatta, Z. Wu, Z. G. Yu, D. J. Srolovitz,P. K. Liaw, Y.-W. Zhang, Nat. Commun. 2021, 12, 4953.[2] E. P. George, D. Raabe, R. O. Ritchie, Nat. Rev. Mater. 2019, 4, 515.[3] Y. Xia, Y. He, F. Zhang, Y. Liu, J. Leng, Adv. Mater. 2021, 33, 2000713.[4] F. D. Jochum, P. Theato, Chem. Soc. Rev. 2013, 42, 7468.[5] S. Divilov, H. Eckert, D. Hicks, C. Oses, C. Toher, R. Friedrich, M.Esters, M. J. Mehl, A. C. Zettel, Y. Lederer, E. Zurek, J.-P. Maria, D. W.Brenner, X. Campilongo, S. Filipovic, W. G. Fahrenholtz, C. J. Ryan,C. M. DeSalle, R. J. Crealese, D. E. Wolfe, A. Calzolari, S. Curtarolo,Nature 2024, 625, 66.[6] C. Oses, C. Toher, S. Curtarolo, Nat. Rev. Mater. 2020, 5, 295.[7] P. Jiao, J. Mueller, J. R. Raney, X. Zheng, A. H. Alavi, Nat. Commun.2023, 14, 6004.[8] C. M. Soukoulis, M. Wegener, Nat. Photonics 2011, 5, 523.[9] Y. Li, W. Li, T. Han, X. Zheng, J. Li, B. Li, S. Fan, C.-W. Qiu, Nat. Rev.Mater. 2021, 6, 488.[10] K. Fan, W. J. Padilla, Mater. Today 2015, 18, 39.[11] X. Zheng, H. Lee, T. H. Weisgraber, M. Shusteff, J. DeOtte, E.B. Duoss, J. D. Kuntz, M. M. Biener, Q. Ge, J. A. Jackson, S. O.Kucheyev, N. X. FANG, C. M. Spadaccini, Science 2014, 344, 1373.[12] J. Bauer, L. R. Meza, T. A. Schaedler, R. Schwaiger, X. Zheng, L.Valdevit, Adv. Mater. 2017, 29, 1701850.[13] J. L. Silverberg, A. A. Evans, L. McLeod, R. C. Hayward, T. Hull, C. D.Santangelo, I. Cohen, Science 2014, 345, 647.[14] J. T. Overvelde, T. A. De Jong, Y. Shevchenko, S. A. Becerra, G. M.Whitesides, J. C. Weaver, C. Hoberman, K. Bertoldi, Nat. Commun.2016, 7, 10929.[15] J. Berger, H. Wadley, R. McMeeking, Nature 2017, 543, 533.[16] X. Zheng, X. Guo, I. Watanabe, Mater. Des. 2021, 198, 109313.[17] T. Frenzel, M. Kadic, M. Wegener, Science 2017, 358, 1072.[18] H. Lee, S. Kwon, J. Min, S.-M. Jin, J. H. Hwang, E. Lee, W. B. Lee, M.J. Park, Science 2024, 383, 70.[19] R. Xu, Z. Zeng, Y. Lei, Nat. Commun. 2022, 13, 2435.[20] M. Malinauskas, A. Žukauskas, S. Hasegawa, Y. Hayasaki, V.Mizeikis, R. Buividas, S. Juodkazis, Light: Sci. Appl. 2016, 5, e16133.[21] D. Gu, X. Shi, R. Poprawe, D. L. Bourell, R. Setchi, J. Zhu, Science2021, 372, eabg1487.Small 2024, 20, 2402685 © 2024 The Author(s). Small published by Wiley-VCH GmbH2402685 (10 of 12) 16136829, 2024, 37, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/smll.202402685 by National Institute For, Wiley Online Library on [19/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons Licensehttp://www.advancedsciencenews.comhttp://www.small-journal.comhttps://sites.google.com/view/rs-tsukubawww.advancedsciencenews.com www.small-journal.com[22] J. Fan, L. Zhang, S. Wei, Z. Zhang, S.-K. Choi, B. Song, Y. Shi, Mater.Today 2021, 50, 303.[23] R. L. Truby, J. A. Lewis, Nature 2016, 540, 371.[24] X. Kuang, D. J. Roach, J. Wu, C. M. Hamel, Z. Ding, T. Wang, M. L.Dunn, H. J. Qi, Adv. Funct. Mater. 2019, 29, 1805290.[25] X. Zheng, W. Smith, J. Jackson, B. Moran, H. Cui, D. Chen, J. Ye,N. Fang, N. Rodriguez, T. Weisgraber, C. M. Spadaccini, Nat. Mater.2016, 15, 1100.[26] X. Zheng, I. Watanabe, S. Wang, T.-T. Chen, M. Naito, Mater. Des.2024, 237, 112548.[27] C. Zhang, D. A. Mcadams, J. C. Grunlan, Adv. Mater. 2016, 28, 6292.[28] Z. Jia, Z. Deng, L. Li, Adv. Mater. 2022, 34, 2106259.[29] H. Dai, W. Dai, Z. Hu, W. Zhang, G. Zhang, R. Guo, Adv. Sci. 2023,10, 2207192.[30] N. K. Katiyar, G. Goel, S. Hawi, S. Goel, NPG Asia Mater. 2021, 13,56.[31] M. C. Fernandes, J. Aizenberg, J. C. Weaver, K. Bertoldi, Nat. Mater.2021, 20, 237.[32] L. Jin, M. Yeager, Y.-J. Lee, D. J. O’Brien, S. Yang, Sci. Adv. 2022, 8,eabq3248.[33] X. Zheng, Z. Fu, K. Du, C. Wang, Y. Yi, J. Mater. Sci. 2018, 53, 10194.[34] D. Yan, J. Chang, H. Zhang, J. Liu, H. Song, Z. Xue, F. Zhang, Y.Zhang, Nat. Commun. 2020, 11, 1180.[35] J.-S. Koh, E. Yang, G.-P. Jung, S.-P. Jung, J. H. Son, S.-I. Lee, P. G.Jablonski, R. J. Wood, H.-Y. Kim, K.-J. Cho, Science 2015, 349, 517.[36] Z. Zhakypov, K. Mori, K. Hosoda, J. Paik, Nature 2019, 571, 381.[37] K. Liu, R. Sun, C. Daraio, Science 2022, 377, 975.[38] H. Cui, Q. Zhao, L. Zhang, X. Du, Adv. Intell. Syst. 2020, 2, 2000138.[39] S. Li, H. Bai, R. F. Shepherd, H. Zhao, Angew. Chem., Int. Ed. 2019,58, 11182.[40] R. O. Ritchie, X. R. Zheng, Nat. Mater. 2022, 21, 968.[41] X. Fang, J. Wen, L. Cheng, D. Yu, H. Zhang, P. Gumbsch, Nat. Mater.2022, 21, 869.[42] Q. Ge, A. H. Sakhaei, H. Lee, C. K. Dunn, N. X. Fang, M. L. Dunn,Sci. Rep. 2016, 6, 31110.[43] A. Vidyasagar, S. Krödel, D. M. Kochmann, Proc. R. Soc. A: Math.,Phys. Eng. Sci. 2018, 474, 20180535.[44] S. Zhao, T. M. Evans, X. Zhou, Powder Technol. 2018, 323, 323.[45] X. Zheng, T.-T. Chen, X. Jiang, M. Naito, I. Watanabe, Sci. Technol.Adv. Mater. 2023, 24, 2157682.[46] X. Zheng, T.-T. Chen, X. Guo, S. Samitsu, I. Watanabe, Mater. Des.2021, 211, 110178.[47] J. Zhou, I. Watanabe, T. Yamada, Compos. Struct. 2022, 302, 116200.[48] H. T. Kollmann, D. W. Abueidda, S. Koric, E. Guleryuz, N. A. Sobh,Mater. Des. 2020, 196, 109098.[49] X. Zheng, X. Zhang, T.-T. Chen, I. Watanabe, Adv. Mater. 2023, 35,2302530.[50] K. Choudhary, B. DeCost, C. Chen, A. Jain, F. Tavazza, R. Cohn, C.W. Park, A. Choudhary, A. Agrawal, S. J. Billinge, E. Holm, S. P. Ong,C. Wolverton, npj Comput. Mater. 2022, 8, 59.[51] S. G. Louie, Y.-H. Chan, F. H. da Jornada, Z. Li, D. Y. Qiu, Nat. Mater.2021, 20, 728.[52] T. D. Sparks, S. K. Kauwe, M. E. Parry, A. M. Tehrani, J. Brgoch, Annu.Rev. Mater. Res. 2020, 50, 27.[53] D. Lee, W. Chen, L. Wang, Y.-C. Chan, W. Chen, Adv. Mater. 2023, 36,2305254.[54] Y. Mao, Q. He, X. Zhao, Sci. Adv. 2020, 6, eaaz4169.[55] S. C.-y. Shen, M. J. Buehler, Commun. Eng. 2022, 1, 37.[56] V. Oommen, K. Shukla, S. Goswami, R. Dingreville, G. E.Karniadakis, npj Comput. Mater. 2022, 8, 190.[57] B. Deng, A. Zareei, X. Ding, J. C. Weaver, C. H. Rycroft, K. Bertoldi,Adv. Mater. 2022, 34, 2206238.[58] J.-H. Bastek, S. Kumar, B. Telgen, R. N. Glaesener, D. M. Kochmann,Proc. Natl. Acad. Sci. 2022, 119, 2111505119.[59] S. Kumar, S. Tan, L. Zheng, D. M. Kochmann, npj Comput. Mater.2020, 6, 73.[60] N. A. Alderete, N. Pathak, H. D. Espinosa, npj Comput. Mater. 2022,8, 191.[61] S. Kench, S. J. Cooper, Proc. Natl. Acad. Sci. 2021, 3, 299.[62] Z. Yang, M. J. Buehler, Front. Mater. 2021, 8, 740754.[63] Y.-C. Hsu, Z. Yang, M. J. Buehler, APL Mater. 2022, 10, 4.[64] Z. Yang, M. J. Buehler, Adv. Mater. 2023, 35, 2301449.[65] X. Shang, Z. Liu, J. Zhang, T. Lyu, Y. Zou, Mater. Today 2023, 70,71.[66] C. S. Ha, D. Yao, Z. Xu, C. Liu, H. Liu, D. Elkins, M. Kile, V.Deshpande, Z. Kong, M. Bauchy, X. (Rayne) Zheng, Nat. Commun.2023, 14, 5765.[67] B. Peng, Y. Wei, Y. Qin, J. Dai, Y. Li, A. Liu, Y. Tian, L. Han, Y. Zheng,P. Wen, Nat. Commun. 2023, 14, 6630.[68] J.-H. Bastek, D. M. Kochmann, Proc. Natl. Acad. Sci. 2023, 5,1466.[69] X. Y. Lee, J. R. Waite, C.-H. Yang, B. S. S. Pokuri, A. Joshi, A. Balu, C.Hegde, B. Ganapathysubramanian, S. Sarkar, Nat. Comp.. Sci. 2021,1, 229.[70] Y. Wang, Q. Zeng, J. Wang, Y. Li, D. Fang, Comp. Methods in AppliedMechanics and Engineering 2022, 401, 115571.[71] M. Maurizi, C. Gao, F. Berto, npj Comput. Mater. 2022, 8, 247.[72] T. Jin, X. Cheng, S. Xu, Y. Lai, Y. Zhang, J. Mech. Phys. Solids 2023,179, 105398.[73] M. Li, Y. Duan, J. Zhou, J. Lu, in Proc. of the IEEE/CVF Conf. on Com-puter Vision and Pattern Recognition, IEEE, Piscataway, NJ 2023, pp.12642–12651.[74] X. Zeng, A. Vahdat, F. Williams, Z. Gojcic, O. Litany, S. Fidler, K.Kreis, Adv. Neural Inf. Proc. Syst. 2022, 35, 10021.[75] R. Fu, X. Zhan, Y. Chen, D. Ritchie, S. Sridhar, Adv. Neural Inf. Proc.Syst. 2022, 35, 8882.[76] K. Chen, C. B. Choy, M. Savva, A. X. Chang, T. Funkhouser, S.Savarese, in Computer Vision–ACCV 2018: 14th Asian Conf. onComputer Vision, Perth, Australia, December 2–6, 2018, RevisedSelected Papers, Part III 14, Springer, Berlin, Heidelberg 2019,pp. 100–116.[77] Z. Liu, Y. Wang, X. Qi, C.-W. Fu, in Proc. of the IEEE/CVF Conf. onComputer Vision and Pattern Recognition, IEEE, Piscataway, NJ 2022,pp. 17896–17906.[78] A. Sanghi, H. Chu, J. G. Lambourne, Y. Wang, C.-Y. Cheng, M.Fumero, K. R. Malekshan, in Proc. of the IEEE/CVF Conf. on Com-puter Vision and Pattern Recognition, IEEE, Piscataway, NJ 2022, pp.18603–18613.[79] N. Mohammad Khalid, T. Xie, E. Belilovsky, T. Popa, in SIGGRAPHAsia 2022 conference papers, Association for Computing Machinery,New York 2022, pp. 1–8.[80] J. Xu, X. Wang, W. Cheng, Y.-P. Cao, Y. Shan, X. Qie, S. Gao, in Proc.of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition.,IEEE, Piscataway, NJ 2023, pp. 20908–20918.[81] J. Gao, T. Shen, Z. Wang, W. Chen, K. Yin, D. Li, O. Litany, Z. Gojcic,S. Fidler, Adv. Neural Inf. Process, Syst. 2022, 35, 31841.[82] Z. Liu, P. Dai, R. Li, X. Qi, C.-W. Fu, ISS: image as stepping stonefor text-guided 3D shape generation, arXiv preprint arXiv:2209.041452022.[83] Z. Zhao, W. Liu, X. Chen, X. Zeng, R. Wang, P. Cheng, B. Fu, T.Chen, G. Yu, S. Gao, Michelangelo: conditional 3D shape genera-tion based on shape-image-text aligned latent representation, arXivpreprint arXiv:2306.17115 2023.[84] A. Nichol, H. Jun, P. Dhariwal, P. Mishkin, M. Chen, Point-E: a sys-tem for generating 3D point clouds from complex prompts, arXivpreprint arXiv:2212.08751 2022.[85] H. Jun, A. Nichol, Shap-E: generating conditional 3D implicit func-tions, arXiv preprint arXiv:2305.02463 2023.Small 2024, 20, 2402685 © 2024 The Author(s). Small published by Wiley-VCH GmbH2402685 (11 of 12) 16136829, 2024, 37, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/smll.202402685 by National Institute For, Wiley Online Library on [19/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons Licensehttp://www.advancedsciencenews.comhttp://www.small-journal.comwww.advancedsciencenews.com www.small-journal.com[86] J. Wei, H. Wang, J. Feng, G. Lin, K.-H. Yap, in Proc. of the IEEE/CVFConf. on Computer Vision and Pattern Recognition, IEEE, Piscataway,NJ 2023, pp. 16805–16815.[87] A. Jain, B. Mildenhall, J. T. Barron, P. Abbeel, B. Poole, in Proc. of theIEEE/CVF Conf. on Computer Vision and Pattern Recognition, IEEE,Piscataway, NJ 2022, pp. 867–876.[88] M. J. Buehler, Oxf. open mater. sci. 2022, 2, itac010.[89] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal,G. Sastry, A. Askell, P. Mishkin, J. Clark, Gretchen Krueger, IlyaSutskever, in Int. Conf. on Machine Learning, PMLR, New York 2021,pp. 8748–8763.[90] F. Chollet, in Proc. of the IEEE Conf. on Computer Vision and PatternRecognition, IEEE, Piscataway, NJ 2017, pp. 1251–1258.[91] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, in Proc. of the 2019Conf. of the North American Chapter of the Association for Compu-tational Linguistics, Human Language Technologies, Minneapolis,Minnesota 2019, 1, pp. 4171–4186.[92] A. Brock, T. Lim, J. M. Ritchie, N. Weston, Generative and discrim-inative voxel modeling with convolutional neural networks, arXivpreprint arXiv:1608.04236 2016.[93] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, S. Hochreiter,Adv. Neural Inf. Process. Syst. 2017, 30, 6629.[94] T. Anvekar, R. A. Tabib, D. Hegde, U. Mudengudi, in Proc. of theIEEE/CVF Conf. on Computer Vision and Pattern Recognition, IEEE,Piscataway, NJ 2022, pp. 2978–2985.[95] C. Nash, Y. Ganin, S. A. Eslami, P. Battaglia, presentedat Int. Conf. on Machine Learning, PMLR, New York 2020,pp. 7220–7229.[96] X. Zheng, X. Guo, Y. Yang, Z. Fu, K. Du, C. Wang, Y. Yi, J. Phys. Chem.C 2018, 122, 16803.[97] G. Dong, Y. Tang, Y. F. Zhao, J. Eng. Mater. Techn. 2019, 141, 011005.[98] T. Gupta, M. Zaki, N. A. Krishnan, Mausam, npj Comput. Mater.2022, 8, 102.[99] F. Yin, X. Chen, C. Zhang, B. Jiang, Z. Zhao, J. Fan, G. Yu, T. Li, T.Chen, ShapeGPT: 3D shape generation with a unified multi-modallanguage model, arXiv preprint arXiv:2311.17618 2023.[100] C. Brauer, D. M. Aukes, in Proc. of the 5th Annual ACM Symp. onComputational Fabrication, Association for Computing Machinery,New York 2020, pp. 1–8.[101] Dawson-Haggerty, et al., Trimesh, version 3.2.0, https://trimesh.org/ (accessed: December 2019).[102] I. Watanabe, D. Setoyama, N. Nagasako, N. Iwata, K. Nakanishi, Int.J. Numer. Methods Eng. 2012, 89, 829.[103] I. Watanabe, A. Yamanaka, Int. J. Mech. Sci. 2019, 150, 314.Small 2024, 20, 2402685 © 2024 The Author(s). Small published by Wiley-VCH GmbH2402685 (12 of 12) 16136829, 2024, 37, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/smll.202402685 by National Institute For, Wiley Online Library on [19/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons Licensehttp://www.advancedsciencenews.comhttp://www.small-journal.comhttps://trimesh.org/https://trimesh.org/ Text-to-Microstructure Generation Using Generative Deep Learning 1. Introduction 2. Results and Discussion 2.1. Deep Generative Model 2.2. Microstructure Generation Using Text Prompt 2.3. Inverse Design of Microstructures 3. Conclusion 4. Experimental Section Supporting Information Acknowledgements Conflict of Interest Data Availability Statement Keywords