# Fileset

[materials-17-02866.pdf](https://mdr.nims.go.jp/filesets/ff491baf-4210-44cf-8fe3-10d5d8c1d873/download)

## Creator

[Paripat Kraisornkachit](https://orcid.org/0000-0002-0796-3665), [Masanobu Naito](https://orcid.org/0000-0001-7198-819X), [Chao Kang](https://orcid.org/0000-0002-9567-0976), Chiaki Sato

## Rights

[Creative Commons BY Attribution 4.0 International](https://creativecommons.org/licenses/by/4.0/)

## Other metadata

[Multi-Objective Optimization of Adhesive Joint Strength and Elastic Modulus of Adhesive Epoxy with Active Learning](https://mdr.nims.go.jp/datasets/9d3aacb6-6673-4bdb-b786-aab00c23ede1)

## Fulltext

Multi-Objective Optimization of Adhesive Joint Strength and Elastic Modulus of Adhesive Epoxy with Active LearningCitation: Kraisornkachit, P.; Naito, M.;Kang, C.; Sato, C. Multi-ObjectiveOptimization of Adhesive JointStrength and Elastic Modulus ofAdhesive Epoxy with Active Learning.Materials 2024, 17, 2866. https://doi.org/10.3390/ma17122866Academic Editor: Mariana CristeaReceived: 19 April 2024Revised: 28 May 2024Accepted: 5 June 2024Published: 12 June 2024Copyright: © 2024 by the authors.Licensee MDPI, Basel, Switzerland.This article is an open access articledistributed under the terms andconditions of the Creative CommonsAttribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).materialsArticleMulti-Objective Optimization of Adhesive Joint Strength andElastic Modulus of Adhesive Epoxy with Active LearningParipat Kraisornkachit 1,2 , Masanobu Naito 1,2,* , Chao Kang 3 and Chiaki Sato 31 Data-Driven Polymer Design Group, Research Center for Macromolecules and Biomaterials,National Institute for Materials Science (NIMS), Ibaraki 305-0047, Japan; kraisornkachit.paripat@nims.go.jp2 Program in Materials Science and Engineering, Graduate School of Pure and Applied Sciences,University of Tsukuba, Ibaraki 305-8577, Japan3 Institute of Innovative Research (IIR), Tokyo Institute of Technology, Kanagawa 226-8503, Japan;kang.c.ab@m.titech.ac.jp (C.K.); csato@pi.titech.ac.jp (C.S.)* Correspondence: naito.masanobu@nims.go.jpAbstract: Studying multiple properties of a material concurrently is essential for obtaining a compre-hensive understanding of its behavior and performance. However, this approach presents certainchallenges. For instance, simultaneous examination of various properties often necessitates extensiveexperimental resources, thereby increasing the overall cost and time required for research. Fur-thermore, the pursuit of desirable properties for one application may conflict with those neededfor another, leading to trade-off scenarios. In this study, we focused on investigating adhesivejoint strength and elastic modulus, both crucial properties directly impacting adhesive behavior.To determine elastic modulus, we employed a non-destructive indentation method for convertinghardness measurements. Additionally, we introduced a specimen apparatus preparation methodto ensure the fabrication of smooth surfaces and homogeneous polymeric specimens, free fromvoids and bubbles. Our experiments utilized a commercially available bisphenol A-based epoxyresin in combination with a Poly(propylene glycol) curing agent. We generated an initial datasetcomprising experimental results from 32 conditions, which served as input for training a machinelearning model. Subsequently, we used this model to predict outcomes for a total of 256 conditions.To address the high deviation in prediction results, we implemented active learning approaches,achieving a 50% reduction in deviation while maintaining model accuracy. Through our analysis,we observed a trade-off boundary (Pareto frontier line) between adhesive joint strength and elasticmodulus. Leveraging Bayesian optimization, we successfully identified experimental conditions thatsurpassed this boundary, yielding an adhesive joint strength of 25.2 MPa and an elastic modulus of182.5 MPa.Keywords: epoxy; adhesive; elastic modulus; machine learning; active learning; experimental testing;multi-objective optimization; sandwich-structured material1. IntroductionIn the realm of materials science and engineering, the study of adhesive materialsplays a pivotal role in advancing technologies across diverse industries [1], for example, au-tomotive [2], aerospace [3] and construction materials [4]. The adhesive joint strength, rep-resenting the force required to break or deform a bonded interface, and the elastic modulus,signifying a material’s resistance to deformation under applied stress, are two fundamentalproperties that have important influence on the performance and reliability of adhesivesystems. The adhesive joint strength serves as a crucial metric in assessing the integrity ofbonded structures. A robust and durable bond is often synonymous with high adhesivejoint strength, ensuring the stability of assemblies in various environments and underdifferent loading conditions [5]. However, this strength is not isolated from the elasticMaterials 2024, 17, 2866. https://doi.org/10.3390/ma17122866 https://www.mdpi.com/journal/materialshttps://doi.org/10.3390/ma17122866https://doi.org/10.3390/ma17122866https://creativecommons.org/https://creativecommons.org/licenses/by/4.0/https://creativecommons.org/licenses/by/4.0/https://www.mdpi.com/journal/materialshttps://www.mdpi.comhttps://orcid.org/0000-0002-0796-3665https://orcid.org/0000-0001-7198-819Xhttps://orcid.org/0000-0002-9567-0976https://doi.org/10.3390/ma17122866https://www.mdpi.com/journal/materialshttps://www.mdpi.com/article/10.3390/ma17122866?type=check_update&version=1Materials 2024, 17, 2866 2 of 17modulus of the adhesive material. The elastic modulus, or stiffness, defines how the ma-terial responds to external forces [6], influencing the distribution of stresses within theadhesive joint. As such, the interplay between adhesive joint strength and elastic modulusbecomes a critical factor in determining the overall performance and longevity of bondedstructures [7]. Understanding the trade-offs and synergies between adhesive joint strengthand elastic modulus is imperative for optimizing material selection based on specific appli-cation requirements. The relationship between adhesive joint strength and elastic modulusis often complex. Several studies have reported that high-elastic-modulus materials have ahigher adhesive joint strength [8,9]. However, lower-modulus adhesives can provide theability to absorb external forces, and this ability is an important factor in adhesive materialsused in bonding parts that are easily broken or damaged [7]. In architectural applicationslike structural glazing systems, low elastic adhesives are used to bond glass panels to thestructural framework. These low modulus adhesives can handle high stress gradients atglass interfaces [10].Achieving an optimal balance between these properties is a multifaceted challenge,as conventional optimization approaches often focus on a singular property, potentiallyneglecting the complex interdependencies that exist. Multi-objective optimization presentsa paradigm shift by simultaneously addressing the enhancement of many properties. Re-cently, high productivity of heat-resistant epoxy matrix systems was successfully achievedusing multi-objective optimization along with machine learning. This accomplishment hasnever been achieved, even though the conventional trial-and-error experiment has beenattempted up to three hundred times [11]. Furthermore, because the required propertiesfrequently conflict with one another in nature, multi-objective optimization can be a can-didate method from the material design point of view. The maximum molecular weightand the maximum of number average degree of branching of polymers were achieved inthe polymerization process with the assistance of a multi-objective optimization approach.This can reduce unnecessary costs and time consumption in the experimental stage [12].Moreover, the multi-objective optimization approach was applied in optimizing materialremoval rate and taper during electrochemical discharge machining of the silicon carbide-reinforced epoxy composites, and it was reported that there was a 15% improvement intaper reduction, together with the material removal rate decreasing by 0.91%. In addition,utilizing multi-objective optimization is able to address the trade-off problem [13]. Thisapproach not only extends the edge of discovery but also accelerates the study processfor humankind.The adhesive joint strength has been measured previously by using single-lap shearjoints [14]. On the other hand, indentation hardness measurements were used to evaluatethe elastic modulus of the same adhesives, owing to the significant convenience in samplepreparation and measurements compared to the widely used tensile test. The relation-ship between hardness and elastic modulus has been thoroughly studied, and severalmethods for calculating elastic modulus from hardness measurements have been reportedpreviously [15]. Nevertheless, before measuring the hardness of polymeric specimens, itis important to prepare appropriate samples to ensure accurate and reliable results. Thepolymeric specimens have to be prepared in standardized shapes and sizes suitable for thespecific hardness testing method being used. Additionally, the surface of the specimensmust be prepared to ensure flatness and smoothness, as irregularities or rough surfaces candirectly affect hardness measurements. Moreover, voids and bubbles lead to inaccurateresults; thus, homogeneity of the polymer material is one of the important factors.The adhesive joint strength and elastic modulus are key mechanical properties that di-rectly influence the adhesion performance of adhesive materials. However, simultaneouslystudying multiple properties can be time-consuming and costly. In this work, we proposeto investigate the adhesive joint strength and elastic modulus of epoxy adhesives using amulti-objective optimization approach. A metal mold is designed and introduced to pre-pare the polymeric specimens according to the French standard NFT 76-142 [16] to eliminateporosity. The elastic modulus of the polymeric specimens is calculated according to ShoreMaterials 2024, 17, 2866 3 of 17hardness which is measured by two types of durometers referring to ASTM-D2240 [17].The experimental conditions are designed and conducted to obtain a small initial dataset ofthe first 32 conditions. The machine learning model is trained and validated for optimizingaccuracy. Then, prediction of the extended 256 conditions is carried out together with theactive learning method. Active learning is a strategy that can be employed to improvemodel accuracy and reduce deviations in predicted results. Active learning is a machinelearning approach that involves selecting the most informative data points for labeling orfurther training, with the goal of enhancing the model performance while minimizing theamount of labeled data needed. After that, the trade-off boundary is obtained after theimproved prediction of the 256 conditions. Finally, Bayesian optimization is employed toidentify experimental conditions with predicted results that can overcome the trade-offboundary. The results are confirmed by checking the actual experiments. Adhesive epoxieswith desired adhesive joint strength and elastic modulus properties can be fabricated withthe provided conditions from the proposed multi-objective optimization approaches.2. Materials and Methods2.1. Experiments2.1.1. MaterialsIn this study, a commercial epoxy resin, Diglycidyl ether of bisphenol A-based epoxyresin (DGEBA) from Mitsubishi Chemical Corporation, Tokyo, Japan, with 4 differentmolecular weights, and a commercial diamine curing agent (Jeffamine™), Poly(propyleneglycol) bis(2-aminopropyl ether) from Sigma-Aldrich, Tokyo, Japan, with 4 differentmolecular weights, were used. The molecular weights of DGEBA and Jeffamine™ areMwE = {370, 1650, 2900 and 3800} g/mol and MwC = {230, 400, 2000 and 4000} g/mol, re-spectively. The appearance of DGEBA with MwE of 230 g/mol is in liquid phase; in contrast,it is in solid phase for MwE of 1650, 2900 and 3800 g/mol. Additionally, the appearanceof Jeffamine™ with MwC of 230 and 400 g/mol is viscous liquid; on the other hand, itis a highly viscous liquid for 2000 and 4000 g/mol. All chemicals were used as receivedwithout further purification or pretreatment. Generally, the stoichiometric mixing ratioof epoxy resin is two moles of epoxy resin per one mole of curing agent. The chemicalstructures and reactions of DGEBA and Jeffamine™ are illustrated in Figure 1.Materials 2024, 17, x FOR PEER REVIEW 4 of 18    Figure 1. Chemical structures of DGEBA, Jeffamine™ and cured epoxy. 2.1.2. Preparation of Specimens A metal mold was designed and fabricated specifically for this study according to the French standard NFT76-142 [16]; the illustration is shown in Figure S1 (Supplementary Materials). In principle, the objective of using this mold is not only to produce cured ad-hesive specimens with a flat and smooth surface according to the standard but also to remove the voids and bubbles entrapped inside the specimens as much as possible. This metal mold consists of a metal lid and a metal base, and the adhesive specimen is fitted into a 6 mm thick silicon rubber frame (obtained from Axel 4-1371-08) at the center of the mold. This silicon rubber is bordered by a metal box which contains 4 small gaps on the topside of each edge; these gaps allow the adhesive mixture to overflow during high-pres-sure curing conditions. An overflow of adhesive mixture is potentially able to remove gas bubbles entrapped in the specimens [18]. This mold created a square-shaped specimen with a dimension of 40 × 40 × 6 mm3 according to the standard test method for rubber property ASTM-D2240 [17]; the illustration is shown in Figure S1. Teflon tape (obtained from Axel 3-5579-09) was pasted onto the metal lid and metal base on the contact side of the adhesive specimen in order to reduce the difficulty in removing the cured adhesive specimens from the mold. DGEBA and Jeffamine™ were separately preheated in the oven at 190 °C for 30 min; this process is to ensure that solid epoxy is melted into liquid prior to the mixing step. After that, it was rapidly mixed together in the disposal bottle glass for a couple seconds until the mixture achieved a homogenous phase, and then, the mixture was poured into a metal mold. In total, four specific amine-to-epoxide ratios were investigated in this study: r = {0.75, 1.00, 1.25 and 1.50}. For example, r = 1 represents the stoichiometric mixing ratio of the epoxy with the curing agent: r = 0.75 represents 25% less amine curing agent than that used in the stoichiometric mixing ratio; on the other hand, r = 1.25 represents 25% excess amine curing agent over epoxy resin. After pouring the mixture into the mold, a metal lid was placed on top, and the assembled mold was put into hot-press machine with applying force of 2 MPa to the mold. The curing time for every condition was set to 60 min. In addition, the effect of curing temperature was investigated. The mixture was cured Figure 1. Chemical structures of DGEBA, Jeffamine™ and cured epoxy.Materials 2024, 17, 2866 4 of 172.1.2. Preparation of SpecimensA metal mold was designed and fabricated specifically for this study according to theFrench standard NFT76-142 [16]; the illustration is shown in Figure S1 (SupplementaryMaterials). In principle, the objective of using this mold is not only to produce curedadhesive specimens with a flat and smooth surface according to the standard but also toremove the voids and bubbles entrapped inside the specimens as much as possible. Thismetal mold consists of a metal lid and a metal base, and the adhesive specimen is fitted intoa 6 mm thick silicon rubber frame (obtained from Axel 4-1371-08) at the center of the mold.This silicon rubber is bordered by a metal box which contains 4 small gaps on the topside ofeach edge; these gaps allow the adhesive mixture to overflow during high-pressure curingconditions. An overflow of adhesive mixture is potentially able to remove gas bubblesentrapped in the specimens [18]. This mold created a square-shaped specimen with adimension of 40 × 40 × 6 mm3 according to the standard test method for rubber propertyASTM-D2240 [17]; the illustration is shown in Figure S1. Teflon tape (obtained from Axel3-5579-09) was pasted onto the metal lid and metal base on the contact side of the adhesivespecimen in order to reduce the difficulty in removing the cured adhesive specimens fromthe mold.DGEBA and Jeffamine™ were separately preheated in the oven at 190 ◦C for 30 min;this process is to ensure that solid epoxy is melted into liquid prior to the mixing step.After that, it was rapidly mixed together in the disposal bottle glass for a couple secondsuntil the mixture achieved a homogenous phase, and then, the mixture was poured into ametal mold. In total, four specific amine-to-epoxide ratios were investigated in this study:r = {0.75, 1.00, 1.25 and 1.50}. For example, r = 1 represents the stoichiometric mixing ratioof the epoxy with the curing agent: r = 0.75 represents 25% less amine curing agent thanthat used in the stoichiometric mixing ratio; on the other hand, r = 1.25 represents 25%excess amine curing agent over epoxy resin. After pouring the mixture into the mold, ametal lid was placed on top, and the assembled mold was put into hot-press machine withapplying force of 2 MPa to the mold. The curing time for every condition was set to 60 min.In addition, the effect of curing temperature was investigated. The mixture was cured atdifferent specific curing temperatures of TC = {90, 130, 170 and 210} ◦C. The mold was thenkept under applying force until it cooled down to room temperature prior to taking out thespecimens for further measurement. The total variable parameters in this study followedour previous work [14] and are summarized in Table 1. For the conditions of liquid DGEBA,the weight of the cured epoxies was set to be sufficient for the metal mold at 16 g, whereasthe weight of the cured epoxies with the conditions of solid DGEBA was set at 24 g.Table 1. Summary of variable parameters for experiments [14].Epoxy Resin Curing Agent Amine toEpoxide Ratio (r) TC (◦C)MwE (g/mol) Appearance MwC (g/mol) Appearance370 Liquid 230 Viscous 0.75 901650 Solid 400 Viscous 1.00 1302900 Solid 2000 Highly viscous 1.25 1703800 Solid 4000 Highly viscous 1.50 230The single-lap shear specimens were prepared and tested for adhesive joint strengthusing the same procedure as described in our previous work. The epoxy resin and curingagent were preheated and then hand-mixed for a few seconds until achieving a homo-geneous phase. Subsequently, the mixture was poured and spread onto an aluminumsubstrate (A6061P-T6, dimensions: 100 × 25 × 2 mm) over an area of 25 × 12.5 mm. Theadhesive thickness was controlled by adding 0.1 parts per hundred resin of spherical glassbeads (Fujiseisakujo, Tokyo, Japan). A second aluminum substrate was then placed ontop of the adhesive overlapping the first substrate, creating a sandwich specimen. Thespecimens were clamped and subjected to the specific curing temperature in an oven forMaterials 2024, 17, 2866 5 of 1760 min. Adhesive joint strength testing was conducted using a 10-kN AG-X plus seriesuniversal tensile testing machine (Shimadzu, Kyoto, Japan). To ensure data reliability, atleast two specimens were fabricated for each measurement. The results are reported as theaverage value along with the standard deviation [14].2.1.3. Modulus Testing TechniqueShore hardness measurements of the specimens were performed using durometerhardness testing tools. After pressing the indentation on top of the specimen surface, theindenter of the durometer pierced the specimens, and then, the hardness values wereobserved by the gauge of the durometer. There are 2 types of durometer hardness testingtools: Shore A and Shore D, which were performed in this study. Shore A is appropriatefor the soft materials that are in the range between 20A and 80A; on the other hand, ShoreD is suitable for the hard materials that are in the range of 80A to 85D. After measuringhardness values, it was converted into S by the following Equation (1). Then, the elasticmodulus was calculated by the relationship between hardness and elastic modulus viaEquation (2) as follows [15]:S ={Shore A 20A < S < 80AShore D + 50 80A < S < 85D(1)logE0 = 0.0235S − 0.6403 (2)The specimens were placed on the rigid surface prior to measuring the hardnessat five points on the topside of the specimens in an effort to minimize variation. Everymeasurement point was at least 6.0 mm away from the other measuring points and at least12.0 mm away from any edge of the specimens as illustrated in Figure S1b [17]. Areas ofthe specimens with trapped gas bubbles must be avoided when measuring the hardness.The elastic modulus of each point was independently calculated, and then, the averagevalue with standard deviation was reported for each specimen.2.2. Multi-Objective Optimization Approaches2.2.1. Dataset PreparationThe total number of possible conditions in this study was 256, which consisted of4 MwE times 4 MwC times 4 amine-to-epoxide ratios times 4 curing temperatures (Table 1).However, the initial dataset for machine learning model training was 32 conditions ac-cording to our previous work [14]. According to our previous study, these 32 condi-tions of the initial dataset were selected by applying experimental techniques using four-by-four Graeco–Latin square design in order to uniformly distribute the experimentalconditions [19,20]. There were two properties studied for each condition, which consistedof adhesive joint strength and elastic modulus. Datasets of adhesive joint strength wereobtained from our previous work; on the other hand, datasets of elastic modulus wereobtained by conducting experiments with the same conditions.2.2.2. Cross-Validation TechniqueMachine learning modeling was carried out using the Python programming language.Several Python packages were applied in this work from the scikit-learn library, for example,data splitting, cross-validation, and machine learning model training and testing. An initialdataset of 32 conditions, each consisting of four variable parameters with two properties,was split into a training set and a testing set in order to train the machine learning modeland evaluate its accuracy. In order to uniformly utilize all 32 conditions in the initial datasetas a training set and testing set, the K-fold cross-validation technique was introduced tosplit these initial datasets. K-fold cross-validation is a technique used for evaluating theperformance of a model by dividing the dataset into multiple smaller folds. This methodhelps to reduce overfitting and provides a more accurate estimate of a model’s performanceon unseen data. The initial dataset was randomly split into K equal-sized folds. In thisMaterials 2024, 17, 2866 6 of 17study, the initial dataset was divided into K = 32 folds. Then, the machine learning modelwas trained on the data from the remaining K − 1 = 31 folds and evaluated for accuracyof models on the data from the remaining fold. This process was repeated K times, witheach fold being used as the validation set only once. After that, the accuracy score of themachine learning models for each K times was calculated and then averaged to obtainthe accuracy score of that model. The model with the highest accuracy score was chosenfor further use. K-fold cross-validation is able to reduce the possibility of overfitting bytraining and evaluating the model on different folds of the data. This technique provides amore reliable estimate of the model’s performance on unseen data, as it reduces the impactof any single-fold peculiarities on the overall performance score [21].2.2.3. Modeling and PredictionSeven machine learning algorithms were investigated in order to find the best modelin this study. The Ridge regression algorithm is generally used to deal with a problemcalled multicollinearity. This problem occurs when the predictor variables—variables thatare used to predict the outcome variable—in a regression model are highly correlated witheach other. As a result, this can be difficult to estimate the effects of each predictor variableon the outcome variable accurately. The Ridge regression solves this problem by adding asmall number to the diagonal of the matrix of predictor variables. By adding this number,the Ridge regression shrinks the estimates of the coefficients towards zero, which reducestheir impact on the outcome variable. Thus, the Ridge regression algorithm is a techniquethat helps to improve the accuracy and reliability of estimates in regression models whenthere is multicollinearity among predictor variables [22]. Likewise, the Lasso regressionalgorithm is commonly used to prevent overfitting in linear regression models. However,the main difference between the Lasso and Ridge regression is the method used to preventoverfitting. Unlike the Ridge algorithm, the Lasso shrinks some of the coefficients to exactly0, effectively performing variable selection and making the model simpler [23]. The Elasticnet algorithm combines the strengths of both the Ridge and Lasso methods, while theRidge regression does not perform variable selection and the Lasso can struggle with highlycorrelated predictors, to provide an improved approach for regularization and variableselection [24]. A new K-nearest neighbor (k-NN) algorithm utilizes the neighborhoodpoints in the training sample. For each new data point, the algorithm finds the K nearestneighbors based on Euclidean distance. Then, the output variable for a new data pointis predicted as the average of the output variables of its K nearest neighbors. This k-NNregression algorithm is a simple and intuitive algorithm that can work well for smalldatasets [25]. In cases of complex nonlinear relationships between the input and outputvariables, the Decision Tree Regression (DTR) algorithm can handle these cases efficiently.The DTR algorithm creates a tree-like structure where each internal node represents adecision based on a feature and threshold value, and each leaf node represents a predictionfor the output variable. Then, the DTR algorithm selects the feature and threshold valuethat best split the data into two subsets that are as homogeneous as possible with respectto the output variable [26]. In order to improve the accuracy of the model, the RandomForest algorithm was introduced as an extension of the DTR algorithm. The principle of theRandom Forest is to construct multiple decision trees at the training step and combine theirpredictions to make a final output variable prediction. Each tree is built using a randomsubset of the features and data points, which helps to reduce the correlation between thetrees and improve the diversity of the forest. By combining the predictions of multiple trees,the Random Forest can capture more of the underlying patterns in the data and make moreaccurate predictions [27]. Lastly, the Gradient boosting algorithm was one of the candidatealgorithms in this study. This algorithm works by iteratively adding decision trees to themodel, with each tree attempting to correct the errors of the previous tree. The algorithmuses gradient descent optimization to minimize the loss function, which measures thedifference between the predicted values and the actual values [28].Materials 2024, 17, 2866 7 of 17All seven machine learning algorithms were trained independently by using the initialdataset with the K-fold cross-validation technique. The accuracy of each algorithm wasevaluated by the calculation of three tools: the coefficient of determination (R2 score), themean absolute error (MAE) and the root-mean-square error (RMSE). The algorithms witha R2 score close to 1 show higher accuracy. On the other hand, the algorithms with alower value of MAE and RMSE show higher accuracy. The calculation of these three toolsutilized the prediction results from the testing dataset and the measured results from theexperiment. Moreover, the predictions of both adhesive joint strength and elastic moduluswere carried out simultaneously in this step.The model with the highest accuracy was selected to perform further prediction of thetotal 256 conditions. Furthermore, the K-fold cross-validation technique was also appliedin this step in order to average the prediction results of each fold in each condition. Hence,the relationship between adhesive joint strength and elastic modulus for each conditionwas observed. In addition, the standard deviation of each prediction result was obtained.Three predicted results with high deviations from three regions: low adhesive jointstrength with low elastic modulus, low adhesive joint strength with high elastic modulusand high adhesive joint strength with high elastic modulus. These were selected to performan active learning approach. The experimental conditions at these three selected pointswere conducted for the experiment. Then, the additional dataset (mi = 3) of the measuredadhesive joint strength and elastic modulus was added to the initial dataset (ni = 32) tomake a new dataset (n = ni + mi = 32 + 3 = 35) for the second active learning cycle, andso on. The overall process started with machine learning model training once again inorder to improve the model accuracy as well as the standard deviation of the predictedresults; this process is called the active learning approach. This active learning approachwas repeated until the average values of the prediction errors (the standard deviation ofpredicted results) were comparable to the experimental errors; then, the active learningloop was terminated.After termination of the active learning loop, machine learning model accuracy andthe standard deviation of the predicted results were collected. In addition, the correlationbetween adhesive joint strength and elastic modulus properties was observed by plottingthe predicted 256 conditions from the last active learning cycle. Moreover, the trade-off lineof these two properties, represented by the Pareto frontier line, was drawn by connectingthe boundary points. These data were kept for investigation in the next step.2.2.4. Bayesian OptimizationIn this study, Bayesian optimization was performed using PHYSBO [29], a Pythonlibrary for Bayesian optimization, in order to search for extended conditions outside thePareto frontier line. In this step, the variable parameters of amine-to-epoxide ratios togetherwith curing temperatures were studied as a continuous value. The amine-to-epoxideratios ranged from 0.75 to 1.50 with 0.01 increment steps. On the other hand, the curingtemperatures ranged from 90 ◦C to 210 ◦C with an increment step of 1 ◦C. Nevertheless, theMwE and MwC parameters were maintained as discrete values because of the limitations inthe supply of these commercially available materials. The summary of variable parametersfor Bayesian optimization is shown in Table 2. All of measured data was utilized as an inputdataset (n) in the PHYSBO with the adjusted settings for a multi-objective optimizationcase with two objective numbers. Additionally, the Thompson sampling method [30] wasselected to search for conditions beyond the Pareto frontier line from the active learningstage. In the end, the conditions proposed by PHYSBO were conducted the experiments toconfirm the results. The pipeline of the overall workflow is illustrated in Figure 2.Materials 2024, 17, 2866 8 of 17Table 2. Summary of variable parameters for Bayesian optimization.Parameters Values Step TypeMwE (g/mol) 370 1650 2900 3800 - DiscreteMwC (g/mol) 230 400 2000 4000 - Discreter 0.75–1.50 0.01 ContinuousTC (◦C) 90–120 1 ContinuousMaterials 2024, 17, x FOR PEER REVIEW 8 of 18   predicted results) were comparable to the experimental errors; then, the active learning loop was terminated.  After termination of the active learning loop, machine learning model accuracy and the standard deviation of the predicted results were collected. In addition, the correlation between adhesive joint strength and elastic modulus properties was observed by plotting the predicted 256 conditions from the last active learning cycle. Moreover, the trade-off line of these two properties, represented by the Pareto frontier line, was drawn by con-necting the boundary points. These data were kept for investigation in the next step. 2.2.4. Bayesian Optimization In this study, Bayesian optimization was performed using PHYSBO [29], a Python library for Bayesian optimization, in order to search for extended conditions outside the Pareto frontier line. In this step, the variable parameters of amine-to-epoxide ratios to-gether with curing temperatures were studied as a continuous value. The amine-to-epox-ide ratios ranged from 0.75 to 1.50 with 0.01 increment steps. On the other hand, the curing temperatures ranged from 90 °C to 210 °C with an increment step of 1 °C. Nevertheless, the MwE and MwC parameters were maintained as discrete values because of the limitations in the supply of these commercially available materials. The summary of variable param-eters for Bayesian optimization is shown in Table 2. All of measured data was utilized as an input dataset (n) in the PHYSBO with the adjusted settings for a multi-objective opti-mization case with two objective numbers. Additionally, the Thompson sampling method [30] was selected to search for conditions beyond the Pareto frontier line from the active learning stage. In the end, the conditions proposed by PHYSBO were conducted the ex-periments to confirm the results. The pipeline of the overall workflow is illustrated in Fig-ure 2. Table 2. Summary of variable parameters for Bayesian optimization. Parameters Values Step Type MwE (g/mol) 370 1650 2900 3800 - Discrete MwC (g/mol) 230 400 2000 4000 - Discrete r 0.75–1.50 0.01 Continuous TC (°C) 90–120 1 Continuous  Figure 2. Illustration of the overall workflow, consisting of three stages: (i) experimental design, (ii) active learning cycle and (iii) Bayesian optimization, where n refers to the total number of datasets, i refers to the first cycle and m refers to the number of additional datasets. Figure 2. Illustration of the overall workflow, consisting of three stages: (i) experimental design,(ii) active learning cycle and (iii) Bayesian optimization, where n refers to the total number of datasets,i refers to the first cycle and m refers to the number of additional datasets.3. Results3.1. ExperimentsThe polymeric specimens were fabricated successfully, ensuring smooth surfaces anduniformity. Eye inspection revealed the absence of voids and bubbles within the specimens.The example of specimens is illustrated in the Figure S2. The specimens were then measuredfor the hardness. The elastic modulus was calculated after measuring the hardness of eachspecimen. The highest elastic modulus was observed at 363.9 MPa, whereas the lowestelastic modulus was observed at 0 MPa because of the incompletely cured specimen. All ofthe experimental results with variable parameters are listed in Table S1 (SupplementaryMaterials). The distribution of elastic modulus was plotted as the percentage of the total32 specimens within specific ranges of elastic modulus values to examine the distributionof the dataset. For example, 22% indicates that 7 out of 32 specimens have elastic modulusvalues in the range of 0 to 46 MPa. This distribution demonstrated a well-spread dataset,as illustrated in Figure 3a. Together with the adhesive joint strength results from previouswork [14], these two properties were observed as the characteristics in Figure 3b.Materials 2024, 17, x FOR PEER REVIEW 9 of 18   3. Results 3.1. Experiments The polymeric specimens were fabricated successfully, ensuring smooth surfaces and uniformity . Eye inspection revealed the absence of voids and bubbles within the speci-mens. The example of specimens is illustrated in the Figure S2. The specimens were then measured for the hardness. The elastic modulus was calculated after measuring the hard-ness of each specimen. The highest elastic modulus was observed at 363.9 MPa, whereas the lowest elastic modulus was observed at 0 MPa because of the incompletely cured spec-imen. All of the experimental results with variable parameters are listed in Table S1 (Sup-plementary Materials). The distribution of elastic modulus was plotted as the percentage of the total 32 specimens within specific ranges of elastic modulus values to examine the distribution of the dataset. For example, 22% indicates that 7 out of 32 specimens have elastic modulus values in the range of 0 to 46 MPa. This distribution demonstrated a well-spread dataset, as illustrated in Figure 3a. Together with the adhesive joint strength results from previous work [14], these two properties were observed as the characteristics in Fig-ure 3b.    (a) (b) Figure 3. Experimental results of adhesive joint strength (MPa) and elastic modulus (MPa); (a) dis-tribution chart of elastic modulus from 32 conditions of the initial dataset, (b) characteristics of ad-hesive joint strength (MPa) and elastic modulus (MPa) properties on 32 conditions. 3.2. Multi-Objective Optimization 3.2.1. Machine Learning Model Selection and Model Training The accuracy of seven machine learning algorithms was evaluated by applying the 32 initial datasets as a training and testing set along with the K-fold cross-validation tech-nique. Three evaluation tools, R2 score, MAE and RMSE, were used to check for accuracy, and the results are reported in Table 3. The Ridge, Lasso and Elastic net algorithms mani-fested a similar level of accuracy. As a result, the fundamental structure of these three algorithms is a linear regression model; thus, they assumed a linear relationship between the input features and the target variables. Even though the regularization techniques of these three models are different, the accuracies are close to each other. Therefore, the ini-tial dataset showed a non-linear relationship. The k-NN algorithm provided a slightly lower accuracy compared to the linear regression models because the k-NN model makes predictions based on the similarity of data points in the input space without assuming a specific functional form for the underlying relationship. Therefore, these four models were not selected for the reason that they are inappropriate algorithms for the dataset and provide low accuracy. The DTR model reported the most obvious lowest accuracy among the others; for this reason, it was discarded. The Random Forest and Gradient boosting models showed high accuracy on both adhesive joint strength and elastic modulus prop-erties; however, the Gradient boosting model could achieve higher accuracy in adhesive Figure 3. Experimental results of adhesive joint strength (MPa) and elastic modulus (MPa);(a) distribution chart of elastic modulus from 32 conditions of the initial dataset, (b) characteris-tics of adhesive joint strength (MPa) and elastic modulus (MPa) properties on 32 conditions.Materials 2024, 17, 2866 9 of 173.2. Multi-Objective Optimization3.2.1. Machine Learning Model Selection and Model TrainingThe accuracy of seven machine learning algorithms was evaluated by applying the32 initial datasets as a training and testing set along with the K-fold cross-validation tech-nique. Three evaluation tools, R2 score, MAE and RMSE, were used to check for accuracy,and the results are reported in Table 3. The Ridge, Lasso and Elastic net algorithmsmanifested a similar level of accuracy. As a result, the fundamental structure of thesethree algorithms is a linear regression model; thus, they assumed a linear relationshipbetween the input features and the target variables. Even though the regularization tech-niques of these three models are different, the accuracies are close to each other. Therefore,the initial dataset showed a non-linear relationship. The k-NN algorithm provided a slightlylower accuracy compared to the linear regression models because the k-NN model makespredictions based on the similarity of data points in the input space without assuminga specific functional form for the underlying relationship. Therefore, these four modelswere not selected for the reason that they are inappropriate algorithms for the dataset andprovide low accuracy. The DTR model reported the most obvious lowest accuracy amongthe others; for this reason, it was discarded. The Random Forest and Gradient boostingmodels showed high accuracy on both adhesive joint strength and elastic modulus proper-ties; however, the Gradient boosting model could achieve higher accuracy in adhesive jointstrength properties. Although the Random Forest and Gradient boosting algorithm areboth ensemble learning techniques, their approaches to building and combining individualmodels are different. Random Forest creates diverse and independent trees in parallel,while Gradient boosting builds trees sequentially, focusing on correcting errors made by theensemble. Considering its better performance, the Gradient boosting model was nominatedfor further prediction.Table 3. Comparison of the accuracy of seven machine learning models represented by R2 score,MAE and RMSE.Ridge Lasso Elastic Net k-NN DTR RandomForestGradientBoostingAdhesivejointstrengthR2 score 0.42 0.43 0.42 0.40 0.18 0.51 0.60MAE 5.7 5.6 5.7 5.4 5.7 4.7 4.3RMSE 7.2 7.1 7.2 7.3 8.5 6.6 6.0ElasticmodulusR2 score 0.57 0.58 0.58 0.54 0.76 0.82 0.82MAE 70.5 69.3 70.8 70.8 43.9 41.9 40.0RMSE 91.7 91.3 91.5 95.3 68.5 60.2 59.53.2.2. Machine Learning Prediction and Proposals for ExperimentsThe Gradient boosting model was trained by using an initial dataset of 32 conditionsas well as applying the K-fold cross-validation technique. Firstly, the averaged predictionon both adhesive joint strength and elastic modulus of a testing dataset from each fold wasreported by comparing it with measured results from the experiment as plotted in Figure S3.The diagonal dash line refers to the same value between the prediction and experimentalresults. Secondly, the averaged prediction results of adhesive joint strength and elasticmodulus for the 256 possible conditions, consisting of four values of the four variableparameters as shown in Table 1, were conducted and reported together with the initialdataset as shown in Figure 4(a1–c1). Lastly, the standard deviation of predicted adhesivejoint strength and elastic modulus at each prediction point was averaged to be reported asa representative of deviation value, and it is shown in the color scale in Figure 4(a2–c2).Materials 2024, 17, 2866 10 of 17Materials 2024, 17, x FOR PEER REVIEW 10 of 18   joint strength properties. Although the Random Forest and Gradient boosting algorithm are both ensemble learning techniques, their approaches to building and combining indi-vidual models are different. Random Forest creates diverse and independent trees in par-allel, while Gradient boosting builds trees sequentially, focusing on correcting errors made by the ensemble. Considering its better performance, the Gradient boosting model was nominated for further prediction. Table 3. Comparison of the accuracy of seven machine learning models represented by R2 score, MAE and RMSE.  Ridge Lasso Elastic Net k-NN DTR Random Forest Gradient Boosting Adhesive joint strength R2 score 0.42 0.43 0.42 0.40 0.18 0.51 0.60 MAE 5.7 5.6 5.7 5.4 5.7 4.7 4.3 RMSE 7.2 7.1 7.2 7.3 8.5 6.6 6.0 Elastic  modulus R2 score 0.57 0.58 0.58 0.54 0.76 0.82 0.82 MAE 70.5 69.3 70.8 70.8 43.9 41.9 40.0 RMSE 91.7 91.3 91.5 95.3 68.5 60.2 59.5 3.2.2. Machine Learning Prediction and Proposals for Experiments The Gradient boosting model was trained by using an initial dataset of 32 conditions as well as applying the K-fold cross-validation technique. Firstly, the averaged prediction on both adhesive joint strength and elastic modulus of a testing dataset from each fold was reported by comparing it with measured results from the experiment as plotted in Figure S3. The diagonal dash line refers to the same value between the prediction and experimental results. Secondly, the averaged prediction results of adhesive joint strength and elastic modulus for the 256 possible conditions, consisting of four values of the four variable parameters as shown in Table 1, were conducted and reported together with the initial dataset as shown in Figure 4(a1–c1). Lastly, the standard deviation of predicted adhesive joint strength and elastic modulus at each prediction point was averaged to be reported as a representative of deviation value, and it is shown in the color scale in Figure 4(a2–c2).   (a1) (a2) Materials 2024, 17, x FOR PEER REVIEW 11 of 18     (b1) (b2)   (c1) (c2) Figure 4. Prediction of the adhesive joint strength (MPa) and elastic modulus (MPa) on 256 condi-tions compared with initial datasets after (a1) active learning cycle 1; ni = 32, (b1) cycle 2; n = 35, and (c1) cycle 3; n = 38, as well as the averaged standard deviation of each predicted result (a2) cycle 1, (b2) cycle 2 and (c2) cycle 3. In the first cycle (Figure S3a), the prediction of the testing dataset showed a high de-viation at the high adhesive joint strength of above 10 MPa; on the other hand, a high deviation could be observed at the middle range of elastic modulus from 100 to 350 MPa. In addition, the prediction of a total of 256 possible conditions on both adhesive joint strength and elastic modulus was successfully carried out as shown in Figure 4(a1). The prediction results showed good distribution along with the initial dataset (ni = 32); how-ever, prediction could not be observed in the area of high adhesive joint strength with low elastic modulus (top-left zone). The missing data indicate the possibility of a trade-off characteristic between these two properties. In the Figure 4(a2), the standard deviation of prediction results from the first active learning cycle showed the highest deviation at 15.5 MPa. Moreover, regarding the percentage of high-deviated prediction results, higher than half of the highest deviation, was observed at 9%. After that, the high-deviation conditions from three regions—low–low, low–high and high–high adhesive joint strength and elastic modulus, respectively—were selected for further experiments as listed in Table 4 for con-dition numbers 33, 34 and 35. The experimental results from the proposed conditions Figure 4. Prediction of the adhesive joint strength (MPa) and elastic modulus (MPa) on 256 conditionscompared with initial datasets after (a1) active learning cycle 1; ni = 32, (b1) cycle 2; n = 35, and(c1) cycle 3; n = 38, as well as the averaged standard deviation of each predicted result (a2) cycle 1,(b2) cycle 2 and (c2) cycle 3.Materials 2024, 17, 2866 11 of 17In the first cycle (Figure S3a), the prediction of the testing dataset showed a highdeviation at the high adhesive joint strength of above 10 MPa; on the other hand, ahigh deviation could be observed at the middle range of elastic modulus from 100 to350 MPa. In addition, the prediction of a total of 256 possible conditions on both adhesivejoint strength and elastic modulus was successfully carried out as shown in Figure 4(a1).The prediction results showed good distribution along with the initial dataset (ni = 32);however, prediction could not be observed in the area of high adhesive joint strengthwith low elastic modulus (top-left zone). The missing data indicate the possibility of atrade-off characteristic between these two properties. In the Figure 4(a2), the standarddeviation of prediction results from the first active learning cycle showed the highestdeviation at 15.5 MPa. Moreover, regarding the percentage of high-deviated predictionresults, higher than half of the highest deviation, was observed at 9%. After that, thehigh-deviation conditions from three regions—low–low, low–high and high–high adhesivejoint strength and elastic modulus, respectively—were selected for further experiments aslisted in Table 4 for condition numbers 33, 34 and 35. The experimental results from theproposed conditions indicated almost similar results to the predictions except for the elasticmodulus of condition number 35. This is because high deviation leads to low accuracy inthe prediction.Table 4. Prediction of adhesive joint strength (MPa) and elastic modulus (MPa) of the proposedconditions and the experimental results.FromCycle No.Variable Parameters PredictedAdhesive JointStrength (MPa)Predicted ElasticModulus (MPa)MeasuredAdhesive JointStrength (MPa)Measured ElasticModulus (MPa)MwE(g/mol)MwC(g/mol) r TC(◦C)133 370 230 1.00 130 28.1 ± 2.5 335.8 ± 8.5 25.2 ± 2.3 361.5 ± 5.234 1650 400 1.25 90 6.8 ± 0.7 314.4 ± 8.0 6.1 ± 1.8 322.7 ± 8.835 2900 2000 1.50 170 6.4 ± 0.8 95.3 ± 24.4 5.3 ± 2.7 312.3 ± 14.9236 370 400 0.75 130 18.4 ± 1.3 253.7 ± 18.0 12.1 ± 1.8 162.8 ± 11.837 2900 2000 1.50 210 7.5 ± 0.8 291.9 ± 36.0 10.3 ± 2.9 279.7 ± 20.138 3800 2000 1.50 90 6.4 ± 1.2 108.3 ± 25.0 6.5 ± 2.1 303.8 ± 18.3The measured results of both adhesive joint strength and elastic modulus from theexperiment according to conditions 33, 34 and 35 were added to the initial dataset for thesecond cycle of active learning. These new initial datasets (n = 32 + 3 = 35) were utilizedto train the machine learning model of Gradient boosting once again. Accompanying theK-fold cross-validation technique, the predictions of a testing set were compared to themeasured testing set itself, and they were plotted as shown in Figure S3b. The predictedadhesive joint strength still performed with a high deviation at the high predicted valueswhich were the same as those of the first cycle. In contrast, the model accuracy couldbe improved by observing a better R2 score, MAE and RMSE after applying this activelearning approach. On the other hand, the elastic modulus properties not only showedhigh-deviation prediction results in the middle range but also obtained lower accuracyon the R2 score, MAE and RMSE. In Figure 4(b1), the prediction of 256 conditions wasreported together with a new initial dataset (n = 35) in this second cycle. The absence ofthe predicted results could still be discovered in the area of low elastic modulus with highadhesive joint strength. However, the predictions were very well distributed along withthe initial dataset. In the deviation point of view, the averaged standard deviation betweenadhesive joint strength and elastic modulus at 18.1 MPa was observed to have the highestvalues as shown in Figure 4(b2). Moreover, the predicted results at the area of high elasticmodulus performed with an outstandingly low deviation according to the prediction of thetesting dataset from Figure S3b. Nevertheless, comparing the percentage of high-deviationprediction results between the first cycle and the second cycle, it could be found that therewas a 2% improvement in decreasing the high-deviation prediction results as shown inMaterials 2024, 17, 2866 12 of 17Figure 5. After that, a new set of three conditions with high deviation, as listed in Table 4for the conditions 36, 37 and 38, were chosen again in order to conduct an experiment for afurther active learning cycle. Additionally, the measured results from the experiment arereported in Table 4. The experimental results indicated an improvement in the deviationcompared to the predictions. Subsequently, these three results were appended to the initialdataset for the third active learning cycle.Materials 2024, 17, x FOR PEER REVIEW 13 of 18    Figure 5. Distribution of averaged deviation from active learning cycle 1, cycle 2 and cycle 3. The total 38 conditions (n = 35 + 3 = 38) were utilized as an initial dataset in this third active learning loop. The process was repeated by training the Gradient boosting model along with applying the K-fold cross-validation technique, then predicting the testing set and evaluating the model accuracy. The third cycle prediction of the testing set is reported in Figure S3c together with the model accuracies of both adhesive joint strength and elastic modulus. The model accuracy of the adhesive joint strength slightly decreased from the second cycle; however, it could exhibit better performance compared to the first cycle. On the other hand, the accuracy of the elastic modulus kept decreasing from the first cycle. In spite of that, it is obvious that the prediction of a testing set deviated from the diagonal dash line much less compared to that of the first and second cycles. Then, the prediction of 256 conditions was carried out and reported together with an initial dataset of 38 con-ditions as shown in Figure 4(c1). The same tendency as seen in the previous cycle was observed: the predictions could not be found in the area of high adhesive joint strength with low elastic modulus. Therefore, this could imply a trade-off between these two prop-erties after three cycles of the active learning loop. Additionally, the Pareto frontier line was drawn to represent a trade-off boundary connecting all of the results at the top-left edge. In Figure 4(c2), the maximum deviation of these two properties is 21.0 MPa, and the extremely high-deviation results show less than 10 conditions. Moreover, the deviation of the overall predicted results improved because of the clearly seen shift in color scale from yellow in the first cycle to light blue in this cycle. Furthermore, a significant enhancement in prediction accuracy was observed, with a 50% reduction in high-deviation outcomes from the first to the third cycle, as illustrated in Figure 5. The active learning cycle termi-nated once the average standard deviation of predictions closely matched that of experi-mental results for both adhesive joint strength and elastic modulus. Thus, the errors in the predictions were acceptable according to the errors from the experiments. In conclusion, the active learning approach was able to balance machine learning model accuracy and the deviation of each single predicted result. Consequently, the Pareto frontier line could be obtained from the predictions of 256 conditions (Figure 4(c1)), and it was highly reliable after achieving three cycles of the active learning loop.  The influence of each variable parameter on the prediction of both properties is illus-trated in Figure S4. It was observed that the molecular weight of the epoxy resin (MwE) exhibited minimal impact on both properties as indicated by its ability to perform 0%10%20%30%40%50%60%70%80%0–6 6–12 12–18 18–24Frequency (%)Averaged deviationCycle 1 Cycle 2 Cycle 3Figure 5. Distribution of averaged deviation from active learning cycle 1, cycle 2 and cycle 3.The total 38 conditions (n = 35 + 3 = 38) were utilized as an initial dataset in this thirdactive learning loop. The process was repeated by training the Gradient boosting modelalong with applying the K-fold cross-validation technique, then predicting the testing setand evaluating the model accuracy. The third cycle prediction of the testing set is reportedin Figure S3c together with the model accuracies of both adhesive joint strength and elasticmodulus. The model accuracy of the adhesive joint strength slightly decreased from thesecond cycle; however, it could exhibit better performance compared to the first cycle. Onthe other hand, the accuracy of the elastic modulus kept decreasing from the first cycle. Inspite of that, it is obvious that the prediction of a testing set deviated from the diagonaldash line much less compared to that of the first and second cycles. Then, the prediction of256 conditions was carried out and reported together with an initial dataset of 38 conditionsas shown in Figure 4(c1). The same tendency as seen in the previous cycle was observed:the predictions could not be found in the area of high adhesive joint strength with lowelastic modulus. Therefore, this could imply a trade-off between these two propertiesafter three cycles of the active learning loop. Additionally, the Pareto frontier line wasdrawn to represent a trade-off boundary connecting all of the results at the top-left edge. InFigure 4(c2), the maximum deviation of these two properties is 21.0 MPa, and the extremelyhigh-deviation results show less than 10 conditions. Moreover, the deviation of the overallpredicted results improved because of the clearly seen shift in color scale from yellow in thefirst cycle to light blue in this cycle. Furthermore, a significant enhancement in predictionaccuracy was observed, with a 50% reduction in high-deviation outcomes from the firstto the third cycle, as illustrated in Figure 5. The active learning cycle terminated once theaverage standard deviation of predictions closely matched that of experimental results forboth adhesive joint strength and elastic modulus. Thus, the errors in the predictions wereacceptable according to the errors from the experiments. In conclusion, the active learningapproach was able to balance machine learning model accuracy and the deviation of eachsingle predicted result. Consequently, the Pareto frontier line could be obtained fromMaterials 2024, 17, 2866 13 of 17the predictions of 256 conditions (Figure 4(c1)), and it was highly reliable after achievingthree cycles of the active learning loop.The influence of each variable parameter on the prediction of both properties is illus-trated in Figure S4. It was observed that the molecular weight of the epoxy resin (MwE)exhibited minimal impact on both properties as indicated by its ability to perform con-sistently across a wide range of predicted adhesive joint strength and elastic modulus(Figure S4a). However, the epoxy resin with molecular weight of 370 g/mol offers signifi-cant advantages in processability due to its liquid phase state. In contrast, the molecularweight of the curing agent (MwC) demonstrated a significant impact on the predictedproperties. Lower-viscosity curing agents with MwC values of 230 and 400 g/mol re-sulted in a prediction of high adhesive joint strength and elastic modulus. Conversely,higher-viscosity curing agents with MwC values of 2000 and 4000 g/mol only achieveda maximum adhesive joint strength of 11.8 MPa. Regarding elastic modulus, predictionswith a modulus below 300 MPa were challenging to obtain using curing agents with anMwC of 230 g/mol, suggesting the necessity for higher molecular weight curing agents forlow modulus adhesives. Moreover, the control of predicted adhesive joint strength andelastic modulus properties is more readily achieved through the molecular weight of thecuring agent (Figure S4b). The impact of the amine-to-epoxide ratio is shown in Figure S4c,indicating its uniform distribution across the range of predicted properties. Higher ratiosnotably enhanced adhesive joint strength, particularly surpassing 16 MPa, while elasticmodulus values between 100 and 200 MPa were predominantly attained with a ratio of0.75. Additionally, it was observed that curing temperatures at 90 ◦C resulted in predictedadhesive joint strengths below 12 MPa. Conversely, curing temperatures exceeding 90 ◦Cfacilitated a more favorable distribution for both predicted adhesive joint strength andelastic modulus properties, as illustrated in Figure S4d.3.2.3. Bayesian OptimizationAccording to the trade-off behavior proposed in the previous section (Figure 4(c1)), ad-hesive epoxy materials exhibiting high adhesive joint strength but low elastic modulus haveyet to be developed through the machine learning approach. However, such a formulation,which shows high adhesive joint strength and low elastic modulus, could offer considerableadvantages, particularly in applications requiring efficient force absorption in adhesivematerials. To address this, a new initial dataset comprising 38 conditions was utilized forfurther investigation, employing Bayesian optimization to identify conditions conducive toachieving an adhesive epoxy with the desired properties. In this stage, PHYSBO which is aPython library for Bayesian optimization studies focusing on the physics, chemistry and ma-terials science fields was performed to optimize this multi-objective problem. Four variableparameters with discrete and continuous types, as summarized in Table 2, were studied.The proposed conditions from Bayesian optimization were separately reported in fourfigures, as shown in Figures S5–S8. The predicted adhesive joint strength and elasticmodulus from Bayesian optimization were plotted individually by varying the continuousparameters of amine-to-epoxide ratio and curing temperature; however, the predictions byvarying discrete parameters of MwE and MwC were plotted one by one in separate charts.The predicted adhesive joint strength and elastic modulus were mainly changed by varyingMwE and MwC. Particularly, changing the MwC along its four values could obtain a varietyof predicted adhesive joint strength and elastic modulus results. This was confirmed bythe influence of MwC on prediction of both properties after the final active learning cycle(Figure S4). On the other hand, the influence of amine-to-epoxide ratio and curing tempera-ture exhibited a notable influence on the predicted results, particularly noticeable at lowervalues of MwE and MwC. Afterward, the conditions proposed by Bayesian optimization,where the predicted results exceeded the trade-off boundary, were selected to overcomethe Pareto frontier line. Considering the discussion from the previous section regardingthe influence of each variable parameter, three conditions with predicted results beyondthe Pareto frontier line were selected as listed in Table 5. Additionally, the experimentalMaterials 2024, 17, 2866 14 of 17results corresponding to these conditions were reported in Table 5 and plotted alongsidethe Pareto frontier line in Figure 6. Notably, conditions 39 and 40 exhibited adhesive jointstrengths of 10.2 MPa and 25.2 MPa, respectively, along with elastic modulus of 74.2 MPaand 182.5 MPa, positioning them further from the trade-off limit. Furthermore, a speci-men with desirable properties as of high adhesive joint strength together with low elasticmodulus was successfully fabricated as polymeric specimen number 40. A comparisonof the appearance between specimen number 40 and the specimen with high adhesivejoint strength and elastic modulus (number 26) was conducted to confirm the difference,as reported in Figure S9. Significantly, specimen number 40 displayed superior ductilecharacteristics compared to specimen number 26, despite both exhibiting the same levelof adhesive joint strength values. Additionally, elasticity behavior was demonstrated, asillustrated in Figure S10. Specimen number 26, with a higher elastic modulus, retained itsshape when subjected to a 100 g load, whereas specimen number 40, with a lower elasticmodulus, exhibited deformation under the same load. Therefore, not only was the trade-offbetween adhesive joint strength and elastic modulus optimized, but also, a specimen withthe desired properties was successfully fabricated by adhering to the suggested conditionsderived from the Bayesian optimization approach.Table 5. Proposed conditions from the Bayesian optimization step, and the experimental resultsaccording to the conditions.No.Variable Parameters PredictedAdhesive JointStrength (MPa)Predicted ElasticModulus (MPa)MeasuredAdhesive JointStrength (MPa)Measured ElasticModulus (MPa)MwE(g/mol)MwC(g/mol) r TC(◦C)39 370 2000 1.46 129 11.5 21.5 10.2 ± 0.6 74.2 ± 3.340 370 400 0.75 168 17.4 177.5 25.2 ± 1.3 182.5 ± 7.941 370 230 1.13 160 32.3 346.8 30.3 ± 0.9 319.5 ± 17.4Materials 2024, 17, x FOR PEER REVIEW 15 of 18   adhesive joint strength together with low elastic modulus was successfully fabricated as polymeric specimen number 40. A comparison of the appearance between specimen num-ber 40 and the specimen with high adhesive joint strength and elastic modulus (number 26) was conducted to confirm the difference, as reported in Figure S9. Significantly, spec-imen number 40 displayed superior ductile characteristics compared to specimen number 26, despite both exhibiting the same level of adhesive joint strength values. Additionally, elasticity behavior was demonstrated, as illustrated in Figure S10. Specimen number 26, with a higher elastic modulus, retained its shape when subjected to a 100 g load, whereas specimen number 40, with a lower elastic modulus, exhibited deformation under the same load. Therefore, not only was the trade-off between adhesive joint strength and elastic modulus optimized, but also, a specimen with the desired properties was successfully fabricated by adhering to the suggested conditions derived from the Bayesian optimiza-tion approach.  Figure 6. The experimental results and proposed conditions for adhesive joint strength (MPa) and elastic modulus (MPa) from multi-objective optimization using PHYSBO along with the predictions from active learning and a Pareto frontier line. Table 5. Proposed conditions from the Bayesian optimization step, and the experimental results according to the conditions. No. Variable Parameters Predicted Adhe-sive Joint Strength (MPa) Predicted Elastic Modulus (MPa) Measured Adhe-sive Joint Strength (MPa) Measured Elas-tic Modulus (MPa) MwE (g/mol) MwC (g/mol) r TC (°C) 39 370 2000 1.46 129 11.5 21.5 10.2 ± 0.6 74.2 ± 3.3 40 370 400 0.75 168 17.4 177.5 25.2 ± 1.3 182.5 ± 7.9 41 370 230 1.13 160 32.3 346.8 30.3 ± 0.9 319.5 ± 17.4 4. Conclusions Multi-objective optimization was employed to analyze the adhesive joint strength and elastic modulus properties of commercially available epoxy resin (DGEBA) with an amine-terminated curing agent (Jeffaime™). Four variable parameters, consisting of mo-lecular weight of DGEBA (MwE), molecular weight of Jeffamine™ (MwC), amine-to-epoxide ratio (r) and curing temperature (TC), were investigated. In this study, the elastic modulus Figure 6. The experimental results and proposed conditions for adhesive joint strength (MPa) andelastic modulus (MPa) from multi-objective optimization using PHYSBO along with the predictionsfrom active learning and a Pareto frontier line.Materials 2024, 17, 2866 15 of 174. ConclusionsMulti-objective optimization was employed to analyze the adhesive joint strength andelastic modulus properties of commercially available epoxy resin (DGEBA) with an amine-terminated curing agent (Jeffaime™). Four variable parameters, consisting of molecularweight of DGEBA (MwE), molecular weight of Jeffamine™ (MwC), amine-to-epoxide ratio(r) and curing temperature (TC), were investigated. In this study, the elastic modulus wasmeasured using a specially designed metal mold apparatus. The polymeric specimenswere fabricated with smooth surfaces and homogeneity. The elastic modulus values werederived from hardness measurements of each specimen. Subsequently, these values, alongwith the adhesive joint strength values from a previous study, constituted the initial datasetof 32 conditions to train the machine learning model. Among the seven machine learningalgorithms evaluated, the Gradient Boosting model exhibited the highest accuracy andwas selected for further analysis. The initial dataset was then utilized in an active learningapproach to train the Gradient Boosting model and make predictions, resulting in a 50%reduction in deviation of predicted results and improved balance in predictive accuracyacross adhesive joint strength and elastic modulus properties after the third cycle of theactive learning approach. Consequently, the Pareto frontier line showing the trade-offboundary between these two properties was able to be reliably presented. The missingprediction area at high adhesive joint strength with low elastic modulus was observed as thetrade-off area. The influence of each variable parameter on both adhesive joint strength andelastic modulus properties was examined. The results indicated that an epoxy resin withMwE of 370 g/mol offered optimal processability because of its liquid phase state, the MwCof the Jeffamine™ curing agent played a crucial role in controlling properties to achievespecimens with high adhesive joint strength and low elastic modulus, an amine-to-epoxideratio (r) of 0.75 was suitable for fabricating adhesive epoxy with lower elastic modulus, anda curing temperature (TC) above 90 ◦C was necessary to maintain adhesive ability.Bayesian optimization was employed to address the trade-off challenges. Utilizinga dataset of n = 38 (initial 32 datasets plus an additional 6), PHYSBO was employed tooptimize the adhesive joint strength and elastic modulus properties of the adhesive epoxysystem. Molecular weights of DGEBA (MwE) and Jeffamine™ (MwC) were treated as dis-crete variables, while amine-to-epoxide ratio (r) and curing temperature (TC) were exploredas continuous variables. The PHYSBO approach effectively predicted and suggested prop-erties for this adhesive epoxy system across a vast array of conditions, up to 147,136 in total.Upon experimentation, the optimized conditions were validated, successfully transcendingthe trade-off boundary (Pareto frontier line) with adhesive joint strength reaching 25.2 MPaand elastic modulus at 182.5 MPa, respectively. The fabricated specimen exhibited superiorelastic behavior. This multi-objective optimization strategy provides valuable insightsinto the curing conditions for the studied adhesive epoxy system, elucidating the impactof each variable parameter on mechanical properties. Moreover, Bayesian optimizationdemonstrates its capability to efficiently suggest conditions for desired properties, acceler-ating the overall process, reducing costs, and time consumption, while also enabling thebreakthrough of trade-off constraints in the adhesive epoxy system.Supplementary Materials: The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/ma17122866/s1, Figure S1: An illustration of (a) a metal mold accordingto French standard NFT76-142, (b) dimension of the specimens and the measuring points accordingto ASTM-D2240; Figure S2: An example of the specimen’s appearance with conditions 26 and 31;Figure S3: The averaged prediction of adhesive joint strength and elastic modulus from 32-foldcross-validation compared to its measured results from experiments as well as the accuracies ofeach active learning cycle; (a) 1st cycle, (b) 2nd cycle and (c) 3rd cycle; Figure S4: The influenceof each variable parameter on the prediction of 256 conditions from the 3rd active learning cycle:(a) molecular weight of epoxy resin, (b) molecular weight of curing agent, (c) amine to epoxide ratioand (d) curing temperature; Figure S5: The proposed conditions from Bayesian optimization for adhesivejoint strength (MPa) and elastic modulus (MPa) by varying three variable parameters—molecular weightof Jeffamine™ (g/mol), amine-to-epoxide ratio and curing temperature (◦C)—with the molecular weighthttps://www.mdpi.com/article/10.3390/ma17122866/s1https://www.mdpi.com/article/10.3390/ma17122866/s1Materials 2024, 17, 2866 16 of 17of DGEBA at 370 g/mol; Figure S6: The proposed conditions from Bayesian optimization for adhesive jointstrength (MPa) and elastic modulus (MPa) by varying three variable parameters—molecular weight ofJeffamine™ (g/mol), amine-to-epoxide ratio and curing temperature (◦C)—with the molecular weight ofDGEBA at 1650 g/mol; Figure S7: The proposed conditions from Bayesian optimization for adhesivejoint strength (MPa) and elastic modulus (MPa) by varying three variable parameters—molecularweight of Jeffamine™ (g/mol), amine-to-epoxide ratio and curing temperature (◦C)—with themolecular weight of DGEBA at 2900 g/mol; Figure S8: The proposed conditions from Bayesianoptimization for adhesive joint strength (MPa) and elastic modulus (MPa) by varying three variableparameters—molecular weight of Jeffamine™ (g/mol), amine-to-epoxide ratio and curing temper-ature (◦C)—with the molecular weight of DGEBA at 3800 g/mol; Figure S9: The appearance ofspecimens after tearing by tensile force of the specimen with condition number 26 (high elastic modu-lus) shows less ductile behavior compared to the specimen with condition number 40 (less elasticmodulus); Figure S10: Demonstration of the elasticity behavior of the specimens: (a) high elastic mod-ulus specimen no. 26 before adding load, (b) after adding 100 g load, (c) low elastic modulus specimenno. 40 before adding load and (d) after adding 100 g load; Figure S11: A picture of actual metal moldconsisting of a metal lid, Teflon tape, metal box, silicon rubber frame and adhesive epoxy specimen;Table S1: Experimental results of adhesive joint strength (MPa) and elastic modulus (MPa) of eachcondition consist of four variable parameters: molecular weight of DGEBA MwE (g/mol), molecularweight of Jeffaime™ curing agent MwC (g/mol), amine-to-epoxide ratio; r, and curing temperatureTC (◦C). Initial dataset size ni = 32 samples. Refs [14,17] are cited in the supplementary materials.Author Contributions: Conceptualization, P.K. and M.N.; data curation, P.K. and C.K.; formalanalysis, P.K.; investigation, P.K. and C.K.; resources, M.N. and C.S.; software, P.K.; supervision,M.N. and C.S.; writing—original draft preparation, P.K. and C.K.; writing—review and editing, P.K.and M.N.; visualization, P.K.; funding acquisition, M.N. All authors have read and agreed to thepublished version of the manuscript.Funding: This work was supported by the Core Research for Evolutional Science and Technology(CREST) program ‘Revolution material development by fusion of strong experiments with the-ory/data science’ of the Japan Science and Technology Agency (JST), Japan, under Grant JPMJCR19J3,KAKENHI Grant-in-Aid for Scientific Research (B): 23H02031 and MEXT Program: Data Creationand Utilization-Type Material Research and Development Project Grant Number JPMXP1122714694.Institutional Review Board Statement: Not applicable.Informed Consent Statement: Not applicable.Data Availability Statement: Data are contained within the article and Supplementary Materials.Acknowledgments: We thank Sirawit Pruksawan, a former member from our group, for the initialdataset of adhesive joint strength.Conflicts of Interest: The authors declare no conflicts of interest.References1. Wei, Y.; Jin, X.; Luo, Q.; Li, Q.; Sun, G. Adhesively Bonded Joints—A Review on Design, Manufacturing, Experiments, Modelingand Challenges. Compos. B Eng. 2024, 276, 111225. [CrossRef]2. Cole, G.S.; Sherman, A.M. Light Weight Materials for Automotive Applications. Mater. Charact. 1995, 35, 3–9. [CrossRef]3. Campbell, F.C. Manufacturing Technology for Aerospace Structural Materials; Elsevier: Amsterdam, The Netherlands, 2006;ISBN 9781856174954.4. Van Straalen, I.J.J.; Van Tooren, M.J.L. Building and Construction—Steel and Aluminium. In Adhesive Bonding; Elsevier:Amsterdam, The Netherlands, 2005; pp. 305–327.5. Xu, W.; Wei, Y. Strength and Interface Failure Mechanism of Adhesive Joints. Int. J. Adhes. Adhes. 2012, 34, 80–92. [CrossRef]6. Jones David, R.H.; Ashby Michael, F. Engineering Materials 1, 5th ed.; Elsevier: Amsterdam, The Netherlands, 2019;ISBN 9780081020517.7. Gordon, T.L.; Fakley, M.E. The Influence of Elastic Modulus on Adhesion to Thermoplastics and Thermoset Materials. Int. J.Adhes. Adhes. 2003, 23, 95–100. [CrossRef]8. Loureiro, A.L.; da Silva, L.F.M.; Sato, C.; Figueiredo, M.A.V. Comparison of the Mechanical Behaviour Between Stiff and FlexibleAdhesive Joints for the Automotive Industry. J. Adhes. 2010, 86, 765–787. [CrossRef]9. Kanani, A.Y.; Liu, Y.; Hughes, D.J.; Ye, J.; Hou, X. Fracture Mechanisms of Hybrid Adhesive Bonded Joints: Effects of the Stiffnessof Constituents. Int. J. Adhes. Adhes. 2020, 102, 102649. [CrossRef]https://doi.org/10.1016/j.compositesb.2024.111225https://doi.org/10.1016/1044-5803(95)00063-1https://doi.org/10.1016/j.ijadhadh.2011.12.004https://doi.org/10.1016/S0143-7496(02)00064-7https://doi.org/10.1080/00218464.2010.482440https://doi.org/10.1016/j.ijadhadh.2020.102649Materials 2024, 17, 2866 17 of 1710. Rocha, J.; Sena-Cruz, J.; Pereira, E. Influence of Adhesive Stiffness on the Post-Cracking Behaviour of CFRP-Reinforced StructuralGlass Beams. Compos. B Eng. 2022, 247, 110293. [CrossRef]11. Taniguchi, S.; Uemura, K.; Tamaki, S.; Nomura, K.; Koyanagi, K.; Kuchii, S. Multi-Objective Optimization of the Epoxy MatrixSystem Using Machine Learning. Results Mater. 2023, 17, 100376. [CrossRef]12. Mogilicharla, A.; Chugh, T.; Majumdar, S.; Mitra, K. Multi-Objective Optimization of Bulk Vinyl Acetate Polymerization withBranching. Mater. Manuf. Process. 2014, 29, 210–217. [CrossRef]13. Antil, P. Modelling and Multi-Objective Optimization during ECDM of Silicon Carbide Reinforced Epoxy Composites. Silicon2020, 12, 275–288. [CrossRef]14. Pruksawan, S.; Lambard, G.; Samitsu, S.; Sodeyama, K.; Naito, M. Prediction and Optimization of Epoxy Adhesive Strength froma Small Dataset through Active Learning. Sci. Technol. Adv. Mater. 2019, 20, 1010–1021. [CrossRef] [PubMed]15. Qi, H.J.; Joyce, K.; Boyce, M.C. Durometer Hardness and the Stress-Strain Behavior of Elastomeric Materials. Rubber Chem. Technol.2003, 76, 419–435. [CrossRef]16. NFT 76-142; Méthode de Préparation de Plaques D’adhésifs Structuraux Pour la Réalisation D’éprouvettes D’essai de Car-actérisation. Afnor EDITIONS: La Plaine Saint-Denis, France, 1988.17. ASTM-D2240; Standard Test Method for Rubber Property-Durometer Hardness. American Society for Testing and Materials(ASTM): West Conshohocken, PN, USA, 2021.18. da Silva, L.F.M.; Dillard, D.A.; Blackman, B.; Adams, R.D. (Eds.) Testing Adhesive Joints; Wiley: Hoboken, NJ, USA, 2012;ISBN 9783527329045.19. Cooper, B.E. Statistics for Experimentalists; Elsevier: Amsterdam, The Netherlands, 1969; ISBN 9780080126005.20. Keppel, G. Design and Analysis: A Researcher’s Handbook, 1st ed.; Prentice-Hall, Inc: Englewood Cliffs, NJ, USA, 1991;ISBN 0-13-200775-4. (In Hardcover)21. Refaeilzadeh, P.; Tang, L.; Liu, H. Cross-Validation. In Encyclopedia of Database Systems; Springer: Boston, MA, USA, 2009;pp. 532–538.22. Hoerl, A.E.; Kennard, R.W. Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics 2000, 42, 80.[CrossRef]23. Tibshirani, R. Regression Shrinkage and Selection Via the Lasso. J. R. Stat. Soc. Ser. B Methodol. 1996, 58, 267–288. [CrossRef]24. Zou, H.; Hastie, T. Regularization and Variable Selection Via the Elastic Net. J. R. Stat. Soc. Ser. B Stat. Methodol. 2005, 67, 301–320.[CrossRef]25. Hastie, T.; Tibshirani, R.; Friedman, J. The Elements of Statistical Learning; Springer Series in Statistics; Springer: New York, NY,USA, 2009; ISBN 978-0-387-84857-0.26. Breiman, L.; Friedman, J.H.; Olshen, R.A.; Stone, C.J. Classification And Regression Trees; Routledge: New York, NY, USA, 2017;ISBN 9781315139470.27. Breiman, L. Random Forests; Springer: Berlin/Heidelberg, Germany, 2001; Volume 45.28. Friedman, J.H. Greedy Function Approximation: A Gradient Boosting Machine. Ann. Stat. 2001, 29, 1189–1232. [CrossRef]29. Motoyama, Y.; Tamura, R.; Yoshimi, K.; Terayama, K.; Ueno, T.; Tsuda, K. Bayesian Optimization Package: PHYSBO. Comput.Phys. Commun. 2022, 278, 108405. [CrossRef]30. Verleysen, M.; Université catholique de Louvain; Katholieke Universiteit Leuven; European Symposium on Artificial NeuralNetworks, Computational Intelligence and Machine Learning; ESANN. Proceedings/23rd European Symposium on Artificial NeuralNetworks, Computational Intelligence and Machine Learning, ESANN 2015, Bruges, Belgium, 22–24 April 2015; Ciaco: Ottignies-Louvain-la-Neuve, Belgium, 2015; ISBN 978-287587014-8.Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individualauthor(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury topeople or property resulting from any ideas, methods, instructions or products referred to in the content.https://doi.org/10.1016/j.compositesb.2022.110293https://doi.org/10.1016/j.rinma.2023.100376https://doi.org/10.1080/10426914.2013.872271https://doi.org/10.1007/s12633-019-00122-8https://doi.org/10.1080/14686996.2019.1673670https://www.ncbi.nlm.nih.gov/pubmed/31692965https://doi.org/10.5254/1.3547752https://doi.org/10.2307/1271436https://doi.org/10.1111/j.2517-6161.1996.tb02080.xhttps://doi.org/10.1111/j.1467-9868.2005.00503.xhttps://doi.org/10.1214/aos/1013203451https://doi.org/10.1016/j.cpc.2022.108405 Introduction  Materials and Methods  Experiments  Materials  Preparation of Specimens  Modulus Testing Technique  Multi-Objective Optimization Approaches  Dataset Preparation  Cross-Validation Technique  Modeling and Prediction  Bayesian Optimization  Results  Experiments  Multi-Objective Optimization  Machine Learning Model Selection and Model Training  Machine Learning Prediction and Proposals for Experiments  Bayesian Optimization  Conclusions  References