Learning Center

Creating a Multiphysics-Driven Deep Neural Network Surrogate Model

---

To illustrate the process of creating a surrogate model for a multiphysics simulation and parametric CAD model, we will create a surrogate model for the displacement of a thermal actuator. In this course, we will ultimately develop a surrogate model that captures virtually all aspects of the thermal actuator. For now, we will begin with a basic surrogate model that requires minimal data. Despite its simplicity, this initial physics-based example will help you better understand the process of developing this type of surrogate model in COMSOL Multiphysics^®. Additionally, this example shows a technique for generating a response surface using a deep neural network.

Thermal Actuator Model Overview

Start by loading the thermal actuator tutorial model from the Application Libraries in the software. Multiple versions of the model are available under the COMSOL Multiphysics > Multiphysics branch. Select and open the simplified model version.

A screenshot of the COMSOL Multiphysics UI with the Application Libraries window open and the thermal actuator simplified tutorial model selected, which displays the tutorial model name and an image of the model along with several details and properties regarding the mode.

The thermal actuator tutorial model in the Application Libraries.

Note that this model is also featured extensively, in different variations, in the Learning Center course "Defining Multiphysics Models".

The thermal actuator design includes two hot arms made of polysilicon, as shown in the figure below.

The thermal microactuator geometry with the parts labeled. The device is made of polycrystalline silicon.

The actuator is activated through thermal expansion. Joule heating (resistive heating) is used to achieve the temperature increase needed to deform the two hot arms and consequently displace the actuator. The greater expansion of the hot arms, compared to the cold arm, causes the actuator to bend. The material properties of polysilicon vary with temperature, leading to fully coupled physics. As electric current flows through the hot arms, it raises the actuator's temperature, triggering thermal expansion. At the same time, the electrical conductivity changes with temperature and the current flowing through the hot arms changes accordingly, in turn affecting the heating and the thermal expansion. Actuator operation thus involves three coupled physics phenomena:

1. Electric current conduction
2. Heat conduction with heat generation
3. Structural stresses and strains due to thermal expansion

The operation of the device is also detailed extensively in the course on defining multiphysics models.

The Parameters node contains the parameters of the model, including geometric dimensions such as length, width, and the applied voltage, as shown in the figure below. These are the parameters that we can use as a basis for creating surrogate models.

The Model Builder with the Parameters 1 node selected and the corresponding Settings window, which contains a tabular list of parameters used in the model.

The parameters driving the thermal actuator model.

Defining the Surrogate Model

Let's now assume that we are interested in creating a surrogate model for predicting the variation in the maximum displacement of the tip of the actuator. Furthermore, assume that we vary just two of the model parameters: the actuator length and the applied voltage, according to the following table:

Parameter	Min Value	Max Value	Unit	Variable Name
Actuator length	150	400
Applied voltage	0.5	10

As output quantity, we have the maximum displacement at the tip:

Quantity of Interest	Type	Unit	Surrogate Model Function Name
Maximum displacement	Scalar

Abstractly speaking, we are defining one surrogate model function:

where and are the input parameters, representing the length and applied voltage, respectively, and is the maximum displacement at the tip.

The Surrogate Model Training Study

In theory, in this simple case, we could perform a parametric sweep over both input parameters to generate a densely sampled data set. However, in many practical scenarios, this approach is computationally prohibitive if there are more than a few parameters. Instead, the parameter space is typically sampled more sparsely, and surrogate models are then used to accurately approximate values at unsampled points. This is why design of experiments (DOE) methods are used in the data generation process for data-driven surrogate models. DOE methods provide an efficient way to explore the parameter space while capturing the important variations needed for accurate model approximation.

We begin the surrogate model creation process by using a Surrogate Model Training study, which takes care of the DOE sampling automatically.

To begin with this study, right-click the Study node and select More Study Extensions > Surrogate Model Training.

Adding the Surrogate Model Training study.

The Surrogate Model Training study has a number of settings, and the first one we will pay attention to is the Quantities of Interest.

The Surrogate Model Training study Settings window.

We want to provide the maximum displacement as the single quantity of interest. However, we haven't yet created a variable that we can use for this purpose. We will do so through the use of a Point Probe. Right-click the Thermal Actuator > Definitions node, and under Probes, choose Point Probe.

Selecting Point Probe for accessing the maximum displacement at a point.

In the settings for the Point Probe node, under Source Selection, select Geometry 1 > Tip. This named selection is already available in the Application Library model we loaded and corresponds to the point at the tip of the actuator.

The Source Selection for the Point Probe feature.

A close-up of the thermal microactuator geometry where the point selected for the Point Probe feature displays a green color with an arrow pointing to it.

The location of the point probe where the maximum displacement is measured.

The default expression that is monitored by the probe is the voltage, V. We will now change this to the magnitude of the displacement. Use the Replace Expression button and menu to locate the expression for Solid Mechanics > Displacement magnitude, or enter solid.disp in the Expression field.

The displacement expression used for the Point Probe.

You may notice that the Type setting of the Point Probe is Average. You can also choose Maximum, Minimum, or Integral. However for a Point Probe, they all generate the same result (the average of the value at a point is the value at that point, etc.).

Next, in the Surrogate Model Training node Settings window, click the Add button to add a row to the table for Quantities of interest. Then, type comp1.point1 for the value of the Point Probe belonging to the component comp1. Alternatively, use the Insert Expression button below the table and browse to the variable comp1.point1.

The Point Probe used as the quantity of interest.

You can also use Ctrl+Space in the Expression field to autocomplete entering the expression and browse to the Point Probe, as shown in the figure below.

Browsing to the expression for the Point Probe using code completion.

In the Quantities of Interest table, the options under Include study-dependent input provide two ways to handle study-dependent results.

The Reduce to single global output option is used when a quantity of interest varies over the study, but the surrogate model should be trained on a single scalar value for each training sample, such as the first, last, minimum, maximum, or summed value over a study parameter.

The Configure study-dependent input option preserves the study dependence by including the study variable itself, such as time in a time-dependent study, frequency in a frequency sweep, or spatial coordinates, as an additional input to the surrogate model. This enables the surrogate model to predict the quantity of interest as a function of both the sampled model parameters and the coordinates or study-dependent variable, instead of reducing each model evaluation to a single summary value.

We are now ready to specify the parameters to vary in the Input Parameters section. Use the Add button in the Input Parameters table to add the parameters L (Actuator length) and DV (Applied voltage). For the parameter L (Actuator length), set 150 for the Lower bound and 400 for the Upper bound, with the unit um (for micrometers). For the parameter DV (Applied voltage), set 0.5 for the Lower bound and 5 for the Upper bound, with the unit V. To set the bounds, you first click on the corresponding row in the Input Parameters table. This will make the fields for Lower bound, Upper bound, and Unit available under the table. Make sure that the length unit of the parameter for the actuator length is set to um, corresponding to micrometers.

The input parameters settings for the Surrogate Model Training study.

Before we start generating the data by running the study, we need to define how many data points we want. The number of data points is another term for the number of parameter tuples generated by the study by solving the full finite element model (or other numerical methods used for the model at hand). It is defined by the Number of input points setting in the Input parameters samplings settings section. Change this setting from the default 20 to 50. This will solve for 50 parameter value tuples for the length and applied voltage with values determined by the underlying DOE method.

The Number of input points setting, which determines the number of times the model is solved.

To speed up the computation, in the Mesh settings window, change the Element size to Coarser.

Changing the _Element size to Normal.

Now it is time to run the computation, which will solve the model 50 times. Right-click the Study node and select Compute. The computation will take about 5 minutes on a standard workstation.

Information on the data generation is displayed in an Accumulated Probe Table as well as a Design Data table, as shown below.

A screenshot of the Messages/Progress/Log window section of the COMSOL Multiphysics UI with the Data Design table displayed.

The Design Data table showing the generated data points.

Note that during the computation you can switch between different tables by selecting from the Display menu button in the Table window toolbar. When the computation is finished, the Design Data table contains the data we need to train a surrogate model. Note that the Surrogate Model Training study gives you the option to define an empty surrogate model automatically after the computation. However, the default is to define it manually. This is controlled by the Surrogate model setting Design of experiments (No surrogate model). In this example, we will not change this setting but will define the surrogate model manually.

The generated table data only supports the base unit system, which is the SI unit system in this case. We will need to compensate for this later on when we train the surrogate model.

Building the Surrogate Model

To add the surrogate model, right-click the Global Definitions node and under Functions select Deep Neural Network.

Selecting the Deep Neural Network option.

For the Data source, select Results table and choose Design Data as the Results table. As described in Part 2, the last layer's Output features setting doesn't need to be changed but is implicitly defined by the number of outputs, or quantities of interest, which in this case is 1, the maximum displacement. For this surrogate mode we can get by with a very small network. Use the Add button under the table to create a network, according to the figure below.

The DNN layer architecture.

Since the Design Data table uses the centralized unit system for the Quantities of Interest, in this case SI units (m), we will define this unit in the Data Column Settings, as shown in the figure below. Thus, the Input Parameters have units um and V, whereas the output, the Quantitiies of Interest, has the base unit m.

Defining SI units for the surrogate model.

In the Training and Validation section, change the Number of epochs to 25,000 and click Train Model at the top of the Deep Neural Network window to start the training. The Number of epochs value is decided by trial-and-error starting from the default value of 1000.

The Training and Validation settings.

The convergence plot for the DNN training showing the loss for the first 7000 epochs.

The training process is quick in this case since the network is very small. After about 20,000 epochs we see a tendency toward overfitting, so anywhere around this point is a good place to stop the training. Recall that we can identify when overfitting sets in by the fact that the training loss keeps decreasing while the validation loss starts increasing.

A line graph containing a blue line and a green line, which both exhibit erratic spikes and dips in the line, with the epoch number on the x-axis and loss value on the y-axis.

The convergence plot showing the loss for the last epochs.

The lowest value of the validation loss is obtained at about 17,000 epochs, and this is the value that will automatically be used by the surrogate model function.

The Information section showing the DNN training results.

We can now plot the surrogate model by selecting Create Plot in the Deep Neural Network window toolbar.

The Model Builder with the Function 1 plot node selected and the corresponding Settings window shown and plot visualization displayed.

A visualization of the DNN function.

Comparing the DNN Function with a Linear Interpolation Function

Similar to the example in Part 2, and the fact that we only have two input parameters and one quantity of interest, we can easily visually and quantitatively compare the surrogate model function to an interpolation function.

To add a linear interpolation function, right-click the Global Definitions node and under Functions select Interpolation. In the Interpolation node Settings window, change the Data source to Result table and make sure the Table from setting is set to Design Data. Change the Type in the table to Argument, Argument, Function value, for L, DV, and QoI1, respectively. This will allow the interpolation function to take the length and voltage as input arguments and output the displacement. Also change the Unit of each variable to be consistent with the earlier settings: um, V, and m, respectively.

The interpolation function based on the Design Data table.

In the Settings window for the interpolation function, click Create Plot.

"The Model Builder with the 2D Plot Group 11 node selected and the corresponding Settings window and plot displayed in the Graphics window.

A visualization of the table data as a linear interpolation function.

Now, to compare with the DNN function, first navigate to Grid 2D 1. Change the Function setting from Deep Neural Network 1 to All. This will allow the visualization of any of the functions available under Global Definitions.

Next, locate the plot group with the Function plot of the DNN function. In this case, it is 2D Plot Group 10 (it could be a different plot group depending on which plots you created earlier). Right-click Function and select Duplicate. In the Expression field, replace the expression with int1(x1,x2) and click Plot. This will generate two superimposed plots.

In order to separate the plots, change the Color table_setting in the Function 2 window to GrayScale. Next, in the Height Expression nodes, under each Function plot node, change the Scale factor to 1e6. For the second Height Expression node, clear the Show height axis check box to avoid duplicate plot labels.

The Scale factor setting in the Settings window for the Height Expression node.

This gives the two function plots the same height scaling, which enables a proper comparison. This produces the result shown in the figure below.

The Model Builder with the Height Expression node selected and the corresponding Settings window and plot in the Graphics window. The Graphics window shows a rectangular-shaped surface plot in 3D space that partially displays a rainbow color distribution from dark red to dark blue and partially displays a grayscale color distribution from white to black.

The DNN function plot compared to the corresponding linear interpolation function plot.

We see that the two functions are close in their values. To further analyze their differences, you can evaluate or visualize the expression abs(dnn1(x1,x2)-int1(x1,x2)) in a separate plot group. You can try this on your own. In cases where only two input arguments are involved, an alternative approach would be to use a dense Parametric Sweep method. However, this method becomes increasingly costly for surrogate models with more input arguments, as the computational expense increases with the number of inputs.

In the next part, we will discuss using a Gaussian Process surrogate model, available in the Uncertainty Quantification Module, highlighting the pros and cons of such a surrogate model.