Parametric identification of technological thermophysics processes based on neural network approach

1. Introduction

This work is devoted to the search for methods for solving inverse problems, formulated in relation to equations of mathematical physics of parabolic type and providing for the definition of an unknown function that sets the boundary conditions when the input data are not fully known. Objects described by equations of this class are characterized by the spatial distribution of the modeled state function, and in some cases, input quantities, and refer to the systems with distributed parameters. Mathematical models based on differential equations in partial derivatives of parabolic type are widely used in many industrial technologies to describe the processes of unsteady heat conduction, heat and mass transfer, unsteady diffusion, electromagnetic waves propagation, and others.

The main difficulty in solving inverse problems of mathematical physics (IPMP) is their belonging to the class of ill-posed problems, which necessitates the use of special numerical regularizing algorithms to obtain a stable solution [1], [2]. In the case of an unavoidable measurement error or other disturbances, the problem of constructing special regularizing algorithms for a stable solution of such problems becomes especially urgent [3], [4].

In the theory of inverse ill-posed problems of mathematical physics, a large number of methods and computational algorithms have been developed that make it possible to obtain regular solutions based, as a rule, on the use of smoothing functionals or additional restrictions on the class of sought solutions [5]-[7]. These approaches require significant computational efforts, depending on factors that are difficult to formalize [1], [2] or the availability of a priori information about the solution, which is usually absent [1], [4]. In such a situation, the development of methods for solving inverse and ill-posed problems of mathematical physics is an urgent problem. It makes possible to obtain stable solutions with satisfactory accuracy while reducing the computational complexity of methods and algorithms.

In the papers [8]-[11], the authors propose an analytical method of parametric optimization, which searches for solutions in the class of physically realizable functions based on their smoothness. The advantages of this approach are an accurate mathematical model that takes into account the qualitative physical laws and basic properties of the process under study, as well as displaying the solution in an analytical form, which provides benefits for its subsequent use in solving technological problems of various directions (control, monitoring, control, forecasting, diagnostics process and other tasks). The disadvantage of the approach may be its limited application under conditions of inevitable disturbing factors due to the sensitivity of the developed method, which uses analytical optimality conditions, to the level of input data error.

This paper presents an approach to the search for physically grounded solutions of the IPMP using the example of parabolic equations on compact sets of polynomial functions under the conditions of disturbed input data based on parametric optimization implemented using artificial neural networks (ANN). The solution to the problem is obtained on the basis of a combination of an accurate analytical approach, which takes into account information about the physical characteristics of processes and systems with distributed parameters, and intelligent technologies that allow finding the quasioptimal solution under conditions of uncertainty in the mathematical description of the process.

The proposed approach allows us to find solutions to inverse problems without the use of laborious numerical regularizing algorithms that depend on factors that are difficult to formalize on a compact set of smooth functions, taking into account the measurement noise acting over a sufficiently wide range, with an accuracy acceptable for practical applications. Difficulties associated with the need to process a large amount of input information affect the training of an artificial neural network, and practically do not affect its performance when it is subsequently used to solve a specific problem of identifying the boundary conditions of the process.

2. Formulation of the problem

We consider a linear one-dimensional homogeneous partial differential equation of parabolic type, given in relative coordinates:

supplemented by initial and boundary conditions of the second kind:

The formulation of the inverse problem provides for the presence of input information $T^{*}\left(t\right)=T\left(x^{*},t,\delta \right)$, where in most real cases the value of the state function is used – temperature $T\left(x,t\right)$, – obtained as a result of its control (measurements) at some fixed point $x^{*}\in$[0,1] at discrete moments in the identification interval $t\in \left.[0:h:t^{\mathrm{*}}\right]$ and containing measurement errors $\delta$. In the inverse boundary problem, it is required to determine the unknown time-dependent heat flux density $q\left(t\right)$, applied to the outer boundary $x=$1, with the remaining parameters and characteristics of the object given.

As a rule, a modern approach to the formulation of inverse problems is their definition in a variational formulation, which provides for the minimization of the objective functional in the space of possible solutions [12]. As a measure of the correspondence of the mathematical model to the real process, the value of the temperature residual is used, written in a quadratic or uniform metric [12]-[14]. Then the inverse problem is considered as an optimal control problem in which the conditions for the extremum of the formulated target functional of the temperature residual are investigated [14].

In this paper the inverse problem is formulated in a variational setting as an optimal control problem, where, based on the experimental data obtained under the conditions of disturbances, it is necessary to restore the unknown characteristic $q_M\left(t\right)=q\left(t\right)$, minimizing the deviation ${\epsilon }_T=T^{*}\left(t\right)–T_M\mathrm{}(x^{*},t)$ between the experimental results $T^{*}\left(t\right)$ and the values of the state function $T_M\mathrm{}(x^{*},t)$, calculated on the basis of the mathematical model (Eq. (1)-(2)) when using found solution $q_M\left(t\right)$ on a given identification interval at the same point $x^{*}$. To estimate the deviation of the model temperature from the experimental one, a functional based on the squared error is used:

where the function to be identified $q\left(t\right)$ is considered as the desired control and is subject to the conditions of belonging to the corresponding set $V$. Quadratic error functional Eq. (3) is chosen as the most widespread in neural network technologies, which are used further in solving the problem of parametric optimization.

3. Parametric optimization of the desired characteristic

To solve the formulated problem (Eq. (3)), a transition is made from the original set $V$ to the class of functions physically substantiated in the process of identifying functions that form a compact set. As physically realizable characteristics specified on the basis of requirements for their smoothness, we consider a set of polynomial functions of the form:

where the exact value $N$ ensures that the identified characteristic belongs to a specific class of functions. Thus, the value $N$ and the corresponding vector of parameters $\mathrm{\Delta }=\left(a_0,a_1,\dots a_N,\right)$ uniquely determine the parametric representation according to Eq. (4) of the sought-for heat flux density $q\left(t\right)=q\left(t,∆\right)$.

Then the mathematical description of the process under study (Eq. (1)-(2)) makes it possible to obtain the calculated value of the state function $T_M\left(x,t\right)=T_M\left(x,t,\mathrm{\Delta }\right)$ as a reaction to the boundary effect $q\left(t,\mathrm{\Delta }\right)$, expressed in parametric form, at given values $N$ and $\mathrm{\Delta }$, which corresponds to the temperature parameterization procedure. The resulting representation $T_M\mathrm{}(x,t,∆)$ leads to the parametric optimization problem:

with respect to the desired vector of parameters $\mathrm{\Delta }$.

Thus, the narrowing of the original class of solutions $V$ to a compact set of polynomial functions according to Eq. (4) given by the number $N$ and the corresponding vector $\mathrm{\Delta }$, allows us to formulate the parametric optimization problem given in a form of Eq. (5), and thus corresponds to the transition from the original ill-posed problem to the conditionally correct formulation, which solution does not require the use of regularization methods.

Proposed approach to identifying an unknown function $q\left(t\right)$ is based on constructing a sequence of functions on a compact set $q\left(t,{\mathrm{\Delta }}^{\left(N\right)}\right)$, that minimizes the quadratic deviation $\sum _{t\in \left[0,t^{*}\right]}^N{\left(T_M\left(x^{*},t,∆\right)-T^{*}\left(t\right)\right)}^2\to \underset{∆}{\mathrm{m}\mathrm{i}\mathrm{n}}$ with increasing number $N$ and, thus, converges to the exact solution $q^0\left(t\right)$. Based on the Weierstrass theorem [15], the solutions obtained are regular. Each successive approximation of the sequence can be considered as an intermediate solution, and the computational procedure can be stopped when the deviation of the calculated value $T_M\mathrm{}(x^{*},t,∆)$ from the measured one $T^{*}\left(t\right)$ meets the requirements of a given accuracy.

At a low level of perturbations, the solution to the problem can be realized using the minimax optimization method [8]-[11], based on the analytical optimality conditions [16]. In a situation where the input data is significantly distorted by measurement errors, it is proposed to solve the problem using artificial intelligence methods that allow processing large data arrays to find vector values $\mathrm{\Delta }$, that ensure the fulfillment of the quadratic optimality criterion Eq. (5).

4. Construction of neural networks for solving inverse problems of mathematical physics

There are no general solution methods that allow one to find the global extremum of functional Eq. (5) under the conditions of the action of perturbations. When using input data $T^{*}\left(t,\delta \right)$, that take into account measurement errors $\delta$, the method of sequential parameterization [8]-[11], which is successfully used to find solutions to inverse problems of heat conduction under deterministic influences, is proposed to be supplemented with the use of ANN. Neural networks are profitably applied to solve problems of approximating time series, constructing dynamic dependencies, predicting the values of functions at future moments in time related to direct problems [17]-[19]. By choosing the type of neural network, the number of layers and neurons in each of the layers, setting the activation function and the network learning algorithm, the behavior of the state function is reproduced with a sufficient degree of accuracy. To search for solutions to problems in mathematical physics, it is recommended to use such neural networks as multilayer perceptron, networks with radial basis functions, probabilistic networks, generalized regression networks [20], [21]. Neural network technologies can also be used to solve inverse problems [22]-[24].

The formulated parametric optimization problem (Eq. (5)) is considered under conditions of interval uncertainties, when the amount of a priori information about the values of the parameters $a_0-a_N$ is limited by information about the boundaries of the interval of their possible change:

A set of possible combinations of parameters $a_i$, $i=\stackrel{-}{0,N}$ forms a ensemble of temperature curves $T_M\left(x^{*},t,\mathrm{\Delta }\right)$, $\mathrm{\Delta }=\left(a_0,a_1,\dots a_N\right)$, that satisfy the conditions (Eq. (6)). Then the problem (Eq. (5)) is considered with respect to a set of temperature realizations that satisfy all possible combinations of parameter values $a_i$, $i=\stackrel{-}{0,N}$. To solve this problem, neural networks are used that allow processing large data arrays containing possible values of the parameters of the identified characteristic and the corresponding realizations of the temperature field.

Solution to the problem (Eq. (5)) based on noisy input data $T^{*}\left(t,\delta \right)$ is reduced to a step-by-step search for the values of the parameter vector ${\mathrm{\Delta }}^{\left(N\right)}=\left(a_0,a_1,\dots a_N,\right)$ for increasing values $N=\mathrm{1,2},\dots ,N^{*}$. The iterative procedure ends at $N=N^{*}$, when the solution obtained at the next step ${\mathrm{\Delta }}^{\left(N\right)}$ ensures the achievement of an absolute $T_M\left(x^{*},t,{∆}^{\left(N\right)}\right)-T^{*}\left(t\right)\le {\epsilon }_1$ or relative $T_M\left(x^{*},t,{∆}^{\left(N\right)}\right)-T_M\left(x^{*},t,{∆}^{(N-1)}\right)\le {\epsilon }_2$ accuracy.

At each iteration, for a chosen number $N$ and a given, thus, the structure of the vector ${\mathrm{\Delta }}^{\left(N\right)}$, the problem of parametric optimization (Eq. (5)) taking into account the perturbed input data $T^{*}\left(t,\delta \right)$ is reduced to the execution of the following algorithm.

1) Based on a priori information about design, technological or other constraints, the permissible range of variation of the desired characteristic is determined $q\left(t\right)$, and then the approximate boundaries of possible intervals of change in values are determined $(a_i^{\mathrm{i}\mathrm{n}\mathrm{f}},a_i^{\mathrm{s}\mathrm{u}\mathrm{p}})$ of every parameter $a_i$, $i=\stackrel{-}{0,N}$. For each of the them, a sampling step is selected $h_i$ and a one-dimensional array $A_i=(a_i^{\mathrm{i}\mathrm{n}\mathrm{f}},a_i^{\mathrm{i}\mathrm{n}\mathrm{f}}+h,a_i^{\mathrm{i}\mathrm{n}\mathrm{f}}+2h_i,\dots ,a_i^{\mathrm{s}\mathrm{u}\mathrm{p}})$ of discrete parameter values is formed each with the dimension $\left[K_i\times 1\right]$, $K_i=\frac{a_i^{\mathrm{s}\mathrm{u}\mathrm{p}}-a_i^{\mathrm{i}\mathrm{n}\mathrm{f}}}{h_i}+1$.

2) Based on one-dimensional vectors $A_i$ an array of sample values of all parameters is formed, i.e. a matrix is drawn up $\mathbf{A}=\left\{A_i\right\}$, $i=\stackrel{-}{0,N,}$ with the dimension $\left[\left(K_0\times K_1\times \dots K_N\right)\times \left(N+1\right)\right]$, containing a set of all possible combinations of discrete parameter values $\left(a_0,a_1,\dots a_N,\right)$ from the corresponding ranges of their variation. Thus, each row of the matrix $\mathbf{A}$ contains some $j$ variant of the admissible realizations of the combination of discrete values ${\mathrm{\Delta }}_j$, $j=\stackrel{-}{1,(K_0\times K_1\times \dots K_N)}$ of the vector $\left\{{\mathrm{\Delta }}_j\right\}=\mathbf{A}$, ${\mathrm{\Delta }}^{\left(N\right)}=\left(a_0,a_1,\dots a_N\right)$, and the complete matrix $\mathbf{A}$ consists of all possible combinations. The matrix $\mathbf{A}$ composed in this way forms the target vector in the development of the ANN.

3) On the basis of the mathematical model of an object with distributed parameters (Eq. (1)-(2)), a set of direct problems of heat conduction is solved for all variants of combinations of parameters from the matrix $\mathbf{A}$ on a given identification interval $t\in \left.[0:h:t^{\mathrm{*}}\right]$. As a result, each $j$-th row of matrix $\mathbf{A}$ is associated with a model temperature realization $T_{Mj}\left(x^{*},t,{\mathrm{\Delta }}_j\right)$, obtained at discrete times of the identification interval for each variant of the combination of parameters ${\mathrm{\Delta }}_j$, $j=\stackrel{-}{1,(K_0\times K_1\times \dots K_N)}$, $\left\{{\mathrm{\Delta }}_j\right\}=\mathbf{A}$.

4) A neural network model of the inverse problem of heat conduction is created. The input data of the ANN is the set of all temperature realizations $T_{Mj}\left(x^{*},t,{∆}_j\right)$, $j=\stackrel{-}{1,(K_0\times K_1\times \dots K_N)}$, and the target values are the corresponding parameters that form the matrix $\mathbf{A}$. The neural network training procedure consists in calculating the weight coefficients (synaptic weights) that minimize the deviation error between the temperature realizations obtained on the basis of the neural network model $T_{ANN}\left(x^{*},t,{\mathrm{\Delta }}_{ANN}\right)$, that correspond to the found solutions Δ _ANN, and the specified temperature realizations $T_{Mj}\left(x^{*},t,{\mathrm{\Delta }}_j\right)$ for the entire data array $\mathbf{A}=\left\{{\mathrm{\Delta }}_j\right\}\text{,}$$j=\stackrel{-}{1,\left(K_0\times K_1\times \dots K_N\right)}$.

The ANN constructed in this way is used further to calculate on the basis of noisy experimental data $T^{*}\left(t,\delta \right)$, obtained at the temperature control point $x^{*}\in$ [0, 1] in the identification interval $t\in \left[t_1,t_2,\dots t_s\right]$; $t_1=$0, $t_s=t^{*}$ the parameter values$A_i=\left(a_0^0,a_1^0,\dots ,a_N^0\right)={∆}^0$, providing the best coincidence of the calculated data and measurement results according to the criterion of the minimum squared error. The obtained solution to the parametric optimization problem (Eq. (5)), corresponding to the conditionally correct formulation of the inverse heat conductivity problem, belongs to a compact set of functions of a given form Eq. (4) and is stable, regardless of the input data error. The magnitude of the error only affects the accuracy of the reconstruction of the identified characteristic. The block diagram reflecting the procedure for solving the inverse problem of technological thermophysics is shown in Fig. 1.

Fig. 1Solution pattern to the inverse heat conductivity problem: ODP – object with distributed parameters, MU – measurement unit, DHCP – direct heat conductivity problem

The formulated parametric optimization problem (Eq. (5)) was solved using radial basis function network (RBFN) [20], [21], which showed efficiency in solving direct problems of mathematical physics. These structures contain an inner layer consisting of radial elements that implement the Gaussian function, and a linear layer that provides a weighted average estimate. The weighting factors are obtained by minimizing the distance between the calculated values $T_{ANN}\left(x^{*},t,{\mathrm{\Delta }}_{ANN}\right)$, corresponding to the found vector of parameters ${\mathrm{\Delta }}_{ANN}$ and the corresponding realizations of the input data array $T_{Mj}\left(x^{*},t,{\mathrm{\Delta }}_j\right)$. A linear combination of the output signals of the radial elements for all elements of the training sample, with the found weight coefficients, makes it possible to obtain the resulting estimate of the values of the parameters of the polynomial representation according to Eq. (4) of the identified characteristic $q\left(t\right)$. A diagram showing the architecture of a radial basic neural network for solving a parametric optimization problem is shown in Fig. 2.

Fig. 2RBF – network architecture, φT*,A – radial basis functions, w – network setup parameters

5. Results and discussion

According to the described method, on the basis of the mathematical model (Eq. (1), (2)), the inverse boundary problem of heat conduction was solved by restoring the unknown function of the heat flux density:

using experimental data $T^{*}\left(t,\delta \right)$ taking into account the additive measurement error $T^{*}\left(t,\delta \right)=T^0\left(x^{*},t\right)+\delta randn$, where $T^0\left(x^{*},t\right)$ – exact solution of the heat conduction problem (Eq. (1), (2)), and “randn” function simulates a normal distribution with zero mean value and standard deviation $\delta$.

Temperature dependence $T^0\left(x^{*},t\right)$ was obtained at the point $x^{*}=$0.9 based on the general solution of the heat problem, expressed in integral form:

where $G\left(x,\xi ,t–\tau \right)$ – Green’s function corresponding to the boundary value problem (Eq. (1), (2)). In accordance with Eq. (7), the flux density is taken to be $q\left(t\right)=q^0\left(t\right)$, and Eq. (8) takes the form:

The parametric optimization problem was solved in the class of polynomial functions according to Eq. (4) with $N=$2, 3. Temperature curves $T_M\left(x^{*},t,{\mathrm{\Delta }}^{\left(3\right)}\right)$ with $N=$3 which are the input data for the ANN, are obtained on the basis of Eq. (8) when specifying $q\left(t\right)=q^0\left(t,{\mathrm{\Delta }}^{\left(3\right)}\right)$ in the form of a polynomial dependence (Eq. (4)) with $N=$3, ${\mathrm{\Delta }}^{\left(3\right)}=\left(a_0,a_1,a_2,a_3\right)$, which has the following form:

The case $N=$2 can be considered as a special case $N=$3 with $a_3=$0.

The problem is solved using RBF – network on the identification interval $t\in \left.[0:0.02:1\right]$ with the following data $k=$2, $\beta =$2 taking into account measurement errors distributed according to the normal law with standard deviation $\delta =$0-15 %.

Some results of solving the problem are presented in Figs. 3-6.

The figures show the change in the configuration of the temperature discrepancy with an increase in the intensity of the interference when searching for solutions in the class of functions $N=$3. With no interference (Fig. 3(a)) alternating extrema are present on the curve $T^0\left(x^{*},t\right)$– $T_M\left(x^{*},t,{\mathrm{\Delta }}^{\left(3\right)}\right)$, and the error in approximating the temperature field is less than 1 %. With an increase in the level of measurement error, the curve of the temperature residual is distorted (Fig. 3(b)), which leads to an increase in the approximation error. A similar situation is when $N=$2 and $N=$3 (Fig. 4, 5). In the general case, the solution in the class of functions with the number $N=$ 3 allows us to obtain greater accuracy than with $N=$ 2 both for approximating the temperature (Fig. 4(a), 5(a)) and for identifying the unknown function – the heat flux density (Fig. 4(b), 5(b)), but in each case the results based on the analysis of random processes using neural networks may differ. At the same time, there is no unambiguous relationship between the error $T^0\left(x^{*},t\right)$-$T_M\left(x^{*},t,{\mathrm{\Delta }}^{\left(N\right)}\right)$ and $q^0\left(t\right)$–$q\left(t,{\mathrm{\Delta }}^{\left(N\right)}\right)$ as in most cases, this happens when using deterministic algorithms [8]-[11].

Fig. 3The error εT3=T0x*,t-TMx*,t,Δ3 in the approximation of the temperature field at N= 3 in a) the absence of interference (δ= 0 %) and b) its action 1-δ= 2 %; 2-δ= 5 %; 3-δ= 10 %

a)

b)

Fig. 4a) Error εTN=T0x*,t-TMx*,t,ΔN in approximating the temperature dependence and b) error εqN= q0t-qt,ΔN of reconstruction of the identified characteristic at δ= 2 %: 1 – N= 2; 2 – N= 3

a)

b)

With an increase in the level of perturbation, the configuration of the temperature field error changes, which leads to a decrease in the accuracy of the temperature approximation and the identified function. This dependence has an ambiguous character for the restoration of both the state function and the desired characteristic. Artificial intelligence methods ensure the accuracy of solving the problem, acceptable for engineering calculations (Fig. 6), in the selected range of measurement error with a given input information.

Fig. 5a) Fitting error εTN=T0x*,t-TMx*,t,ΔN and b) departure εqN=q0t-qt,ΔN in approximating the desired effect at δ= 15 %: 1 – N= 2; 2 – N= 3

a)

b)

Fig. 6Comparison of the solution obtained with the experimental temperature 1 – T*t, δ= 15 %; 2 – TMx*,t,Δ3, N= 3

6. Conclusions

In this paper, using the example of the boundary value inverse problem of heat conduction, an approach to solving inverse problems of mathematical physics is presented, which combines the advantages of the exact analytical method, which is the basis for the identification of objects with distributed parameters on a compact set of polynomial functions, and artificial intelligence methods that allow processing large amounts of data and used for solving the problem of parametric optimization in conditions of perturbation of input information. The results obtained confirm the possibility of using neural networks in solving the problem of parametric optimization using the example of the inverse heat conductivity problem. The effectiveness of the results obtained depends on the level of measurement error and the specified values of the boundaries of the intervals of the probable change in the parameters.

To improve the accuracy of the solution, it is necessary to increase the degree $N$ of the approximating polynomial and, accordingly, the dimension of the sought vector of parameters $\mathrm{\Delta }\left(N\right)$, each of which $a_i$, $i=\stackrel{-}{0,N}$ under data uncertainty is set on a certain admissible interval. With an increase of the number $N$, the complexity and cumbersomeness of the neural network rapidly occurs. Moreover, some a priori information about the technological limitations of the device or process under study must be known for the adequate parameter setting.

The computational complexity of the proposed approach is associated with a large volume of processed data with an increase in the number of estimated parameters, which requires significant computing resources and time. These requirements are necessary only at the stage of training the neural network. The trained neural network is versatile and can be used repeatedly in the study of typical heat conduction processes.