Laser speckle contrast imaging and machine learning in application to physiological fluids flow rate recognition

1. Introduction

Analysis of changes in the properties of physiological fluids, in particular blood, can be used to diagnose various diseases. It is shown in the works [1-3] that changes in blood viscosity are associated with the regulation of immune responses. It is demonstrated in paper [4] that an increase in blood viscosity is associated with a number of diseases, such as acute nonlymphocytic leukemia, chronic lymphocytic leukemia, chronic myeloid leukemia, sickle cell disease. The study of the rheological and kinematic properties of physiological fluids is an actual task, and of particular interest are minimally invasive and non-invasive diagnostic methods. The laser Doppler flowmetry (LDF) and the laser speckle contrast imaging (LSCI) methods make it possible to carry out studies in the “in vivo” conditions to evaluate microcirculation in biological tissues [5-7]. LSCI is used in rheumatology, dermatology, ophthalmology, neurology, and many other areas of medicine [8-10]. LSCI is an effective method for full-scale monitoring of particle dynamics in heterogeneous media However, there is a problem in the determination of blood flow quantitative parameters. The blood vessel visualization using multispectral optoacoustic tomography is considered in Ref. [11]. As an alternative to manual segmentation of the obtained images, the authors propose the use of “sparse-UNET”. The test accuracy in the form of the “Dice” estimate is 0.88. The stenosis of coronary arteries by coronary angiography were considered by M. Zreik and coauthors [12]. The authors propose the use of coronary CT angiography to reduce invasive procedures. The application of a convolutional neural network for segmentation and a support vector machine for classification is used. The classification accuracy according to the “ROC” estimate is 0.74.

This paper deals with the application of artificial neural networks (ANN) to the recognition of the average flow rate. The physiological fluid flow images obtained using the LSCI method are under study. In the future, the developed method is expected to be used as a part of the viscosity measurement procedure [13].

2. Experimental setup and methodology

The aim of the study is a combination of the determined and stochastic factors including the flow rate, the intralipid concentration, the camera exposition time, and the camera positioning for the following feature extraction using an ANN and several experts to compare their accuracy.

The study was implemented with the experimental setup that combines the fluid flow circuit and the LSCI equipment (see Fig. 1), designed earlier [6, 8, 9]. The enforced fluid flow through the medical glass capillary of 1.6 mm in diameter was investigated. The studied fluid is 20 % intralipid (Fresenius Kaby, USA). The electric pump DSh-08 (Visma, Belarus) provides a flow rate of a known value while the UI-3360CP-NIR-GL Rev.2 (IDS GmbH, Germany) CMOS camera combined with the extension adapter and MVL25TM23 lens (Thorlabs, Inc., USA) records the intralipid flow illuminated by a 785 nm laser (LDM785, Thorlabs Inc.).

Fig. 1The experimental setup: a) schematic and b) photograph

a)

b)

The plan of the study consisted of 225 tests including 5 parallel tests at each combination of the factors. The tests were performed in random order. Each test lasted 1 minute. The camera recorded the flow with the resolution of 2048×512 pixels at the frequency of 30 FPS. The flow rate was of 0, 0.5, 1, 1.5, and 2 mm/s. The intralipid was diluted to concentrations of 4, 8, and 12 %. The camera exposition time was 5, 15, and 33 ms.

3. Data processing

In LSCI, the image of dynamic inhomogeneities is obtained by analyzing the local speckle contrast in the image plane [14]. If a scattering object is illuminated by the coherent light, the randomly changing light intensity pattern, produced by random interference inside the medium and on its surface, appears. This granular structure of light intensity is widely known as the speckle pattern. Movement of particles inside the illuminated medium causes the fluctuations in the position of speckles and results in a blurred image due to the averaging during the exposure time of the detector. Further, temporal and spatial statistics of the speckle pattern can be used for obtaining information on the motion of scattering particles$.$ The spatial speckle contrast (SSC) uses a small square domain of several pixels in an image for the calculations. The optimal domain is a 7×7 pixels window. The calculation convolves the input image to the SSC image by moving the window along the image vertically and horizontally and computing the speckle contrast using:

where $n$ is the calculation domain size; ${\sigma }_n$ is the intensity standard deviation; $⟨I_n⟩$ is mean intensity.

The $K$ ranges from 0 to 1 (see Fig. 1). To calculate the temporal speckle contrast (TSC), the value of one pixel intensity in several frames is used. The spatio-temporal speckle contrast (STSC) combines both methods discussed above.

The images obtained with the LSCI are subjected to further processing by a feed-forward ANN. The ANN receives contrast values of all pixels in an image. An input matrix $\mathbf{X}$ of $[n\times {(l}^2+1)]$ elements represents $n$ images with the size of $l\times l$ each. The matrix $\mathbf{X}$ also includes the unit column as a bias. An output matrix $\mathbf{Y}$ of $[n\times 1]$ elements represents $n$ values of the flow rates that correspond to $n$ images. The available data helps to build the classification model that predicts the flow rate in the capillary based on the image of this flow. The classification model is a complex parametrized function $H(\mathbf{\theta },\mathbf{X})$ which approximates the class prediction probabilities. A feed-forward ANN is used to implement this model (see Fig. 2). The input layer has ($l^2+1)$ neurons that correspond to the size of the images. The number of neurons in the hidden layer $n_{hid}$ is a hyperparameter. The number of layers in the output layer is 5 that corresponds to the number of flow rate levels (see Section 2). The weight matrices of the input and output layers are ${\mathbf{\theta }}_{\left[\right(n_{input}+1)\times n_{hid}]}^{hid}$ and ${\mathbf{\theta }}_{[{(n}_{hid}+1)\times n_{out}]}^{out}$, respectively. The “ReLu” function is the activation function in the hidden layer. The hidden layer neurons output the following values:

The output layer has the “softmax” activation function. The output values are:

So, the probability of belonging to each class is approximated by the ANN shown in Fig. 2. The cross-entropy loss with the $L_2$-regularization is used as an error function:

where ${\tilde{\mathbf{Y}}}_k$ is a column matrix with the size of $[n\times 5]$ and with the components corresponding to the logic equation $y_i=[y_i==k]$, given for each image.

Fig. 2A 3-layers feed-forward neural network for a multiclass classification

The gradient methods are applied to minimization of the loss function Eq. (4) and finding the weights. The “Adam” algorithm is one of the most popular [15].

4. Results

This section deals with the experimental results obtained for the intralipid concentration of 8 % and the camera exposition time of 15 ms (see Section 2). A set of speckle contrast images was generated using the SSC method (see Fig. 3). Then the 5-classes classification problem was studied. At the first stage, a subset of speckle contrast images was used to estimate the accuracy of visual pattern recognition by 6 experts. Their average recognition accuracy was 59 % and the highest accuracy was 65 %.

Fig. 3Fluid flow visualization: a) an initial image and b) the corresponding speckle contrast image obtained with the SSC method

a)

b)

Experts’ low accuracy can be explained by the fact that the pump of the experimental setup has some flow rate fluctuations when operating. Fig. 4 demonstrates this periodical fluctuation.

Fig. 4The SSC values in the capillary central part domain of 40×40 pixels. The flow rate is 1.5 mm/s

At the next stage, the newer sets of TSC and STSC images were calculated with the depth parameter of 120 frames, as well as the mean SSC across the 120 frames (see Fig. 5).

Fig. 5Speckle contrasts calculated in the the capilary central part domain of 40×40 pixels at the different fluid flow rates: a) SSC, b) TSC, c) STSC, and d) mean SSC

a)

b)

c)

d)

All plots demonstrate outliers. This may be caused by the presence of a transient flow regime in the first few seconds of the video recording. Such data was removed from the data set.

The dataset of 38000 images was divided into 3 parts in proportion of 0.55:0.23:0.25 for training, validation, and test, respectively. Fig. 6 demonstrates examples of images for each class. Standard scale normalization was applied.

Fig. 6Training images represent the capillary central part domain of 128×128 pixels

The ANN was trained with different number of hidden neurons: $l_{hid}$ = [8, 16, 32, 64]. The number of classes and the regularization parameter value $\lambda$ (see Eq. (4)) were also varied. The parameters were selected based on the validation results. Fig. 7 demonstrates the classifier confusion matrices.

Fig. 7The confusion matrices calculated for the mean SSC images

Quantitative analysis of the ANN and the accuracy of 6 experts depending on the data preprocessing method are presented in Table 1.

The Table 1 demonstrates that the ANN accuracy is higher than the accuracy of the experts at approximately 10 %. The best ANN accuracy of 88.6 % on the test set was obtained for the case of data preprocessing using the mean SSC method. The proposed technic or any of its elements are applicable to the LSCI or LDF methods in the “in vivo” conditions. For example, when studying soft tissues microcirculation [5-7] and when the obtained data samples can be represented in form of one- or multi-dimensional sequences of 10²-10³ elements. For the bigger samples, the deep neural networks should be met.

5. Conclusions

The proposed approach demonstrated that the mean spatial speckle contrast calculation allows highlighting important features and decreasing the negative effect of pulsations that helps to prepare images for the following flow rate recognition. The best ANN settings, e.g. the number of hidden neurons, were determined in the training and validation cycles. The trained ANN demonstrated high accuracy up to 90 % and exceeded the accuracy of 6 experts by about 10 %. The combination of the mean spatial speckle contrast imaging and machine learning methods to the physiological fluids flow study is promising in many applications including health monitoring.