Unsupervised Clustering in Epidemiological Factor Analysis

2.1. Data

Evidently, the time of the local exposure to the epidemics is one of the critical parameters of the impact, so the case data was adjusted and aligned at a similar phase in the development of the epidemics, based on the availability of data at approximately local Time Zero + two months, i.e., approximately two months after the first local exposure to the infection. In the study, this translates to the beginning of April, 2020, for Wave 1 case (i.e., those with local Time Zero at the end of January, 2020) and the end of April - beginning of May, 2020 for Wave 2 (LTZ end of February to early March, 2020).

A combined dataset of approximately forty national and subnational cases was constructed based on the criteria of reliability and consistency, essentially, bringing together cases with similar social and economic parameters to minimize the number of potentially influencing factors, along with the expectation of a certain minimal level of exposure to the epidemics and reliability of the reported data.

The dataset was constructed from the publicly available current data on the epidemic’s impact per case, i.e., reporting jurisdiction. As previously published statistical data in the public domain containing no individual identification, the data used in the study were excerpted from a review by the Institutional Review Board [18]. The dataset comprised the current value of the epidemiological impact recorded in the jurisdiction (case) and measured in mortality per capita m(t) (M.p.c.), per million of population, and a number of observable parameters selected, as described further in this section, with the hypothesis of a certain level of correlation between the observable parameter set and the severity of the outcome.

On the relative scale of impact by jurisdiction, the “explosive” cases were normally identified as those with the relative impact (i.e., relative to the maximum among all reporting jurisdictions at the time of reporting) of around and above 0.3. This subgroup of cases included all reported cases of high epidemic’s impact at the time of writing.

In the evaluation of distribution in the coordinates of principal components, two higher impact clusters of cases were identified by relative impact: explosive cases with a relative impact above 0.8 groups included the well-known first wave cases: Italy, Spain, and New York with the highest impact worldwide observed to date. In the second group, milder-impact cases, including the United Kingdom, France, Belgium, Netherlands, Ireland, and Quebec (Canada), with relative impact in the range from 0.3 to 0.8, were included.

The outcome parameter was not used in the training of the unsupervised learning models (i.e., excluded from the training dataset) but only for identification of the regions of interest (i.e., those with higher epidemiological impact) in the latent representations produced by the models as a result of training.

2.2. Observable Parameters

The examples of factors of influence are genetic differences, population density, social traditions and cultural practices, past widespread public policy, such as immunization, smoking habits, and the epidemiological policy of the jurisdiction aimed at controlling the spread of the disease.

In addition to the common measurable factors, such as population density, age demographics, smoking prevalence, a number of additional factors with potential impact on the severity of the epidemic pattern were considered in this study, as described in this section. Due to the limitation of time and resources, a rating scale approach was chosen for those factors that could not or would be challenging to measure directly. Understandably, such an approach can be influenced by subjective perceptions; however, we believe that more robust and objective techniques can be developed over time, improving the quality of the analysis and the resulting conclusion.

It needs to be noted that in the framework of unsupervised analysis, the epidemiological impact is not known a priori and for that reason, it was not used in the evaluation of data with the selected methods. It was used, however, to analyze distributions obtained with the models and identify regions of potential interest, such as combinations of observable parameters associated with the areas of higher epidemiological impact.

The resulting dataset of 40 national and subnational cases with the identified observable parameters and the recorded epidemiological impact at the time of preparation is presented in Appendix Table 1.

Reservations and qualifications:

1. Consistency and reliability of data reported by the national, regional, and local health administrations.

2. Alignment at the time of reporting may not be consistent between all jurisdictions due to possible differences in reporting practices.

Sources:

Google coronavirus map [20], World statistical data [21, 22], World BCG atlas [23], National and subnational jurisdictions COVID-19 information and statistics sources [24-26].

Media reports and others sources were used.

2.3. Methods of Unsupervised Machine Learning

To evaluate the hypothesis of the correlation between the identified parameters and the epidemiological outcome of the cases in the dataset, two well-known and commonly used machine learning methods were used. Different methods were used to verify the consistency of the findings and eliminate the influence of the specific choice of the models, the possibility of fluctuations in the data, and so on, such as:

1. Principal Component Analysis and identification of principal informative factors (Table 1).

2. Unsupervised deep neural network-based dimensionality reduction and selection of dominant informative factors.

Principal Component Analysis [13] produces a linear transformation of the data to the coordinates with the highest variance. The method is based on the internal characteristics of the data and does not require prior knowledge of the outcome.

A deep neural network autoencoder performs a non-linear dimensionality reduction of the observable data to the lower-dimensional representation with identified informative features. The diagram of the architecture of the unsupervised autoencoder model is given in Fig. (1).

The neural network model of a deep autoencoder used in this analysis had 5 fully interconnected layers of size 3 – 30, with a total of approximately 8,000 trainable parameters. The data were scaled to the interval (0, 1), and “sigmoid” activation in the output layer with Mean Squared Error (MSE) cost function was used for unsupervised training of the model with the dataset observable parameters, excluding the impact. Models of this and similar architecture were found to be effective in the earlier studies of unsupervised latent representations, including the Internet [27] and different types of visual data [28, 29]. A detailed description of the deep neural network model architecture similar to those used in the study is provided previously [27]. It is expected that with larger datasets with higher numbers and complexity of the parameters, the complexity of the models, including the size and depth can be extended in future studies.

In the unsupervised training phase, the neural network model is trained to reproduce the data in the training dataset with good accuracy (i.e., an incentive to reduce the deviation of the output and input) and does not require labels marked with the outcome; the same applies to PCA. Achieving an improvement in the accuracy of the reproduction of the input data, which can be measured by a number of training metrics, indicates that the model has learned some essential characteristics of the initial distribution. The aim of unsupervised learning is thus to minimize the deviation of the original training sample from its regeneration created by the model.

3.1. Principal Component Analysis

The principal component analysis identified three principal components with overall influence above 95%, as shown in Table 2. The highest influence factors in the PCA analysis were mostly aligned with the results of the linear regression analysis, such as policy-timing, connection hub, social proximity, BCG, and smoking prevalence.

PCA transformation is an inherently unsupervised method of learning, which means that the prior known outcome labels are not required to learn the principal components as well as representation of the input data in the coordinates of identifying principal component eigenvectors. By plotting the data in the coordinates of the identified principal component vectors, interesting results can be obtained, indicating the cases with the highest recorded impact of the epidemics.

Fig. (2) presents visualizations of the distribution of the dataset of epidemiological cases in the latent coordinates of the three leading principal components {p₁, p₂, p₃} with the highest variation identified by PCA analysis, corresponding to the axes described in Table 1. The cases and approximated region of the high-impact cluster are shown in blue, defining the region of principal coordinate values with the highest recorded impact of the epidemics; in a similar way, a cluster with medium impact is shown in magenta.

A clear separation of the high-impact case clusters from the general background cases can be clearly observed in the diagrams. It allowed identifying the region where the cases with potentially higher impact, including the “explosive” pattern, are distributed in the latent coordinates of the principal component representation.

3.2. Data Analysis with Unsupervised Neural Network Model

A similar approach can be demonstrated with an unsupervised neural network autoencoder model that reduces the number of parameters by compressing the observable data space into a lower-dimensional representation in an unsupervised training process aimed at improving the accuracy of regeneration from the compressed representation. Models of a similar type were used to create structured unsupervised representations of different data types via unsupervised autoencoder training with minimization of generative error [15-17, 27].

The dimensionality of the unsupervised representation for the models in the study, which is defined by the size of its central encoding layer, was chosen based on the results of the Principal Component Analysis in the previous section, indicating the three most informative components. The identified latent coordinates {q₁, q₂, q₃} represented activations of the neurons in the central, “encoding” layer of the neural network model.

Presented in Fig. (3) are direct visualizations of the distributions of data in the coordinates of the latent representation created by an autoencoder neural network model trained in an unsupervised process without outcome labels.

The highest impact cluster of three cases is shown in green, whereas the medium one (7 cases), in orange. Again, a similar pattern of clear separation of high-impact cases from the general background can be observed with these models, consistent with the results of PCA analysis in the previous section, as indicated in Table 2.

The consistency between the results of two independent and unrelated methods applied to the same dataset's observable parameters indicates with high confidence that the observed effect represented a genuine relation of parameters in the dataset and not a spurious artifact.

It is worth noting that as with PCA, autoencoder models, though essentially non-linear, also allow identifying the higher-impact regions in the coordinates of the observable parameters. This can be achieved by forward-propagating through the generative part of the model, the identified region of interest, defined by a set of characteristic points in the latent representation, defining the corresponding region in observable parameters. The combinations of observable parameters that produce the effect of interest can be identified proactively and used in the development of an effective preventative or mitigating epidemiological policy.

3.3. Observable Regions of Interest

It is essential to note that the regions of interest, such as those with a higher impact of the epidemics identified in the analysis with unsupervised models can be translated into the observable parameters, identifying the regions in the observable space with potentially high impact {R_imp}.

In the case of PCA, which is a linear model, the set of identifying vortices in the coordinates of principal components can be expanded to the initial dimension with a number of strategies, for example by adding the median values of other components. The obtained n-dimensional polyhedron region can then be transformed into the observable coordinates via inverse PCA transformation.

Where P is the linear PCA transformation operator.

With artificial neural network-based unsupervised models that are essentially non-linear, the transformation is not as straightforward but still possible. With a sufficiently accurate generative model (Fig. 1) produced by a successful unsupervised training process, the observable image of any point of interest in the latent space of the model X_lat can between approximated by its regeneration by the generative model, Gen(X_lat).

Then, the region of interest in the observation space can be approximated by a polyhedron of generating points:

Where Gen is the generative submodel of the autoencoder.

Thus, in both considered cases, the identified regions of interest in the informative parameters can be translated into the observable ones, providing important early input for the development of preventative policies.

3.4. Trend Analysis with Identified Informative Factors

The informative factors identified in the unsupervised phase of the analysis can be used to evaluate early trends in the development of the epidemiological scenario, providing an essential input in the development of the containment policy.

Linear regression with identified observable parameters of highest influence [30] produced a trend with a strong correlation to the outcome, with the value of 0.9 out of 1.0 maximum, as shown in Table 3.

Policy factors, described in a number of studies [31], have a strong influence on the outcome that is confirmed by the results of the linear regression analysis. Furthermore, the importance of other factors, such as connection intensity, social proximity culture, immunization record, and smoking prevalence was observed.

Influencing factors identified with methods of unsupervised machine learning can be used in trend analysis of the factor(s) of interest, such as epidemiological outcome, with standard statistical methods. For example, trend analysis [32] produced a clear exponential trend with a similar dataset of epidemiological cases (Fig. 4).

In Fig. (4a) (left), the epidemiological outcome shows a clear exponential trend with respect to the sum of identified main factors of influence, like policy, connectivity, and immunization. Fig. (4b) (right) shows the logarithmic outcome with a clear linear trend. A number of the outlier data points that are present in the linear diagram on the right can be explained by the influence of other factors in the data, factors that have not yet been identified, and/or statistical fluctuations.