Not yet tested.

python_Factor_Analyzer

This project is to implement methods about factor analysis.

• Rotator
  • Varimax rotation.
  • Promax rotation.
• Factor extraction method
  • Maximum-likelihood estimation.

Usage

See programme comments.

Requirement

• Python 3.8 =<
• numpy

pip install numpy

Algorithm

Rotator

Varimax Rotation

This section describes the implementation of Varimax rotation. Given a matrix $\Phi$, representing the initial loadings, Varimax rotation aims to produce a rotated matrix $\Lambda$. The rotation matrix used for this transformation is denoted as $R$.

Therefore, the relationship between $\Phi$, $R$, and $\Lambda$ is given by:

$$\Lambda = \Phi R$$

The objective is to find the rotation matrix $R$ that maximizes the variance of the loadings in $\Lambda$.

Solution Method

The gradient ascent method is employed to solve for $R$. The gradient of $\Lambda$ is defined as:

$$A = \Lambda ^3 - \frac{\gamma}{p} \Lambda (\text{diag}(\Lambda^T \Lambda))$$

Note:

• $\Lambda^3$ refers to each element of $\Lambda$ raised to the power of 3. It amplifies each loading in $\Lambda$.
• $\text{diag}(\Lambda^T \Lambda)$ is the diagonal matrix obtained from the product of $\Lambda^T$ and $\Lambda$, enhancing the diagonal elements of $\Lambda$.
• $p$ represents the number of variables. The term $\frac{1}{p}$ scales down the components of $\Lambda \text{diag}(\Lambda^T \Lambda)$.
• $\gamma$ acts as a scaling factor.

Next, decompose $\Phi^T A$ into singular values:

$$A = U\Sigma V^T$$

Here, $U$ and $V$ are the orthogonal matrices from the Singular Value Decomposition (SVD), representing the structure of $A$.

The rotation matrix $R$ is updated as $R = UV^T$. The process iterates until $R$ converges.

Promax Rotation

In the context of Promax rotation, let's denote the Promax loadings as $\Lambda^\kappa$ (commonly, $\kappa$ defaults to 4).

The rotation matrix $T$ is initialized as an identity matrix.

The updating rule for $T$ is as follows:

$$ T_{next} = (\Lambda^\kappa)^T \Lambda T \text{diag}\left(\frac{1}{(\Lambda^\kappa)^T \Lambda T}\right) $$

This equation adjusts the correlation matrix $(\Lambda^\kappa)^T \Lambda$ and normalizes the scale of $T$.

The iteration continues until the difference between $T_{next}$ and $T$ falls below a certain threshold, indicating convergence.

Factor extraction method

Maximum-Likelihood Estimation (MLE)

The MLE approach in Factor Analysis aims to find the factor loadings (matrix $\Lambda$) and unique variances (vector $\Psi$) that maximize the likelihood of the observed data given the model.

Steps:

1. Standardization:

   • Standardize the input data $X$ by subtracting the mean of each variable.

2. Initialization:

   • Initialize the factor loadings matrix $\Lambda$ with random values. $\Lambda$ has dimensions $p \times m$, where $p$ is the number of variables and $m$ is the number of factors.
   • Initialize the unique variances vector $\Psi$ with random values. $\Psi$ has length $p$.

3. Iteration:

   • E-step (Expectation step):

     • Compute the expected factor covariance matrix: factor_cov $= \Lambda \Lambda^T + \text{diag}(\Psi)$

     • Calculate its inverse: factor_cov_inv $=$ (factor_cov) $^{-1}$

   • M-step (Maximization step):

     • Update factor loadings $\Lambda$: $\Lambda_{\text{new}} = S \Lambda \times$ factor_ cov_ inv , where $S$ is the sample covariance matrix of the observed data.
     • Update unique variances $\Psi$: $\Psi_{\text{new}} = \text{diag}(S) - \text{diag}(\Lambda_{\text{new}} \Lambda_{\text{new}}^T)$

   • Check for convergence. The algorithm stops if the change in $\Lambda$ and $\Psi$ is smaller than a pre-defined tolerance level.

4. Output:

   • The factor loadings matrix $\Lambda$, which represents the relationship between observed variables and underlying latent factors.
   • The unique variances $\Psi$, which represent the variances in the observed variables not explained by the common factors.
   • Communalities, which represent the proportion of variance in each observed variable that is accounted for by the common factors.

The MLE approach for Factor Analysis seeks the factor loadings and unique variances that best explain the observed covariance (or correlation) matrix of the input data.

Directory

.
├── README.md
├── sandbox
│   └── README.md
└── src
    └── rotator.py