Anomaly Detection: Unsupervised or Semi-Supervised?

🔑 Enhanced Key Takeaways

• •The methodology described is formally classified in academic literature as 'One-Class Classification' (OCC), where the model learns a decision boundary around the target class to reject outliers.
• •The use of labeled validation data for threshold tuning introduces a 'leakage' of supervision, which is why many researchers argue this approach is technically 'weakly-supervised' rather than purely unsupervised.
• •Modern implementations often utilize Deep SVDD (Deep Support Vector Data Description) or Autoencoder-based reconstruction error, where the latent space representation is optimized to minimize the volume of the hypersphere containing normal data.

🛠️ Technical Deep Dive

• •Architecture: Typically employs Autoencoders (AE), Variational Autoencoders (VAE), or Generative Adversarial Networks (GANs) where the generator is trained to reconstruct normal inputs.
• •Loss Function: Often utilizes Mean Squared Error (MSE) for reconstruction-based models, or a custom hypersphere loss function in Deep SVDD to minimize the distance of normal samples to a center point.
• •Thresholding: Post-training, the anomaly score is calculated as the reconstruction error or distance from the hypersphere center; a validation set is then used to find the threshold that optimizes the F1-score or Precision-Recall AUC.
• •Data Requirements: Requires a clean dataset of 'normal' samples; contamination of the training set with anomalies significantly degrades the decision boundary.

🔮 Future ImplicationsAI analysis grounded in cited sources