openmoa.classifiers

Full name: Feature Evolvable Streaming Learning  ·  Hou et al., NeurIPS 2017

FESL detects feature-space shifts by measuring the Jaccard distance between consecutive active feature index sets (threshold < 0.8). It maintains two sparse linear models (w_curr, w_old). During a transition, it accumulates instances in an overlap buffer and learns a linear mapping matrix M (via Ridge Regression) projecting new features onto old ones. Final prediction is a weighted ensemble: y = μ_curr · f_curr(x) + μ_old · f_old(M·x).

FESLClassifier(
    schema: Schema,
    alpha: float = 0.1,
    lambda_: float = 0.1,
    window_size: int = 100,
    random_seed: int = 1,
)

Full name: Online Active Sparse Feature Learning  ·  Chen et al., 2024

OASF uses Passive-Aggressive (PA) updates for incremental and decremental feature spaces (Theorems 1 & 2). It maintains a ring-buffer weight matrix W of size (n_features, L), applying L₁,₂-norm group sparsity shrinkage on each update. The ring buffer avoids expensive full-matrix copies. W auto-expands to handle arbitrarily high-dimensional streams.

OASFClassifier(
    schema: Schema,
    lambda_param: float = 0.01,
    mu: float = 1.0,
    L: int = 100,
    random_seed: int = 1,
)

Full name: Robust Sparse Online Learning  ·  Chen et al., SDM 2024

RSOL shares the same ring-buffer and L₁,₂ sparsity architecture as OASF but uses a significantly larger default lambda_param (50.0 vs. 0.01) for stronger sparsity, and a larger default window L=1000. Designed for robustness to feature-space changes with emphasis on aggressive feature pruning.

RSOLClassifier(
    schema: Schema,
    lambda_param: float = 50.0,
    mu: float = 1.0,
    L: int = 1000,
    random_seed: int = 1,
)

Full name: Forward-Backward Splitting  ·  Duchi & Singer, JMLR 2009

FOBOS combines a forward gradient step with a proximal operator for online regularization. Supports L1 (sparse), L2 (ridge), L1+L2 (elastic net), or no regularization. Supports binary (logistic loss) and multi-class (softmax loss). Weight matrix auto-expands for dynamic feature streams. Learning rate decays as α/√t (sqrt schedule) or α/t (linear schedule).

FOBOSClassifier(
    schema: Schema,
    alpha: float = 1.0,
    lambda_: float = 0.001,
    regularization: str = "l1",
    step_schedule: str = "sqrt",
    random_seed: int = 1,
)

Full name: Follow the Regularized Leader — Proximal  ·  McMahan, AISTATS 2011

FTRL-Proximal maintains per-coordinate adaptive learning rates via three arrays: z (gradient accumulator), n (squared-gradient sum), and w (weights). The per-coordinate learning rate is (β + √n[i]) / α + L2, enabling features updated rarely to receive larger steps. L1 sparsification: features where |z[i]| ≤ L1 are zeroed. Arrays auto-expand for dynamic feature streams.

FTRLClassifier(
    schema: Schema,
    alpha: float = 0.1,
    beta: float = 1.0,
    l1: float = 1.0,
    l2: float = 1.0,
    random_seed: int = 1,
)

Full name: Online Learning in Variable Feature Spaces with Mixed Data  ·  He et al., ICDM 2021

OVFM handles streams with mixed feature types (continuous + ordinal) and missing values caused by evolving feature spaces. It uses a Gaussian Copula EM algorithm to transform mixed data into a shared latent space, then trains dual classifiers: w_obs (observed space) and w_lat (latent space). Batch-based EM updates run every batch_size instances. The ensemble weight between the two classifiers is updated exponentially based on cumulative loss.

OVFMClassifier(
    schema: Schema,
    window_size: int = 200,
    batch_size: int = 50,
    evolution_pattern: str = "vfs",
    decay_coef: float = 0.5,
    num_ord_updates: int = 2,
    max_ord_levels: int = 14,
    ensemble_weight: float = 0.5,
    learning_rate: float = 0.01,
    l1_lambda: float = 0.0,
    l2_lambda: float = 0.01,
    sparsity_threshold: float = 0.01,
    random_seed: int = 1,
)

Full name: Online Semi-supervised Learning with Mix-Typed Streaming Features  ·  Wu et al., AAAI 2023

OSLMF extends OVFM with semi-supervised label propagation. It uses a Gaussian Copula for mixed-data imputation and a Density Peak Clustering algorithm to propagate labels from labeled to unlabeled instances within the buffer. Dual classifiers (w_obs, w_lat) are trained with pseudo-labels from density-peak assignment.

OSLMFClassifier(
    schema: Schema,
    window_size: int = 200,
    buffer_size: int = 200,
    batch_size: int = 50,
    learning_rate: float = 0.01,
    decay_coef: float = 0.5,
    max_ord_levels: int = 14,
    ensemble_weight: float = 0.5,
    l2_lambda: float = 0.001,
    random_seed: int = 1,
)

Full name: Online Random Feature Forests for Varying Feature Spaces  ·  Schreckenberger et al., AAAI 2023

ORF3V builds an independent decision stump forest for each feature. Per-feature forests predict class distributions, weighted by each feature's accuracy history. Handles dynamic feature spaces naturally — forests are initialized lazily after grace_period instances and new feature forests are created when new features appear. Stumps are periodically replaced by newly generated candidates with better Gini gain.

ORF3VClassifier(
    schema: Schema,
    n_stumps: int = 10,
    alpha: float = 0.1,
    grace_period: int = 100,
    replacement_interval: int = 100,
    replacement_strategy: str = "oldest",
    window_size: int = 200,
    delta: float = 0.001,
    compression: float = 1000,
    d_max: int = 1000,
    enable_pruning: bool = False,
    random_seed: int = 1,
)

Full name: Online Learning Deep models from Data of Double Streams  ·  Lian et al., IEEE TKDE 2024

OLD3S supports lifelong learning across multiple sequential feature-space transitions. It maintains a Variational Autoencoder (VAE) for feature extraction and a Hedge Backpropagation (HBP) MLP for classification. A reactive state machine detects three phases — STABLE, OVERLAP, STABLE_NEW — and applies knowledge distillation during OVERLAP to align new and old latent spaces. Ensemble weights between current and previous models are updated exponentially based on prediction loss.

OLD3SClassifier(
    schema: Schema,
    latent_dim: int = 20,
    hidden_dim: int = 128,
    num_hbp_layers: int = 3,
    learning_rate: float = 0.001,
    beta: float = 0.99,
    eta: float = 0.01,
    random_seed: int = 1,
)

Full name: Online Learning from Open-World Soft Sensing  ·  Lian et al., ICDM 2024

OWSS uses a bipartite graph neural network (GNN) where nodes represent features and instances, and edges encode feature-instance relationships. Learnable feature embeddings capture universal feature representations shared across instances. A Feature Reconstruction Loss (Eq. 2) aligns the latent GNN output back toward the initial embedding, preventing representational drift. Updates are batch-based using a sliding window buffer.

OWSSClassifier(
    schema: Schema,
    window_size: int = 100,
    hidden_dim: int = 32,
    learning_rate: float = 0.01,
    rec_weight: float = 0.1,
    sparsity_threshold: float = 0.05,
    random_seed: int = 1,
)

L_total = CrossEntropyLoss(logits, y) + rec_weight · MSELoss(latent_z, initial_z.detach())

Where initial_z = instance nodes before GNN refinement, and latent_z = after. The reconstruction term (Eq. 2) prevents feature embeddings from drifting arbitrarily.