Concept Drift Detection

OpenMOA provides a complete concept drift detection framework covering 12 detectors, drift stream construction, drift-aware data generators, and evaluation tools for detector performance — all integrated with the broader stream learning pipeline.

1. What is Concept Drift?

In data streams, the underlying data distribution can change over time — a phenomenon known as concept drift. When drift occurs, a learner's previously acquired knowledge may become outdated, leading to degraded prediction performance.

OpenMOA addresses this through a dedicated drift detection module (openmoa.drift) that provides:

2. Common Detector Interface

All detectors share the same base interface defined in BaseDriftDetector. You call add_element() on each new data point, then query detection state.

Base Class Attributes

Base Class Methods

Basic Usage Pattern

from openmoa.drift.detectors import ADWIN

detector = ADWIN(delta=0.001)

for i, (x, y) in enumerate(stream):
    y_pred = learner.predict(x)

    error = int(y_pred != y)
    detector.add_element(error)

    if detector.detected_change():
        print(f"Drift detected at instance {i}")
        learner.reset()
    elif detector.detected_warning():
        print(f"Warning at instance {i}")

    learner.train(x, y)

print(detector.detection_index)   # [1011, 3025, ...]
print(detector.warning_index)     # [985, 2990, ...]

3. Drift Detector Catalogue

Maintains an adaptive window that automatically shrinks when it detects that two sub-windows have statistically different means. Non-parametric and parameter-light.

ADWIN(delta=0.002)

Monitors the error rate of a classifier. Drift is detected when the error rate increases significantly beyond the minimum observed error rate, using statistical control limits.

DDM(min_n_instances=30, warning_level=2.0, out_control_level=3.0)

Accumulates deviations from a target mean. When the cumulative sum exceeds threshold lambda_, drift is flagged and the counter resets.

CUSUM(min_n_instances=30, delta=0.005, lambda_=50)

Applies exponential smoothing to the error stream. Drift is detected when the smoothed value exceeds a control limit.

EWMAChart(min_n_instances=30, lambda_=0.2)

Uses a geometric moving average with a forgetting factor to track the error rate. Drift is flagged when the average exceeds a threshold.

GeometricMovingAverage(min_n_instances=30, lambda_=1.0, alpha=0.99)

Applies Hoeffding's inequality to bound the probability that the error rate has increased significantly from its historical minimum, using cumulative average statistics.

HDDM_A(drift_confidence=0.001, warning_confidence=0.005, test_type="Two-sided")

Weighted variant of HDDM_A. More recent observations receive higher weight via an exponential decay factor lambda_.

HDDM_W(drift_confidence=0.001, warning_confidence=0.005, lambda_=0.05, test_type="Two-sided")

Extra attributes: .estimation (current mean estimate), .delay (detection delay)

Sequential change-point detection using cumulative sums with both upper and lower bounds. Drift is flagged when the difference between the maximum cumulative sum and the current value exceeds the threshold.

PageHinkley(min_n_instances=30, delta=0.005, lambda_=50.0, alpha=0.9999)

An extension of DDM that resets when a long sequence of instances passes without drift detection, preventing false adaptation to temporary fluctuations.

RDDM(
    min_n_instances=129,
    warning_level=1.773,
    drift_level=2.258,
    max_size_concept=40000,
    min_size_concept=7000,
    warning_limit=1400
)

Detects drift by monitoring volatility shifts in the error stream using block-level entropy. Suited for detecting changes in error variability rather than just mean shift.

SEED(delta=0.05, block_size=32, epsilon_prime=0.01, alpha=0.8, compress_term=75)

Uses a statistical test comparing the proportion of correct predictions in a recent window versus the overall accuracy to date.

STEPD(window_size=30, alpha_drift=0.003, alpha_warning=0.05)

Unique among OpenMOA detectors — ABCD operates on raw feature vectors (not prediction errors). It trains a dimensionality reduction model (AutoEncoder, PCA, or Kernel PCA) and monitors reconstruction loss over time using the Bernstein statistical test. It can also identify which features are drifting.

ABCD(
    delta_drift=0.002,
    delta_warn=0.01,
    model_id="ae",              # "ae", "pca", or "kpca"
    split_type="ed",
    encoding_factor=0.5,
    update_epochs=50,
    num_splits=20,
    max_size=np.inf,
    subspace_threshold=2.5,
    n_min=100,
    maximum_absolute_value=1.0,
    bonferroni=False
)

Additional Methods (ABCD only)

from openmoa.drift.detectors import ABCD
import numpy as np

detector = ABCD(model_id="ae", delta_drift=0.002)

for instance in stream:
    detector.add_element(instance.x)   # full feature vector, not error

    if detector.detected_change():
        print(f"Drift at feature dims: {detector.get_drift_dims()}")
        print(f"Severity: {detector.get_severity():.3f}")

4. Choosing a Detector

5. Evaluating Drift Detectors

When the true drift positions are known, OpenMOA provides EvaluateDetector to measure the quality of a detector's output.

Import: from openmoa.drift.eval_detector import EvaluateDetector

EvaluateDetector(max_delay: int)

results = evaluator.calc_performance(preds, trues)

Returned Metrics (pd.Series)

import numpy as np
from openmoa.drift.detectors import ADWIN
from openmoa.drift.eval_detector import EvaluateDetector

data = np.concatenate([
    np.random.randint(0, 2, 1000),    # Before drift: low error
    np.random.randint(3, 8, 1000),    # After drift: high error
])

detector = ADWIN(delta=0.001)
for val in data:
    detector.add_element(float(val))

evaluator = EvaluateDetector(max_delay=200)
results = evaluator.calc_performance(
    preds=detector.detection_index,
    trues=np.array([1000])
)
print(results)
# mean_time_to_detect          11.0
# missed_detection_ratio        0.0
# mean_time_btw_false_alarms    NaN
# no_alarms_per_episode         0.0

6. DriftStream — Composing Streams with Drift

Import: from openmoa.stream.drift import DriftStream, AbruptDrift, GradualDrift

AbruptDrift

AbruptDrift(position: int, random_seed: int = 1)

Instantaneous switch at a given instance index.

GradualDrift

# Specify by center position + width
GradualDrift(position=10000, width=2000, random_seed=1)

# Or specify by start + end
GradualDrift(start=9000, end=11000, random_seed=1)

Instances are probabilistically drawn from either concept over the drift window.

from openmoa.stream.drift import DriftStream, AbruptDrift, GradualDrift
from openmoa.stream.generator import SEA

stream = DriftStream(stream=[
    SEA(function=1),                           # Concept 1
    AbruptDrift(position=5000),                # Sudden switch at instance 5000
    SEA(function=2),                           # Concept 2
    GradualDrift(position=10000, width=2000),  # Gradual drift from 9000–11000
    SEA(function=3),                           # Concept 3
])

print(stream.get_num_drifts())   # 2
print(stream.get_drifts())       # List of Drift objects

RecurrentConceptDriftStream

from openmoa.stream.drift import RecurrentConceptDriftStream

stream = RecurrentConceptDriftStream(
    concept_list=[concept1, concept2, concept3],
    max_recurrences_per_concept=2,
    transition_type_template=AbruptDrift(position=2000),
)

7. Built-in Drifting Generators

from openmoa.stream.generator import RandomRBFGeneratorDrift

stream = RandomRBFGeneratorDrift(
    model_random_seed=1,
    number_of_classes=2,
    number_of_centroids=50,
    number_of_drifting_centroids=5,
    magnitude_of_change=0.5,
)

8. Drift Benchmark Datasets

from openmoa.datasets import CovtFD, RTG_2abrupt

stream = CovtFD()
stream = RTG_2abrupt()

9. Visualizing Drift in Evaluation Results

When a DriftStream is passed to prequential_evaluation, the resulting plot automatically marks drift locations.

from openmoa.evaluation import prequential_evaluation
from openmoa.evaluation.visualization import plot_windowed_results
from openmoa.classifier import HoeffdingTree

stream = DriftStream(stream=[
    SEA(function=1),
    AbruptDrift(position=5000),
    SEA(function=2),
    GradualDrift(position=10000, width=2000),
    SEA(function=3),
])

learner = HoeffdingTree(schema=stream.get_schema())
results = prequential_evaluation(stream=stream, learner=learner, window_size=500)

plot_windowed_results(results, metric="accuracy")