I nearly fooled myself validating a wearable IMU classifier — here's the bug and the fix

Most of the validation work on set (Altun, Barshan & Tunçel, 2010): 8 subjects, each wearing five Xsens IMU units — torso, both arms, both legs — 9 axes per unit (accelerometer, gyroscope, magnetometer × x/y/z), sampled at 25Hz. 45 channels total. It includes both a sedentary activity (sitting) and dynamic sport activities (basketball, rowing), which gives a clean, checkable question: does a classifier that's never seen this data correctly tell apart "person sitting still" from "person playing basketball," using channel statistics alone?

import pandas as pd



# Mirrored subset: github.com/AniMadurkar/Daily-Activities-and-Sports-Biomechanics-Analysis

df = pd.read_csv("sports_science_dataset_subset.csv")

channels = [c for c in df.columns if c not in ("subject", "activity", "timestamp")]

print(len(channels), "channels")  # 45

First attempt — and the mistake

My first pass pooled all 8 subjects together per activity and ran it through the profiler in one shot. The result came back backwards: sitting showed up with more "significant" channels than basketball. That's not just unexpected, it's physically nonsensical — a person sitting still should be one of the lowest-variance activities in the entire dataset.

The bug wasn't in the classifier. It was in the test. Pooling subjects together means each subject's own sensor baseline and IMU orientation differences get mixed into the between-subject variance for every channel — including sitting, where there's no real movement to swamp that noise. For basketball, the actual movement signal is large enough to dominate regardless. Pool the data and you're accidentally measuring "how different are these 8 people's resting sensor baselines" more than "how much does this channel move during this activity."

# Wrong: pools all subjects, cross-subject baseline noise contaminates

# the low-motion activity's variance more than the high-motion one

df_sitting_pooled = df[df["activity"] == "sitting"][channels]

The fix: one continuous stream per subject, per activity

That's also just what a real deployment looks like — one wearable, one athlete, one continuous stream. So I reran it that way: for each of the 8 subjects independently, feed their sitting stream and their basketball (or rowing) stream through the profiler as two separate single-device sessions.

from vaasx.bootstrap import StatisticalProfiler, SchemaClassifier



def profile_subject_activity(df, subject_id, activity):

    subset = df[(df["subject"] == subject_id) & (df["activity"] == activity)]

    profiler = StatisticalProfiler(device_id=f"subject_{subject_id}_{activity}")

    for _, row in subset[channels].iterrows():

        profiler.observe(row.to_dict())

    return SchemaClassifier().classify(profiler.snapshot())



results = {}

for subject_id in df["subject"].unique():

    for activity in ["sitting", "basketball", "rowing"]:

        results[(subject_id, activity)] = profile_subject_activity(df, subject_id, activity)

The result, once the methodology was fixed

Across all 8 subjects, sitting left an average of 14.8 of 45 channels classified stable, 22.4 flagged significant. Basketball and rowing collapsed to zero stable channels in every single subject, with 40–44 of 45 flagged significant every time. Consistent, in the right direction, in all 8 people — with no channel labels or activity hints ever given to the classifier.

Check it against something outside the SDK entirely

import numpy as np



def independent_variance(df, subject_id, activity):

    subset = df[(df["subject"] == subject_id) & (df["activity"] == activity)][channels]

    return subset.var().mean()



sitting_var = np.mean([independent_variance(df, s, "sitting") for s in df["subject"].unique()])

basketball_var = np.mean([independent_variance(df, s, "basketball") for s in df["subject"].unique()])

rowing_var = np.mean([independent_variance(df, s, "rowing") for s in df["subject"].unique()])



print(f"basketball / sitting: {basketball_var / sitting_var:.0f}x")

print(f"rowing / sitting: {rowing_var / sitting_var:.0f}x")

Plain pandas .var(), nothing from the SDK. Basketball came out roughly 154x higher variance than sitting, rowing roughly 27x — computed completely independently of whatever the classifier decided. The classifier's "significant vs. stable" split is tracking a real, independently measurable physical signal, not an artifact of how it happens to score things.

What this doesn't show

I want to be as clear about this here as the whitepaper is: this validates that zero-config channel profiling correctly separates a sedentary activity from a dynamic one, on real wearable data, in a domain the classifier was never tuned for. It does not validate injury prediction, movement-quality scoring, or that a flagged pattern means anything clinically. That's a much harder, differently-evidenced claim, needs labelled injury-outcome data specific to a sport and population, and I haven't tested it — so I'm not claiming it.

The mistake in my first pass is the part I'd actually want another engineer to take away from this: pooling data across subjects/devices/sessions before checking a per-entity classification claim is an easy way to quietly validate the wrong thing, and get a plausible-looking wrong answer out the other end.

Full runnable steps, same code as above, against the real UCI zip (not a mirror): . If anyone reproduces this against activities I didn't test, I'd like to know if the pattern holds outside basketball and rowing.