NaoX TechnologiesDeepWave2
Local recordingPrototype · 0.8.07 Physiological Binary

MULTIMODAL MONITORING

Video, In-Ear EEG & Attention/Behavior Monitoring

Record NaoX in-ear EEG and facial video in the same session.

Connect an earbud and enable the camera to prepare a session.
Runtime · waiting for JavaScript…

RECORDING CONTROL

Record synchronized EEG + video · select a protocol after starting
Face calibration not yet saved
00:00Waiting
MAIN MODES
CALIBRATION / ADVANCEDShow calibration protocols
⚙ Difficulty Medium · 16 × 16 Choose before starting the task.
Start recording to enable the selected protocol or model.

02 / FACIAL VIDEO

Facial context

Eye visibility · not evaluated Camera off
Camera image settings (hardware controls where supported)

Enable camera to check which hardware settings are available.

Important: Camera hardware settings change the real feed, including VideoAI. Preview brightness/contrast only change what you see, not the recorded pixels or the raw images analysed by MediaPipe. Adjust hardware and lighting before starting a session.

◉

Your camera stays on this device

Enable it to check framing.
No video is uploaded to a server.

SIGNAL QUALITY

Check contact

Waiting
Average quality—First estimate after 5 s of signal.

Waiting for signal…

DeepWave index: amplitude, RMS and 1/f slope (exploratory limit −4 to −0.2) · 5 s window · average of 10 estimates. Recording remains available with degraded signal so artefacts can be studied.

03 / LIVE EEG

Live signal

Display filter: PITA Default 0.5–35 Hz · changing a cutoff switches to Custom · left-to-right sweep

04 / SPONTANEOUS RECORDING

Free behaviour recording

Not started

Record natural behaviour without instructions, prompts or a predefined task duration. No behavioural label is created during acquisition: facial expressions are proposed afterwards by the local AI on the review page.

Ready after EEG + video recording starts.

The segment has no experimental instruction and no fixed task duration. The prototype's general session safety limits still apply.

05 / SINGLE BEHAVIOR CALIBRATION

Single behavior calibration

Not started

Record one behavior at a time. DeepWave2 alternates Neutral and the selected behavior, then Review estimates whether the behavior is separable from rest in Video AI features and in the two in-ear signals.

Ready after EEG + video recording starts.

Review uses the same VideoAI → EEG learning workflow as all other sessions. The experimental Neutral / Behavior instructions are retained as independent reference, never used directly as EEG training labels. Automatic EEG Behavior teaching currently supports the 15 VideoAI categories; arbitrary FACS/custom actions are recorded but do not automatically create a new VideoAI class.

06 / SINGLE COGNITIVE CALIBRATION 1 · FOCUS

Single cognitive calibration 1: Focus

Not started

Focus 1: eyes-open relaxation (calm landscape image), conjunction visual search (find the one green X among green O, red X and red O; then keep looking at it), and eyes-closed rest. No clicking or hand movement. Experimental task conditions are saved separately from the VideoAI-derived EEG teacher labels.

2 s transitions between conditions; 1 s target reveal after search. High single BIP = start, two low BOP-BOP cues = end. Transitions and feedback are excluded from analysis; no mouse or keyboard response is needed. Start EEG + camera recording first.

Ready after EEG + video recording starts.
Coming later · SART protocol to be defined

07 / STANDARDIZED BEHAVIOR CALIBRATION

Standardized behavior calibration

Not started

Fifteen standardized conditions with automatic start/end markers and audio cues. Planned task time: 9 min 21 s (plus a 3 s preparation countdown).

1 · Neutral / relaxed · 20 s 2 · Repeated blinking · 4 × 5 s 3 · Eyes left · 4 × 4 s 4 · Eyes right · 4 × 4 s 5 · Eyes up · 4 × 4 s 6 · Eyes down · 4 × 4 s 7 · Eyes closed · 20 s 8 · Eyebrow raise · 4 × 5 s 9 · Smile · 4 × 5 s 10 · Jaw clench · 4 × 5 s 11 · Open mouth · 4 × 5 s 12 · Talking · 4 × 10 s 13 · Nodding · 4 × 5 s 14 · Head left · 4 × 4 s 15 · Head right · 4 × 4 s

Repeated actions are separated by 5 s of neutral relaxation. Eyes-closed timing is controlled by beeps; do not count mentally.

Ready after EEG + video recording starts.

08 / IN-EAR LIVE RECOGNITION

Personalized in-ear classifier

No model

Uses only the in-ear signal(s) required by the loaded model for the live decision. A unilateral model uses one earbud; a bilateral model uses both. The video may remain recorded for later validation but is not an input to this classifier.

All classes: report the top class only when confidence and separation from the runner-up are high enough; otherwise report UNKNOWN.

Current prediction — Confidence —
No personal model saved in this browser.

No model is loaded. Use Model → Use EEG model in Session, or Import model JSON.

In Focus class mode the live output has three explicit states: TARGET DETECTED for the selected target, OTHER BEHAVIOR / OTHER when another known class is confidently preferred, and NO CONFIDENT DECISION / UNKNOWN when the decision is ambiguous. Live recognition requires an active recording session. For a unilateral model, only the trained side is required; a bilateral model requires both earbuds. Device 1 is treated as Left and device 2 as Right.

Understand synchronization and exports

Bluetooth arrivals, rendered frames and protocol/spontaneous segment events use the same browser clock. Render time is burned into the recorded video. It is not exactly the camera exposure time.

EEG device times and arrival times are preserved. Missing samples are not invented. Absolute camera–EEG alignment still requires calibration; this prototype does not guarantee millisecond accuracy. Rendered frames are not an exact index of encoded frames.

Keep this tab visible during acquisition. Moving it to the background stops the session to avoid continuing without reliable video. Interrupted sessions remain exportable, but an incomplete video may require repair.

The video carries a t=…s render time and frame counter. They are also stored in video-frames.jsonl. session.json describes clocks and limits. Version 0.6 keeps portable ZIP transfer and a synchronized Session/Review workflow. PITA Default display filtering is available in both acquisition and Review; raw EEG remains unchanged. Left EEG, right EEG and video/protocol labels share the browser session timeline after per-device clock fitting. Exported EEG values remain raw, with no EDF conversion.

QUICK GUIDE

Technical parameters

EEG acquisition

Sampling: 500 Hz per connected earbud. For paired modeling, the first connected device is Left and the second is Right; use Swap L/R before recording if necessary.

Raw received samples are stored without interpolation.

Live display filter: PITA Default is used by default: a 35 Hz Kaiser FIR low-pass (80 dB, 5 Hz transition) followed by a 0.5 Hz running-baseline subtraction. Raw and Custom display modes are also available. Display filtering never changes stored EEG, QC, EDF or model features.

Signal-quality index: 5 s windows, FFT 2–30 Hz for amplitude/RMS, 1/f slope from 2–30 Hz, averaged over the latest 10 estimates.

Video & synchronization

Video: facial video without audio, 720p or 1080p requested resolution.

Timing: PITA retains the 32-bit counter, learns packet intervals and aligns sample positions with detected gaps; BLE arrival, rendered video frames and protocol/spontaneous events share the browser clock. Review fits each earbud clock independently and rejects excessive drift/boundary mismatch/data loss.

Recommendation: create a visible/EEG reference event near the beginning and end of a session when fine alignment is required.

Standardized behavior calibration

15 conditions: neutral, repeated blinking, eyes left, eyes right, eyes up, eyes down, eyes closed, eyebrow raise, smile, jaw clench, open mouth, talking, nodding, head left and head right.

Actions are time-locked with start/end beeps and stored as experimental intervals. Repeated actions use 5 s neutral recovery periods. Eyes closed lasts 20 s without mental counting.

Talking: a fixed French passage is displayed for standardization.

Single Behavior Calibration

Explore one behavior at a time with alternating Neutral and Behavior blocks. Presets include the 11 active standardized behaviors, FACS-inspired facial actions and free custom behaviors.

Review keeps experimental task labels separate from VideoAI teacher labels; the EEG models learn exclusively from accepted VideoAI labels.

Spontaneous recording

Free behaviour recording has no prompts and no predefined task duration. DeepWave2 stores only the segment boundaries; it does not assign behavioural labels during acquisition.

After recording, the local facial AI proposes expression intervals on the review page for human validation.

Local facial AI

Model: MediaPipe Face Landmarker 1.1.1, running in the browser. It provides facial landmarks and blendshapes; DeepWave2 derives compact temporal features and heuristic prelabels.

Prelabel categories: the same 15 categories as the guided protocol: neutral, blinking, eye_gaze_left, eye_gaze_right, eye_gaze_up, eye_gaze_down, eyes_closed, eyebrow_raise, smile, jaw_clench, open_mouth, talking, nodding, head_left, head_right. Face tracking must pass quality control for automatic labels to be accepted by default.

Important: experimental protocol labels and AI prelabels are kept separate for review.

Personal in-ear classifier

After VideoAI labeling of any session type, Review can train a lightweight personalized model from the in-ear EEG channels that pass the existing signal/timing QC. Two valid earbuds produce a bilateral model; one valid earbud produces a unilateral model.

On a later recording, live recognition uses only the required in-ear signal(s). Video is retained only for validation.

Review, export & transfer

After recording, the review page synchronizes video, EEG, landmark-derived features and behavioural intervals with a common cursor.

Use Download Session to transfer a complete session to another phone or computer. On the destination device, choose Import Session; the session is restored into that browser and can be reviewed normally.

Human corrections can be saved locally and exported as video-labels.csv, video-ai-features.csv and ai-review.json.