Open-source · Python · ecosystem agnostic

Sensor synchronization you can inspect.

embodied-sync is an open-source Python library for synchronizing and validating multimodal robot-learning data. Align cameras, robot state, actions, audio, force, and tactile streams across different rates and clock domains—then see the skew, missing samples, stale observations, and alignment choices before they reach a policy.

Install from PyPI View source

pip install embodied-sync

One timing model

Recorded datasets and live robots fail differently

Align robot-learning datasets

Build policy-rate frames with nearest-neighbor, zero-order hold, linear interpolation, or a per-stream policy. Every chosen sample keeps its skew, confidence, and missing status.

Python dataset synchronization guide →

Validate timestamps and sync quality

Measure missing rate, median and tail skew, source frequency, and alignment decisions. Generate portable HTML and JSON reports for dataset review or CI gates.

Validate a LeRobot dataset →

Stress-test latency and jitter

Inject known fixed latency, receive jitter, clock drift, dropped frames, burst stalls, duplicates, non-monotonic delivery, and missing intervals into deterministic recordings.

Test robot-policy timing robustness →

Handle clocks and late packets

Keep acquisition time separate from receive time, map device clocks explicitly, and synchronize causally without assuming all timestamps share an epoch.

Multi-clock and late-packet guide →

Fits around the robotics tools you already use

embodied-sync normalizes timing without requiring ROS or replacing the recorder, dataset format, visualization stack, or sensor SDK you already chose.

LeRobot v3 ROS 2 / rosbag2 MCAP LSL / XDF UMI-style data Rerun SurgSync Custom datasets

Scope matters: this is temporal calibration and alignment, not camera intrinsics, extrinsics, or LiDAR-to-camera spatial calibration. Adapter depth varies by format; the current-scope table documents what works today.

A synchronization workflow with evidence

Preserve both times. Keep when a measurement was acquired distinct from when its packet arrived.
Map clock domains. Estimate offset and drift from physical events instead of treating unrelated device nanoseconds as comparable.
Choose alignment semantics. Select causal or offline alignment and a method appropriate to each modality.
Validate the result. Review skew distributions, missingness, staleness, and the actual neighboring frames the aligner rejected.
Rehearse failures. Apply known timing corruptions and test how preprocessing, observation assembly, and policy evaluation respond.

Start with a reproducible timing failure

embsync synth --out runs/clean --seed 0 --duration-s 10
embsync corrupt runs/clean \
  --profile configs/corrupt_kitchen_sink.yaml \
  --out runs/bad
embsync align runs/bad --out episodes/bad_10hz \
  --target-rate-hz 10 --check-ground-truth
embsync report episodes/bad_10hz \
  --out reports/bad.html --json-summary reports/bad.json

The corruption ground truth makes the example testable: you know what delay, drift, drops, or stalls were introduced, and can compare that truth with the aligned episode and sync-quality report.

Is embodied-sync the right library?

Need	Fit	Why
Validate temporal alignment in a robot-learning dataset	Yes	Reports retain per-frame skew, confidence, missingness, and alignment policy.
Synchronize heterogeneous live Python callbacks without ROS	Yes	`SyncSession` assembles causal bundles and surfaces late, stale, or unmapped streams.
Estimate device-clock offset or drift	Yes	Clock-domain mappings can be fitted from event trains, claps, or visual timestamps.
Simulate inference latency and network jitter	Partial	The corruption engine changes recorded timing for reproducible offline tests; it is not a network or physics simulator.
Calibrate camera intrinsics or sensor extrinsics	No	Use a spatial calibration toolbox; embodied-sync focuses on time.
Hard real-time scheduling or PTP configuration	No	The library validates and aligns data but does not configure hardware clocks or promise real-time deadlines.

Frequently asked questions

Does embodied-sync require ROS?

No. The core is a Python library with NumPy and PyYAML dependencies. ROS 2 recordings can enter through MCAP, but live synchronization also works around ordinary SDK callbacks and polling loops.

Can it check LeRobot dataset synchronization?

Yes. The LeRobot v3 path preserves timestamps and episode boundaries, produces a normalized run, and supports alignment plus HTML or JSON sync-quality reports.

How does it handle clock skew and clock drift?

Each stream declares a clock domain. A mapping relates source time to reference time with an offset and, when the evidence supports it, a rate term for drift. Unmapped clocks are treated as a quality violation rather than silently compared.

Does it support LSL clock synchronization?

It reads LabRecorder XDF and retains LSL time-correction information so corrected and raw timing can be inspected. LSL performs online clock synchronization; embodied-sync adds replay, robot-learning alignment, and validation around the recording.