Egocentric data for physical AI

Real first-person work for training robots.

Clips with video, hand pose, head motion and annotation, recorded by the people who do the work and licensed in writing. Depth is measured on devices with LiDAR.

Request a sample See what's in a clip
  • Video, hands, IMU, pose and steps
  • JSON, LeRobot or MCAP
  • Per-clip consent

Data from peopledoing the real work

Robots learn by watching hands at work: cutting, fitting, folding, stacking. We pay people to film what they already do, at the real pace of a shift, and turn it into data a lab can use on Monday.

What's in a clip

Five layers from the same device, synchronized to one clock. Every clip ships ready for training.

  • Hands chopping vegetables on a cutting board, seen from the cook's point of view
    Video

    First person

    Head-mounted camera, landscape, uncut. What the person sees is what the robot will see.

  • Hands fitting a part, with joint keypoints drawn on top
    Pose

    Hands and wrists

    Twenty-one keypoints per hand, frame by frame, each with its confidence score.

  • Hands stacking boxes in a warehouse, with the head trajectory traced
    Motion

    Head and body

    Accelerometer and gyroscope at 100 to 200 Hz depending on the phone, aligned to the video on the same clock.

  • Workshop bench shown as a depth map, from near to far
    Depth

    Distance per pixel

    Measured by LiDAR on devices that have it. Estimated in processing on the rest.

  • Hands folding laundry, with the task annotation written alongside
    Annotation

    Task, step and object

    Every clip states what was done, in what order and with what. In JSON, LeRobot or MCAP.

Anatomy of a clip

Six tracks on one clock. Hover or tap a track to see its file and columns.

slice_onion · kitchen00:00.0

Metadata

Everything that travels with the video, on the same clock. This is what turns a recording into training data.

Video
1920×1080, 30 frames per second, landscape, no stabilization. Audio included.
Frames
Sensor timestamp, actual exposure and ISO for every frame.
Motion
Accelerometer, gyroscope, gravity and device attitude at 100 to 200 Hz depending on the phone, on the video clock.
Calibration
Camera intrinsics, field of view, distortion and the camera-to-IMU rotation.
Hand pose
21 keypoints per hand in every frame, with a confidence score per keypoint.
Camera pose
Head position and orientation across the clip, in meters, with tracking state.
Depth
Measured by LiDAR on devices that have it, at 256×192. Estimated in processing on the rest.
Annotation
Task, step and object, with the start and end of each step.
Session
Task, environment type, country, device model and how it was mounted.
Quality
The review result for each clip.
License
The recorder's consent, privacy and license, attached to every clip.
Illustrative example of a clip
eh_01JC8Z.../
  rgb.mp4            1920x1080, 30 fps
  frames.csv         timestamp_sensor_ns, exposure_ns, iso
  imu.csv            accel, gyro, gravity, q_wxyz
  pose.csv           camera-to-world, tracking_state
  calibration.json   intrinsics, distortion, camera_to_imu
  steps_marked.csv   t0, t1, source
  qc_live.json       hands, light, dropped_frames
  metadata.json
  depth/             LiDAR only (256x192, mm)

{
  "schema_version": "0.1.0",
  "task": "slice_onion",
  "environment": { "type": "kitchen", "indoor": true },
  "device": { "class": "smartphone", "mount": "head_strap" },
  "sensors": { "imu": { "rate_hz": 199.6, "sync": "video_locked" } },
  "camera": { "fx": 1450.1, "fy": 1450.1, "fov_h_deg": 68.2 },
  "consent": { ... }, "qc": { ... }, "license": { ... }
}

Where we film

Real manual work, where it happens. Each sector with its own tasks, tools and mess.

  • Hands portioning meat in a professional kitchen
    Cut and portion

    Kitchen

    Prep, plating, dishes.

  • Hands tightening a bolt with a wrench in a workshop
    Tighten and fit

    Workshop

    Assembly, part replacement, fastening.

  • Hands picking products from a warehouse shelf
    Pick and pack

    Warehouse

    Picking, packing, labeling.

  • Hands picking fruit in a field
    Harvest and sort

    Field

    Harvesting, sorting, crating.

  • Hands plugging in a connector on an assembly line
    Assemble and check

    Assembly

    Fitting, cabling, testing.

  • Hands folding a towel on a bed
    Fold and put away

    Home

    Laundry, dishes, tidying.

How it works

From briefing to delivery in five steps. The first sample arrives before any larger contract.

  1. Briefing

    You tell us the task, the environment and the format. We come back with a capture plan in two days.

    • Tasks
    • Environments
    • Volume
    • Format
    Notebook with a capture task list, seen from above

  2. Pilot

    A few hours recorded, reviewed and delivered in your format. You test it in your pipeline before scaling.

    • Sample
    • Review
    • Adjustments
    Phone on a head strap on a table, next to a notebook

  3. Scale

    Recorders recruited by sector, trained and recording on real shifts. Hundreds of hours per month.

    • Recruiting
    • Training
    • Real shifts
    Mosaic of recorders' hands in kitchens and workshops

  4. Review

    Every minute goes through automated and human checks before it counts. Faces and personal data are removed before delivery.

    • Light
    • Hands in frame
    • Sensors
    • Privacy
    Review screen with a clip and its check marks

  5. Delivery

    Clips in JSON, LeRobot or MCAP, in your bucket or ours. Written license per clip, with consent attached.

    • JSON
    • LeRobot
    • MCAP
    • Bucket
    Dataset folders open on a screen, with video, pose and annotations side by side

The platform

From the recorder's phone to your lab's bucket, every clip rides one line, and we own all of it: capture, enrichment, review and delivery. A complete infrastructure, ready to scale anywhere in the world.

  1. Recorder web app

    In the browser

    Recorders pick an activity, reserve a slot and follow their submissions and earnings.

    Activity accepted

  2. Capture app

    iOS and Android

    Records 1080p30 H.264 with synchronized IMU and device metadata. Guides the task step by step and warns live when hands leave the frame or the light is too low.

    Video + IMU

  3. Enrichment server

    Processing pipeline

    Adds the layers on top of the raw clip: hand pose, 6-DoF camera pose, depth, steps and a quality score. The privacy pass blurs faces, license plates and personal data in text.

    + pose, depth, steps

  4. QA and operations backoffice

    EgoHands team

    Our team reviews every delivery before it ships and decides the privacy cases that are in doubt.

    Reviewed by a person

  5. Partner portal

    partnerdata.egohands.com

    The lab places orders, follows progress batch by batch and receives deliveries in LeRobot, MCAP or JSON.

    Delivered to the lab

A person reviews every delivery before it ships.

Request a sample

Field notes

Head strap with a phone, side view, on a workbench Frame from a clip with the hands leaving the shot A person's face in the background, blurred, behind working hands Three data format folders side by side on a screen

  1. Method · · 4 min

    Why we record in first person, not on a tripod

    The robot won't have a camera in the corner of the room. It will have the view of the person doing the task.

  2. Quality · · 5 min

    What review rejects, and why

    Light, hands out of frame, a missing sensor. Six fixed reasons, always explained to the recorder.

  3. Privacy · · 3 min

    How we protect people who end up in the shot

    Faces blurred before delivery, a notice on site and the right to withdraw a clip at any time.

  4. Format · · 6 min

    JSON, LeRobot or MCAP: which to ask for

    All three come from the same clip. The difference is what your pipeline already reads.

Questions

How much does a dataset cost?

It depends on the sector, the volume of hours and whether it is exclusive. We send a proposal after the briefing, together with the sample.

How does a project start?

With a one-hour briefing and a pilot of a few hours recorded in your format. You test it in your pipeline before signing anything larger.

What format does it come in?

JSON, LeRobot or MCAP, in your bucket or ours. All three come from the same clip, with video, hands, motion and steps on the same clock. Depth is included in clips recorded with LiDAR.

Did the people in the video consent?

Recorders sign a consent form in the app, per campaign, and can withdraw consent at any time. Third-party faces are blurred before delivery and a notice is posted at the recording site.

Where do you operate?

Worldwide. Recorders film where the work happens and delivery is remote, to your bucket or ours.

Let's talk

Tell us the task your robot needs to learn. A person reads every message and replies within two business days.

partner@egohands.com
EgoHands
A product of Codemart LLC
30 N Gould St # 39741
Sheridan, WY 82801, USA
Operations
Recorders where the work happens
Remote delivery, worldwide