Autonomous Driving Data Annotation Requirements

Perception annotation is the labelling of camera, LiDAR and radar recordings so that a driving model can learn to detect, locate, classify and track the objects around a vehicle.

A single accuracy percentage across a delivery therefore tells you almost nothing. This guide specialises the companion criteria for choosing AI annotation services; the vendors that work in this category are compared in the list of top autonomous driving annotation companies.

Intersection over union (IoU) is the area where a predicted box and a reference box overlap divided by the area they cover together, so a score of 1.0 means a perfect match and 0 means no overlap at all.

IoU = Area of overlap ÷ Area of union
F1  = 2 × (Precision × Recall) ÷ (Precision + Recall)

• Cuboids and boxes — IoU thresholds, stated separately for near and far range. Distant objects are sparse in the point cloud and a single global threshold either over-penalises far range or under-measures near range. Public benchmarks set the precedent for class-specific thresholds: KITTI's 3D detection benchmark requires 70% box overlap for cars and 50% for pedestrians and cyclists.
• Segmentation — mean IoU per class, not overall. Rare classes are where the value is; an overall figure is dominated by road and sky.
• Tracking — identity persistence through occlusion, and identity-switch count per sequence. This is not measurable frame by frame, which is why vendors who audit per-frame miss it entirely.
• Classification and attributes — F1 per class, plus a confusion matrix. Which classes get confused with which is more actionable than the headline number.
• Subjective tasks — chance-corrected agreement between independent annotators, because on intent and behaviour there is no gold answer, only a defensible consensus.

Ask what the vendor does below threshold: who pays for rework, at what turnaround, and how the root cause feeds back into annotator training rather than being fixed silently batch by batch.

An operational design domain (ODD) is the set of operating conditions, including environment, geography, time of day and roadway characteristics, under which a driving automation system is designed to function.

• Lighting — daylight, dusk and dawn, night, low sun directly into the sensor, tunnel entry and exit.
• Weather — rain, spray, fog, snow, wet reflective surfaces.
• Density — empty road through dense urban traffic.
• Actors — pedestrians including children, cyclists, motorcycles, wheelchairs, animals, unusual vehicles, people carrying or pushing objects.
• Occlusion — partial, heavy, and re-appearance after full occlusion.
• Road structure — construction zones, temporary markings, unmarked roads, complex junctions, roundabouts.
• Regional variation — signage, markings, vehicle types and driving conventions differ by country, and a model trained in one region degrades in another.

Stratum coverage = Annotated sequences in stratum ÷ Target sequences in stratum

Report the minimum across strata, not the mean: the mean says the programme is on schedule; the minimum names the condition in which the model will fail.

Sensor-fusion annotation labels one scene across camera, LiDAR and radar so that every object carries a single identity and a geometrically consistent position in each modality.

Three checks worth writing into the specification:

• Cross-modal identity — the same object carries the same identity in camera and LiDAR.
• Projection consistency — a 3D cuboid projected into the image plane lands on the object. Cheap to check automatically and a reliable detector of calibration drift.
• Disagreement handling — a defined rule for what happens when modalities conflict, rather than an annotator's ad hoc choice. Conflicts are informative and should be logged, not silently resolved.

The nuScenes dataset shows what multi-sensor ground truth looks like: 1,000 scenes from six cameras, five radars and one LiDAR, with 3D boxes for 23 classes and tracking metrics that count identity switches. What a fusion shift looks like in practice is described in the account of how a LiDAR annotation shift actually runs.

Temporal consistency means an object keeps the same identity, stable dimensions and a physically coherent trajectory across every frame of a sequence, including frames in which it is hidden.

Require sequence-level review, and ask specifically how interpolation is used and validated. Interpolation between keyframes is a legitimate efficiency and a common source of quiet error. Sequence-level review also changes pricing; see the image, video and 3D/LiDAR annotation pricing guide.

Ask for annotator retention on projects of comparable length, and what happens to quality when a team turns over. Then ask about specialist review for safety-critical categories: who adjudicates ambiguous cases, and whether "I am not sure" has a defined escalation path. A programme without one produces confident wrong labels, which are more dangerous than gaps because they pass an acceptance check. How two managed providers differ on these points is worked through in the comparison of Lifewood and iMerit for physical AI annotation.

• Residency — can work be confined to a named jurisdiction or facility? Recorded-in-public data frequently cannot cross certain borders.
• Privacy processing — face and plate blurring, at what stage, and whether the original is retained and under what control.
• Access model — least privilege, revocation, audit logs; physical controls where warranted, including secure rooms and no removable media.
• Per-item provenance — who labelled it, who reviewed it, under which guideline version.
• Certifications — ask for the certificate and its scope statement, not the logo. Scope is where these usually fail: a certification covering a head office says nothing about the delivery centre doing your work.

The controls that apply to any sensitive annotation programme are set out in the guide to enterprise data annotation security, privacy and compliance.

Build the pilot set deliberately: a majority of ordinary sequences, plus a deliberate minority of hard ones — night rain, heavy occlusion, a construction zone, an unusual actor, a sequence with a long full occlusion and re-appearance. Include at least one sequence you have already annotated internally to a standard you trust, and do not tell the vendor which one it is.

Score on: per-task metrics against your thresholds; identity switches across the occlusion sequence; projection consistency between modalities; per-class F1 on rare classes; how ambiguous cases were escalated and documented; and the guideline questions the vendor raised — a vendor asking sharp questions in week one is reading the taxonomy properly. Independent AI data validation scores pilot output against a gold set.

Two structural points matter for this category specifically. Owned centres, 40+ delivery centres across 30+ countries, make it practical to confine processing to a named jurisdiction, which recorded-in-public driving data often requires. And regional coverage across Asia, Europe, North America and Africa means datasets can be annotated by people who recognise local signage, markings and driving conventions rather than inferring them. Automotive and vision engagements span AI compute vendors, autonomous-mobility developers and computer-vision suppliers.

The company-reported scope on Lifewood's autonomous driving annotation page covers LiDAR, camera and radar annotation, including multi-frame tracking, lane markup, sign and signal recognition and fusion alignment, delivered from dedicated autonomous-vehicle centres in Malaysia and Indonesia. Quality is governed by a 95%+ accuracy SLA, a 95%+ inter-annotator agreement threshold against a customer-approved gold set, and two independent review passes with timestamped approval records, which supply the per-item provenance. Lifewood also reports 414,120 training hours across its Bangladesh workforce in 2025.