Short answer. An L4 LiDAR annotation shift runs in five stages — intake and pre-labelling, the human annotation pass, peer review, QA sampling, then rework and delivery — and the human pass is the least automatable part of it. Pre-labelling covers roughly 70–80% of a typical urban frame, but its plausible-and-wrong outputs are harder to correct than a blank frame, which is why annotators are trained not to accept them. An experienced annotator on complex urban scenes processes about 8–15 frames per hour, and tracking-ID assignment at crossing paths is the error that compounds fastest across a sequence.
Walk into most AI conferences and you will hear a lot about models. About architecture choices, about training runs, about benchmark numbers. What you hear much less about is the operational layer underneath all of it: the rooms full of people making judgment calls at a pace that determines whether a perception system works in the real world.
LiDAR annotation for L4 autonomous driving is one of the most demanding annotation tasks in production. It is not like labelling images, where a mistake produces a wrong caption. A mislabelled pedestrian, a missed cyclist in a sparse point cloud, a bounding box that drifts by 15 centimetres across a 10-frame sequence: any of these can corrupt the training signal for a safety-critical system. The stakes are high and the volume is relentless. Modern autonomous vehicle fleets generate terabytes of sensor data every day, and every frame of LiDAR needs to be processed, labelled, verified and delivered before it can do anything useful.
This is a walkthrough of how a production shift at a Lifewood LiDAR delivery centre actually runs: from the moment a frame batch arrives to the moment it ships.
The shift before the frames arrive
Before anyone opens an annotation tool, the shift lead has already reviewed the task specification for that day's batch.
LiDAR annotation for autonomous driving clients is governed by a detailed ontology: which classes exist (vehicles, pedestrians, cyclists, vulnerable road users, construction elements, static obstacles), what the labelling conventions are for each, how to handle occlusion, how to treat objects that partially exit the field of view, and what to do when a LiDAR return is too sparse for confident classification.
This last question comes up constantly. Sparse returns happen at distance, in rain, around reflective surfaces and at the edges of sensor range. A vehicle 80 metres away might produce 12 to 15 LiDAR points. A pedestrian at the same distance might produce 4 to 6. The specification determines whether this is an annotatable object, a "low confidence" flag, or a deliberate omission, and the right answer changes based on the client, the training objective and the programme phase.
The shift lead runs a 10-minute briefing: edge cases from the previous session that surfaced in rework, any guideline updates pushed by the client, and the day's batch characteristics. If a new driving scenario is in the dataset (a construction zone type not in previous batches, an unusual weather condition, a geographic region with unfamiliar road markings), that gets discussed before anyone touches the tools.
Stage 1: Intake and pre-labelling
The batch arrives as raw sensor data: point clouds in a binary format, timestamped and synchronised with camera feeds and radar where applicable. Before human annotators touch it, the batch passes through automated processing.
At Lifewood, this means a pre-labelling pass using models trained on the client's previously labelled data. The output is a set of candidate annotations: 3D bounding boxes placed automatically around clusters that look like annotatable objects, with predicted class labels and confidence scores attached to each.
The honesty about this step matters. Pre-labelling is not annotation. At best it is a fast first draft that a skilled annotator can validate and correct; at worst, on frames with unusual conditions, it can produce confident, plausible, wrong outputs that are harder to correct than a blank frame would have been. Production teams know this and calibrate accordingly.
On a typical batch of urban driving in reasonable visibility, pre-labelling achieves acceptable coverage on maybe 70 to 80% of objects. The remaining 20 to 30%, the distant objects, the partially occluded ones, the novel scenarios the training data did not cover, need to be found and labelled from scratch. The pre-labeller also misclassifies regularly at low confidence: a motorcycle classified as a vehicle, a rubbish bin classified as a pedestrian. These show up in the confidence scores, but the annotator has to look at all of them.
Frames with confidence scores below a threshold (set per programme, usually around 0.6 to 0.7 on the pre-labeller's internal scale) are flagged for priority human attention before the rest of the batch is processed.
Stage 2: The human annotation pass
This is the core of the shift. Annotators work in a 3D point cloud viewer, with the pre-labelled boxes already placed and the raw cloud visible underneath. The task is not to accept the pre-labels; it is to audit every object in every frame.
Practically, this means checking every placed box for correct class, tight fit and consistent heading across frames. It means looking at the raw cloud for objects the pre-labeller missed entirely. It means deciding, for every ambiguous return, whether this is an annotatable object or background noise.
The cuboid-first approach is standard: create or correct the 3D bounding box first, then adjust in bird's-eye view (BEV) to fine-tune position and orientation. For sequences rather than single frames, the annotator also tracks objects across time, ensuring that a pedestrian in frame 1 carries the same tracking ID in frames 2 through 10, and that the bounding box moves consistently with the object's motion.
Tracking is where a lot of annotation time goes, and where a lot of errors compound. A tracking ID swap at frame 5, where two pedestrians cross paths and the annotator assigns the IDs incorrectly, will propagate through the rest of the sequence.
It is a small decision that looks minor and costs significant rework to find later.
Annotator specialisation matters here. At L4 quality levels, annotating LiDAR well requires understanding 3D geometry, motion dynamics and sensor physics: how to interpret sparse returns at range, what a LiDAR shadow indicates, how to handle retroreflective surfaces that produce artificially bright returns. Lifewood trains annotators specifically for LiDAR work rather than rotating them through task types, precisely because the specialised knowledge is what the quality level depends on.
An experienced annotator working at production pace will process somewhere in the range of 8 to 15 frames per hour on complex urban scenes, substantially fewer on difficult conditions. This is slower than many clients expect, and it is exactly right: faster annotation on this task means missed objects and wrong classifications, which cost far more to fix downstream than the annotation time saved.
Stage 3: Peer review
Completed annotations do not go directly to QA. They go to peer review first.
A second annotator on the same team opens the annotated batch and checks it against the raw cloud. This is not a full re-annotation; it is a verification pass looking specifically for missed objects, class errors and tracking inconsistencies. The reviewer uses a checklist aligned with the programme's known failure modes: distant objects in sparse returns, objects at sensor edges, occluded pedestrians behind vehicles, and tracking ID swaps at crossing paths.
Peer review catches approximately half the errors that would otherwise reach QA. It is also faster than a full QA review, because the reviewer is checking a labelled output against a known specification rather than re-labelling from scratch. The cost is that it requires annotators to spend part of their shift reviewing rather than annotating, and a production team needs to plan for that overhead in throughput calculations.
Items flagged in peer review go to the original annotator for correction before the batch moves on. The correction, the flag reason, and the reviewer ID are all logged. This log becomes part of the quality history of the batch.
Stage 4: QA sampling
Not every frame in every batch gets a full QA check. That would be prohibitively expensive and is also not how statistical quality assurance works. Instead, a stratified sample of frames is reviewed by a senior QA specialist who did not work on the annotation or peer review.
The sampling rate varies by programme phase and annotator track record. For established annotators with a clean history on the programme, a 10 to 15% sample rate per batch is typical. For new annotators, or after a programme specification change, the rate rises to 25 to 30%. Frames flagged as low-confidence by the pre-labeller, or containing rare scenario types, are always included in the sample regardless of the base rate.
The QA specialist's review is more thorough than peer review. It uses the programme's precision and recall criteria rather than a checklist, meaning the specialist is actively looking for missed objects rather than only checking the ones that were placed. A QA specialist might spend 20 to 30 minutes on a complex frame that an annotator labelled in 8.
The output of QA is an accuracy score per batch and per annotator. Batches that fall below the programme accuracy threshold are rejected and returned to rework. Annotators whose individual accuracy consistently falls below threshold are paused for calibration.
Stage 5: Rework and delivery
Rejected frames come back with specific flags: missed object, wrong class, tight-fit error, tracking ID inconsistency, or heading error. The annotator addresses each flag individually and resubmits.
Rework rates are a production efficiency metric as much as a quality one. A programme with high rework is a programme where guidelines are ambiguous, pre-labelling is performing badly, or annotator calibration is drifting. Tracking rework reasons over time, rather than just rework volume, is what allows the team to distinguish between a guideline problem (affects many annotators consistently) and an individual calibration problem (affects one annotator in specific scenario types).
Once a batch passes QA, it is packaged with its quality report: per-batch accuracy, IAA scores where applicable, QA sample rate, rework history and the guideline version against which it was labelled. The client receives both the annotations and the documentation. The documentation is what makes the dataset auditable: if a model trained on this data shows unexpected behaviour on a specific scenario type, the provenance trail exists to investigate whether the annotation was correct.
Why this matters beyond LiDAR
The specific details of a LiDAR annotation shift are LiDAR-specific. But the structure, five stages, each catching different problems, with documented decisions and provenance at every point, reflects a principle that applies across annotation work.
The accuracy figure that a client sees on a data delivery is the output of a process. Without understanding the process, the number is opaque: you cannot know what went into producing it, what it would cost to improve it, or why it degrades on specific scenario types. Understanding the process is what allows a client to have an informed conversation about quality rather than accepting a headline figure at face value.
At Lifewood, this is why our LiDAR programmes are run through dedicated trained teams rather than general annotators, why the QA layer is a separate specialist function from annotation, and why every batch ships with a quality report rather than just a label file. The accuracy Lifewood delivers to L4 clients is not a claim to be benchmarked once and forgotten; it is the result of a shift structure that is designed to produce it reliably.
Key takeaways
- A LiDAR annotation shift for L4 autonomous driving begins before frames arrive: the shift lead reviews edge cases, guideline updates and batch characteristics in a briefing.
- Pre-labelling places candidate annotations automatically, achieving roughly 70 to 80% coverage on typical urban frames, with the remainder requiring human annotation from scratch.
- Pre-labelling outputs that are plausible and wrong are harder to correct than a blank frame; experienced annotators know not to simply accept them.
- The human annotation pass uses a cuboid-first approach in 3D point cloud viewers, with BEV refinement for position and orientation, and temporal tracking across frame sequences.
- Tracking ID assignment at crossing paths is among the most common compounding errors in sequence annotation.
- Annotators at L4 quality levels are specialised in LiDAR, not rotated through task types: the required knowledge of 3D geometry, motion dynamics and sensor physics takes time to build.
- An experienced annotator on complex urban scenes processes roughly 8 to 15 frames per hour.
- Peer review catches approximately half the errors before QA, using a failure-mode checklist, with corrections logged against the reviewer and annotator IDs.
- QA sampling runs at 10 to 15% for established annotators and 25 to 30% for new ones, with low-confidence and rare-scenario frames always included.
- Rework reasons, tracked over time by category, distinguish guideline problems from individual calibration problems.
- Every delivered batch includes a quality report covering per-batch accuracy, QA sample rate, rework history and guideline version.
Sources and further reading
- Label Your Data, "Autonomous Vehicle Data Collection", on the hybrid annotation model (automated pre-labelling followed by human QA) and the importance of catching metadata errors early
- Label Your Data, "LiDAR Annotation: What It Is and How to Do It in 2026", on annotation guidelines, QA feedback loops and scale challenges
- Keylabs, "LiDAR Point Cloud Annotation for Autonomous Driving", on the cuboid-first workflow, BEV refinement, and QA steps in 3D annotation
- Kognic, "Best LiDAR Annotation Platforms 2026", on multi-sensor calibration-aware pipelines and L4 programme requirements
- Robosoft, "Guide to Data Annotation for Autonomous Vehicles", on the annotation workflow (annotation, quality review, rework, final review, delivery) and annotator tooling
- Yahoo Finance / Globe Newswire, "Multi-Sensor Data Labeling and AI Data Operations", April 2026, on AV annotation market growth and human-in-the-loop requirements at scale
- Lifewood, autonomous driving data annotation and delivery network