Short answer. An L4 LiDAR annotation shift runs in five stages — intake and pre-labelling, human annotation, peer review, QA sampling, then rework and delivery. Pre-labelling covers roughly 70–80% of a typical urban frame, but its confident, wrong outputs are harder to correct than a blank frame. An experienced annotator processes about 8–15 frames per hour, and tracking-ID swaps at crossing paths are the error that compounds fastest across a sequence.
Walk into most AI conferences and you will hear a lot about models. About architecture choices, about training runs, about benchmark numbers. What you hear much less about is the operational layer underneath all of it: the rooms full of people making judgment calls at a pace that determines whether a perception system works in the real world.
LiDAR annotation for L4 autonomous driving is one of the most demanding annotation tasks in production. It is not like labelling images, where a mistake produces a wrong caption. A mislabelled pedestrian, a missed cyclist in a sparse point cloud, a bounding box that drifts by 15 centimetres across a 10-frame sequence: any of these can corrupt the training signal for a safety-critical system, an aspect of autonomous driving data annotation requirements that buyers underestimate until it shows up in delivered data. Modern autonomous vehicle fleets generate terabytes of sensor data every day, and every frame of LiDAR needs to be processed, labelled, verified and delivered before it can do anything useful.
This is a walkthrough of how a production shift at a Lifewood LiDAR delivery centre actually runs: from the moment a frame batch arrives to the moment it ships.
Key takeaways
- Pre-labelling automatically places candidate annotations on roughly 70 to 80% of objects in a typical urban LiDAR frame, but confident-and-wrong outputs are harder to correct than a blank frame.
- An experienced annotator on complex urban scenes processes roughly 8 to 15 frames per hour, and tracking-ID assignment at crossing paths is among the most common compounding errors.
- Peer review, a second annotator checking placed annotations against the raw point cloud, catches approximately half the errors that would otherwise reach QA.
- QA sampling rates run 10 to 15% for established annotators and 25 to 30% for new ones, with low-confidence and rare-scenario frames always included regardless of the base rate.
- Every delivered batch ships with a quality report covering per-batch accuracy, QA sample rate, rework history and the guideline version the annotations were produced against.
What happens before an annotation shift begins?
Before anyone opens an annotation tool, the shift lead has already reviewed the task specification for that day's batch and briefed the team on edge cases and guideline updates.
LiDAR annotation for autonomous driving clients is governed by a detailed ontology: which classes exist (vehicles, pedestrians, cyclists, vulnerable road users, construction elements, static obstacles), what the labelling conventions are for each, how to handle occlusion, how to treat objects that partially exit the field of view, and what to do when a LiDAR return is too sparse for confident classification.
This last question comes up constantly. Sparse returns happen at distance, in rain, around reflective surfaces and at the edges of sensor range. A vehicle 80 metres away might produce 12 to 15 LiDAR points. A pedestrian at the same distance might produce 4 to 6. The specification determines whether this is an annotatable object, a "low confidence" flag, or a deliberate omission, and the right answer changes based on the client, the training objective and the programme phase.
The shift lead runs a 10-minute briefing: edge cases from the previous session that surfaced in rework, any guideline updates pushed by the client, and the day's batch characteristics. If a new driving scenario is in the dataset (a construction zone type not in previous batches, an unusual weather condition, a geographic region with unfamiliar road markings), that gets discussed before anyone touches the tools.
How does intake and pre-labelling work?
The batch arrives as raw sensor data and passes through an automated pre-labelling pass before any human annotator touches it.
The batch arrives as raw sensor data: point clouds in a binary format, timestamped and synchronised with camera feeds and radar where applicable. Pre-labelling is a model-generated first pass that places candidate 3D bounding boxes around clusters that look like annotatable objects, with predicted class labels and confidence scores attached to each — it is a draft for a human to validate, not a finished annotation, a distinction covered in more depth in coverage of model-assisted labelling and active learning.
The honesty about this step matters. At best pre-labelling is a fast first draft that a skilled annotator can validate and correct; at worst, on frames with unusual conditions, it can produce confident, plausible, wrong outputs that are harder to correct than a blank frame would have been. Production teams know this and calibrate accordingly.
On a typical batch of urban driving in reasonable visibility, pre-labelling achieves acceptable coverage on maybe 70 to 80% of objects. The remaining 20 to 30%, the distant objects, the partially occluded ones, the novel scenarios the training data did not cover, need to be found and labelled from scratch. The pre-labeller also misclassifies regularly at low confidence: a motorcycle classified as a vehicle, a rubbish bin classified as a pedestrian. These show up in the confidence scores, but the annotator has to look at all of them.
Frames with confidence scores below a threshold (set per programme, usually around 0.6 to 0.7 on the pre-labeller's internal scale) are flagged for priority human attention before the rest of the batch is processed.
What happens during the human annotation pass?
The human annotation pass audits every pre-labelled object and finds the ones the model missed, using a cuboid-first workflow with temporal tracking across frame sequences.
Annotators work in a 3D point cloud viewer, with the pre-labelled boxes already placed and the raw cloud visible underneath. The task is not to accept the pre-labels; it is to audit every object in every frame. Practically, this means checking every placed box for correct class, tight fit and consistent heading across frames. It means looking at the raw cloud for objects the pre-labeller missed entirely. It means deciding, for every ambiguous return, whether this is an annotatable object or background noise.
The cuboid-first approach is standard: create or correct the 3D bounding box first, then adjust in bird's-eye view (BEV) to fine-tune position and orientation. For sequences rather than single frames, the annotator also tracks objects across time using a tracking ID, a persistent identifier that must carry a pedestrian or vehicle through every frame it appears in, with the bounding box moving consistently with the object's motion.
Tracking is where a lot of annotation time goes, and where a lot of errors compound. A tracking ID swap at frame 5, where two pedestrians cross paths and the annotator assigns the IDs incorrectly, will propagate through the rest of the sequence. It is a small decision that looks minor and costs significant rework to find later.
Annotator specialisation matters here. At L4 quality levels, annotating LiDAR well requires understanding 3D geometry, motion dynamics and sensor physics: how to interpret sparse returns at range, what a LiDAR shadow indicates, how to handle retroreflective surfaces that produce artificially bright returns. Lifewood trains annotators specifically for LiDAR work rather than rotating them through task types, precisely because the specialised knowledge is what the quality level depends on — the same reasoning that shapes how annotators are recruited, trained and certified for other specialist domains.
An experienced annotator working at production pace will process somewhere in the range of 8 to 15 frames per hour on complex urban scenes, substantially fewer on difficult conditions. This is slower than many clients expect, and it is exactly right: faster annotation on this task means missed objects and wrong classifications, which cost far more to fix downstream than the annotation time saved.
How does peer review catch errors before QA?
Peer review is a second annotator checking a completed batch against the raw point cloud, and it catches roughly half the errors that would otherwise reach QA.
Completed annotations do not go directly to QA. A second annotator on the same team opens the annotated batch and checks it against the raw cloud. This is not a full re-annotation; it is a verification pass looking specifically for missed objects, class errors and tracking inconsistencies. The reviewer uses a checklist aligned with the programme's known failure modes: distant objects in sparse returns, objects at sensor edges, occluded pedestrians behind vehicles, and tracking ID swaps at crossing paths.
Peer review is also faster than a full QA review, because the reviewer is checking a labelled output against a known specification rather than re-labelling from scratch. The cost is that it requires annotators to spend part of their shift reviewing rather than annotating, and a production team needs to plan for that overhead in throughput calculations.
Items flagged in peer review go to the original annotator for correction before the batch moves on. The correction, the flag reason, and the reviewer ID are all logged. This log becomes part of the quality history of the batch.
How does QA sampling work?
QA sampling reviews a statistical sample of frames, not every frame, using a senior specialist who did not work on the annotation or peer review.
Not every frame in every batch gets a full QA check. That would be prohibitively expensive and is also not how statistical quality assurance works. Instead, a stratified sample of frames is reviewed by a senior QA specialist. The sampling rate varies by programme phase and annotator track record: for established annotators with a clean history on the programme, a 10 to 15% sample rate per batch is typical, rising to 25 to 30% for new annotators or after a programme specification change. Frames flagged as low-confidence by the pre-labeller, or containing rare scenario types, are always included in the sample regardless of the base rate.
The QA specialist's review is more thorough than peer review. It uses the programme's precision and recall criteria rather than a checklist, meaning the specialist is actively looking for missed objects rather than only checking the ones that were placed — a distinction between review methods explored further in gold sets, audit sampling and consensus. A QA specialist might spend 20 to 30 minutes on a complex frame that an annotator labelled in 8.
The output of QA is an accuracy score per batch and per annotator, cross-checked in some programmes against inter-annotator agreement (IAA), the rate at which independent annotators label the same object the same way. Batches that fall below the programme accuracy threshold are rejected and returned to rework. Annotators whose individual accuracy consistently falls below threshold are paused for calibration.
How are rejected frames reworked and delivered?
Rejected frames come back with a specific flag reason, the annotator corrects that flag, and the batch ships only once it passes QA along with its full quality report.
Rejected frames come back with specific flags: missed object, wrong class, tight-fit error, tracking ID inconsistency, or heading error. The annotator addresses each flag individually and resubmits.
Rework rates are a production efficiency metric as much as a quality one. A programme with high rework is a programme where guidelines are ambiguous, pre-labelling is performing badly, or annotator calibration is drifting. Tracking rework reasons over time, rather than just rework volume, is what allows the team to distinguish between a guideline problem (affects many annotators consistently) and an individual calibration problem (affects one annotator in specific scenario types).
Once a batch passes QA, it is packaged with its quality report: per-batch accuracy, IAA scores where applicable, QA sample rate, rework history and the guideline version against which it was labelled. The client receives both the annotations and the documentation. The documentation is what makes the dataset auditable: if a model trained on this data shows unexpected behaviour on a specific scenario type, the provenance trail exists to investigate whether the annotation was correct.
Why does this process matter beyond LiDAR?
The five-stage structure — each stage catching different problems, with documented decisions and provenance at every point — reflects a principle that applies across annotation work, not only LiDAR.
The accuracy figure that a client sees on a data delivery is the output of a process. Without understanding the process, the number is opaque: you cannot know what went into producing it, what it would cost to improve it, or why it degrades on specific scenario types. Understanding the process is what allows a client to have an informed conversation about quality rather than accepting a headline figure at face value, which is also why buyers evaluating autonomous driving annotation providers should ask how a shortlisted vendor structures QA, not only what accuracy number it quotes.
At Lifewood, this is why LiDAR programmes are run through dedicated trained teams rather than general annotators, why the QA layer is a separate specialist function from annotation, and why every batch ships with a quality report rather than just a label file. The accuracy Lifewood delivers to L4 clients is not a claim to be benchmarked once and forgotten; it is the result of a shift structure designed to produce it reliably, supported by the same inter-annotator agreement discipline and validation practice Lifewood applies across its autonomous driving annotation and broader AI data validation work.