Short answer. Horizontal LLM training data builds general capability: broad coverage across many domains, languages and task types, sourced at scale and judged on breadth and consistency. Vertical LLM training data builds domain competence: narrow, deep, expert-produced material for one field, judged on correctness by someone qualified to know. Enterprises almost always need both, and the costliest mistake is buying horizontal volume when the model's failure is vertical.
Key takeaways
- Horizontal LLM training data is produced at scale by generalists and fixes fluency, format, general reasoning and language coverage; vertical LLM training data is produced by verified specialists and fixes terminology, procedure, edge cases and regulatory specifics.
- The producer row drives everything else: generalist production scales, specialist production does not, and vertical cost per item is set by the market rate for the expertise rather than by annotation rates.
- A model that is fluent and wrong on domain specifics has a vertical gap, and no volume of general data will close it.
- A structured error review that classifies real failures as horizontal or vertical takes a few days and routinely redirects a training-data budget by an order of magnitude.
- Heavy vertical fine-tuning can degrade general capability, so separate evaluation sets for general and domain performance are needed to detect interference.
What is the difference between horizontal and vertical LLM training data?
Horizontal LLM training data is a corpus assembled from broad general material across many domains, languages and task types, while vertical LLM training data is a corpus written by qualified specialists in a single field. The two differ in how they are sourced, who produces them, what they cost per item, and how their quality is measured.
Two products, one label. Treating them as one purchase is where most enterprise programmes go wrong.
| Criterion | Horizontal | Vertical |
|---|---|---|
| Goal | General capability, breadth, robustness | Domain competence and correctness |
| Coverage | Many domains, languages, task types | One field, in depth |
| Producer | Generalist annotators and writers, at scale | Qualified domain specialists |
| Volume | High | Low relative to horizontal |
| Cost per item | Lower | Substantially higher |
| Quality measured by | Consistency, coverage against a stratification plan, chance-corrected agreement | Expert review; factual correctness; currency of practice |
| Main risk | Skew toward whatever was easiest to source | Too narrow; a model expert in one sub-area and weak beside it |
| Fixes | Fluency, format, general reasoning behaviour, language coverage | Terminology, procedure, edge cases, regulatory specifics |
The producer row is the one that drives everything else. Generalist production scales; specialist production does not, and its cost is set by the market rate for the expertise, not by annotation rates. The published evidence points the same way: scaling the diversity of instruction-tuning tasks improves general held-out performance, a horizontal effect, while BloombergGPT's domain results came from adding a curated in-domain corpus to general data. The guide to RLHF, SFT and distillation covers which training methods each corpus feeds.
When does a model need horizontal training data?
A model needs horizontal training data when its weakness is general rather than specific: poor instruction following, inconsistent formatting, weak reasoning across all topics, or capability that degrades outside its strongest language. The remedy is coverage produced at scale.
The signatures of a horizontal gap:
- The model is generally weak across all topics rather than in one.
- Language coverage is the gap. A model that works in English and degrades in your other markets needs breadth, in those languages, produced natively.
- You are training or substantially adapting a base model rather than fine-tuning a strong one.
- Robustness is the complaint. The model handles the expected phrasing and falls over on the awkward variants.
Quality here is about coverage design and consistency, not depth. The recurring failure is a corpus that is large and skewed: over-representing whatever was easy to source, which is typically news, encyclopaedia and forum text, and under-representing the long tail of domains and the languages with the least available material. Stratify deliberately and report the minimum coverage per stratum, not the mean. The same discipline governs multilingual LLM training data.
When does a model need vertical training data?
A model needs vertical training data when it produces confident, well-formed output containing errors a practitioner would catch instantly. That is the diagnostic signature of a vertical gap, and it is not improved by general data at any volume.
The signatures of a vertical gap:
- Fluent and wrong, with errors a practitioner would catch instantly.
- Terminology drift. Using a term correctly in the general sense and incorrectly in your field's sense.
- Procedural gaps. Knowing what a process is called and not the order or the exceptions.
- Regulatory and currency problems. Stating a rule that was true, or that is true in a different jurisdiction.
- Edge cases that only a practitioner recognises as significant.
Quality here is correctness judged by someone qualified, which changes the sourcing problem entirely. Three requirements follow:
- Verified expertise. Qualification checked, not self-declared. The demonstration is a ceiling: a writer who cannot produce expert-quality output produces data that teaches the model to be a non-expert.
- Currency. Fields move. A corpus reflecting practice from five years ago will train a model to be confidently out of date, and there should be a review cadence for time-sensitive material.
- Jurisdiction and market specificity. Regulated fields differ by market. "Finance" is not one domain; it is one domain per regulatory regime.
Sourcing and screening specialist writers is covered in the guide to domain-expert SFT datasets.
How do you diagnose whether a model needs horizontal or vertical data?
Run a structured error review of real failures before buying anything, classify each failure as a horizontal or vertical gap, and compute the share of errors that are domain-specific. If that share is high, general volume will not move it; if it is low, expensive specialist writing is the wrong purchase.
Sample real failures and classify each one:
| Symptom | Likely gap |
|---|---|
| Wrong format, ignored instruction, rambling | Horizontal (behaviour) |
| Correct in English, poor in another language | Horizontal (language coverage) |
| Reasoning breaks on multi-step problems generally | Horizontal (capability) |
| Fluent output, domain-specific factual errors | Vertical |
| Right general term, wrong field-specific meaning | Vertical |
| Correct for one jurisdiction, wrong for yours | Vertical (market specificity) |
| Fails only on rare inputs, fine otherwise | Either; check coverage before buying depth |
Vertical share of errors = Domain-specific errors ÷ Total errors reviewed
The review costs a few days and routinely redirects a budget by an order of magnitude. Buying volume before the diagnosis is the most common way this money is wasted. Where the sample comes from production traffic, the guide to turning production logs into training data covers unbiased sampling.
How do horizontal and vertical training data combine in one programme?
They combine sequentially more often than simultaneously: general behaviour is established first with horizontal data, vertical depth is added narrowly afterwards, and both are evaluated separately because each can regress without the other's benchmark noticing.
- Establish behaviour horizontally. Format, instruction following and language coverage first, because vertical data cannot fix a model that will not follow instructions.
- Add vertical depth narrowly, targeted at the specific error classes the review found.
- Build evaluation sets for both, separately. A general benchmark will not detect a domain regression, and a domain benchmark will not detect a general one.
- Re-test for interference. Heavy vertical fine-tuning can degrade general capability; catastrophic forgetting during continual fine-tuning is documented in the research literature, and a model that became excellent at your field and worse at everything else is a known and avoidable outcome. Only a general evaluation set will reveal it.
A useful budgeting heuristic: vertical data buys correctness where you are judged, horizontal data buys the competence that makes the model usable at all. Programmes that skip the horizontal layer produce a model that is expert and unusable; programmes that skip the vertical layer produce one that is pleasant and wrong. Designing paired evaluation sets is covered in the guide to enterprise evaluation benchmarks.
What should you ask a training-data supplier?
Ask whether the supplier produces general-scale and specialist data in-house, how it verifies domain expertise, how it stratifies horizontal coverage, how it keeps vertical content current, and whether it measures quality differently for the two. A supplier that cannot answer these separately is probably producing both with the same people.
A checklist for the conversation:
- Which do you actually produce in-house: general-scale production, specialist production, or both?
- How is domain expertise verified for vertical work?
- What is your stratification plan for horizontal coverage, and what is the minimum coverage per stratum?
- How do you keep vertical content current, and at what cadence?
- For multilingual work, is vertical content produced in-market or translated?
- How is quality measured differently for the two, and can I see both figures?
- Who owns the corpus, and can we export it in full?
Red flags: one blended price for both, which usually means specialist work is being produced by generalists; expertise described as self-declared; a horizontal corpus with no stratification plan; vertical content in non-English markets produced by translating English source material, which imports the wrong jurisdiction along with the language. The ranked list of LLM training data companies compares suppliers on scale, languages and specialist depth.
How does Lifewood approach horizontal and vertical LLM training data?
Lifewood delivers horizontal and vertical LLM training data as distinct services rather than one blended offering, because they are different production problems: horizontal LLM data is a scale-and-coverage operation, vertical LLM data is a specialist-sourcing operation, and pricing them identically means one of them is being produced by the wrong people.
The multilingual dimension cuts across both and is where the delivery model matters most. With 100+ languages, 40+ delivery centres across 30+ countries and 56,000+ registered contributors, both breadth and depth can be produced in-market rather than translated. For vertical content that is not a quality preference but a correctness requirement, since a translated domain corpus imports the source market's regulatory assumptions. Lifewood's enterprise LLM training data scope extends to RLHF, SFT, data distillation and response evaluation, and in-market production for the horizontal layer draws on its multilingual data collection operation. Lifewood was founded in 2004 and has worked in AI data for over two decades.
The two offerings are published as separate service pages, type B horizontal LLM data and type C vertical LLM data, alongside type A data servicing, listed in the sources below.