The Minimum Viable Dataset Spec: What to Define Before You Scale Collection

Before you scale data collection, define what the buyer will actually accept. That is the Minimum Viable Dataset (MVD) spec: the smallest dataset that passes acceptance tests and proves the commercial path is real.

Most AI data collection efforts fail the same way: teams build sophisticated capture infrastructure, collect thousands of samples, then discover buyers won't accept the data. The problem isn't the infrastructure. It's that they scaled before defining what "done" looks like.

The solution is a Minimum Viable Dataset (MVD) specification—the smallest dataset that passes buyer acceptance tests and validates commercial viability. Not the smallest dataset that trains a model. Not the dataset you can collect fastest. The smallest dataset that proves someone will pay for what you're building.

Why "Collect Everything and Figure It Out Later" Fails

The intuition seems sound: gather comprehensive data, worry about requirements later. But three forces make this expensive:

Storage costs compound. A single hour of multi-modal robotics data—RGB-D cameras, joint encoders, force sensors, audio—can generate 50GB compressed. Scale to thousands of hours before validating format requirements, and you're paying for storage you can't use.

Buyer rejection is binary. If a hospital robotics team needs consent trails for every patient interaction and you didn't capture them, the entire dataset is worthless to that buyer. There's no partial credit. You can't retrofit consent after collection.

Access windows close. Many high-value data collection opportunities have finite access. A factory pilot might give you two weeks of floor time. A hospital might grant access to specific procedures for one month. If you spend that window collecting data in the wrong format or at the wrong granularity, you can't rewind.

The MVD framework inverts this: define acceptance criteria first, validate with minimal samples, then scale confidently.

If you want the practical buyer-side version of this checklist for surgical or robotics data, start here and then compare it with our working-with-us page and the broader robotics data posts.

The MVD Framework: Five Required Specifications

A complete MVD specification includes five components:

1. Task Scope and Boundaries

Define exactly what actions the dataset covers and what's explicitly out of scope. For a hospital restocking robot dataset, this might be:

• In scope: Navigating hallways, operating service elevators, opening supply room doors, identifying shelf locations, retrieving items.
• Out of scope: Patient rooms, clinical equipment, sterile environments, emergency procedures.

The boundary matters because buyers evaluate datasets against specific deployment contexts. A dataset marketed as "hospital navigation" that includes only empty hallways at 3 AM won't validate daytime obstacle avoidance.

2. Sensor Modalities and Format Requirements

Specify which sensors are required, their configurations, and output formats. Common gaps:

• Camera specs: Resolution, frame rate, field of view, RGB vs RGB-D, fisheye vs rectilinear.
• Synchronization: Are camera frames synchronized with joint encoder readings? What's the acceptable drift?
• Coordinate systems: How are transformations between sensors represented? Using ROS transforms? Custom formats?

For robotics datasets, the Robotics Language Dataset Standard (RLDS) provides a reasonable baseline format. But buyer-specific requirements often diverge—some want raw sensor streams, others want pre-processed observations. Define this before building ingest pipelines.

3. Annotation Schema and Quality Thresholds

What labels or metadata must accompany each sample? For manipulation tasks:

• Action labels: Discrete actions, continuous control vectors, or both?
• Object annotations: Bounding boxes, semantic masks, 6D poses?
• Failure modes: How are recoveries, interruptions, and edge cases labeled?

Quality thresholds prevent ambiguity. Instead of "high-quality annotations," specify: "Bounding boxes must have IoU ≥ 0.85 with ground truth, verified by two independent annotators."

For autonomous systems, one emerging pattern is the acceptance test annotation: labeling only the features a buyer's model must predict, not everything the sensor captured. A navigation dataset might annotate only traversable surfaces and dynamic obstacles, ignoring static furniture.

4. Consent and Provenance Requirements

Consent is a first-class schema requirement, not an afterthought. For any dataset involving people or regulated environments, define:

• Granularity: Per-session consent, per-location consent, or per-interaction consent?
• Audit trail format: Timestamped consent events? Cryptographic signatures?
• Revocation handling: If someone withdraws consent, which samples become invalid?

Medical and EU deployments may require consent trails that link every data sample to a specific signed agreement. Retrofitting this after collection is often impossible.

5. Validation and Acceptance Criteria

How will you know if the dataset works? Define testable acceptance criteria:

• Holdout test performance: A baseline model trained on the dataset must achieve X accuracy on a predefined test set.
• Distribution coverage: The dataset must include samples from Y distinct environments or Z failure modes.
• Buyer acceptance test: A specific task demonstration that the buyer requires before purchase.

The acceptance test is the most important. If a buyer requires that a navigation dataset proves hallway traversal during visitor hours, your 100-sample pilot must include that scenario. Scaling without validating the acceptance test is building unvalidated infrastructure.

Reverse-Engineering the Spec from Buyer Requirements

Most buyers don't hand you a complete specification. They describe deployment constraints and ask if your data can support them. Translating those constraints into collection requirements requires structured inquiry:

For task scope:

• "What's the failure mode you're most worried about?"
• "What edge cases have caused problems in your current system?"
• "What environment variability do you expect in production?"

For formats:

• "What's your model's observation space?"
• "Are you training end-to-end from pixels or using pre-processed features?"
• "What coordinate frame conventions do your existing tools expect?"

For quality:

• "What's the minimum accuracy your deployment requires?"
• "How do you currently evaluate model performance?"
• "What's an example of a data sample you'd reject?"

For consent:

• "What are your data retention and deletion policies?"
• "Do you need per-sample consent trails for compliance?"
• "What happens if a data subject requests deletion?"

Document answers in a shared spec artifact. Ambiguity here costs months later.

Validating the Spec: The 100-Sample Pilot

Once you have a candidate MVD specification, validate it with a small pilot before scaling infrastructure. The goal is not to train a production model—it's to prove the spec is complete and testable.

A hospital restocking dataset pilot might look like:

• 100 samples: 10 restocking episodes across 2 hospital floors, 2 times of day (morning rush, evening quiet).
• Full pipeline: Capture, ingest, annotation, consent logging, acceptance test.
• Success criteria: Baseline navigation model achieves >90% obstacle avoidance on held-out hallway segments. Consent trails validate for all samples. Buyer reviews and accepts 3 sample episodes.

If the pilot fails any criterion, the spec is incomplete. Common failure modes:

• Under-specified quality: Annotations pass review but model fails acceptance test (thresholds too loose).
• Over-engineered formats: Ingest pipeline works but buyer can't parse the output (format mismatch).
• Missing consent: Samples are high-quality but unusable in regulated environments (consent not captured).

Fix the spec, re-run the pilot. Do not scale until the pilot passes buyer acceptance.

Common Failure Modes

Failure Mode 1: Scaling Before Validation

A team collects 5,000 samples, then discovers the buyer needs pose annotations they didn't capture. The entire dataset requires manual re-annotation or is discarded. Cost: 6 months and $200K.

Failure Mode 2: Ignoring Consent Upfront

A dataset includes hospital staff interactions but no consent trails. Legal review blocks sale to EU buyers. The dataset is valuable only in jurisdictions without GDPR-equivalent requirements. Market size cut by 60%.

Failure Mode 3: Mismatch Between Test and Deployment

A dataset is collected in a clean, well-lit lab but marketed for factory deployment. Buyer tests in production environment and rejects due to poor generalization. The spec didn't include environment variability requirements.

Each failure mode is preventable with a validated MVD specification.

The Validation Loop

The MVD framework is iterative:

1. Draft spec from buyer requirements and deployment constraints.
2. Pilot collection: 100-500 samples, full pipeline.
3. Acceptance test: Buyer review, baseline model performance, compliance check.
4. Iterate: If pilot fails, update spec and re-test.
5. Scale: Once pilot passes, build production infrastructure confidently.

The pilot is cheap. Scaling unvalidated infrastructure is expensive. The MVD spec is the tool that prevents the latter.

When to Use an MVD Spec

Use an MVD specification when:

• You're building a data collection program for a specific buyer or vertical.
• The cost of collecting a single sample is high (robotics, medical, regulated environments).
• Format or consent requirements are strict and non-negotiable.
• You need to validate commercial viability before infrastructure investment.

Don't use an MVD spec when:

• You're doing exploratory research with no buyer.
• Samples are cheap and re-collection is trivial.
• Requirements are genuinely unknown and discovery is the goal.

The MVD framework is about risk reduction, not rigidity. If your goal is to discover what's useful, collect broadly. If your goal is to sell data, define acceptance criteria first.

What to Define Before You Scale

Most data collection failures trace to one root cause: building before defining what "done" means. The Minimum Viable Dataset specification forces that definition upfront.

Task scope. Sensor formats. Annotation schema. Consent trails. Acceptance criteria. Define these in a 100-sample pilot. Validate buyer acceptance. Then scale infrastructure.

The alternative is scaling first and discovering later that you built the wrong thing. That's not a data collection strategy. It's expensive hope.

← Back to Field Notes