What CLEAR is solving in noisy instruction-tuning data
Annotation noise is not a edge case in large-scale retrieval datasets — it is the default condition. Human annotators miss relevant passages, mark ambiguous documents arbitrarily, and apply inconsistent relevance thresholds across long labeling queues. When you train a retrieval model on these labels, the noise is not averaged away; it compounds through contrastive loss objectives that push hard negatives apart and pull positives together, meaning a mislabeled negative actively suppresses a passage the model should learn to rank highly.
CLEAR (Consistent Labeling Enhanced by LLM-driven Automated Re-labeling) attacks this directly. As stated in the OpenReview abstract, it "estimates the reliability of existing annotations using LLMs and rectifies potential labeling errors" — a two-stage automated data curation pipeline that scores each annotation for trustworthiness, filters out likely errors, and corrects suspicious negatives by promoting them to verified positives. The method was evaluated on MS MARCO Passage, MS MARCO Document, Natural Questions, and TriviaQA, where it "consistently outperforms existing baseline models."
At a Glance: What it is: An LLM-driven automated re-labeling pipeline for information retrieval training data. Core mechanism: Reliability scoring → Auto-Filter (drop low-confidence annotations) → Auto-Correct (recover likely true positives from the negative pool). Validation scope: MS MARCO Passage, MS MARCO Document, Natural Questions, TriviaQA — retrieval benchmarks only, not general-purpose SFT corpora. Primary constraint: Cannot recover information absent or systematically biased in the original annotation signal.
The scale justification is concrete within retrieval benchmarking. The original MS MARCO release comprises 1,010,916 anonymized questions and 8,841,823 passages from 3,563,535 web documents. At that volume, even a 1% false-negative rate means roughly 100,000 relevant passages mislabeled as irrelevant. Manual correction is infeasible; automated reliability estimation is the only tractable path for retrieval-label cleanup.
One precision the article will maintain throughout: CLEAR is an information retrieval data curation method. Its validated application is to retrieval labels. The connection to instruction fine-tuning is conceptual — the same reliability-scoring and relabeling logic applies to any labeled corpus where LLM confidence can serve as a proxy for annotation quality — but the paper's evidence base does not extend to general SFT corpora.
How CLEAR's Auto-Filter and Auto-Correct pipeline works
CLEAR's pipeline decomposes into two operationally distinct stages that operate sequentially on the raw annotation set. The architecture is conceptually reconstructed from the OpenReview abstract's description; exact threshold values and scoring formulas are not exposed at the abstract level.
flowchart LR
A["Raw Annotated\nDataset\n(query, doc, label)"] --> B["LLM Reliability\nScorer"]
B --> C["Reliability\nAssessment"]
C --> D["Auto-Filter\n(discard likely errors)"]
C --> E["Auto-Correct\n(relabel suspicious cases)"]
D --> F["Curated\nTraining Set"]
E --> F
F --> G["Retrieval Model\nFine-tuning"]
The pipeline consumes (query, document, original label) triples. The LLM reliability scorer processes each triple and emits a numeric score representing how trustworthy the existing human annotation is. This score gates two downstream operations: Auto-Filter discards annotations whose reliability is low, while Auto-Correct routes suspicious cases to a second LLM pass that makes an explicit relabeling decision.
The critical insight — and the competitive-gap this architecture addresses — is that CLEAR operates at the label level, not the document level. It is not removing duplicate or near-duplicate passages (deduplication). It is questioning whether the relationship between a specific query and a specific document was labeled correctly by a human annotator. That distinction determines what the pipeline can and cannot fix.
BSDetector contributes the conceptual scaffolding for the reliability scorer. As described in the BSDetector arXiv paper, it introduces "a method for detecting bad and speculative answers from a pretrained Large Language Model by estimating a numeric confidence score for any output it generated." Mapping this to CLEAR's context: the LLM scores its own confidence in the assertion "this document is relevant to this query," and that confidence score proxies the annotation's reliability.
Where annotation reliability is estimated
The reliability estimation stage is where CLEAR's design commits to a specific epistemic claim: an LLM's confidence in a relevance judgment is a valid signal for whether a human annotator got that judgment right. This is defensible when the LLM's training distribution overlaps with the annotation task, and it degrades when the domain is narrow or the LLM has systematic gaps in the relevant subject matter.
The mechanics mirror BSDetector's framing. BSDetector asks: "how confident should we be that a particular LLM answer is correct?" (ACL 2024 paper). Applied to annotation triage, the LLM is presented with the query-document pair and asked to assert relevance. The confidence of that assertion becomes the reliability signal that CLEAR uses to gate the filter and correct stages.
Pro Tip: Separate label confidence from model confidence. Label confidence is a property of a specific (query, document, annotation) triple — it asks whether the existing human label is correct. Model confidence is a property of the retrieval model's current state — it asks how certain the model is about its own ranking. CLEAR operates on label confidence. If you conflate the two, you will filter examples the retrieval model currently ranks poorly but that are genuinely relevant — which is exactly the noise you need to keep for hard-positive training.
No official source exposes CLEAR's exact reliability-feature set or thresholding protocol at the abstract level. The diagram above treats thresholds as conceptual operations rather than specific numerical values.
How likely label errors are filtered out
Auto-Filter removes annotations whose reliability score falls below the low-confidence threshold. These are (query, document) pairs where the LLM disagrees with the human label confidently enough that retaining the label creates more noise than signal.
This is categorically different from deduplication. Deduplication removes near-identical documents regardless of their labels; it addresses corpus redundancy. Auto-Filter addresses label correctness: a unique, non-duplicate document can still carry a wrong annotation. A passage that answers a related but distinct query might have been labeled relevant by a human annotator, but that passage is not a duplicate, and its positive label can still be wrong, teaching the retrieval model a spurious relevance association.
The scale of MS MARCO makes the over-filtering risk concrete. With 8,841,823 passages, true positives are sparse enough that an aggressive reliability threshold can discard genuinely relevant passages that happen to express relevance in an atypical or domain-specific way — exactly the hard positives that most improve retrieval quality at the margin.
Watch Out: Over-filtering is a structural risk when positive examples are sparse or topically narrow. Auto-Filter operates on reliability scores, and those scores are only as good as the LLM's coverage of the domain. For queries in low-resource verticals (legal, biomedical, multilingual), the LLM may assign low confidence to correct annotations simply because it lacks domain fluency. Before deploying a filter threshold in production, audit the filter's false-positive rate on a domain-stratified sample — not just on the aggregate distribution.
How Auto-Correct recovers true-positive documents
Auto-Correct is the more consequential half of the pipeline for retrieval training quality. Its job is to salvage examples that human annotators labeled as negative but that are, in fact, relevant. The OpenReview abstract states that CLEAR "rectifies potential labeling errors" — which at the mechanism level means reclassifying suspicious negatives as verified positives and adding them to the training set.
Why this matters for retrieval training specifically: false negatives in the negative pool can corrupt the loss signal and push genuinely relevant passages away from the query representation. When the model receives a (query, positive, [negatives]) triple and one of those "negatives" is actually a relevant document, the loss function actively penalizes the model for ranking that document highly. Auto-Correct stops this inversion from propagating through the training run.
The mechanism is deliberately conservative in the public abstract: documents flagged as suspicious by the reliability scorer can be re-evaluated by the LLM, and if they are judged relevant they are flipped to positive and added to the curated training set. If the LLM confirms the negative, the document remains in the hard-negative pool, which is itself valuable for contrastive training.
Production Note: Auto-Correct changes labels without a human in the loop. For any production pipeline, preserve before/after label provenance for every corrected example — specifically the original human label, the reliability score that triggered correction, and the LLM's relabeling decision with its confidence value. The OpenReview abstract does not confirm whether the authors' pipeline exposes this provenance natively, so teams adopting the approach should implement their own audit trail. Without it, downstream debugging of a retrieval model that underperforms on a specific query cluster becomes extremely difficult because you cannot distinguish Auto-Correct artifacts from original annotation errors.
Why BSDetector-style uncertainty estimation belongs in the mental model
Uncertainty estimation, filtering, and relabeling are three distinct operations that CLEAR composes into a single pipeline. Engineers who conflate them misapply the method — using a filtering signal where a relabeling signal is needed, or applying relabeling when the confidence is too low to justify it. BSDetector makes the conceptual separation explicit, which is why it belongs in the mental model for CLEAR.
| Operation | Output | Role in CLEAR | Main risk if misconfigured |
|---|---|---|---|
| Uncertainty estimation (BSDetector-style) | Numeric confidence score per annotation | Always, before filter and relabel decisions | Poor calibration misroutes downstream decisions |
| Auto-Filter | Discard or retain label | Score low enough to reject | Over-filtering removes true positives |
| Auto-Correct (relabeling) | New label with confidence | Suspicious cases routed for relabeling | Teacher bias propagates into corrected positives |
The table above maps the three roles to their operational position and failure mode. Uncertainty estimation is upstream of both other operations; its calibration quality determines the fidelity of every downstream decision. A well-calibrated scorer routes most examples correctly with high confidence; a miscalibrated one creates a large uncertain mid-range that overwhelms Auto-Correct with ambiguous cases.
What BSDetector adds conceptually
BSDetector's core contribution to this mental model is its framing of confidence as a first-class measurable quantity for LLM-generated assertions. The ACL 2024 paper states the goal is to "estimate how confident we should be that a particular LLM answer is correct (or simply 'good' for more general LLM responses)" (ACL Anthology). Applied to annotation noise triage, this means the LLM is not just making a binary relevance judgment — it is also communicating how much to trust that judgment.
The practical implication is that uncertainty scores give you a priority queue for human review. High-confidence scores (in either direction) need no adjudication. Low-confidence scores that fall in the uncertain band are the candidates most worth routing to a human annotator when the cost of a wrong correction is high.
Pro Tip: Treat the uncertainty score as a calibration diagnostic, not just a filter gate. Plot the reliability score distribution for a held-out sample with known-correct labels. If the distribution is well-calibrated, confident-positive scores should cluster near genuinely relevant documents and confident-negative scores near genuinely irrelevant ones. A bimodal distribution with a thin middle band indicates a discriminating scorer; a flat or unimodal distribution indicates a miscalibrated scorer that will flood the Auto-Correct stage with noise. Run this diagnostic before committing to a threshold.
What CLEAR does not solve downstream
CLEAR was validated on information retrieval benchmarks — specifically MS MARCO Passage, MS MARCO Document, Natural Questions, and TriviaQA — which measure whether a retrieval model ranks relevant passages above irrelevant ones. Gains on these metrics do not imply equivalent gains on instruction-following, answer formatting, reasoning chains, or safety supervision.
The distinction is structural. Retrieval labels are binary relevance judgments (relevant / not relevant) between a query and a passage. Instruction fine-tuning labels are (prompt, response) pairs evaluated on dimensions including factual accuracy, instruction adherence, tone, format, and safety. Label noise in retrieval means a wrong relevance bit. Label noise in SFT can mean a subtly incorrect reasoning step, a hallucinated entity, an unsafe response that a human annotator approved, or a high-quality response that a human annotator rated too harshly. The error modes differ enough that a curation method optimized for binary relevance correction may not address — and could even amplify — the dominant noise sources in SFT corpora.
Watch Out: A retrieval benchmark win does not transfer to instruction-following gains without independent validation. Teams that adapt CLEAR's reliability-scoring logic to SFT corpora should treat this as a new experimental condition requiring its own held-out evaluation on instruction-following benchmarks. The paper provides no evidence that the method improves answer quality, reasoning accuracy, or alignment metrics — and applying it to SFT data under the assumption that "better labels always help" ignores the domain shift between retrieval annotation and instruction annotation.
How CLEAR was validated on retrieval benchmarks
CLEAR's validation covers four benchmark families, each testing a distinct slice of retrieval performance. The scope is reported in the OpenReview abstract: "We evaluate CLEAR on several widely-used IR benchmarks, including MS MARCO Passage, MS MARCO Document, Natural Questions, and TriviaQA."
| Benchmark | Task type | Annotation style | Data scale context |
|---|---|---|---|
| MS MARCO Passage | Passage retrieval | Human relevance judgments | MS MARCO-scale passage corpus |
| MS MARCO Document | Document retrieval | Human relevance judgments | MS MARCO-scale document corpus |
| Natural Questions | Open-domain QA retrieval | Wikipedia-grounded | Short factoid queries |
| TriviaQA | Open-domain QA retrieval | Web + Wikipedia evidence | Trivia-style queries, multiple evidence docs |
The abstract confirms CLEAR "consistently outperforms existing baseline models" across these families. The abstract does not expose per-dataset metric deltas, error bars, or ablation magnitudes — those require the full paper PDF and supplementary materials.
What the benchmark results actually support
The benchmark evidence supports one specific claim: CLEAR-style label curation improves retrieval model performance on standard IR benchmarks relative to baselines trained on the raw, uncurated annotation sets. That is a meaningful result. It validates the core mechanism — that LLM-driven reliability scoring and relabeling produce a training set that generalization better on held-out retrieval tasks.
What the results do not support: that the same curation policy improves instruction fine-tuning on instruction-following benchmarks. The supervision format differs (relevance bit vs. full response), the evaluation metric differs (MRR, Recall@K vs. MT-Bench, AlpacaEval, MMLU), and the dominant noise modes differ. Retrieval and SFT are separate experimental conditions.
Pro Tip: When reading benchmark wins in data curation papers, check whether the paper reports absolute metric values or relative improvement over a weak baseline. "Consistently outperforms existing baselines" is a relative claim — it is compatible with small absolute gains (e.g., +0.5 MRR on MS MARCO) that may or may not translate to production-relevant improvements. Before adopting a curation method, replicate the result on your own data distribution, not just the paper's benchmark families.
Why the paper's validation is not a general-purpose SFT proof
The experimental scope is information retrieval, which imposes two constraints on generalization. First, the annotation error types that CLEAR is calibrated to detect — mislabeled relevance bits in query-passage pairs — are not the same as the annotation error types dominant in SFT corpora. SFT label noise includes incorrect reasoning steps, factual errors in model-generated responses that human raters approved, safety boundary calls that are inherently ambiguous, and format mismatches between prompts and responses. None of these map cleanly onto the binary relevance judgment that CLEAR's reliability scorer is trained to question.
Second, teacher bias is an amplification risk specific to LLM-driven relabeling. When an LLM re-labels a suspicious annotation, it does so using its own prior over what constitutes a good answer. If the LLM scorer has systematic gaps — domain blindspots, training data cutoffs, or ideological tendencies — those gaps propagate into the corrected labels. For retrieval data, this risk is bounded: relevance judgments are relatively objective. For SFT data, this risk is unbounded: the LLM's prior over "what is a good instruction response" can encode arbitrary biases.
Watch Out: Domain shift and teacher bias interact. If the curation LLM was trained primarily on English web data and the SFT corpus contains domain-specific technical queries, the scorer will exhibit low confidence on correct domain-specific annotations and high confidence on generic but shallow answers. The result is a curated SFT set that is more fluent but less technically accurate than the original — a failure mode that is difficult to detect without domain-expert spot audits.
Where CLEAR fits among practical curation pipelines
CLEAR occupies a specific niche in the data curation stack: automated label correction for datasets where the annotation signal is noisy but recoverable through LLM judgment. It is not a universal data quality solution.
Choose automated relabeling (CLEAR-style) when: - Your dataset is large enough (>100K examples) that manual review of all questionable annotations is infeasible. - The annotation task is well-defined enough that an LLM can make reliable relevance judgments (binary relevance, factual grounding, instruction adherence — not subjective quality). - The cost of a wrong automated correction is lower than the cost of a missed true positive in training. - You can audit a random sample of corrections to validate the reliability scorer's calibration.
Choose active filtering (without relabeling) when: - You have enough training data that discarding uncertain examples does not starve the model of signal. - The annotation noise is concentrated in a specific slice of the data (e.g., a specific annotator batch or time period) that you can identify and quarantine. - Relabeling infrastructure (LLM API calls, quality control loop) is not available.
Choose synthetic data generation when: - The positive-to-negative ratio is so imbalanced that filtering would leave an insufficient number of positives. - The noise source is systematic (e.g., the annotation guidelines changed mid-project) and correction is not possible without re-specifying the task entirely.
Choose manual review when: - The error cost of a wrong label is high (safety-critical, legal, medical). - The data volume is small enough that expert review is tractable. - The annotation task is inherently subjective and cannot be reduced to an LLM confidence signal.
| Noisy data condition | Recommended curation action | CLEAR applicable? |
|---|---|---|
| Mislabeled relevance bits at scale | Auto-Correct (relabeling) | Yes — primary use case |
| Near-duplicate documents | Deduplication (semantic or exact) | No — distinct problem |
| Systematic annotator disagreement | Filtering + re-annotation | Partial — filter yes, re-annotation manual |
| Ambiguous positives, subjective quality | Manual review | No — LLM confidence unreliable here |
| Insufficient positive examples | Synthetic generation | No — out of scope |
| Label drift across annotator cohorts | Cohort-stratified audit | Partial — can inform triage priority |
When automated relabeling is the right first move
Automated relabeling is defensible as a first-pass triage mechanism at scale when manual annotation costs exceed the cost of a miscalibrated correction pass. BSDetector's confidence-score framing — producing a numeric signal for any LLM-generated output — provides the right architecture: generate scores, sort by confidence, and act on the low-confidence tail rather than treating all annotations equally.
Bottom Line: Use automated relabeling as the first move when your dataset exceeds the scale where manual triage is tractable, you can audit a stratified sample of corrections (minimum 200–500 examples across confidence deciles) before committing the full correction pass to training, and the reliability scorer's domain coverage matches your annotation task. If those three conditions are not met, invest in calibration or manual review before running Auto-Correct at scale. A miscalibrated first pass that systematically corrupts true positives in a specific query cluster is worse than no curation at all.
When manual review still wins
Manual review remains necessary in three failure modes that automated relabeling cannot address. First, adversarial labels — annotations deliberately manipulated to degrade model quality — carry the surface characteristics of correct labels and fool reliability scorers that only measure LLM confidence, not annotation provenance. Second, ambiguous positives in domains where even expert humans disagree (legal interpretation, medical triage, open-ended creative tasks) cannot be resolved by LLM confidence because the LLM's prior over the correct answer is no more reliable than a human's. Third, examples where the business cost of a wrong correction is asymmetric — a false positive in a safety filter, a missing negative example in a high-stakes retrieval system — require human sign-off regardless of the scorer's confidence level.
Watch Out: Automated relabeling systems have no mechanism to detect adversarial or politically manipulated labels; they detect inconsistency with LLM priors, not malicious intent. If your data collection pipeline has any exposure to adversarial label injection — crowdworker gaming, prompt injection in web-scraped data, or annotation vendor fraud — add a provenance audit layer before and after any automated relabeling pass. CLEAR's reliability estimation assumes annotations are noisy but not adversarial; violating that assumption degrades the pipeline in ways that aggregate metrics will not surface.
Practical takeaways for instruction fine-tuning teams
CLEAR's retrieval-label logic translates to instruction fine-tuning data curation through the same two-stage architecture: score each (prompt, response) pair for annotation reliability using an LLM judge, filter out low-reliability examples, and correct uncertain examples by relabeling with the LLM. The mechanism is portable; the evidence base is not.
Teams should treat this as a research hypothesis for their own data, not an established result. The path from hypothesis to validated curation policy requires running the reliability scorer on a held-out labeled sample where ground-truth quality is known, measuring the scorer's precision and recall at different threshold values, and validating that post-curation instruction-tuning performance improves on task-specific benchmarks — not just retrieval benchmarks.
| Noisy-data symptom | Likely cause | Recommended curation action |
|---|---|---|
| Model ignores formatting instructions | High rate of format-violating responses in training set | Reliability filter on format adherence |
| Model hallucinates facts it was not taught | Factually incorrect responses labeled as correct | Auto-Correct with factual grounding check |
| High variance across evaluation runs | Mixed-quality annotations from multiple annotator cohorts | Cohort stratification + reliability scoring per cohort |
| Model regresses on held-out tasks after SFT | Label noise concentrated in the fine-tuning domain | Held-out contamination check + filtering |
| Model performance plateaus despite more data | Dataset-level bottleneck, not model capacity | Full reliability audit + spot-check of bottom decile |
Signals that your dataset, not your model, is the bottleneck
The clearest signal that dataset quality is the bottleneck is a validation performance swing after data curation that exceeds the swing from hyperparameter tuning or architecture changes. If changing your learning rate from 2e-5 to 3e-5 moves your benchmark score by 0.5 points but removing the bottom reliability decile of your training data moves it by 2 points, the dataset is the active constraint.
At the scale of MS MARCO — 1,010,916 questions and 8,841,823 passages — label noise effects are measurable because they affect millions of gradient updates. At smaller SFT dataset scales (10K–100K examples), noise effects are concentrated and hit harder proportionally.
Pro Tip: Label-noise fingerprints appear in specific evaluation slices, not just aggregate metrics. If your fine-tuned model performs well on common query patterns but degrades sharply on rarer query types or domain-specific topics, check whether those slices are underrepresented in the retained positives after filtering. A reliability scorer that discards rare positives at a higher rate than common ones produces exactly this pattern — strong aggregate performance, poor tail performance — and it looks identical to an overfitting problem until you stratify the evaluation results.
What to measure after curation
Post-curation evaluation should move three things if the pipeline is working correctly: training label consistency (measured by inter-annotator agreement on a held-out sample before and after correction), downstream task performance on benchmark families relevant to your use case, and the rate of manual spot-check disagreements with Auto-Correct decisions.
For retrieval use cases, the relevant evaluation families are the same ones CLEAR was validated on: MS MARCO Passage and Document, Natural Questions, and TriviaQA. For instruction fine-tuning, select benchmarks that match your actual task distribution — not generic benchmarks that your training data already covers well.
Watch Out: Small evaluation sets produce false positives in post-curation analysis. If you validate curation effectiveness on fewer than 500 held-out examples per evaluation category, apparent metric lifts can be statistical noise. The minimum viable evaluation set for a curation comparison depends on the effect size you expect and the metric variance in your task; for retrieval, MRR variance on a 200-example set is high enough that a 1-point apparent gain may be within the confidence interval of zero. Size your eval sets before, not after, the curation experiment.
FAQ
How does CLEAR work?
CLEAR applies a two-stage automated data curation pipeline to retrieval training data. In the first stage, an LLM scores each (query, document, label) triple for annotation reliability — estimating how trustworthy the existing human label is. In the second stage, low-reliability annotations are either discarded (Auto-Filter) or reclassified by a second LLM pass (Auto-Correct). The result is a curated dataset with fewer mislabeled negatives and more verified true positives, which improves retrieval model training.
What is data curation in machine learning?
Data curation is the systematic process of improving training data quality before model training. It encompasses deduplication, filtering (removing low-quality or incorrect examples), relabeling (correcting wrong labels), and augmentation (adding new examples). CLEAR focuses specifically on label correction for retrieval data.
What is the difference between filtering and correcting noisy labels?
Filtering removes examples whose labels are unreliable — the example is discarded from training entirely. Correcting noisy labels changes the label on an uncertain example rather than discarding it. Correction recovers examples that carry a wrong label but contain genuine signal; filtering is appropriate when the example itself cannot be confidently relabeled.
Can LLMs reliably identify label noise?
LLMs can identify label noise when the annotation task is well-defined and within their domain coverage. Their reliability as a noise detector degrades in low-resource domains, on subjective tasks, and when their training distribution does not cover the annotation domain. BSDetector's numeric confidence framing provides a practical way to quantify that reliability per example rather than applying a binary good/bad judgment to the entire scorer.
Does data curation improve instruction fine-tuning?
For retrieval-specific training data, CLEAR's results demonstrate consistent improvement over uncurated baselines. For general instruction fine-tuning, the evidence is not established — CLEAR was validated on IR benchmarks, not SFT corpora. The same logic should generalize, but requires independent validation on instruction-following benchmarks before adoption.
Sources and references
Production Note: The primary evidence source for CLEAR is the OpenReview paper page, which exposes the abstract and peer review discussion but not the full implementation details, threshold values, or metric tables. For exact ablation numbers and algorithm specifics, consult the full paper PDF linked from the OpenReview page. MS MARCO scale statistics are drawn from the original 2016 arXiv release. BSDetector methodology is from the 2023 arXiv preprint and its ACL 2024 camera-ready version.
- CLEAR: Consistent Labeling Enhanced by LLM-driven Automated Re-labeling for Improved Information Retrieval — Primary source for CLEAR's mechanism, scope, and benchmark validation claims.
- MS MARCO: A Human Generated MAchine Reading COmprehension Dataset (arXiv:1611.09268) — Canonical reference for MS MARCO scale: 1,010,916 questions and 8,841,823 passages.
- BSDetector: Detecting Bad and Speculative Answers from Large Language Models (arXiv:2308.16175) — Source for BSDetector's numeric confidence-score framing for LLM-output reliability estimation.
- BSDetector ACL 2024 camera-ready (ACL Anthology) — ACL 2024 version with the calibration and confidence-estimation methodology.
Keywords: CLEAR, BSDetector, OpenReview, MS MARCO Passage, MS MARCO Document, Natural Questions, TriviaQA, information retrieval, LLM-driven relabeling, annotation noise, confidence estimation, contrastive learning, label rectification, cleanlab, Hugging Face Datasets
