arXiv preprint ·
Data Curation and Model Souping: Unlocking Qwen3-0.6B for MS MARCO Passage Ranking
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
This report studies passage retrieval and re-ranking on the MS MARCO benchmark using lightweight Qwen3-0.6B models. Across three experimental settings --- BM25-then-reranking task, corpus-wide dense retrieval task, and dense retrieval then re-ranking task --- the results demonstrate that competitive ranking performance can be achieved without complex architectural modifications, custom loss functions, or cross-encoder distillation. Instead, substantial gains stem from data-centric strategies: corpus-targeted continued pre-training, rigorous data curation, hard-negative mining, training--inference distribution alignment, and weight merging via Model Soup. In particular, Model Soup delivers marked performance gains while preserving the inference efficiency of a single model. These findings highlight how data curation and weight merging can push sub-billion-parameter models to state-of-the-art performance. My models can be downloaded from Hugging Face: https://huggingface.co/shuguangh/models
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Information Retrieval
Co-adaptive Rule Routing and Constraining for Structured Knowledge Retrieval
Retrieving Complementary Evidence for Multi-Hop RAG with Semantic Gain Graphs
RES-MR: Risk-Aware Reasoning for Explainable and Safe Medication Recommendation
Understanding ChatGPT Citations: Evidence from Production Web Search Across Model Generations
EVISTA: Unified Multimodal Video Search with Evidence-Aware Temporal Path Optimization
Get the few papers worth your time in Information Retrieval every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.