arXiv preprint ·
Large Language Models for Automated Cross-Domain Machine Learning Task Type Identification: A Benchmark Dataset and Evaluation
- Published
- Source
- arXiv preprint, posted before peer review
- Published in
- arXiv
- DOI
- 10.48550/arxiv.2609.35335
- Original
- Read it at arXiv
- Cited by
- 0 works, as OpenAlex counts them
- Topics
In brief
LLMs Take the Guesswork Out of ML Tasks
This study introduces a novel approach using LLMs to automatically infer task types from dataset-level information. It also presents a benchmark dataset of 625 public datasets, providing a solid foundation for evaluating this new method.
What they found: The research demonstrates that LLMs can significantly enhance the accuracy of machine learning task type identification. In tabular datasets, the LLM-based method achieved an impressive F1 macro score of 0.98, surpassing AutoGluon's 0.93. For cross-domain evaluations, the best model reached 0.90, while smaller models maintained a score of 0.75.
AI-written summary. Check anything you rely on against the original paper.
Abstract
Machine learning task type identification is essential for constructing valid ML pipelines, yet in practice it is typically specified manually. We investigate whether large language models (LLMs) can infer both the data domain and the downstream prediction task directly from dataset-level information when only the target feature is provided by the user. Together with our LLM-based system we also release an annotated benchmark comprising 625 public tabular and time series datasets. We evaluate the proposed approach in three settings: (i) tabular datasets in comparison with established AutoML heuristics, (ii) cross-domain evaluation across tabular and time series datasets, and (iii) a practical deployment scenario using smaller local models. The results show consistent advantages for LLM-based task type identification, with increasing difficulty in heterogeneous and resource-constrained settings. LLM-based approaches outperform AutoGluon in the tabular setting, reaching 0.98 F1 macro compared to 0.93. In the cross-domain setting, the best model achieves 0.90 F1 macro, while smaller locally deployable models reach 0.75, indicating a trade-off between deployment feasibility and accuracy.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
Large Language Models for Structured Clinical Data Analysis: Dual-Agent Grounding and Validation
From Retrieval to Reasoning: Agentic Mechanism Prediction from Cell Painting Profiles
ARCH-B: Architectural Representation, Comprehension and Hierarchy Benchmark
Longer Records, Broader Invariance: The Hidden Scaling Problem in Longitudinal Contrastive Learning
Thinking Outside the Box: Retention and Transmission of Information in Sliding-Window KV Inference
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.