arXiv preprint ·
LabFactory: Building and Evaluating Executable AI Labs
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Scientific tasks specify a desired capability, but realizing it often requires building a computational system tailored to the task---acquiring data, designing representations, training models, implementing tools, and deciding how they are used at inference. We present, a framework in which an AI builder turns a scientific brief into an executable AI lab: a task-specific solver that integrates models, knowledge resources, tools, and a controller behind a fixed interface. The builder develops and packages the lab in a metered workspace; a separate host then executes the delivered artifact on held-out inputs, with reference labels kept outside the solver's input interface, and scores its outputs under the task's protocol. This makes the delivered system, rather than the builder's account of its progress, the object of evaluation. We document 10 selected constructions across six scientific task categories---from molecular and genomic prediction to medical imaging, clinical decision support, and biomedical text---whose delivered labs exceeded their configured reference values on all 12 subtests under host-side execution. Four contain predictive models fitted during construction; the others assemble executable analysis environments, knowledge resources, and tool-driven workflows around a fixed platform LLM. Together they show that an AI agent can carry a scientific brief all the way to a working lab that can still be invoked, inspected, and checked after construction ends.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
Artificial intelligence surrogates for treatment effect estimation with before-and-after data
Multimodal Routing and Region Refinement for Language-Guided Medical Image Segmentation
Data-driven discrete-time deep recurrent neural network-based modeling for dissipative systems
Image Fidelity is Not Field Fidelity: Joint Thermodynamic Reconstruction and Error Localization in Neural Tomography
ZO-COSMO: Index-Free One-Hop Mixing for Decentralized Zeroth-Order Optimization
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.