arXiv preprint ·
Building a Method: Challenges and Lessons from Developing an Impact Evaluation Framework for Public Investment in Open Source Software
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Modern societies and institutions depend on an invisible layer of open digital infrastructure: foundational, openly accessible digital technologies that support a broad range of social and economic activities. Open source software is a core component of this infrastructure, underpinning everyday technologies and critical systems in various critical sectors. As open source software becomes increasingly recognised, funded, and governed as open digital infrastructure, policymakers face growing demands for accountability, transparency, and credible evidence of impact. However, methods for evaluating public investment in open source software remain underdeveloped, with no widely accepted methodological standard. This paper describes the Sovereign Tech Agency's transition from qualitative reporting and funding-recipient self-assessment towards a mixed-methods approach that incorporates quantitative, goal-oriented evaluation to inform funding decisions. Drawing on the development of this framework, it examines its initial design, the practical and structural challenges encountered, and the responses adopted. The work identifies three central challenges: evaluation indicators must reflect heterogeneous project goals and contexts; quantitative assessment relies on fragmented and fragile data infrastructure; and observed changes before, during, and after investment do not establish causal attribution without additional research designs. The paper argues that evaluating investment in open source software is not only a methodological task but also a governance practice, dependent on sustainable open research infrastructure, community knowledge, and ongoing methodological refinement.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computers & Society
Sure Enough to Share? Overconfidence in GenAI Video Discernment is Associated with Sharing Intentions
Code, Commits, and Complexity in the GenAI Era: A Longitudinal Study of Team-Based Capstone Projects
Thick Alignment: A Social Framework for Values and AI Systems
Digital Security Under Compound Threats: Surveillance, Threat Reasoning, and Habituation Among North Korean Defectors in China
InterSim: Innovative Intersection Digital Twins for Human-in-the-Loop Driving Simulation
Get the few papers worth your time in Computers & Society every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.