Post-Hoc Feature Selection Layer for Neural Networks Interpretability

Abstract

The interpretability of complex neural networks remains a critical challenge, especially for models already deployed in high-stakes domains. To address this, we adapted the Feature Selection Layer (FSL) to optimize feature weights post-training, a variant we term post-hoc FSL. Our approach reframes the FSL as a lightweight, trainable module that integrates with already frozen pre-trained models on tabular datasets to highlight the features the original model considers most important. This post-hoc FSL learns feature relevance by fine-tuning its weights based on the original model's learned outputs. Crucially, this process is non-invasive, operating without altering the original model's architecture or its learned parameters. We conducted our experiments using both statistical and visual metrics, including accuracy, F1 score, recall, precision, weighted t-SNE and silhouette score, and also analyzed the stability of the post-hoc FSL on high-dimensional synthetic and real-world tabular datasets. We compared the post-hoc FSL feature weighting method using these metrics against the original embedded FSL and other post-hoc interpretability methods, such as Integrated Gradients, Noise Tunnel, DeepLIFT, Gradient SHAP, and Feature Ablation. Experimental results demonstrate that the post-hoc FSL feature weighting method successfully identified relevant features across the different datasets, maintaining the predictive power of the original neural network while enhancing its interpretability.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Machine Learning

• arXiv preprint · 22 October 2026

  Governed Self-Evolution for Autonomous Investing: A Multi-Agent LLM System with an Invariant Core

  Yebo Feng, Jingyang Dai, Liangrui Pan and 3 others
• arXiv preprint · 23 October 2026

  MI-MIDI 2: From Observation to Control with Sparse Autoencoders in Text-to-MIDI Generation

  Jakub Poćwiardowski and Mateusz Modrzejewski
• arXiv preprint · 22 October 2026

  Lifetime-valid decisions by continually evolving AI analysts

  Xiaojing Sun, Yuhan Zeng and Xiao Wang
• arXiv preprint · 23 October 2026

  CEMIR vs. FISER: a comparative study of reasoning-based and enrichment-based approaches to pragmatic intent recognition

  Ziya Hosseynzadeh, Farjad Fallah, Farnoosh Fallah and 3 others
• arXiv preprint · 22 October 2026

  Diagnosing Personalization Failures in Mobile GUI Agents

  Zhixin Lin, Dongliang Xu, Jungang Li and 6 others

All new papers in Machine Learning