The method proposed in this Letter selects a feature subset that preserves the data quality in the viewpoint of information theory. Using an efficient information‐theoretic evaluation, the proposed method identifies the final feature subset significantly faster than conventional methods.