In this paper, we propose a novel approach to detection and localization of abnormal sound sources for robotic inspection in oil refineries. Such environments are difficult environments with high noise from multiple machines and where swift detection of anomalies is critical. The rarity of anomalies hinders the gathering of a balanced training dataset for the common supervised learning approach. Our previous work, based on autoencoders, bypassed this issue but lacked the ability to locate the abnormal sound source. Our proposed method first learns a spatial map of the normal sounds, allowing to predict what sound should be present at each robot position. This enables a detection based on a comparison between the predicted and observed sound. Localization can then be conducted based on this comparison using optimization. Experiments conducted in laboratory conditions showed the effectiveness of the proposed method. Additionally, experiments in field conditions in an actual oil refinery further showed the potential of the proposed method.