Deep learning has exhibited a promising performance in a series of task-specific bearing fault diagnosis problems. However, the current literature lacks a versatile model that could serve as a general-purpose feature extractor for a wide range of downstream fault diagnosis tasks. To fill this gap, this paper develops a theoretically well-grounded approach to build an \bl{enhanced} backbone model for various key downstream tasks. The proposed methodology follows a three-step procedure. In the first step, using approximation theory, we prove that incorporating neural heterogeneity by combining linear and quadratic neurons leads to a more efficient approximation of any univariate real coefficient polynomial. Compared to conventional neural networks, HNNs exhibit enhanced representation capability while using fewer model parameters. Secondly, building upon this theoretical foundation, we design an \bl{enhanced} feature extractor termed as heterogeneous neural blind deconvolution (HBD). At a high level, HBD is comprised of two time domain blind deconvolution branches in parallel: one using regular convolutional networks and the other using quadratic convolutional network. The inclusion of diverse neurons in HBD facilitates to learn discriminative features for reliable fault diagnosis that neither neuron type could achieve independently. Following the dual time domain blind deconvolution branches, a frequency-domain BD module complements the feature extraction capability of the time domain blind deconvolution by performing signal filtering in the frequency domain. Finally, to illustrate the general-purpose nature of HBD, we explore the application of HBD across various downstream fault diagnosis tasks, including anti-noise fault diagnosis, cross-domain fault diagnosis, and lightweight model for fault diagnosis on edge devices. Extensive experiments and comparisons with state-of-the-art baselines clearly show the advantage of HBD in enhancing the accuracy and interpretablity for bearing fault diagnosis.