Abstract: A mixing matrix estimation method based on local directional density of time-frequency coefficients and Hough transform was proposed for underdetermined blind source separation. The signals were transformed to time-frequency domain by STFT to improve sparsity; and then Hough transform was applied to transform the coefficients of directional line in scatter plot. Mixing matrix estimation was accomplished by determining the numbers of local maximum values and calculating their values. In order to eliminate the peak cluster in Hough transform, a method was proposed, i.e. detecting directional density of time-frequency coefficients to determine these outlier points and then removing them, taking Hough transform again to improve Mixing matrix estimation accuracy. The experiment results show this algorithm can accomplish the source numbers estimation under unknown source numbers, and the mixing matrix estimation accuracy is obviously higher than those normal K-means methods.