Automatic music genre classification is a research area that is increasing in popularity. Most researchers on this research area have been focusing on combining information from different sources than the musical signal itself. This paper presents a novel approach for the automatic music genre classification problem using audio signal for the context of Sri Lankan Music. The proposed approach uses two feature vectors and Support Vector Machine (SVM) classifier with radial-basis kernel function. More specifically, two feature sets for representing frequency domain, temporal domain, cepstral domain and modulation frequency domain audio features are proposed through this work. Music genre classification accuracy of 74.5% was recorded as the highest overall classification accuracy on our dataset containing over 100 songs over the five musical genres. This approach shows that it is possible to implement a genre classification model with a reasonably good accuracy by using different types of domain based audio features.