Issues in Automatic Musical Genre Classification

1. Issues in Automatic Musical Genre Classification Cory McKay

2. Introduction to musical genre • Practical importance • Radio stations • Libraries • Retailers • Theoretical importance • How we construct genre taxonomies • Mechanisms we use to classify music • Mechanisms we use to distinguish between categories

3. Introduction to musical genre • No universally accepted set of categories • Genre descriptions are rarely consistent, comprehensive, clear or objective • Genre constructed by a complex interaction of • Marketing strategies • Historical conventions • Choices made by music librarians, critics and retailers • Interactions of groups of musicians and composers

4. Introduction to musical genre • Difficulties with classifying by musical genre • What categories should be used? • What are the boundaries between categories? • How are different categories related to each other? • What are characteristics of a particular genre? • What genre(s) do individual pieces belong to? • Genres constantly changing and being created • Main problems in automatic genre classification • Which features should be used? • What taxonomy should be used?

5. Symbolic vs. audio representation • Using a symbolic representation of music rather than an audio representation • Allows one to think of music in terms of musical features rather than signal processing features • Also allows one to classify scores for which no audio recordings are available • Future advances in automatic transcription systems will allow use of both types of features

6. Feature extraction • Features: • Characteristic pieces of information that can be extracted from music and used to describe or classify it. • Features are very important, as they are the only percepts available to classification systems. • Want features that demonstrate differences between categories.

7. Feature extraction • Sophisticated theoretical analyses • Too genre-specific • Automatic analysis often an unsolved problem • Want features that can be represented as simple numbers that can be fed to classification system. • Want features with musicological meaning if possible. • Want a large catalogue of features so that classifier can choose ones best suited to particular types of classification and sub-classification (hierarchal).

8. Feature extraction • Have devised 160 features based on: • Instrumentation • Texture • Rhythm • Dynamics • Pitch Statistics • Melody • Chords • Scope of these features not limited to genre classification. • Could be used for a variety of classification, clustering and analysis tasks.

9. Feature extraction • Future research: extract non-musical features • Lyrics • Clothing • Album art • etc. • Research of Whitman & Smaragdis (2002) a good start in this direction.

10. Automatic classification techniques • Three main automatic classification paradigms: • Expert Systems: Use pre-defined rules to process features and arrive at classifications. • Require explicit a priori knowledge of rules • Great deal of effort required to change once implemented • Unsupervised Learning: Cluster the data based on similarities that the systems perceive themselves. No model categories are used. • Categories generated “objective” and not likely to correspond to categories used by humans

11. Automatic classification techniques • Supervised Learning: Attempt to formulate classification rules by using machine learning techniques to train on model examples. Previously unseen examples are classified into one of the model categories using the patterns learned during training. • Require a pre-defined taxonomy and pre-classified training examples • Supervised learning is the best option for the particular problem of genre classification • Several possible implementations: nearest neighbor, neural networks, induction trees, etc.

12. Forming genre taxonomies • Using hierarchal taxonomy allows the inclusion of both broad and specialist categories • Could devise rational but artificial categories • Not realistic and therefore not useful • Experimental approach • Music industry categories (e.g. Billboard, Grammies, etc.) • Specialty shows on TV and radio • Specialist interviews (DJs, music reporters, etc.) • Retailers, including on the Internet

13. Forming genre taxonomies • Data-mining techniques • Computers automatically search text resources on the web and attempt to form categories and correlations • Holds a great deal of potential, but is difficult to implement and still untested

14. Existing automatic genre classification systems • Most experiments to date have been with audio rather than symbolic recordings • Success rates of between 61% to 93% when dealing with between 3 and 10 categories • Only a few studies of symbolic classification • 63% to 84% for between 2 and 3 categories

15. Classification experiment • Did initial experiment to test viability of symbolic classification • Used 225 MIDI recordings divided into 3 parent genres and 9 sub-genres: • Classical • Baroque, Romantic, Modern • Jazz • Swing, Cool, Funky • Pop • Rap, Country, Punk • Categories were just roughly chosen for test purposes • Could just have easily used other formats, such as Humdrum or GUIDO

16. Classification experiment • Performed classification with 8 neural networks and a coordination system. • Factors increasing difficulty: • Only 20 recordings per genre were used for training for each run to represent a wide range of musics within each category: increased difficulty. • Only 20 features were implemented

17. Classification experiment • Average success rates: • 85% for parent genres • 58% for sub-genres • Results were fairly consistent across training runs • These rates comparable to existing audio classification systems using similar numbers of categories and better than existing systems using symbolic data. • Encouraging

18. Classification experiment

19. Classification experiment • Future improvements: • Use realistic taxonomy • Larger training and sample set • More features • More sophisticated classification methodology • Feature selection sub-system for each level of classification hierarchy

20. Software interface • A user-friendly interfaced is being developed that will be ported to the classification system. • Easy to use and flexible so that it can be used for a variety of research and applied purposes by people with little technical expertise. • Allows user to: • Input and edit arbitrary taxonomies and lists of recordings • Choose which features to extract • Evaluate the usefulness of particular features in different contexts • Evaluate effectiveness of different classification techniques • Additional features can be designed and added to the software easily and painlessly by anyone with some basic Java programming skills.

23. Conclusions • Could use system such as this to study • Particular taxonomies • How well different features perform in different contexts • Differences between and definitions of particular genres • Could easily adapt system to other types of classification • Composer / performer • Historical / geographical / cultural characteristics • Personal preferences • Practical applications • On-line musical databases of any kind

24. Questions