Title

1. PCI 2014 Quality in Education Technologies Title Authors SotiriosKontogiannis IoannisKazanidis Stavros Valsamidis AlexandrosKarakos Course opinion mining methodology for knowledge discovery, based on web social media

2. Outline PCI2014 Quality in Education Technologies • Introduction • Method • Case study • Results • Proposed framework • Proposed opinion mining system architecture • Discussion and conclusions

3. Introduction PCI2014 Quality in Education Technologies • Primarily, authors used the LMS platforms of academic institutions where course knowledge and course interest collide (apart from the classroom) • Then the authors focused on student-course appreciation by using questionnaires and course grades • The results of this twofold evaluation were sometimes contradicting

4. Introduction PCI2014 Quality in Education Technologies • Since LMS course evaluation is based on a scientific evaluation approach, authors concluded that a similar approach is needed to replace the questionnaires with a less guided and manipulous scientific methodology • That is, knowledge extraction and discovery from existing social networks where people express freely and non guided opposition for an academic course

5. Introduction PCI2014 Quality in Education Technologies • This paper • This paper proposes a framework for applying opinion mining in social networks. • The goals of the proposed framework is to • (a) extract useful textual information from social networks of blogs regarding learning course activities or processes and • (b) apply opinion mining techniques on the extracted text in order to discover the positive or negative opinions concerning each course.

6. Framework PCI2014 Quality in Education Technologies The 4 steps for opinion mining in a social network

7. Case study PCI 2014 Quality in Education Technologies • Study population and context • a microblog by following posts relative to the educational level and institutional services of a Greek Technological Educational Institute (TEI) • a period of six months • comments of different commentators from the same department in the Greek language • It can be accessed at https://www.facebook.com/ groups/69887509784/

8. Case study PCI 2014 Quality in Education Technologies View of the experimental microblog

9. Case study PCI 2014 Quality in Education Technologies 1st process Create, train and store the classification model for automatic categorization of text as positive or negative

10. Case study PCI 2014 Quality in Education Technologies 2nd process Apply the model to new data to automatically be categorized into positive and negative

11. Results PCI 2014 Quality in Education Technologies • Words like “painful”, “discourage”, “damage”, “unemployment”, etc. were found to have negative sentiment whereas words like “profitability”, “prosperity”, “interest”, “success”, etc. were found to have positive sentiment in comments. • The comments from the microblog of our study were split in half between positive and negative about the educational institute offered services and knowledge. • In other words our sample was split evenly on feelings.

12. PCI 2014 Quality in Education Technologies Proposed Framework • Twofold methodology for the evaluation of an academic course • LMS course web usage mining evaluation process with the use of a three tier evaluation architecture and the measures, metrics and algorithms • Opinion mining process

13. PCI 2014 Quality in Education Technologies Proposed Framework • Opinion mining process • Step 1 • Source selection and monitoring of UGC sources • educational institution channels • general source channels • Step 2 • Source crawling engine and initialization mechanism - crawling design and crawled content storage capabilities

14. PCI 2014 Quality in Education Technologies Proposed Framework • Opinion mining process • Step 3 • Semantic enrichment engine – semantic enrichment design • If the text content is semi-structured, then the use of either natural language processing (NLP) or other text analysis techniques in order to interpret (grammatically and syntactically process) each sentence, prior to the assignment if possible a sentiment to it • The effectiveness of the different approaches largely depends on the quality of the raw text to be analyzed; in general, NLP and therefore semantic enrichment is effective on syntactically-correct texts while it falls short on ill-formed sentences or when Internet dialects are used

15. PCI 2014 Quality in Education Technologies Proposed Framework • Opinion mining process • Step 4 • Sentiment Analysis processes, metrics and algorithms • This step involves the use of opinion mining over a adequate level of enriched sentences of user text. For this process to be accurate a very well trained dataset of opposite and negative sentences is required, with a high level of polarity among those datasets • For such functionality to be performed in an automatic and real-time manner or even to be a self trained feedback mechanism, appropriate metrics or scores need to be defined as well polarity judgment algorithms need to be proposed and validated

16. PCI 2014 Quality in Education Technologies Proposed Framework • Opinion mining process • Step 4 • Sentiment Analysis processes, metrics and algorithms • If a polarized dataset of high confidence is pertained, then using a Part Of Speech sentence (POS) tagger (or NLP clustering of a sentence) and a trained Bayesian sentence classifier, we can pinpoint that a sentence tokens belong to a class of sentences by looking at the tokens probability • The likelihood of a sentence can be calculated to be negative as the number of negative sentences in that class over the total number of negative sentences in all classes • Likelihood_sentence_Negative= Number of class negative sentences / Total number of Negative sentences

17. PCI 2014 Quality in Education Technologies Proposed Framework • Opinion mining process • Step 5 • Visualization and courses ranking mechanism based on opinion results

18. PCI 2014 Quality in Education Technologies Discussion and Conclusion • This paper proposes • a framework and a testbed system used for applying opinion mining in blogs regarding course educational content. • a context sensitive sentiment analysis methodology which provides human like sentiment analysis based on semi supervised learning structures

19. PCI 2014 Quality in Education Technologies Discussion and Conclusion • The expected benefits of applying such a framework are the following: • Qualitative presentation of people concerns over a course and course user preferences. • Recording user’s problems and negative or positive user opinions concerning educational courses they are interested in without spatial and temporal restrictions

20. Discussion PCI 2014 Quality in Education Technologies • Research Limitations • The accuracy of the WSD (Word Sense Disambiguation) program (OpenNLP) within this approach, so that the exact sense of each term can be identified and exact sentiment scores can be calculated • The framework has been only applied to a specific microblog for a set of three courses. In order to better benchmark it, it must be also applied to other blogs as well

21. PCI 2014 Quality in Education Technologies THANK YOU FOR YOUR ATTENTION Course opinion mining methodology for knowledge discovery, based on web social media SotiriosKontogiannis IoannisKazanidis Stavros Valsamidis AlexandrosKarakos