Plan for Session 3: Annotation and Analysis of Data II (continuation of Session 1) FIRST PART: fix XML files and finish annotation --------------------------------------- - fix the wrong XML files. - make sure that the fixed files can be opened - finish annotation - those that have finished, go over the annotation with the "search" tool to spot possible mistakes SECOND PART : calculate inter-annotator agreement with kappa ------------------------------------------------------ - pairs that have annotated the same file, compress the directory and send it by email to your partners (and to me) - short intro to kappa statistics - how to use the statistics facility of MMAX - calculate kappa: - speaker role (it should be 1, perfect agreement) - marker (idem) - dysfluency type - forward looking - backward looking - analysis of results: - where did you encounter disagreements? - hypothesis of why? - come forward in pairs and present results. THIRD PART: statistical comparison of conditions with t-test -------------------------------------------------- - short intro to t-test - how to use the online t-test calculator - each of the two groups has 6 files with 3 different conditions that can be compared: - Turn taking (3 FTT/3 P2T) - Speaker role (6 players / 6 executors) - Gender (3 female players/ 3 male players) - possible measures (open list - creativity welcome!) - average of dysfluencies per turn across conditions - number of forward looking acts across conditions - number of backward looking acts across conditions - ... - make hypothesis to try to explain the results obtained (especially when there is an statistically significant difference across conditions) - come forward in groups and explain results.