This is a report on the final phase of our 4-year grants-in-aid project which was aimed at creating a reliable metric for assessing Japanese youths’ readiness for global citizenship. The metric is based on the 40 descriptors validated in the preceding study. These 40 descriptors for global citizenship for future Japanese global resources were previously ascertained by an extensive online survey with more than 400 international workers who are currently working in various sectors of industry. The 40 descriptors cover 4 areas: knowledge of language and culture, intercultural competence, critical thinking skills, and generic competencies. First, a portfolio-type assessment tool consisting of essay-type test items and checklists was constructed after a series of validation experiments, in which different combinations of them were tried out, considering the different nature of skills and competencies being assessed. A simplified version was also developed for younger learners (aged below 15) by adjusting the difficulty and complexity of the items included in the essay test and checklist. This study reports on further in-depth analysis results of the essay-test items for both general and simplified versions of the assessment tool, in order to elucidate how much of the abilities represented by 40 descriptors the language-medium essay-test items (both in English and Japanese) could measure, using approximately 300 subjects consisting of university, high school, and elementary school students. In the presentation, the statistical test power of essay-test items in terms of validity, reliability and practicality will be discussed, together with the correlations between student self-evaluation and objective teacher evaluation. In addition, detailed linguistic analyses of representative responses were done by both quantitative and qualitative methods. It was found that lexical density and rhetorical sophistication are the two strongest features leading to higher scores, though for younger students, the length of response also affects the score significantly. We tentatively concluded that the essay-test items exhibited strong test power for approximately one-third (12) of the 40 descriptors, while moderate test power for 6 more descriptors. However, the overall judgment of the readiness for global citizenship should be done by incorporating other types of assessment such as checklist evaluation by students and teachers and ideally, supplementary interviews.

Keywords: Global citizenship, essay-test, validation.