Oral Summary Performance Based on the Corpus of Interpreting Competition and its Application to Language Evaluation

doi:10.3969/j.issn.1674-8921.2020.05.011

Abstract

Abstract:

Summaries have long been seen as a main measure for developing and testing listening competence. The present study proposed a set of post listening summary measures and applied them to summary performance by three groups of students (higher level group, medium level group, lower level group) in a nationwide interpreting competition. The results showed that while measures like fluency, connectives, primary propositions and secondary propositions can effectively differentiate three student groups, other measures like accuracy, mistranslation and narration structure are found to be less effective. Based on these findings, this article provided suggestions and guidelines for teaching and listening testing.

CLC Number:

H319

SU Wei. Oral Summary Performance Based on the Corpus of Interpreting Competition and its Application to Language Evaluation[J]. Contemporary Foreign Languages Studies, 2020, 20(5): 101-109.

Figures/Tables 4

References 24

[1]	Bax, S. & S. Chan. 2019. Using eye-tracking research to investigate language test validity and design[J]. System 83:64-78. doi: 10.1016/j.system.2019.01.007 URL
[2]	Brennan, R. L. 2001. Generalizability Theory[M]. New York: Springer-Verlag.
[3]	Hidi, S. & V. Anderson. 1986. Producing written summaries:Task demands,cognitive operations, and implications for instruction[J]. Review of Educational Research(4):473-493.
[4]	Hood, S. 2008. Summary writing in academic contexts: Implicating meaning in processes of change[J]. Linguistics and Education 19(4):351-365. doi: 10.1016/j.linged.2008.06.003 URL
[5]	Johnston, P. 1984. Prior knowledge and reading comprehension test bias[J]. Reading Research Quarterly 19(2):219-239. doi: 10.2307/747364 URL
[6]	Khalifa, H. & C.J. Weir. 2009. Examining Reading: Research and Practice in Assessing Second Language Reading[M]. Cambridge: Cambridge University Press.
[7]	Kintsch, W. & T. A.van Dijk. 1978. Toward a model of text comprehension and production[J]. Psychological Review 85(5):363-394. doi: 10.1037/0033-295X.85.5.363 URL
[8]	Setton, R. & A. Dawrant. 2016. Conference Interpreting: A Complete Course[M]. Amsterdam/Philadelphia: John Benjamins.
[9]	Su, Wei. 2019a. Interpreting quality as evaluated by peer students[J]. The Interpreter and Translator Trainer 13(2):177-189. doi: 10.1080/1750399X.2018.1564192 URL
[10]	Su, Wei. 2019b. Exploring native English teachers’ and native Chinese teachers’ assessment of interpreting[J]. Language and Education (6):577-594.
[11]	Taylor L. 2013. Testing Reading Through Summary: Investigating Summary Completion Tasks for Assessing Reading Comprehension Ability[M]. Cambridge: Cambridge University Press.
[12]	Wang L & C. Chen. 2016. Features of formulaic sequences used by Chinese EFL learners in performing a story retelling assessment task [A]. In G.Yu & Y. Jin (eds.). Assessing Chinese Learners of English [C]. London: Palgrave Macmillan.101-120.
[13]	Yu, G. 2008. Reading to summarize in English and Chinese: A tale of two languages?[J]. Language Testing 25(4):521-551. doi: 10.1177/0265532208094275 URL
[14]	Yu, G. 2009. The shifting sands in the effects of source text summarizability on summary writing[J]. Assessing Writing 14(2):116-137. doi: 10.1016/j.asw.2009.04.002 URL
[15]	方岚. 2008. 建立外语听力困难的诊断指标体系——一项基于中国英语学习者的实证研究[J]. 外语电化教学(5):51-56.
[16]	符荣波. 2013. 口译方向性对译语非流利产出的影响[J]. 现代外语(2):198-205.
[17]	何莲珍、陈大建. 2017. 中国英语能力等级量表结构探微——听力描述语的横向参数框架与纵向典型特征[J]. 外语界(4):12-19.
[18]	金怡. 2016. 中学生英语概要写作研究: 问题与对策[J]. 外语测试与教学(4):38-42.
[19]	李久亮. 2014. 不同文章体裁概要写作任务的Rasch模型分析[J]. 外语与外语教学(4):30-35.
[20]	柳明明. 2016. 英语听后口头复述任务构念研究[J]. 外语测试与教学(2):17-22.
[21]	王海贞、王永双. 2014. 英语专业四级口语考试对学习的反拨效应[J]. 外语测试与教学(2):34-43.
[22]	王蕾. 2005. 英语复述过程的语篇意识[J]. 西安外国语学院学报(2):41-43.
[23]	吴雪峰. 2018. 英语概要写作评分量表研制的实证研究[J]. 中国外语教育 11(2):57-66, 86.
[24]	余东涛. 2016. 一个英汉口头叙述语篇中衔接词语使用的实证研究——以“梨子故事”为例[J]. 西安外国语大学学报(3):9-13.

构念	一级指标	二级指标	含义
听后概述	文本内指标(产出文本)	1. 语言准确性	无语言错误句子占比
	文本内指标(产出文本)	2. 语言流利性	每分钟非流利频次
	文本间指标(产出文本与原文本对比关系)	3. 衔接词	正确译出原文衔接词比率
		4. 错译句	原文内容错译的句子数量
		5. 主要命题	原文主要命题提取数量
		6. 次要命题	原文次要命题提取数量
		7. 叙事结构	原文叙事结构完整度

概述指标	高水平组(n=15)		中水平组(n=15)		低水平组(n=15)
概述指标	均值	标准差	均值	标准差	均值	标准差
语言准确性	0.86	0.07	0.80	0.08	0.73	0.07
语言流利性	2.73	1.03	5.07	1.28	6.67	1.05
衔接词	0.90	0.08	0.69	0.11	0.31	0.09
错译句	3.47	1.06	3.80	0.86	4.27	0.80
主要命题	5.73	1.16	4.60	0.74	3.33	0.82
次要命题	8.07	1.34	5.40	1.24	3.67	1.59
叙事结构	0.97	0.09	0.87	0.13	0.82	0.18

	F值	显著水平(sig.)
语言准确性	10.144	0.060
语言流利性	46.333	0.000
衔接词	42	0.009
错译句	2.901	0.066
主要命题	25.320	0.000
次要命题	37.805	0.000
叙事结构	4.742	0.044

指标	水平组	均值差	显著小平(sig.)
语言流利性	高水平:中水平	-2.33	0.000
	中水平:低水平	-1.60	0.001
	高水平:低水平	-3.93	0.000
衔接词	高水平:中水平	0.21	0.000
	中水平:低水平	0.39	0.000
	高水平:低水平	0.59	0.000
主要命题数	高水平:中水平	1.13	0.005
	中水平:低水平	1.27	0.002
	高水平:低水平	2.40	0.000
次要命题数	高水平:中水平	2.67	0.000
	中水平:低水平	1.73	0.004
	高水平:低水平	4.40	0.000