Abstract: The Advanced Writing Test of the General English Proficiency Test (GEPT), developed and administrated by the Language Training and Testing Center, has adopted a computer-based test (CBT) mode since 2010, providing test-takers an alternative to traditional paper-based tests (PBT). As CBT and PBT are both currently administrated, the questions as to whether test-takers' performance under the two different testing modes are comparable when test tasks and time limits are equal, and whether there is a mode effect on raters are yet to be investigated empirically. This preliminary study compared test-takers' performance under the two testing modes, including score comparability and an analysis of textual and linguistic features. The results indicate that the two tests are comparable, and it is practical to continue to offer both CBT and PBT formats. However, more evidence is required to defend the comparability between the two tests.