Kalaja, P. & L. Nieminen (toim.) 2000. Kielikoulu ssa – kieli koulussa.
AFinLAn vuosikirja 2000. Suom en soveltavan kielitieteen yhdistyksen julkaisu ja no. 58. Jyväskylä. s. 265–280.
––––
THE RE-INTRODUCTION OF ORAL ENTRANCE EXAMINATIONS AT A TRANSLATION STUDIES DEPARTMENT
Mike Garant & Leena Immon en University of Helsinki
At the Univ ersity of Helsinki Translation Department in 1999, oral interviews were re-introduced for the top candidates from the written entrance examination in an attempt to offer study places to applicants m otivated enough to graduate. This article presen ts a descriptive analysis of an on-going study that addresses both q ualitative and quantitative issues. Results suggested that oral exam ination did not improve no-show statistics and that waiting lists wou ld be useful. This article presen ts teacher perceptions of what makes a poten tially good student and a breakdown o f th e m an-ho urs used.
Results indicate that on-going workshop training is the most effective way to produ ce consisten t, systema tic oral examin ations. Interviewers who participated in all phases of preparation, execution and evaluation were observed to have a positive attitude toward changes in departmental policy.
This study shou ld be useful to those considering alternatives for promoting effectiveness in translation and foreign language education.
Keyw ords: translation studies, entrance examination, motivation, action research
1 INTRODUCTION
In the summer of 1999, the Translation Studies Department in Kouv ola of the University of Helsinki introduced an oral entrance examination as a component of their admissions proce dure in order to improve the department's retention and graduation rates. This study is an action research project (van Lier 1988, Cohen and Manion 1985: 174) reporting on the effects of such intervention in our department's educational system. We, the memb ers of the entrance test planning committee, believed that higher retention and graduation rates could be achieved by recruiting students with higher levels of motivation and better communication skills. Oral entrance examinations were seen as one possible method of ac- hieving this goal.
Since it is still too early to gather data on retention rates, this article will describe and analyze the methods used in carrying out the tests including pre- and post-test interviewer training, question- naire construction and interview techniques. Then it will describe faculty attitudes toward student selection. It will also present a break-down of the time involved in the planning and execution of such an ende avor. It is hoped that the present study will be useful for other departments in Finland, and possibly elsewhere, who may be contemplating the introduction of oral entrance examinations for student selection. The reader should bear in mind that this is a prelimi nary report of an on-going research project which concent- rates only on certain issues related to the introduction of our oral entrance examination.
2 BACKGROUND
Oral entrance tests were phased out of Finnish translation studies departments in 1988 because of budget restrictions, the work that they entailed and the feeling among some of the faculty memb ers that they were not beneficial. The University of Turku's Translation
Department retained their oral examination because of their emphasis on interpretation. By 1998, many of the faculty in the department in Kouv ola felt that the re-introduction of an oral com- ponent might be a worthwh ile ende avor.
Huhta (1992) sugges ts that the length and number of sections in an examination increase its reliability. With the re-introduction of this section, our departmental entrance examination consisted of (a) the upper high school matriculation examination scores, (b) a test on preassigned readings, (c) a three part second language (L2) section, and (d) an oral interview. This addition, we believed, would significantly enhance the reliability of our examination.
In 1998, we offered study places for 78 students in our depart- ment after the written examinations were completed. Table 1 shows that 77.2 % of the students who were offered places in our depart- ment actually began their studies. This phenomenon is common and expected in Finnish universities where students apply to a specific department as opposed to applying to the university as a whole as in countries like the United States.
TABLE 1. Offered study places vs. Intake in 1998.
Student places Enrolled
Eng lish 28 22
Swed ish 20 17
Russian 12 10
German 12 10
Total 88 68
In Finland, students have the academic freedom to enroll in one major subject and study another major subject. For example, high school graduates with excellent L2 skills have been known to take the entrance test to both the Law Department and the English De- partment. When they fail the law test and pass the English test they enter the university and study law until they are able to pass the entrance test for the law major. Such students may or may not major or minor in English. This has a negative effect on statistics as they are now calculated. More concretely, in 1998 in the German
section of our department, only six of the ten students who enrolled in the department in 1998 actually took a course there. As of Febru- ary 2000, the four "missing students" are still in the University of Helsinki registry as enrolled students in the translation department.
Therefore, it is no wonder that the average graduation rate in the Faculty of Humanities is only 43 percent (Piekala 199 9). This calls into question the practice of calculating graduation statistics based on offered study places, especially when such statistics serve as the basis for funding.
3 FINNISH UNIVERSITY ENTRAN CE EXAMINATIONS AND SUCCESS IN STUD IES
University entrance in Finland is usually determined by a com- bination of scores from the matriculation examination, which is a national examination taken at the end of high school, and depart- mental entrance examinations (Garant 1997, Patosalmi & Pakkanen 1996, Patosalmi 199 6). Test security makes systematic research into the reliability and validity of Finnish university departmental entrance tests somewhat problematic. In some cases, the test desig- ners themselves may not feel comfortable allowing external resear- chers examine what they test and how they test it. At the moment, there is very little public or private discussion related to the matter and these tests are generally not questioned. It is fair to say that these tests are generally perceived as being reliable and valid as well as difficult but that this perception is based more on a general feeling than researched fact.
How ever, some studies have been conducted that are relevant to this topic. Hakoniemi (1995) studied the Histo ry Department at the University of Turku and suggests that according to his data, entrance examinations are more important than matriculation exa- mination scores in predicting which students will be successful in the university. In contrast, Lonka's (1986) work on data from the University of Helsinki Department of Psychology suggested that
some of the entrance test data she examined may show gender bias.
Huttunen and Ravinen (1989) studied translation department ent- rance examinatio ns at the University of Joensuu and sugges t, among other conclusions, that matriculation examination scores correlate with success in studies but there is no connection between these and the students' performance on the entrance examination.
They also suggest that the mother tongue score on the entrance test can be used to predict who will do well in their studies.
Liikka's (2000) comprehensive study comparing success on entrance examinations and success in studies at the Helsinki Uni- versity Translation Department in Kouv ola for the intake years 1988–1991 showed that there were significant differences in suc- cess rates depending on the year. She also found that the English and Finnish tests had a negative correlation and that the Finnish test correlated with the matriculation examination. Like Hakoniemi (199 5), she found that the matriculation test seemed to be the best indicator of success in studies among the students who entered the department between 1988 and 1991. Among her recommendations, she sugges ts that waiting lists be adopted. Her study also indicates that research and development within this area is worthwhile.
4 PLANNING AND CONDUCTING ORAL INTER VIEW S
One potential problem with using oral interviews and testing in general as part of a selection process is what Takala (1997) refers to as inter-tester irregularity. It is unden iable that this Rashomon effect, where no two people see the same event the exact same way (Fanslow 1977, 1992, 1993) applies to oral testing. To ensu re test reliability, the entrance test planning committee decided to take a series of steps to minimize the differences between interviewers. A prelimi nary symposium with outside experts was held in order to discuss the re-introduction of these tests and to share ideas. Oral examinations from the translation department at the University of
Turku were first considered as a model (see Tommola 199 8); howe- ver, a different approach to the examination process was thought to be more suited to our needs, mainly because Turku concentrates more on interpretation than Kouvola.
Early in the planning process, we decided to use only faculty for the oral examinations and not hire part-time interv iewers for the day of the examination. This meant that the interv iewers would be around before, during and after the examinations for training and feedback. It also meant that they would have a personal stake in selecting who they felt would be the most successful students. Nine of the eleven interv iewers who participated had more than ten years experience in the department. Many had been teaching for 20 or 30 years. The reason for preferring experienced teache rs was that their years of classroom experience was thought to serve as a sufficient basis for judgement that would better able them to select potential- ly successful students.
Another step to increase reliability was to introduce on-going training for all interviewers. Before the oral examinations, two workshops were conducted with the faculty memb ers who would conduct the interviews to discuss the oral interview pro cess in detail. Prelimin ary question naires were formulated and prelimi nary sample interviews were conducted with four local upper high school students. The sample interviews were videotaped so that they could be watched for training purposes (Takala & Huhta 199 7). At our workshops, grading was discussed in great detail in order to try to move toward as much inter-rater standardization as possible. Still, since we are dealing with human beings, there will probably always be some variation in observer-interviewer percep- tions. The teache rs who participated in the workshops also helped to improve the questionnaires and interview process. This made them part of the process of innovation within the department and increased their willingness and motivation to participate.
It was decided that two intervi ewers would be present in each session: one to conduct the interview and one to observe and take notes. This was done in order to provide a clear written record of
each interview and because an observer might be able to see things that the actual interviewer might miss. The written records that were produced were useful in order to provide transparency to the examination. Following the interviews, the interv iewers decided on the mark that each applicant would be given. In case of disagree- ment, both graders' marks would be averaged. How ever, inter- viewe rs did not report cases of assessment disagreement in post- examination feedback forms or the workshop.
Three times the number of students that we had study places for were invited to the oral examination. All but one of the appli- cants who were invited to the oral examination came and participa- ted in the interview. The oral tests were conducted in one day one month after the written examination. This was done because the university bureaucracy mandates that one month elapse between the time of the written examinations and the oral entrance tests to allow time for possible appeals. The interviews were videotaped to provide transparency in case a student felt they were treated unfair- ly in the selection process as well as to provide material for trai- ning and improveme nt.
A residential follow-up workshop to assess the results of the examination was conducted three months after the oral exa- minations. This gave the participants time to think about the pro- cess and get an impression of the incoming students. A residential workshop was thought to be more effective than a series of mee- tings in the department, because it gave participants a more relaxed format for real reflection and discussion away from the normal distractions of the workplace. It also contributed to team building.
At the workshop we discussed how the tests went. The overall impression among the faculty was that they were quite successful.
A number of the videotaped interviews were watched and stand ard grading was discussed in great detail. It was felt that a waiting list was of primary importance because of the number of no-shows.
Based on these recommendations, the central administration has now agreed to allow us to introduce a waiting list beginning with our year 2000 intake.
1The qu estionnaire is not inclu ded in the ap pendix for test security. Copies may be o btained from the authors.
Another result of the workshop was that the basic questions on the 1999 ques tionn aire1 was thought to achieve the purpose of structuring the interview and eliciting an adequate amount of infor- mation from the applicant. One drawback that was pointed out was that the pape rs involved were difficult to fillout within the time allotted. To remedy this, a form that could be marked easily was adopted for the 2000 examination. We also came to the conclusion that the interview should be as much like a conversation as possi- ble.
The 1999 selection process was conducted in the applicant's L1 because of university regulations. They state that if questions are asked in the L2, a separate test shall be conducted. The 2000 examination will have the following structu re table 2):
TABLE 2. Interview Structure.
Warm-up L1 2-3 minutes
Interview L1 6-10 minutes
Follow-up L2 3-5 minutes
The time allotted for each interview is 10–15 minutes. The warm- up (approximately 2 minutes) will include general questions about the applicant's background with the goal of creating a more relaxed atmosphere. The L1 interview will consist of a series of questions related to motivation and should last 6–10 minutes. The L2 follow- up will include a few questions in the target language to get a small sampling of the student's abilities. Intervi ewers will have to bear in mind that the follow-up sampling is limited and is not intended to assess the student's overall abilities. This is done in the written examination.
5 LESS POPULAR LAN GUA GES
Another factor which affected the interview structu re was the different proficiency levels of incoming students in different langu- ages. Beginner or elementary levels at the time of admission are common for some of the languag es that are taught in Kouv ola and conducting full-scale L2 interviews in the language would not be fruitful. This influenced our overall test construction because of the desire for more uniformity across language sections. The reason for the low levels is most likely the choice of languages studied in comprehensive school (table 3).
TABLE 3. Choice of foreign languages in t he comprehensive school in 1997.
Foreign Language Primary education (grades 3–6 )
% of pup ils
Low er second ary education
(grades 7–9)
% of pup ils
Eng lish 93.2 98.5
Swed ish 6.0 91.7
German 12.9 21
French 2.9 8.3
Finnish 5.4 5.4
Russian 0.4 1.2
Other, e.g. Sami, Spa- nish, Italian, Hebrew
0.3 0.9
Source: Statistics Finland 2000.
For example, Russian translation majors often begin at elemen tary levels because of language selection trends in primary education.
Unless these trends change in elemen tary and secon dary schools, we will have to continue training Russian transla tors from elemen- tary levels. This naturally affects the intake process and how we conduct the interviews.
In addition, we feared that the oral interview process would become less effective if separate L1 and L2 interviews were con- ducted, as they were before 1988, because personnel shortages
would make it neces sary to hire part-time outside interviewers.
Beginning in 2000, we have been granted permission to add a few question at the end of the interview in the target language. It was felt that this could give the interviewer some idea of the applicant's L2 ability.
6 TEACH ER PERCEPTIONS OF A POTENTIA LLY SUCCESSFUL APPLICANT OR A GOOD
TRANSLATION STUDENT
A primary question throughout the planning and examination pro- cess was and is "What is a good student?" or "What are we looking for?" This question has been discussed extensively in the field of language teaching in general (see Berman 2000, Green 1995, Nai- man et al. 197 8). Discussions about what influences retention rates and predicts who will become assets to our department entail a variety of factors. The work group on our oral entrance exa- minations were able to verbalize some of these factors. It was felt that studen ts who come to the interview show some motivation.
As stated previously, all but one of the invited applicants participa- ted in the interview. This was taken as an indication of at least some degree of motivation.
Learners should demo nstrate communicative competence (Hymes 197 2), including gramm atical, sociolingu istic and stra- tegic competence. It was thought that L1 interviews may be better for judging these qualities, especially for languages like Russian where, as previously stated, the proficiency level of incoming students is generally quite low. After all, we were not trying to recruit transla tors and interp reters but rather undergraduate stu- dents with the potential to become transla tors and interpreters.
Communicative ability was seen as favorable. Proper pronun- ciation and enunciation were seen as favorable. Students who tended to volunteer information by giving complex, multi-
layered answers were seen as more favorable. This was a so- mewhat open question, because some people are more talkative than othe rs and one need not talk all the time to have the potential to become a translator or interp reter. Let it suffice to say that one word answers were generally seen as less favorable, if they were all the applicant offered during the entire interview. The rate of speech was not taken into account for assessment, since slow speech does not necessarily mean a slow rate of thinking and there may be longer pauses and slower uptake as part of the communica- tion system in Finland (Lehtonen & Sajav aara 198 5). For example, an American with little experience with Finnish learners may mi- sinterpret and make a negative value judgement about certain common Finnish communication style characteristics. The commit- tee thought that the content of answ ers should be taken into account and we were more interested in whether and to what degree a com- municative act was actually done and performed (Hymes 1972:
281)
Specific interests and hobbies were seen as more favorab le because it indicates that an individual is more active in the com- munity. An interest in reading and literature was seen as favora- ble. One of the auth ors conducted his part of the interviews with a lecturer who had about 30 years of teaching experience. He put forth that in the pre-1988 interviews, a large number of the candi- dates indicated a keen interest in literature. Interestingly enough, of the 35 candidates that one pair of raters interviewed in 1999, only three went into any length explaining what they read and why. It was an indication, the senior colleague informed us, that times had changed and not for the better.
Computer skills were regarded as favorab le because of all of the employment opportunities in documentation, localization and other translation-related fields. Of course, these skills can be and are taught in our department so computer skills must be seen more as a plus than as a determining factor.
Still other factors appeared to influence the selection process in oral entrance examinations. These are abstract and difficult to
put on pape r. They have to do with the overall perc eption of the candidate which may include body language and the overall projec- tion that the applicant gives rather than the content of his or her answers. Such factors may reflect bias. At the moment, we are conducting research into what these factor are and how they can be put into a standardized, written format.
7 MAN-HOU RS
When the re-introduction of oral examinations was first discussed, it is fairly certain that the administration's first question was how much this new innovation would cost. How ever, the conven iently timed introduction of a comprehensive work-time schedule of 1600 hou rs per year, including everything, made it possible to re-intro- duce oral entrance examinations at little or no out-of-pocket cost to the university. Table 4 indicates that a number of hou rs were invol- ved in the planning and execution of the examination.
TABLE 4. Man-h ours in the 1999 oral entrance examination.
Preliminary symposium 20 participan ts x 4 hou rs 80 hou rs Test interviews 2 teachers x 4 hou rs 8 hou rs 2 seminars 2 x 11 interviewers x 3 hou rs 66 hou rs Oral entrance test 12 interviewers/staff x 9
hou rs
108 hou rs Residential follow-up
workshop
8 interviewers x 8 hou rs 64 hou rs
Subtotal 326 hours
Planning, copying, other work 124 hou rs
Sup port staff 100 hou rs
Total 550 hours
In the case of our department, serving on the oral entrance exa- mination was included in the overall number of hou rs involved in the yearly contract. Participants did not receive extra pay for the interviews. The residential workshop that we held in Tallinn, Esto- nia and the pay for the outside experts at the first symposium we
held were the project's main out-of-pocket expenses. Before the tests were discontinued in 1988, hourly wages had been paid to the oral exami ners and they had become relatively expensive affairs.
Table 4 is meant to give a rough idea of how many man-h ours the test included. One can see from such a table how conducting sepa- rate L1 and L2 examinations could grow into a labor-in tensive operation. Departments that may be contemplating the introduction of their own oral entrance examination should take into account that planning time and other variables may differ, depending on the individual department.
8 ENROLLMENT
The introduction of oral examinations may or may not improve the retention rates and graduation statistics of our department. It is still too early to tell. Howev er, it has provided a framewo rk for the faculty to define what they are looking for in student selection.
This, in turn, has helped us define the goals of translation education in our department.
The oral examination introduced by the Department of Trans- lation in Kouv ola did not achieve the goal of increasing the number of students who actually enrolled in classes in the Department.
Table 5 shows that even though oral interviews were conducted, the number of students who were offered places and enrolled re- mained constant.
Table 5. Offered study places vs. Intake in 1999.
Student places Enrolled
Eng lish 25 18
Swed ish 20 15
Russian 20 13
German 12 6
Total 77 53
The number of students who did not enroll remained fairly consis- tent with previous years. This has made it possible for our depart- ment to establish a waiting list for incoming students beginning in fall 2000. Hopefully, this will reduce the number of unfilled study places at the department. This would make for a more efficient use of the tax-payers' money.
9 CONCLUSION
Since the number of students who were offered places and enrolled remained more or less constant, any improvement in this year's intake is qualitative rather than quantitative. The general opinion in the department is that the communicative abilities of the new stu- dents this year is quite good and an improvement over last year's intake. How ever, these sorts of conclusions at this stage are subjec- tive and preliminary.
This action research project has already achieved one major result: the adoption of a waiting list for incoming students. This was neces sary for some of the language sections within our depart- ment. This project has also provided a prelimi nary taxonomy of criteria for assessing a good translation student.
Another positive aspect that the oral examination has brought about is bringing the staff and faculty closer toge ther. One lecturer summed it up by saying that he enjoyed interviewing the students and seeing who was interested in studying in our department and why. Thus, instru ctors gave the re-introduction of the oral exa- mination overwhelmin gly positive reviews and felt that it was wort h continuing in the future. Further research into the area is currently being conducted and should bring about a greater unders- tanding of oral testing in Finnish university entrance examinations and oral testing in general.
REFEREN CES
Berman, D. 2000. National Resource Center for the First-Year
Experience and Stud ents in Transition – WWW Resource Center.
Univ ersity of Sou th Carolina. >http://www .sc.edu/fye> (Cited May 11, 2000)
Cohen, L. and L. Manion. 1985. Research methods in education (2nd ed.).
London: Croom Helm.
Fanslow, J. 1993. Smile, your on candid camera. The Language Teacher 17, 3–5.
Fanslow, J. 1992. Contrasting coversations. New York: Long man
Fanslow, J. 1977. Beyond Rashoman-conceptualizing and describing the teaching act. TESOL Qua rterly 11, 17–39
Garan t, M. 1997. Intercultural teaching and learning. Eng lish as a Foreign Language Education in Finland and Japan. Univ ersity of Jyväskylä.
Jyväsk ylä Studies in Communication 8.
Green, D. 1995. Good learner studies revisited. The Language Teacher 19, 9–12.
Hak oniem i, J. 1995. Valmistuu ko kuuden laud aturin ylioppilas myös yliopisto sta erinom aisin arvosanoin? Turun yliopiston historian koulutusohjelmaan vuoden 1980 jälkeen valittujen pääsykoe- ja opinto menestys. Turun yliopisto. Historian laitoksen julkaisu 33.
Huhta, A. 1992. Suullisen ja kirjallisen kielitaidon testauksen kehitys- linjoja viime vuosina. In J. Tom mola (ed.) Kääntäjänko ulutuksen opiskelijav alintaseminaari 1992. Turku: Univ ersity of Turku.
Kääntämisen ja tulkkauk sen keskus, 21–60.
Huttunen, M. and I. Ravinen 1989. Savonlinnan kääntäjä nkoulutu slaitoksen vuoden 1983 valintakoe opintom enestyksen selittäjänä. Joensuu
University. Unpu blished Master’s thesis.
Hym es, D. 1972. On communicative competence. In J.B. Pride & J.
Holmes (eds.) Sociolingu istics. Hardm ond s: Penguin.
Lehtonen, J. and K. Sajavaara 1985. The silent Finn. In D.Tann en &
M. Saville-Troike (eds.) Perspectives on silence. New York: Ablex, 193–200.
Liikka, S. 2000. Success in entrance examination vs. success in studies:
a case study on Eng lish Translation Studies. Univ ersity of Helsink i, Translation Studies Department in Kouv ola. Unpu blished Master’s thesis.
Lonk a, K. 1987. Psykologian valintakokeet Helsing in yliopistossa 1986 – testien toimivuu s. Helsing in yliopisto. Opintoasiaintoimiston
julkaisu ja 26.
Naiman, N., M. Frölich, H. Stern, & A. Todesco. 1978. The good language learner. Research in Education 7. Toronto: Ontario Institute for
Studies in Education.
Patosalm i, I. 1996. Yliopiston opinnot 1997–1997: Korke akou lu ja tiedeosasto. Opetusministeriö. Helsink i: Edita.
Patosalm i, I. & P. Pakkan en 1996. Korkeakou lujen valintaopas 1996.
Opetusministeriö. Helsink i: Edita.
Piekkala, J. 1999. Aloittaneet ja tutkinnon suorittaneet – keskiarvot aineittain. Hum anistinen tiedekunta.
>http ://www .helsink i.fi/hum /tdk/al-tu -ka.htm l> (Cited Febru ary 2, 199 9).
Richard s, J.C. & T.S. Rod gers 1986. Appro aches and methods in language teaching. Cambrid ge: Cambridge Univ ersity Press.
Rutherford, W. & M.S. Smith 1988. Grammar and second language teaching. A book of readings. Boston, MA: Heinle & Heinle.
Statistics Finland. 2000. Education in Finland: Statistics and Indicators.
Helsink i. >http ://www .stat.fi/tk/he/ed ufinlan d/edu t.html> (Cited Febru ary 24, 200 0).
Stern, H.H. 1983. Fundamental concep ts of language teaching. Oxford:
Oxfo rd Univ ersity Press.
Takala, S. and A. Huh ta 1997. Presentation at the National Certificate Interviews. National Board of Education, March 12, 1997.
Takala, S. 1997. Key note Speech at the AFinLA Autumn Symposium, Tallinn, October 14–15, 1997.
Tom mola, J.1998. Oral entrance examinations at the Univ ersity of Turku Translation Dep artmen t. Presentation at the Fall Symposium on Oral Entrance Tests, Kouv ola, Octob er 14, 1998.
van Lier, L. 1988. The classroom and the language teacher. London:
Longman.
