Influence of Language and Task Complexity on Internet Searching Performance

The Web now provides instant access to an unprecedented amount of information that was unthinkable even 20-30 years ago. However, the full potential of the contents available through the Internet can only be realized when one can speak the lingua franca online, English. According to a recent survey (Web Technology Surveys, 2015), more than 55% of web contents are in English. Previous studies have shown that non-native speakers’ English proficiency affects their searching behaviors and performance (Bilal, 1989, Liao, Finn & Lu, 2007, Onwuegbuzie & Jiao, 1997). In this study, we try to investigate the effect of search task languages and task complexity on searching performance. In other words, we want to understand the extent to which the use of a foreign language becomes a barrier – when one’s native language is not English but one has to use English for searching on the Web for personal or work related tasks.
Primary research questions are as follow: RQ1) Will search tasks in Korean (native language) and tasks in English produce different search performance? RQ2) Will subjects’ proficiency in English and task complexity affect search performance?
A total of thirty students enrolled at a top private university in Korea were recruited as study subjects. These students are considered to be prime candidates for the study in the sense that they are quite likely to utilize the Web’s English contents for their work as well as for personal use.
For the operationalization of the study, the following independent and dependent variables were measured.
Independent variables: task language (Korean, English), task complexity (Simple, Complex), English proficiency level (High/Intermediate/Low corresponding to TOEIC, TOEFL test scores)
Dependent variables: SERP (search engine results page) time, number of queries, total search time, correctness of search results
We set up a quasi-experimental design in which thirty subjects are randomly assigned to a set of eight different search tasks (modified from Singer, Norbisrath & Lewandowski, 2013) containing an equal number of simple and complex tasks and an equal number of tasks in Korean and in English. The subjects went through a requisite pre-test instruction and informed consent was obtained. The subjects are asked to identify a set of web sites as their answers to queries. The answers were evaluated by two research assistants independently in three different levels (correct, partly correct and wrong/did not submit). The entire search process was recorded using screen recording software. The recordings form the primary data from which we measure search performance metrics. In addition, a survey was administered after subjects complete the tasks with regard to their personal evaluation of the search process and outcomes.
Preliminary results show that there is a significant difference between simple and complex tasks in terms of SERP, number of queries used, correctness of results and total search time. However, task language does not seem to have affected search performance for this study group. In addition, students with high English proficiency test scores show comparable search performance in English tasks compared with lower test scores. But we note differences in behavioral patterns (different search engines used and search tactics) in two groups.

Keywords: Web searching, Task language, Task complexity, Search performance

Poster Session
Location: Lecture Hall Aula Magna Date: May 20, 2015 Time: 4:30 pm - 4:33 pm Jeayeon Byun Wonsik Shim