TEMPTING TO TAG: AN EXPERIMENTAL COMPARISON OF FOUR TAGGING INPUT MECHANISMS
IMPLICATIONS FOR PRACTITIONERS
In this study, several tagging input mechanisms were compared to test which one encouraged users to supply most tags in the context of video tagging. These involved a chatbot mechanism (chatting with a chatbot so tags can be derived from the logs), a bookmarking condition (where users can organize their bookmarks by using tags), and a tag & vote condition (where users can tag videos and rate other users’ tags), as well as a traditional tagging method (tag input box) as a comparison condition. The results show that the advanced tagging input mechanisms do not improve users’ motivations to tag. Therefore, designers of tagging applications have two options. The first is to use a standard tag input box where users can type tags. This is an easy and cheap solution and yielded the same results as the more advanced tagging input mechanisms we tested. The second option is to motivate users to tag by means of implementing other tagging input mechanisms. For instance, tag recommenders or tagging as a game as incorporated in an ESP game2 have been shown to be more promising than the mechanisms we tested in this study.
Taking into account the results from our earlier studies, reported in Van Velsen and Melenhorst (2009), we can state that motivating users’ to tag video content requires the careful selection of the right focus and instruments. The primary motivation to tag a video is to make a video (or other medium) easier to find for others or yourself. This activity can be simplified by using the use of tag recommenders (Melenhorst et al., 2008). An alternative approach may be to make tagging fun by means of a game. Which approach works best will be dependent on the system, the context, and the user. Even though the interplay between these dependencies is a research topic in itself, system developers need to determine the best
approach by exploring the system’s context of use and identifying its target group. Based on this knowledge the most appropriate approach can be selected.
A second finding from this study is that usability is related to users’ motivation to tag.
Therefore, it is important to ensure that a tagging input mechanism is usable. In order to achieve this goal, we recommend applying a user-centered design perspective while creating tagging input mechanisms and testing the interface and interaction design of a mechanism before launch.
Finally, in the case of video tagging, it is very important that tagging input mechanisms do not distract the user too much from watching the video. A novel or relatively complex tagging input mechanism might prove too distracting and has implications for the tagging process and effectiveness. It is paramount to present the users’ primary goal—that is, watching a video—as the main activity in the interface and interaction design of a video application.
ENDNOTES
1. See www.movielens.org 2. See www.espgame.org
REFERENCES
Ames, M., & Naaman, M. (2007, April-May). Why we tag: Motivations for annotation in mobile and online media. Paper presented at the Computer-Human Interaction conference (CHI 2007), San Jose, CA, USA.
Beenen, G., Ling, K., Wang, X., Chang, K., Frankowski, D., Resnick, P., & Kraut, R. E. (2004, November).
Using social psychology to motivate contributions to online communities. Paper presented at the Computer Supported Cooperative Work conference (CSCW ’04), Chicago, IL, USA.
Berendt, B., & Hanser, C. (2007, March). Tags are not metadata, but “just more content” to some people. Paper presented at the International Conference on Weblogs and Social Media, Boulder, CO, USA.
Cheng, R., & Vassileva, J. (2005, January). User motivation and persuasion strategy for peer-to-peer communities. Paper presented at the the 38th annual Hawaii International Conference on System Sciences, Big Island, HI, USA.
Chi, E. H., & Mytkowicz, T. (2008, June). Understanding the efficiency of social tagging systems using information theory. Paper presented at the 19th ACM Conference on Hypertext and Hypermedia, Pittsburgh, PA, USA.
Deci, E. L., Koestner, R., & Ryan, R. M. (1999). A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin, 125, 627–668.
Deci, E. L., & Ryan, R. M. (1985). Motivation and self-determination in human behavior. Heidelberg, Germany: Springer.
Drenner, S., Shen, S., & Terveen, L. (2008, October). Crafting the initial user experience to achieve community goals. Paper presented at the the 2008 ACM Conference on Recommender Systems, Lausanne, Switzerland.
Golder, S. A., & Huberman, B. A. (2006). Usage patterns of collaborative tagging systems. Journal of Information Science, 32, 198–208.
Gould, J. D., & Lewis, C. (1985). Designing for usability: Key principles and what designers think.
Communications of the ACM, 28, 300–311.
245
Heckner, M., Heilemann, M., & Wolff, C. (2009, May). Personal information management vs. resource sharing: Towards a model of information behaviour in social tagging systems. Paper presented at the International AAAI Conference on Weblogs and Social Media (ICWSM), San Jose, CA, USA.
Jäschke, R., Marinho, L., Hotho, A., Schmidt-Thieme, L., & Stumme, G. (2008). Tag recommendations in social bookmarking systems. AI Communications, 21, 231–247.
Lange, P. G. (2008). Publicly private and privately public: Social networking on YouTube. Journal of Computer-Mediated Communication, 13, 361–380.
Logan, R. A., Nuttall, R. J., Hazelwood, S. E., Parker, J. C., Johnson, J. C., Hewett, J. E., & Reid, J.C. (2000).
Audience motivations to use an arthritis website. Arthritis Care and Research, 13, 320–329.
Marlow, C., Naaman, M., Boyd, D., & Davis, M. (2006, August). HT06, tagging paper, taxonomy, Flickr, academic article, to read. Paper presented at the Seventeenth Conference on Hypertext and Hypermedia, Odense, Denmark.
Maslow, A. H. (1943). A theory of human motivation. Psychological Review, 50, 370–396.
Mathes, A. (2004). Folksonomies: Cooperative classification and communication through shared metadata.
Retrieved September, 10, 2010, from http://www.adammathes.com/academic/computer-mediated-communication/folksonomies.html
Matusiak, K. K. (2006). Towards user-centered indexing in digital image collections. OCLC Systems &
Services: International Digital Library Perspectives, 22, 283–298.
Melenhorst, M., Grootveld, M., & van Setten, M. (2008). Tag-based information retrieval of educational videos.
EBU Technical Review, 314, 1–18.
Melenhorst, M., & van Setten, M. (2007, October). Usefulness of tags in providing access to large information systems. Paper presented at the IEEE Professional Communication Conference, Seattle, WA, USA.
Morrison, P. J. (2008). Tagging and searching: Search retrieval effectiveness of folksonomies on the World Wide Web. Information Processing and Management, 44, 1562–1579.
Norris, C. E., & Colman, A. M. (1994). Effects of entertainment and enjoyment of television programs on perception and memory of advertisements. Social Behavior and Personality, 22, 365–376.
Nov, O., Naaman, M., & Ye, C. (2008, April). What drives content tagging: The case of photos on Flickr. Paper presented at the the 26th annual SIGCHI Conference on Human Factors in Computing Systems, Florence, Italy.
Preece, J. (2000). Online communities. Designing usability, supporting sociability. New York: John Wiley &
Sons, LTD.
Rashid, A. M., Ling, K., Tassone, R. D., Resnick, P., Kraut, R., & Riedl, J. (2006, April). Motivating participation by displaying the value of contribution. Paper presented at the CHI 2006, Montréal, Canada.
Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions.
Contemporary Educational Psychology, 25, 54–67.
Sen, S., Lam, S. K., Rashid, A. M., Cosley, D., Frankowski, D., Osterhouse, J., Harper, F.M., & Riedl, J. (2006, November). Tagging, communities, vocabulary, evolution. Paper presented at the Conference on Computer Supported Cooperative Work, Banff, Canada.
Shang, R., Chen, Y., & Shen, L. (2005). Extrinsic versus intrinsic motivations for consumers to shop on-line.
Information & Management, 42, 401–413.
Sood, S. C., Owsley, S. H., Hammond, K. J., & Birnbaum, L. (2007, March). TagAssist: Automatic tag suggestion for blog posts. Paper presented at the International conference on weblogs and social media, Boulder, CO, USA.
Stafford, T. F., Stafford, M. R., & Schkade, L. L. (2004). Determining uses and gratifications for the Internet.
Decision Sciences, 35, 259–288.
Teo, T. S. H., Lim, V. K. G., & Lai, R. Y. C. (1999). Intrinsic and extrinsic motivation in internet usage.
Omega, 27, 25–37.
Van Velsen, L., & Melenhorst, M. (2008, July). User motives for tagging video content. Paper presented at the Adaptation for the Social Web workshop, Hannover, Germany.
Van Velsen, L., & Melenhorst, M. (2009). Incorporating user motivations to design for video tagging.
Interacting with Computers, 21, 221–232.
Von Ahn, L. (2006). Games with a purpose. Computer, 39, 92–94.
Von Ahn, L., & Dabbish, L. (2004, April). Labeling images with a computer game. Paper presented at the Conference on Human Factors in Computing Systems, Vienna, Austria.
Wartena, C. (2010). Automatic classification of social tags. In M Lalmas, J. Jose, & I. Frommholz (Eds.), Lecture notes in computer science: Research and advanced technology for digital libraries (Vol. 6273; pp.
176–183). Berlin, Germany: Springer-Verlag.
Wartena, C., & Brussee, R. (2008, September). Topic detection by clustering keywords. Paper presented at the the 19th International Conference on Database and Expert Systems Application, Turin, Italy.
Wu, H., Zubair, M., & Maly, K. (2006, August). Harvesting social knowledge from folksonomies. Paper presented at the the Seventeenth Conference on Hypertext and Hypermedia, Odense, Denmark.
Authors’ Note
We thank the students from the Utrecht University of Applied Science who participated in the software development and data collection for this study. The research was financed through the European Commission FP7 Project MyMedia (www.mymediaproject.org) under the grant agreement No. 215006.
All correspondence should be addressed to Mark Melenhorst
Novay PO Box 589
7500 AN Enschede, the Netherlands Mark.Melenhorst@novay.nl
Human Technology: An Interdisciplinary Journal on Humans in ICT Environments ISSN 1795-6889
www.humantechnology.jyu.fi
247 APPENDIX A Chatbot condition
Control condition
This is the control condition.
In the text box, tags can be entered that appear in the tag
list after pressing the Plaats tag button
Tag & Vote condition
Below Uw gegevens the user’s statistics and position in the higschore
is displayed. The green and red arrows represent positive and negative votes for tags. Below Tag deze video, new tags can be entered.
Bookmarking Condition
The user’s collection of tags is displayed below Mijn tags.
Folders can be found below Mijn mappen. In Mijn favorieten new
video clips and tags can be entered.
A collage of screendumps of the tagging mechanisms.
APPENDIX B
Titles and URLs of YouTube Videos.
Title URL
Frozen Grand Central http://www.youtube.com/watch?v=jwMj3PJDxuo Japanese way of folding t-shirts http://www.youtube.com/watch?v=b5AWQ5aBjgE
Dove evolution http://www.youtube.com/watch?v=iYhCn0jf46U
OFFICIAL - Terry Tate Office Linebacker
"Superbowl Spot" http://www.youtube.com/watch?v=Kg5cdZ-Fnpc Learn Popular Magic Illusions : The Penetrating
Pinky Illusion Magic Trick Explained http://www.youtube.com/watch?v=RDGCR4W7Yn8 Fonejacker: Latest Episode: Bank Robber Vs
Locksmith http://www.youtube.com/watch?v=rr2d7YYUHEI
Bud Light Swear Jar http://www.youtube.com/watch?v=JI3Y1auTFpU
Big Band Explained With Mince Pies http://www.youtube.com/watch?v=WdCqtnS_cOA
249 APPENDIX C
After each movie clip the following questions were asked:
What do you think of this clip? Please indicate what you think of this clip on a scale from 1 to 5 on the following points.
Entertaining 1 – 2 – 3 – 4 – 5 Exciting 1 – 2 – 3 – 4 – 5 Humorous 1 – 2 – 3 – 4 – 5 Amusing 1 – 2 – 3 – 4 – 5 Nice 1 – 2 – 3 – 4 – 5 Funny 1 – 2 – 3 – 4 – 5
What mark between 1 and 10 would you assign to this clip?
Suppose you would see this clip on a video site. Please indicate to what extent you would like to provide this clip with tags.
I would definitely not tag this clip
I would probably not tag this clip
I am not sure if I would tag this clip
I would probably tag this clip
I would definitely tag this clip
Apart from questions about the movie clip, after the second movie of each pair associated with a tagging condition, the following questions were asked about the tagging input mechanism:
Please provide a mark between 1 and 10 for this interface:
Please indicate for the statements below to which extent you agree with them:
This interface is easy to use Strongly disagree – Disagree – Neutral – Agree – Strongly agree It is easy to learn how to use this interface Strongly disagree – Disagree –
Neutral – Agree – Strongly agree When I saw the interface, I could see what Strongly disagree – Disagree – with it I could do immediately Neutral – Agree – Strongly agree The interface is fun to use Strongly disagree – Disagree –
Neutral – Agree – Strongly agree If this website would exist in real life, Strongly disagree – Disagree – I would definitely use it Neutral – Agree – Strongly agree The interface encourages to tag Strongly disagree – Disagree –
Neutral – Agree – Strongly agree Tagging has added value for me Strongly disagree – Disagree –
Neutral – Agree – Strongly agree
www.humantechnology.jyu.fi Volume 6 (2), November 2010, 250–268