CATA, DATA, & Cheating

A Synthesis, Analysis, and Evaluation of Text Analysis, Big Data, and Online Integrity

THERE IS CATA DATA

Technology has come a long way in the past few years and that includes how technology has changed education. Computer-based text analysis is becoming increasingly popular as a tool in the educational sphere as it can be used to analyze student work and to give students helpful feedback, but what exactly is computer-based text analysis? This type of analysis is the use of natural language processing, or NLP, to automatically extract meaning from text. NLP allows a computer to understand language, analyze content, and draw meaning from written or spoken text. “Computer-aided text analysis (CATA) offers great promise for scholars who aspire to capture the beliefs, cognitions, and emotions of individuals as reflected in their narratives and written texts,” (Short et al., 2018). Using this type of analysis can give teachers insights into their students' work, allowing them to spot errors or misunderstandings and provide timely, personalized feedback to their students. Furthermore, computer-based text analysis can identify key words or themes that can be used for grading or student assessment. Although, not everyone is on board with using computer-based text analysis in education. It has been argued that relying on algorithms to assess student writing may not be reliable, as these algorithms may not always understand nuance and may misinterpret students' work (Gordon et al., 2020). In addition, algorithms can only judge text on a very limited number of criteria, which means that some areas may be ignored, such as the organization of a student’s work or their grammar usage. In spite of the criticisms, computer-based text analysis can still be used effectively in the educational sphere. In my opinion it should not replace teacher assessments, but rather it should supplement them, helping teachers identify potential issues in student writing more quickly and efficiently.

AND THEN THERE IS BIG DATA

“Big data is the analysis of massive sets of information collected by individuals, businesses, governments and other organizations,” (Van Winkle, 2018). In the world of education, big data is transforming the way educators and students learn. It’s revolutionizing educational decisions and helping students, educators, administrators and policy makers get a better understanding of learning outcomes. “Big data is an essential aspect of innovation which has recently gained major attention,” (Baig et al., 2020). Big data allows us to gain insight into student learning outcomes that would have previously been difficult, if not impossible to gather. With big data, educational institutions can use metrics to make informed decisions about what strategies to adopt, which resources to allocate and how to improve educational experiences. For example, universities can analyze student data such as performance in past courses, what topics were the most engaging, which methods are the most effective, or where resources could be used more efficiently. By using such metrics, schools can make data-driven decisions about the learning process. It also provides access to data about student progress, behaviors, engagement and achievement. Educators are increasingly utilizing such data to inform curriculum design, instruction and assessment. By looking at detailed data sets, educators can identify areas of concern, as well as which students may be struggling or excelling. This data can be used to provide additional support to students who are in need and also to accelerate learning opportunities for those who are excelling. Finally, big data can also help institutions of higher education become more cost effective. With access to vast amounts of data, universities can analyze and determine which resources should be allocated in the most cost effective way. This information can also be used to help faculty members be more effective in their instruction, giving them data that allows them to better evaluate their methods and resources. “Big data and analytics have been powering the consumer giants in the global marketplace. That’s why it’s no surprise institutions of higher learning expect positive outcomes from big data in education,” (Big Data in Education, 2020). Through better understanding of student progress, resource allocation, and curriculum design, educational institutions can take full advantage of big data to further improve learning experiences for students.

Addressing the Issue of Online Integrity and Cheating

As the shift to remote learning continues to gain momentum, the challenges of preventing cheating and plagiarism have grown. While educators may feel they lack the time and resources to address these issues, there are ways they can make their assessments as safe as possible. One of the most effective ways to discourage online cheating is to use software that flags plagiarism. Programs like Turnitin are becoming popular as a means of stopping online cheating. These tools detect similarities between two pieces of writing and can be integrated into an online exam environment. They also offer a variety of reports, such as originality and similarity scores, so that educators can see what elements of the test were plagiarized and how to properly credit any legitimate source material. It is also a good idea to mix up the question types when creating online assessments. Having multiple choice, short answer, and essay-based questions will reduce the likelihood that a student can cheat. As well, use questions that require higher-level thinking and more critical analysis in order to encourage independent thinking and make it more difficult for someone to look up answers on the internet. Finally, provide your students with explicit instructions on the guidelines of the exam. Specifying exactly how they are allowed to take the exam, and the consequences of not following these rules, is key to preserving academic integrity and preventing online cheating.

References

Short, J. C., McKenny, A. F., & Reid, S. W. (2018). More Than Words? Computer-Aided Text Analysis in Organizational Behavior and Psychology Research. Annual Review of Organizational Psychology and Organizational Behavior, 5(1), 415–435. https://doi.org/10.1146/annurev-orgpsych-032117-104622

Gordon, J., Donnelly, L., Weissenberger, J., Geffen, A., Hollingsworth, C., Bakhshandeh, N., ... Stasko, J. (2020). Algorithmic Assessment of Free-Response Student Writing: Risks, Limitations, and Considerations. Journal of Research on Technology in Education, 53(1), 35–55.

VanWinkle, M. (2018). Using big data to understand student progress, support improvement, and reduce costs. The Evolllution.

Big Data in Education. (2020, December 7). Maryville Online. https://online.maryville.edu/blog/big-data-in-education/#:~:text=The%20Benefits%20of%20Big%20Data

‌Baig, M.I., Shuib, L. & Yadegaridehkordi, E. Big data in education: a state of the art, limitations, and future research directions. Int J Educ Technol High Educ 17, 44 (2020). https://doi.org/10.1186/s41239-020-00223-0

Resources

https://www.turnitin.com/

https://youtu.be/Jfv2uU6xl64