Executive Summary
With $310,000 of support from NSF, Dr. Alan Sherman and his CoPIs, Drs. Dhananjay Phatak and Linda Oliva, are developing educational Cybersecurity Assessment Tools (CATS) to measure student understanding of cybersecurity concepts. In particular, they are developing and validating two concept inventories (CCI, CCA): one for any foundational or introductory course in cybersecurity, and then one for college graduates looking to begin a career in cybersecurity. This project is collaborative with the University of Illinois at Urbana-Champaign.
Technical Challenge/Activitites
To develop evidence for the validity of the CCI’s and CCA’s measurement of student conceptual understanding of cybersecurity, the project will complete six tasks for each assessment tool: (1) conduct at least thirty cognitive interviews of students thinking aloud while answering the draft questions, and use results to improve the validity of the questions; (2) collect and use feedback on the draft questions from at least twenty experts in cybersecurity or cybersecurity education; (3) administer the draft tool to at least 200 students and use psychometric methods of analysis on the results; (4) revise the draft questions in light of tasks 1-3; (5) administer the revised tool to at least 1,000 students and use psychometric analysis on the results; and (6) hold workshops at four cybersecurity education meetings to promote awareness of, and to seek additional feedback on the tools. By following well-established rigorous methods for developing CCIs, this project will help the cybersecurity educational community to develop and improve curricula and instructional methods and materials.
Potential Impact
These inventories will provide science-based criteria by which different approaches to cybersecurity education can be assessed (e.g., competition, gaming, hands-on exercises, and traditional classroom).