Item writer training: Planting seeds rooted in practice
As they sit in front of a computer screen taking an ASWB social work licensing exam, test-takers might not think much about each question’s origin story. But ASWB puts abundant thought, effort, expertise, and careful monitoring into making sure every exam item contributes to a valid and reliable exam.
“The ASWB social work examinations measure competencies to safely practice from day one of a social worker’s career,” said Lavina Harless, director of examination development, “so involving subject matter experts who understand the elements of safe practice in a firsthand, personal way is crucial.” Consequently, the first step in exam development involves a practicing licensed social worker, working under contract with ASWB, writing an item to test an assigned KSA—a statement of knowledge, skills, and abilities.
“Our item writers are responsible for planting the seeds that grow into scored test items. It’s hard to overstate their importance to a program that depends on a steady flow of test questions that are rooted in the dynamics of current practice.”
Writing items is a skill that requires training, practice, and mentorship. All of these were plentiful when a diverse group of 25 social workers, selected from 96 applicants, were trained as item writers in Alexandria, Virginia, August 23–25, 2019. ASWB staff members and consultants served as instructors and mentors during the two-and-a-half-day training designed to prepare new item writers to contribute high-quality items to the ASWB exams.
The sessions began with an overview of the exam development process. Next, writers participated in a how-to session on creating items from scratch. For much of the rest of the training, groups viewed items onscreen, working together to improve questions that the new writers had drafted.
Onscreen review serves as an excellent teaching tool because the process helps writers understand what the Examination Committee looks for when approving items for pretest. Item writers learned to ensure that each of their items is critical to practice and has a clearly written stem that ends in a question, up-to-date language, and a single correct answer or “key.” They also learned to include at least one accurate and recent reference and to make sure that the item aligns with its assigned KSA.
This item-writing primer wasn’t the only major topic addressed in the training, however. Item writers must also master a certain mindset. “Item writers are asked to do two things at once: We tell them that their subject matter expertise, diverse backgrounds, and critical thinking abilities are key to their success, but we also stress the importance of creating an emotional distance between the writer and the items she or he may create,” said Harless. “We depend on the individual contributions of our writers, but as we edit and make other adjustments at each step of the process, items can and do change. Writers need to be okay with that and proud of the contributions they make even if the final scored item bears little resemblance to the item they originally drafted.”
Relationship-building is another goal of the training. “During our short time together, we deliver a lot of information, but we also build lasting relationships between staff, writers, and consultants,” Harless said. “I think those relationships are key to our program’s success and to the fact that once social workers get involved in ASWB item development, they tend to stick around—we now have 90 writers across the United States and Canada.” Because every member of the Examination Committee is a former item writer, Lavina explained, “Everyone is invested in the process.”
The August training was the first time ASWB had trained new item writers since the 2017 practice analysis was completed. Many of the individuals who helped with the practice analysis and standard setting expressed interest in becoming item writers, so most members of the new class were already familiar with ASWB.
New item writer Tony Bibus of Minnesota, a former regulator, is one of those with past ASWB experience. “During two decades of involvement with ASWB, I have come to appreciate the high quality of the association’s work, so it was not surprising that the training was conducted as professionally as it was,” he said. “What did surprise me was how complex and wide-ranging a task it is to keep refining and updating the exam’s content and performance. I didn’t realize that so much time, effort, and expertise are needed at each stage.”
Every scored item on an ASWB exam begins with an idea inspired by practice experience and is shaped by social work practitioners, subject matter experts, psychometricians, and editors as it makes its journey to the test center screen. Item writers hold a crucial place in the overall exam development process because they connect the exam to the practice of social work. “Our item writers are responsible for planting the seeds that grow into scored test items,” Harless said. “It’s hard to overstate their importance to a program that depends on a steady flow of test questions that are rooted in the dynamics of current practice.”