Exam development team refines processes for 2024
When ASWB partnered with PSI to administer the social work licensing exams starting in January 2024, the most obvious changes were for exam candidates — new scheduling systems, new check-in procedures, and new exam software. Behind the scenes, however, another big transition was taking place. ASWB’s item bank — a database of exam questions, called “items” — moved into PSI’s system so that ASWB can take advantage of PSI’s item development database for creating, reviewing, and approving items.
We all work together to make sure these projects are successful and uphold the highest standards.
To allow time for both teams to adjust to the new system, ASWB paused item development and exam form review for a few months. “This has given us an opportunity to look at every piece of our program and rethink them,” says Exam Development Director Carl Hokanson, LGSW.
Hokanson cites the form review process — an exam’s version’s final review before publishing — as a good example of this rethinking. A form is a full set of exam items, made up of 150 scored items and 20 unscored (pretest) items that are being considered for inclusion on an exam. The content of a form is determined by the exam blueprint, with each form assigned a pass point based on the difficulty of the items it includes. ASWB typically has four different forms of each category of exam available.
Previously, pairs of volunteers with experience as Examination Committee members would meet several times a year to review the newest forms. Their tasks include conducting a sensitivity review and assuring that each item is testing knowledge that is critical to practice. This year, ASWB has created a team of six form reviewers for each exam category who independently review a form asynchronously. The team then comes together to review the forms in person prior to publishing. “That’s adding seats at the table,” Hokanson says, “adding perspectives, more voices, around the form finalization process.”
Hokanson and his team have also revised the standard contract for item writers, reducing the required number of items for a completed contract. Contracts previously required 30 items to be written before an item writer was paid the $1,500 contract fee. The new contracts will provide a $500 payment after 10 items are written. Hokanson says the shorter-term contract will make it easier for participating social workers to complete contracts while fulfilling their other personal and professional obligations. The new contracts also prohibit the use of artificial intelligence tools like ChatGPT in the creation of exam items.
Two new item development consultants have been brought on board to review item writers’ submissions and to collaborate with them to ensure the items meet standards. New consultants Susanna Sung of Maryland and Stephanie Washington of Texas previously served as item writers, Examination Committee members, and form reviewers. “We rotate consultants regularly,” Hokanson says, “to make sure that we are bringing in new perspectives to that part of the process.”
Hokanson leads a team of three ASWB staff members, two of whom are also licensed social workers. “The staff are the shepherds of the process,” Hokanson says, “not the creators of the exam.” Each step of creating the exams is done by practicing social workers from diverse backgrounds:
- Item writers write questions.
- Item development consultants review drafted items.
- Examination Committee members approve, edit, or reject items.
- Form reviewers approve the final forms.
Meanwhile, the Social Work Census, which will be the largest analysis of the practice of social work ASWB has ever conducted, is underway through May 31. This major survey combines practice analysis data — the bedrock dataset for the social work licensing exams — and a comprehensive workforce study.
With independent research firm HumRRO collecting the Census data and PSI’s collaboration on exam development, ASWB’s examination development team continues to manage the equally important processes that result in valid, reliable, and fair licensure exams. “Partnerships with our vendors are key,” says Hokanson. “We all work together to make sure these projects are successful and uphold the highest standards.”