The ASWB social work licensing examinations follow strict guidelines for development and maintenance. The end result is a set of valid and reliable measures of minimum competency.
What are the steps?
They are the practice analysis, the linking to KSAs (Knowledge, Skills and Abilities), the exam blueprint, and the many steps in developing items. Then comes the work of the Examination Committee. The testing contractor, Pearson Vue, then does its part, from banking and analyzing items and pulling out questions for each form, and finally the administration--and more analysis. After all the number crunching, it's back to the volunteer content experts, the volunteer social workers, for more decisions.
The practice analysis
ASWB does a practice analysis, a survey of the profession to determine the knowledge and skills needed by entry level social workers at each stage of practice, about every seven years. The most recent one was completed in 2009. A task force of volunteers served as content experts and did most of the real work. A new practice analysis is underway.
The survey instrument is drawn up and sent to a random sample of social workers that represent an accurate sample of the profession. Depending on the overall response rate, additional, targeted mailings are sometimes used to ensure that sampling is sufficient.
Overall importance ratings for each task statement on the survey are determined for each of four groups (corresponding to the four exams) through the rating scales, marked for frequency and criticality. The tasks are then linked (by volunteers, again) to the KSA statements. KSA are grouped into content areas.
Exam blueprint
The content areas become the components for the outline for each examination, each of which consists of 150 multiple choice questions. The number of test items assigned to each content area is based on the total weight and number of KSAs in that area. Each exam has its own blueprint, which consists of four or five broad content areas, further divided into competencies.
While there is psychometric oversight of a job analysis, the science has to be tempered with the judgment of social work subject matter experts. This is provided by social workers, volunteers who are often members or former members of social work regulatory boards. They spend time really dissecting their profession, trying to break down a humanistic job into measurable components-once people have done this, they often say, they do have an understanding of the exams that stays with them.
Passing scores
Recommended passing scores are established by a volunteer panel using a modified Angoff method. Using a so-called "anchor exam," each committee member estimates for each item on the test what percentage of minimally competent social workers should get the item correct. Their responses are examined by the psychometric experts, and minor adjustments can be made by the ASWB Board of Directors. The set anchor exam becomes the yardstick by which all other forms of an exam are measured. This method ensures that overall difficulty remains consistent, even though individual exam items may change.
The test items
Once all this groundwork is done, the difficult part is to fill in the blueprints with questions that 1. fit, 2. are pertinent to the profession, 3. are just hard enough but not impossibly difficult, and 4. do not present unfair obstacles to candidates of different genders, ethnic groups or geographic locations, or for whom English is a second language.
Item writers
A network of social worker across North America work under annual contracts to produce exam questions for ASWB. They are chosen for a mix of demographics, trained in a three-day weekend session, and sent home to write. Working with them are five item development consultants, who edit, advise and sometimes reject items.
Examination Committee
The next step for the item is the Examination Committee. No matter how hard the writer has tried, or how much additional work has been put in by the consultant, the Exam Committee is more than likely to find changes to make. A distractor is weak; another distractor is too good, and may be arguably correct; the item depends on a law that is not nationwide or too obscure to be fairly used; or someone argues that the entire question is a giveaway. Only when there is general agreement is the question accepted.
Pretest
New items are banked by the test contractor, Pearson Vue, and are sorted into pretest blocks of 20 and put on a form of the appropriate examination. The pretest items do not count, but they are mixed in so candidates must respond to them as if they were part of the exam. Once enough candidates have answered the item, it is replaced, and psychometricians analyze the ways in which people responded to the question. If women do better than men, or whites better than African Americans, or if people who do well on the exam generally pick a distractor that isn't the key, or any number of other scenarios in which the statistics are not good, the item is pulled out and returned to the Exam committee. Otherwise, it goes into the active bank, and can be used to fill out the blueprint on future exams.
Monitoring
The performance of the examinations and the Examination Committee are constantly being monitored. Inattention by a testing company, by staff, or by the stakeholders on regulatory boards in all jurisdictions can mean an exam that no longer does its job.
Examination scoring
There is no set passing score for the ASWB examinations.
Because the actual items change from examination form to examination form, there is no way to establish a rigid passing score regardless of the version taken. Some administrations of the examinations will contain individual items that may be slightly harder or easier than other items on other administrations not by much, but by enough to make the establishment of one unalterable passing score impractical, and unfair.
To compensate for these variations, test administrations are equated, a psychometric process that accounts for the varying difficulties, and moves the passing score up or down accordingly. As a result, overall difficulties remain the same from test to test.
Candidates receive either a "pass" or "fail" score. Failing candidates receive diagnostic information on their performance on each content area. |
