Legal Considerations When Using Testing in the Hiring or Promotion Process

Title IV of the Civil Rights Act prohibits employers from discriminating in employment decisions on the basis of race, color, religion, sex and national origin. Using a test to help determine an applicant's qualifications is entirely lawful, provided that the testlike all other aspects of the hiring or promotion processdoes not discriminate against a protected class of applicants. The Equal Employment Opportunity Commission's Uniform Guidelines for Employee Selection Procedures, 29 C.F.R. Part 1607[1] state as follows:

"Nothing in these guidelines is intended or should be interpreted as discouraging the use of a selection procedure [i.e., a test] for the purpose of determining qualifications or for the purpose of selection on the basis of relative qualifications, if the selection procedure had been validated in accord with these guidelines for each such purpose for which it is to be used."

The Guidelines provide a framework to help insure that a test used as part of the hiring, evaluation or promotion process will be employed in a non-discriminatory manner. In general, a testing process that has an "adverse impact" on the selection rates for any race, sex or ethnic group is considered to be discriminatory, unless it has been "validated" in accordance with the Guidelines.

"Adverse impact" is determined by a statistical measurement of selection rates for various applicant groups. In most cases, if the selection rate for a particular race, sex or ethnic group is less than 4/5 (80%) of the selection rate for the group with the highest rate, it is considered evidence of discriminatory adverse impact.[2] The "4/5 Rule" applies first to "bottom line" selection ratesthat is, the percentage of applicants hired or promoted as a result of the total application, evaluation and selection process. An analysis of the impact of any particular selection procedure such as a testis required only if the "bottom line" selection rates show an adverse impact.

For example, an employer may evaluate applicants for a particular position or class of positions on the basis of relevant education, relevant experience, proficiency in tested skills and a personal interview. If the range of selection rates for males and females, as well as for all races and ethnic groups, complies with the "4/5 Rule," no further analysis is necessary. However, if the selection rate for a particular group is less than 80% of the highest selection rate, the individual components of the selection processincluding any testsmust be evaluated to determine if they have had an adverse impact. Nevertheless, the fact that a test has resulted in a selection rate for one applicant group that is less than 80% of the highest rate does not mean that the employer has acted unlawfully. If the testing procedure has been "validated" in accordance with the EEOC Guidelines, it will not be found to be discriminatory.

"Validation" of a test or similar selection process can be accomplished in several ways. Tests such as those employed by Brainbench, Inc., which measure an applicant's technical knowledge in specified areas, are subject to a "content validity" study.[3]

"Content validity" requires that the employer have data showing that the contents of the test are representative of important aspects of the job for which the applicants are being selected. Two factors are involved in this showing. First, the test must consist of, and measure, a representative sample of the knowledge, skill or ability it purports to evaluate. Second, the knowledge, skill or ability must be used in, and be a necessary prerequisite to, critical or important aspects of the job. It is important that the tested knowledge or skill be "operationally" definedthat is, it should be the body of learned information or work behavior that actually will be used in performing the job.[4]

The EEOC emphasizes, therefore, that any content validity study should be based on a job analysis. This analysis should identify the important work behaviors that are required for successful performance of most of the aspects of the job, and their relative importance. If those behaviors result in identifiable work products, the work products should also be analyzed. The areas to be tested should relate as closely as possible to the analyzed work behaviors and products.

In many cases, an employer will use test procedures for screening purposes, on a pass/fail basis. In that situation cutoff scores should be set at a level that is reasonable and consistent with the employer's normal expectations of proficiency in the workforce. If the employer intends to use test scores to rank candidates, however, a more complete showing is required. Based on its job analysis or other information, the employer must be able to demonstrate that a higher score is likely to result in better job performance. In doing this, the employer should identify and measure candidates' performance in those areas of knowledge, skill or ability which, if done well, relate most directly to better job performance.

Content Validity of Brainbench Tests

Brainbench tests are designed to satisfy the content validity requirement that a test consist of, and measure, a representative sample of the knowledge, skill or ability they purport to evaluate. Each Brainbench test module is tuned to measure knowledge relevant to typical activities performed by individuals on the job. For example, the Oracle DBA test covers the typical activities performed by an Oracle Database Administrator on the job. Brainbench's software programming assessments provide many software code samples for the examinee to interpret, simulating work typically performed on the job. Additionally, test time limits are set to allow the candidate to consult a handy reference if necessary to answer the question (but not to completely learn the topic), again simulating the way the person would act on the job. In this way, the assessments are built to relate directly to job performance.

The processes Brainbench uses to develop and administer tests are consistent with computer adaptive testing approaches studied during the past 30 years by experts in the field of psychometrics.[5] Computer adaptive testing is currently employed by several major testing organizations, such as the U.S. Department of Defense, and the Educational Testing Service.[6] Today, the SAT, LSAT, GRE and similar standardized tests are being converted to this more efficient and accurate method.

In the Brainbench development process, at least four credentialed experts are employed to develop a content outline and set of questions for each module that measure typical job-related knowledge and skills. After content, editing, and psychometric reviews, the tests are beta tested by at least 100 individuals for both reliability and score validity before final calibration and release into production. Today, thousands of users have taken Brainbench assessments. The test content outlines, credentials of the test developers, and summary results from beta tests and subsequent analysis are retained and available, if necessary.

Reliable Administration over the Internet

To ensure that Brainbench skill tests are "safe" for multiple administrations over the Internet, each test module is actually a pool of 125 450 questions. Individual tests only use 16-40 questions each. In each test, the next question is selected from the available pool of questions based on how the examinee has answered the previous questions. By using a large pool, the same test can be administered multiple times without exposing the same question twice. This allows employers to administer the test both before and after training, for example, to verify training effectiveness. The size of each question bank is set so that it is easier for a person to learn the topic than to memorize the questions, if they were available, which they are not.

Summary

Use of testing in hiring, promotion, or retention is an established and accepted practice. More than 30 percent of all for-profit organizations in the U.S. and over 60% of the fortune 100 use some form of testing in their employment practices today, and this percentage is growing. If an employer's selection rate for a protected group violates the four-fifths rule, all components of the employer's selection process should be investigated to determine if they have an "adverse impact" on the affected group. If testing is used and shown to have an "adverse impact", the test must be validated according to EEOC guidelines. In general, this validation process requires that the employer confirm that the test criteria are directly related to job performance, and that the test used actually measures these criteria.

All Brainbench tests are developed to ensure they test the specific, work-related criteria they are intended to address, and that these criteria are focused on typical job functions. Data supporting this process are collected and retained.

[1] The Guidelines were adopted jointly by the EEOC, the Department of Labor (including its Office of Federal Contract Compliance Programs) and the Department of Justice.

[2] The Guidelines require employers to maintain records sufficient to show the impact of selection procedures by sex, and the following races and ethnic groups: Blacks, American Indians, Asians, Hispanics, and whites other than Hispanics.

[3] "Content validity" cannot be used for tests that purport to predict job performance, or to measure characteristicssuch as leadership or judgmentthat are important to successful performance of the job.

[4] Knowledge or skills which the employee is expected to learn on the job, especially those that can be learned through a brief orientation period, should not be the subject of selection tests.

[5] Item Response Theory (Lord, 1980a), Random selection of test items from a proportioned pool of items (Lord and Norvich, 1968), Stradaptive process for test item selection for use in computer adaptive testing (Sands, Waters, McBride 1997). [6] Armed Services Vocational Aptitude Battery (ASVAB) (Sands, Waters, McBride, 1997).