 |
 |
 |
 |
 |
 |
 |
||||||
|
||||||||||||
| ||||||||||||||||||||
|
United States Department of Agriculture Forest Service Washington
Office 14th & Independence SW
File Code: 5160/6100 Date: January 25, 1999
As you are aware, all the Regions participated in a comprehensive pilot test program of the Work Capacity Tests (WCT), particularly the Pack Test, in order to better determine if the WCT should become the sole fitness used by the Forest Service. Of specific interest was whether any potential candidate groups were adversely impacted by the Pack Test, and if so, to ascertain the reasons for such impacts as well as to identify appropriate mitigation measures that could be used without jeopardizing the test's integrity. The pilot test program was conducted in accordance with an agreement between the Forest Service Partnership Council (FSPC) and Washington Office Fire and Aviation Management (FAM) staff. The data gathered from over 4,350 test candidates was analyzed by a team of agency employees, who presented their findings and recommendations to the FSPC on December 2, 1998. Representatives from the FAM staff participated in the discussion at that meeting. A summary of the Assessment Team's recommendations are enclosed. After extensive discussion and use of an interest based problem solving process, the FSPC reached consensus, concurring with team's recommendation that the Pack Test should become the sole fitness test for arduous fire duties. The Council also agreed that all four recommended mitigation measures be put in place before the Pack Test is fully implemented. Until that time, fitness testing for firefighter positions will be conducted in accordance with provisions of item 6 of the January 8, 1998, agreement between the FAM and FSPC (enclosed). The FSPC also agreed with the team's recommendation specific to Region 8. Work is now underway to develop and put the mitigation actions agreed
to by the FSPC into place by February 1, 1999. However, until we
analyze the final report on the fatality and the subsequent Board of Review
recommendations, all units should only use the step test or the
/s/ Phil Janik
Enclosures: Attachment 1: WCT-Pack Test Assessment Team
Recommendations
Work Capacity Tests - Pack Test
Background and Discussion: On January 1998, an assessment team consisting of representatives from WO Fire and Aviation Management (FAM), WO Civil Rights (CR), WO Human Resources Management (HRM), WO Research and the FSPC was formed to conduct a comprehensive pilot test program for Pack Test during 1998, consistent with terms of a Memorandum of Understanding between the FSPC and WO FAM dated January 8, 1998. The primary purpose of the pilot was to provide additional information for use by the FSPC and FAM to determine whether the pack test should be the sole fitness test used by the Forest Service for arduous ICS fire positions. Specifically, the pilot test program was to: collect and analyze data on a servicewide basis to ascertain whether any candidate groups were adversely impacted by the Pack Test, the reason(s) for such impacts, and recommendations for appropriate mitigation measures that could be used without jeopardizing the integrity of the test. The team coordinated the preparation and distribution of instructions, collection of data, statistical analysis, and its interpretation. A copy of the statistical and narrative analysis for the 4,353 total usable data sheets collected is enclosed. This information will also be used to amend the Civil Rights Impact Analysis (CRIA), dated December 31, 1997. As part of their analysis, the team also screened out non Forest Service employees as well as incomplete surveys, to determine whether results differed from those obtained using all respondents. Results for the 3,274 FS employees differed very little. Consequently, the team used the larger number on which to base its analysis, conclusions and recommendations. The team also reviewed narrative input received from various test administrators and regional fire staff commenting on WCT pilot testing. During one of two teleconferences to discuss results, the team consulted with Dr. Sharkey. Conclusions 1. No employee group or protected class was adversely impacted under EEO statistical guidelines. However, males passed at higher rates (93.9%) than females (81.9%), and certain protected classes performed better than others. 2. Age was not a significant factor in pass rates. 3. Height, especially below 5'3" was a significant factor in pass rates. In that category only 67.3% of candidates passing (versus an overall 91.5% pass rate). 4. Weight below 109 lbs. and over 230 lbs. had a negative effect on test scores, with 64.7% and 83.2% pass rates respectively (versus an overall 91.5% pass rate). 5. Disability was not a major factor in results. 6. Very few self exemptions were taken (i.e. total of 21; 7 for medical reasons and 14 for other unspecified reasons). 7. Fitness training over 12 hours in the month preceding the test yielded high pass rates (95.1%) than those who did not training (89.7%). 8. Altitude of test sites did not appear to affect test results, with employees from regions with typically higher elevations scoring as well as those in lower elevation regions. Although the need for an altitude adjustment appeared to be unsubstantiated by test results, scientific data on oxygen consumption at different altitudes necessary to do work, provides a well documented basis for continued use of the adjustment. 9. Region 8 had a significantly lower passing rate than other
regions, with a pass rate of 81.4% when compared to a service wide average
of 91.5%. Except for R-9, where 7 out of 8 employees passed, the
next lowest region was R-1 (783 participants) with a 91.7% pass rate.
When compared to other regions, the following factors appeared to
affect test results in Region 8: gender (female - 59.3% passed),
appointment type (permanent - 81.2% passed v. temporary - 87.0%),
race (African-American males - 69.7% passed), and weight (overweight
- over 230 lbs. 64% passed in R-8 v. 83% agencywide).
Recommendations: 1. Because the pack test is found to be a more valid fitness test than the step or run tests, five of the six team members recommend the Pack Test as currently administered be used by the Forest Service as the sole fitness test for arduous fire duties provided certain procedures and arrangements be provided to employees to mitigate impacts which were identified. Those mitigations include: a. Establishing an agency policy and regulations to allow current employees otherwise authorized for arduous ICS assignments to take 3 hours of official time per week for the four weeks preceding administration of the pack test for suitable fitness training. Grant employees who possess a red card with an arduous fitness level 3 hours of official time per week to maintain their fitness standard. Fitness activities should be consistent with local wellness programs where they exist. b. Preparing a Q/A document for employees addressing the basis for changing to the pack test, its job relatedness, the science behind the test, and other misunderstanding about the test. c. Adding, if supported by statistical data and analysis, a time adjustment for height. The amount of adjustment would be determined by the same analysis. d. Establishing a test monitoring process for the first year that would permit the detection of any systemic administration problems or test result impacts on employees. Results of this monitoring would be reported to the FSPC and WO-CR by December '99. Any issues identified would be addressed by the national parties in a partnership mode. 2. A more thorough evaluation and follow up be made to determine
the reasons for the test results in Region 8, and to set up whatever additional
monitoring processes deemed necessary in conjunction with recommendation
1.d. Findings and appropriate mitigations should be determined
between the Regional Fire Director and the R-8 partnership council, consistent
with service wide policy.
Assessment Team Members/Role: Robert Ragos - WO-CR (Civil Rights Impact Analysis)
Attachment 1: Analysis of the Pack Test Survey Results
Forest Service Partnership Council (FSPC)
This statement contains a consensus response from the FSPC to the proposal on pilot testing of the Pack, Field and Walk tests during 1998, as contained in a letter dated December 5, 1997, from the Director of Fire and Aviation Management. If the terms and conditions as set forth in this document are agreed to by the parties, this Statement will become a memorandum of understanding (MOU) under which the parties will comply during the 1998 pilot year. This statement is formatted to show F&AM's proposal followed by the FSPC's response. 1. Fire and Aviation Management will work with the Civil Rights Staff to develop a Civil Rights Impact Analysis. Robert Ragos of the Civil Rights Staff is leading the project. FSPC: We agree that a CRIA to identify the anticipated impacts on employees based on such factors as gender, race, age, etc., and mitigations for both the pilot year testing and that prepared for the eventual decision on long term use of the tests are critical. It is the Council's understanding that the CRIA for pilot year testing must be completed before a final decision is made to implement testing. The FSPC requests the opportunity to review the CRIA before the pilot test is initiated to insure the anticipated impacts and pilot year mitigations offered by the Council have been addressed. Further, it is the Council's understanding that as data and information related to employee impacts is obtained during the course of the pilot year, such data/information will be provided to the Civil Rights staff for incorporation in to the CRIA as well as being used to provide periodic updates to FS employees on the pilot test. In order to preclude introducing any biases into the pilot test study, any information released during the pilot to the workforce will be reviewed and approved by the group referred to in proposal #2 (below) who will beleading the study. 2. A group consisting of Tom Martin, Pete Rockx, David Randall, Dave Aldrich and Chris Earnest (FSPC representative) will work with Civil Rights to develop the procedures to obtain data and do the statistical analyses to identify impacts of the tests on firefighters. FSPC: We agree with the composition and purpose of this group, recognizing that the procedures developed will impact and in most cases be requisite to the proper execution of pilot year testing as described in other items of the F&AM proposal. Thus it is our understanding that pilot year testing would not begin until this group has had opportunity to develop the procedures for incorporation into testing instructions and associated data collection processes. It is also understood that the FSPC will have access to data and information being collected, through their representative, during the course of the pilot year. 3. Fire and Aviation Management will distribute detailed instructions
and explanatory notes for giving the Pack, Field and Walk tests to insure
uniformity and fairness in testing.
FSPC: We agree, understanding of testing instructions as well as data collection responsibilities as established by the group noted in item 2 above, are critical both to insure uniformity/fairness and validity of data being collected. 5. Fire and Aviation Management will distribute existing information related to Pack, Field and Walk tests and recommended physical fitness programs to Forest Service field units ("Fitness and Work Capacity - Second Edition"). FSPC: We agree information contained in these documents will provide useful background and technical information to enhance understanding the need for measuring fitness/work capacity and the genesis of the pack test. However, employees should also be made aware that the pack test is still a "work in progress". Although approved for use in the Forest Service during this pilot year, a final determination on its impacts on employees and use as the sole test to measure fitness for firefighting assignments has yet to be made. Hence, the importance of test administration and data collection on pack during 1998. 6. Fire and Aviation Management will develop and distribute a pamphlet describing the Pack, Field, and Walk tests and how employees can prepare for them. FSPC: We agree, recognizing that for the benefits of such information to be realized by those to be tested, the pamphlets need to be received in sufficient time prior to testing so that employees have time to prepare. In addition, as part of the F&AM's instructions, employees must be made aware of the following requirements, options and opportunities associated with testing during the pilot year: A. It is understood that the Pack, Field, or Walk tests will be administered to employees for them to meet the fitness requirements established for the applicable ICS position. Those passing the test will be certified under the red card system. Those failing will be provided the opportunity to take the Step or Run tests in order to be certified. Subsequent testing under the Pack, Field, or Walk test will not be required unless deemed necessary for pilot test assessment purposes by the F&AM group listed in proposal # 2 above. B. Prior to taking the Pack or Field tests, employees will be advised of their right to self-exempt from the test by providing an explanation which is medically-based and reasonably describes the basis or potential physical risks to themselves that could result if given the test. If the reasons for self-exemption would not pose a health risk, and the employee is both agreeable and deemed to be physically capable by the test administrator, he/she would be given the opportunity to take the step or run tests. It is understood that any self disclosed medical information would be treated as confidential in accordance with federal regulations, but may be used to assess whether the employee is medically/physically qualified to perform arduous ICS jobs. Further, costs associated with any requirement to have employees furnish medical evidence to determine if the employee is physically capable of performing an ICS job, will be borne by the FS. NOTE: Intent of the self-exemption process is to reduce the possibility of test-related injuries to employees, who because of their unique medical/physical circumstances should not take the pack test but could otherwise safely perform arduous firefighting duties. It is not intended to establish a more rigorous physical capability screening of employees who have self exempted, and subsequently express a desire to take the Step or Run tests. C. For employees required to take the Pack test, they will be
given the opportunity to select and use any pack weighing 45 pounds during
the test, including the use of personal packs.
7. A video tape program will be developed and distributed to explain the Pack, Field, and Walk tests to people who will be taking the tests. A companion video will be developed to assist people who will be administering the test. FSPC: We believe these training/orientation aids will be very useful to employees and test administrators. In the case of test administrators, training/orientation on their responsibilities should be conducted before they assume that role. Recognizing the probable lag time to prepare and distribute the subject video, it is recommended that F&AM encourage Regional F&AM Directors to conduct some interim form of pre-testing orientation for those forest level fire staff members responsible fot test administration to insure their receipt, readiness and understanding of pilot year testing instructions. In addition to the conditions and arrangements contained in the FSPC's response to F&AM's proposal, the Council agreed that there is an over-arching understanding. That is; if it becomes apparent to the analysis group (proposal 2) based of data/information being reported from testing sites that the pack test is creating significant negative impacts to employees being tested and/or the agency's firefighting readiness, modifications to the pilot fitness testing procedures will be made. Such modifications will be made in conjunction with the FSPC, recognizing that modifications made to deal with emergency situations may be necessary before the FSPC becomes involved. Approved by the FSPC: January 8, 1998 For the FSPC: /s/ George E. Christopher
/s/ Pete Rockx
For Washington Office Fire and Aviation Management Staff: /s/
Mary Jo Lavin
|
