Needs Assessment
Organization and Background
The U.S. Coast Guard (USCG) is one of five branches of the U.S. Armed Forces and has a fleet of 259 cutters, which were the subject of this project. Specifically, the project focused on the Afloat Training Organization (ATO) of the Force Readiness Command (FORCECOM) branch of the USCG, which focuses on enabling mission success and readiness through leadership development, training, and assessments.
ATO evaluates USCG’s small and large cutter crews for their area of responsibility. The ATO team we worked with in San Diego is made up of 20 members that travel to evaluate the cutter training teams, assist with correcting discrepancies, and improve training techniques. ATO completes two evaluations: Command Assessments of Readiness & Training (CART) and Tailored Ship’s Training Availability (TSTA). These evaluations aim to determine if onboard cutter training teams can effectively train crews to combat shipboard emergencies and ensure mission readiness within FORCECOM standards and policies.
Another important group is the onboard training teams (OBTTs). These folks are crew members with the added responsibility of ensuring the crews are mission ready and continually trained. CART/TSTA essentially evaluate how well OBTTs are performing their training duties.
Opportunity and Value
Two major benefits to the USCG were determined by comparing the actual and desired levels of performance:
Increasing crew readiness: the CART/TSTA evaluation process includes several gaps contributing to an inaccurate representation of mission readiness.
Reducing costs: by making changes to CART/TSTA, the length of the evaluations can be reduced, thus reducing costs and time invested in evaluating.
Analyses
We completed five levels of analyses by asking key questions, applying appropriate frameworks and models, identifying data sources and collection methods, documenting our analyses, and determining due dates:
Presented problem: client request to increase readiness and reduce costs.
Organizational analysis: mission, vision, values, goals, strategies, and critical issues.
Environmental analysis: world, workplace, work, and worker.
Gap analysis: actual and desired performance.
Cause analysis: systemic and individual performer factors contributing to the gap.
Causes
Through this analysis, the team identified several causes for the gap:
Evaluation procedures: TSTA, which stands for Tailored Ship’s Training Availability, is not accurately tailored to a crew’s needs. Based on the extant data, it seems nearly impossible to fail CART or TSTA. ATO cannot provide direct feedback to crews and can only communicate with OBTT.
Cultural acceptance: the current process is accepted because it provides passing scores, providing the appearance of mission readiness. However, questions remain about whether CART/TSTA truly measures readiness.
Poor data quality: drills that are passed because of onsite instructions by OBTT are not recorded and tracked. Failed drills are not always calculated correctly. Overall, trends are not tracked.
Standards for OBTT: the passing score for OBTT-specific evaluation items is only 50%, far below the 80% required for entire-crew drills.
Qualifications: ATO evaluators are recommended for the position by higher-ranking officers, regardless of experience. New evaluators often have no experience aboard a cutter, impacting their ability to assess cutter crew performance accurately.
Incentives: there are no clear incentives or punishments for good or poor performance.
Intervention Selection and Recommendations
A multicriteria matrix was used to guide the decision process. Six potential interventions were identified and ranked across five categories, which were then weighted. The two most beneficial solutions the team identified include:
Tailor CART/TSTA: Tailor the type and number of CART/TSTA drills to the platform and crew.
Increase OBTT standard: Increase the OBTT performance standard from 50% to 80%.
Limitations
The team observed several limitations over the course of the project. The most critical include:
Timing: the 15-week project time frame did not align with scheduled CART/TSTA evaluations, reducing the data collection methods available to the project team.
Data collection methods: it was not possible to complete direct observations of crews or to complete surveys. It was also not possible to schedule interviews with commanders.
Confidentiality: classified and even unclassified data requires approvals that the team could not obtain at all or in time for the project to be completed.
You can view the report below, open it in a new tab, or download a PDF copy here.
ATO Final Report v2.0sms.pdf