Partner TRDC ADDIE Page

Overview

Once a Training Provider receives certification of their course and an official NTED course number, they should contact their NTED/TPP PM to set up an account in the Registration and Evaluation System (RES).  

Level 1 Evaluation

Once an account is established, the training partner will be provided  Level 1 evaluation forms, administrative instructions on the distribution, collection, and processing of the forms, and a username and password to access their evaluation data.  The Level 1 Form is where participants can provide reactions and comments to the course, instructors, benefits, and challenges.

Level 2 Evaluation

The Level 2 evaluation is an objective measure of student knowledge, skills, and abilities acquired through training. Training partners are required to develop, administer, track, and report a Level 2 evaluation for each course they offer to the public. The instrument may be either a pre and post-examination, or a post-course practical exercise for performance-level courses that do not lend themselves to a pre-test.  The Level 2 testing methodology must be approved by the NTED/TPP PM.  Tests or practical exercises must measure the individual, not the class as a whole.

Level 2 results will be posted on a bi-weekly basis in the RES.

Level 3 Evaluation

The Level 3 Evaluation Form is currently in review.  The template and instructions will follow when available.

Overview

Implementation is a broad phase that contains many diverse tasks and process considerations.

Once the course is certified and operational, it requires continuous support and maintenance to ensure that it operates effectively and cost-efficiently and produces learners who meet job performance requirements. 

Each training partner may have their own procedures; however, all procedures must be in accord with all Programmatic Terms and Conditions of the applicable Contract or Cooperative Agreement. Training partners should address all programmatic queries to their NTED/TPP PM.

Scheduling

Training partners are required to comply with the requirements of NTED/TPP for scheduling courses.  The training partner should coordinate and seek approval from the State Administrative Agencies (SAAs) / Training Points of Contact (TPOC) prior to scheduling or delivering in-resident or mobile training within any State and local jurisdiction of the training location.  The SAA and/or TPOC will approve the student registration, admittance; attendance and billing before the student can attend the training.  For mobile deliveries, after the training partner receives approval from the SAA and/or TPOC, delivery of training is authorized and the training partner may coordinate with the municipality points of contact for scheduling. 

Instructors

All training partners who develop and deliver training on behalf of NTED must ensure that their instructors meet the standards and protocols of an Instructor Quality Assurance Program.

A successful instructor quality assurance program (IQAP) requires effective hiring protocols, ongoing monitoring, and continued professional development.  It is imperative that training partners develop standards and processes to ensure that individuals selected to become instructors possess the necessary knowledge, skills, and abilities to deliver quality instruction and maintain their skills.

Selection of qualified instructors should be based on the following criteria: 

• Education
• Experience/Technical Expertise
• Special Qualifications
• Certifications and Licenses
• Instructor Competencies

Training partners must develop formal instructor requirements that spell out the criteria, which must be met for each course.

Registration

Each training partner may have their own registration procedures; however, all procedures must be in accord with all Programmatic Terms and Conditions of the applicable Contract or Cooperative Agreement. Training partners should address all programmatic queries to their NTED/TPP PM.

Reporting

Training partners are responsible for administering a standardized Level I evaluation to all course participants to measure reactions to the course and instructor(s) and a customized Level 2 evaluation to measure student learning.  Training partners must submit all training participant Level 1 and Level 2 data electronically into the Registration and Evaluation System (RES) on a biweekly basis.  Additionally, training partners will conduct and report Level 3 evaluations, as directed by NTED/TPP.  Level 3 evaluations are conducted following a student's return to home jurisdiction or place of employment (reporting requirements for Level 3 evaluations TBD).

Overview

At this point in the process (the Development phase) the CDD has been written and approved, and the course structure/content outline defined. It is time to build the training materials, beginning with a prototype. 

prototype

Developing a prototype is the first step of the Development phase.

• For an instructor-led course prototype, it is recommended that one lesson be produced that is most representative of the entire course (i.e., select the lesson containing different visuals or supporting materials versus the short and least complicated lesson). The course prototype must include all materials needed to conduct the lesson (e.g., instructor and participant guides, supporting materials, practical exercises, etc.).
• For a web-based course prototype, one completed lesson of the course needs to be developed. It is recommended that the lesson chosen for the prototype be one that is most representative of the entire course (i.e., select the lesson containing different types of interactivity, incorporates discussion boards, online chat, other collaboration tools, or practice exercises).  Storyboards may be submitted in lieu of the preferred fully functioning lesson if approved in advance by the NTED/TPP PM.

Once the prototype is approved by the NTED/TPP PM, complete the rest of the lessons and modules in the same manner.  When all of the draft materials are complete, submit them to the NTED/TPP PM for approval to move forward to the third party review.

Third Party Review

Instructor-Led Third Party Review:

A new course requires three subject matter experts (SMEs) to review the materials and attend a pilot presentation of instructor-led courses.  An existing course requires one SME to review the materials without attending any sessions of the course unless there have been major changes from the most recent certification.  

Web-Based Third Party Review:

Before WBT can be implemented for the target audience as a whole, it must first be integrated into the identified delivery (host) environment where it must be tested to ensure all components function as expected and that learners are able to access all instructional elements easily and without additional assistance.

There are a number of steps involved in this process, including proper content integration into the Learning Management or Learning Content Management System (LMS/LCMS) environment, ensuring that the Sharable Content Object Reference Model (SCORM) functionality regarding interoperability is working correctly from both a content and LMS perspective, that the course is functioning as expected, and that the content is accessible from a Section 508 perspective.

A new web-based course needs three SMEs to review the course in a live environment, while an existing course only requires one SME.

All Third Party Reviews

The NTED/TPP PM may request review of the course materials by federal employees who are experts in the subject.

The Level 2 evaluation instrument must be submitted at the time that other course materials are submitted for the course review process. The instrument will be evaluated during the course review process, based on its adherence to instructional design principles for testing, and to ensure that test questions or checklists (for post-course practical exercises –with NTED/TPP PM approval) map to learning objectives and critical “must-know” aspects of the course.

The results and comments of the SME review, which also includes a National Incident Management System (NIMS) review need to be addressed and approved by the NTED/TPP PM and the NTED/TPP Instructional Systems Designer (ISD) before the revised course materials are submitted to a third party 508 review. 

When it is determined that all 508 compliance issues are resolved, the Training Support Package (TSP) is submitted to the NTED/TPP PM for final review and Certification.  The TSP includes:

• 508 Compliance Form
• Core Capability Mapping
• Course Catalog Form
• Course Design Document
• Course Review Change Document (Spreadsheet)
• Electronic Copy of Course Curriculum
• Letter of Request
• Needs Assessment
• NIMS Review Letter

THE NTED/TPP PM requires a re-writable DVD with the complete TSP for the TPP files.

Overview

The design of training courses and materials is based on the results obtained during Analysis.  Course goals and objectives, structure, and content are created, using a design strategy which considers the following components:

• Learning taxonomy
• Structure
• Learning sequence
• Progression
• Learner pacing
• Learner participation/interactivity
• Content presentation
• Learner feedback
• Supplemental information

The next step is creating the instructional strategy, which is also informed by the results of the earlier analyses.   

Course Design Document (CDD)

The purpose of the CDD) is to serve as the blueprint for developing the course materials. It provides a skeleton of the course content, as well as documents that establish expectations. The CDD finalizes the course goals and learning objectives and establishes the course instructional and assessment strategies. The CDD also includes design and development standards and guidelines, as well as technical standards, for production and delivery appropriate for the selected training delivery solution.

Overview

Analysis is broad and contains many diverse activities and tasks that are extremely important to the overall success of the learning solution. Through a proper analysis, the "Who, What, Where, Why, and By Whom" is determined. Instructional courses and systems that are developed, updated, or revised based on objective, valid, and reliable analyses are more likely to provide accurate, appropriate, and effective instruction for the target population.

During Analysis, the Instructional Development Team conducts various types of analyses (e.g., target audience, content, task, learning, and media).

The Instructional Development Team should carefully review project requirements to ensure that they conduct the appropriate types of analyses and that they collect the data required to make effective instructional decisions.

During Analysis, it is important for the Instructional Development Team to determine if the course will be “new” (i.e., new content), or if it will be a course conversion or update. If the course is to be converted or the course materials already exist, the Instructional Development Team should begin by validating the existing content using the following templates:

• Needs Assessment
• Planning and Analysis (P/A)
• Course Mapping Tool

Needs Assessment (NA)

A needs assessment is the process of determining if there is a need to improve performance and, if so, in what area and to what extent. Conducting a thorough needs assessment is critical to the development of successful instruction.

The purpose of a needs assessment is to ensure that the stated problem or identified deficiency can be solved with instruction and, if so, determine what instruction is needed. Instruction will only solve performance deficiencies that occur when a learner lacks the knowledge, skills, abilities, and/or attitudes required to successfully perform the task.

If the identified performance deficiency is not a result of a lack of knowledge, skills, abilities, or attitudes, instruction cannot solve the deficiency, and thus there is no need to proceed further with the course.

Planning and Analysis (P/A)

Defining the project scope involves identifying the project objectives and the work that must be performed to complete a project. The purpose of determining the scope is to clearly define the deliverables or end product of a project, as well as to focus the project team’s goals. A successful project scope should define and interweave project objectives, deliverables, milestones, technical requirements, limitations, and review cycles.

To assist in determining scope, consider the following:

• Resources
• Project Schedule
• Tasks
• Budget

To complete the Planning and Analysis, address the following:

• Learner/Environment considerations
• Job/Task analyses
• Content analyses
• Potential learning objectives
• Delivery/Media analyses
• Course mapping tool output

Resource:

Please see the Planning/Analysis, Delivery Analysis, and Course Mapping tools in the Templates section of the Training Resource and Center (TRDC).  The completed Planning/Analysis tool requires review and approval by the NTED/TPP PM to continue development of the course.