Test Administration Observations in Kite Collector

Purpose

Dynamic Learning Maps^® (DLM^®) test administration observations are conducted across consortium states on an annual basis. Observations provide anonymous information about how DLM testlets are delivered to help the consortium evaluate the process of administering assessments and to inform improvements. Data from test administration observations also provide evidence of validity for the assessment system.

Collection

Observations should be conducted by state or local education agency staff when observing the testing session for informational purposes regarding the assessment. The Collector tool should not be used when the primary purpose of the agency staff’s visit is to evaluate the teacher, monitor student performance, or coach the teacher. Observations collected through Collector are completely anonymous and not linked to either teacher performance or student assessment results.

Using the Collector App/Web Version

Observations are completed through a free mobile application called Kite^® Collector or through the web version at https://collector.kiteaai.org/. The Kite Collector mobile app can be downloaded onto devices from the Apple App Store or from Google Play. The App Store offers both iPhone/iPod Touch and iPad versions of the app. Apple devices must be running iOS^® 10 or higher, and Google devices must be running Android 5 or higher.

All state and local education agency staff will use a single login to collect anonymous data about DLM testlet administration. A table of all consortium state credentials is provided at the end of this document.

The mobile device must be connected to the internet to log in. After login, the mobile application may be used without internet connectivity. Data will be stored on the device until reconnected to the internet as long as Submit is selected before closing the application. Submit completed surveys before closing the application to avoid losing entered data.

The web version of the app (https://collector.kiteaai.org) does not support offline submission of data. While you can fill out the survey fields after logging in, your device will need to be reconnected to the internet at the time you click Submit to submit your observation if using the web version.

Questions about completing or submitting questionnaires may be sent to dlm-support@ku.edu.

Conducting Observations

After logging in, select the generic School and Participant from the dropdown fields. For Questionnaire, select either computer-delivered or teacher-administered depending on the type of testlet administered during the observation.

Set-Up

1. Remind the teacher that you are not there to evaluate their teaching or monitor student performance. You are there simply to observe what s/he normally does during DLM assessments. The purpose of what you are recording is to support documentation of the alternate assessment.
2. Ask the teacher to arrange the session as s/he typically would for that student.
3. Based on that arrangement, place yourself where you can unobtrusively observe – preferably outside the student’s line of vision and where you can still see the screen (for computer-delivered assessments) or behind the teacher/student pair, facing the computer (for teacher-administered assessments).
4. If the student completes multiple testlets, complete separate questionnaires for each testlet.

During the session

1. Do NOT include information that could be used to identify the student by name.
2. Document your observations in each section as thoroughly as possible.
3. Avoid interrupting the testing process.

After the session

1. Follow up with the teacher if any clarifications need to be made on the questionnaire.
2. Allow the teacher to ask any questions if they would like to.
3. If you completed the questionnaire(s) without internet connectivity through the mobile app, select Submit to save the observation on your mobile device. When your observation is successfully saved to your device you will to receive the following message:

Upon reconnecting to the internet, select the three-line icon (hamburger) on the top- left corner. Choose Send data.

Alternatively, you can reopen the application once online and submit your response without logging in.

Tap the banner at the bottom of the screen to submit your observation data. You will receive the following confirmation message indicating that your response has been successfully submitted: