Back to my Ocean Literacy Web Page Back to Cynthia Cudaback's Web Page

Preliminary validation of the survey instrument, data collection, data analysis and further validation of the instrument progress in a cyclic fashion. Student responses to open ended questions are used both as qualitative data in their own right, and to refine the questions for future use. See also my website on conducting rigorous educational research.

Ocean Literacy Data Collection

1. Fall, 2005, 10 honors students completed and commented upon two preliminary surveys
2. Janary, 2006, my 120 students completed open format survey pre-test in class
3. January, 2006, 20 students in E. Knowles distance-learning class completed pre-test online
4. February, 2006, 20 high school students competing in North Carolina Ocean Sciences Bowl completed post-test
5. April, 2006, My 120 students completed several surveys
   • My open-format survey post-test
   • AAAS' Attitude Survey
   • Ocean Project's Content Questions
   • Extra content questions as a "retention quiz"
   • Open ended question: "where did you learn about the ocean?"
6. April, 2006, E. Knowles' 20 students completed open format survey as post-test
7. August, 2006, 12 honors students completed 3 surveys
   • Open Format pre-test
   • Science Attitude Questions CLASS
   • Where did you learn about the ocean? Educational Demographics
8. December, 2006, 12 honors students completed revised survey
   • post-test: Content, science attitudes, conservation attitudes
9. January, 2007, 16 students in my First Year Inquiry Course pre-test
10. January, 2007, 13 students in Ernie's course, pre-test
11. April, 2007 post tests, my course
12. April, 2007, post tests, Ernie's course
13. May, 2007, 250 post tests, T. Garrison
14. Fall, 2007, lotsa stuff

Data Analysis

Qualitative Data

• thematic content analysis
• what gets students interest

Validation and Reliability

• types of validity & reliability
• qualitative Vs quantitative data
• cognitive Vs affective domain
• iterative process

Validation and Reliability

Content Validation: Work with Experts

• Essential Principles and Fundamental Concepts from COSEE, 2005
• focusing on EP1 (size of ocean), EP4 (habitability), EP5 (biology) and EP6 (human connections)

Construct Validation: Work with Students

• choices from open ended questions: ocean depth, where learned
• whole class discussion of wording coriolis question

Criterion Related Validation

• Survey use in different settings => different types of responses
• correlate literacy scores with my grades & Ernie's

Reliability

• inter-rater reliability for ecosystem question