Content originally presented by Gianina贝克, Ph.D., National Institute for Learning Outcomes 评估 (NILOA)
2018年1月, The National Institute for Learning Outcomes 评估 (NILOA) shared a report analyzing a national study of institutional effectiveness communication called “评估 that Matters.” One of the major findings and implications of this study is:
“Communicating effectively about student learning remains a challenge. Colleges and universities must more clearly and persuasively communicate relevant, 及时的, and contextualized information on their impact on students and value to society.”
When drilling down into this data to find what types of information and outcomes to publicly share and the challenges around that, 出现了一些主题. Specifically, provosts said that sharing these outcomes are most important:
- Information on accreditation retention, persistence, graduation, & 毕业率
- 许可 & certification exam pass rates
- 就业安置 & 工资
And, what was most important to provosts was not 分享什么, but 如何分享 这个信息. Most institutions have tons of data, 报告, 研究, and opportunities to communicate all of 这个信息 publicly. So how have institutions traditionally approached presenting outcomes?
Current Institutional Communication Approaches
One of the ways institutions try to communicate is by flooding their audience with data. They present comprehensive findings by item, 仪器或测量, where there is little connection between each 仪器或测量. Their idea is to make all of their information available in the name of ”transparency,” with very little meaning or analysis of what the data represents in the overall narrative. Other examples where there are opportunities for better communication include:
- Overwhelming audiences with access to data without meaning-making
- Scattershot bullet lists of processes attempting to guess what people want
- Archives of 报告 that document the processes – but provide a confusing history of data collection or changes made
- Posting individual student stories of success (internships) without any context or groups of stories
- Providing limited publicly available information on their websites. Institutional websites are most often prime spaces to communicate effective student learning results, but are often forgotten about spaces
One of the biggest presentation and interpretation gaps between audience understanding and assessment evidence is best stated by one provost from the study:
“We can provide all the assessment results or data we like, but if others cannot interpret them accurately there is no benefit to transparency or accountability.”
Communicating outputs is not sufficient. Outcome results must be made more meaningful for target audiences.
“This is not easy to share in ‘sound bites,’ and communicating outputs such as employment rates and beginning 工资 does not serve as a proxy for student learning and quality of programs…we need to find ways to help the general public make meaning of the results.”
We need to be much more explicit and meaningful about sharing the good work of assessment student learning on campuses. Specifically, we need to be intentional about 分享什么, 我们如何分享 (mostly externally), and most importantly, 如何沟通 nuanced, complex picture of student learning that couples evidence of learning outcomes with student success data.
Here is a step-by-step continuum of how most institutions can be categorized as they go about communicating learning outcome assessment to the general public. As you think about how and what your institution communicates, where do you fall on this scale?
- Compliance posting of outcomes & select (favorable) results
- Oversharing of easy-to-measure outcomes & results, just to do something
- More tailored posting of outcomes, by program/major experience & results for different audiences
- More complete picture of all the places learning occurs, posting outcomes in student affairs/services & results for different audiences
- 细致入微的, tailored framework for communicating outcomes and improvements in a comprehensive, student-focused and culturally responsive way
Effectively communicating information about student learning clearly remains a target of opportunity. So how do we effectively communicate?
Why 评估 Storytelling Matters – Evidence-Based Storytelling
A story in itself tells others a bit about us and of alternate theories of others, and seeks to make believers out readers. Stories allow us to go assumption hunting. Evidence-based storytelling is defined by the evidence of student learning used in support of claims or arguments about improvement and accountability, told through stories to persuade a specific audience.
When starting to construct an evidence-based story, here is a guide to be able to effectively tell our story and help students tell theirs.
Supportive claims or arguments. Here we are using Toulmin’s Argumentation Model to construct the beginnings of the story. This means we are selecting a variety of evidence of student learning, which may change depending on the audience, and providing the end relation to the claim that is made about the improvement of student learning. This justifies why this change will lead to the improvement we want to see in student learning.
Improvement or accountability. What argument do you want to make about your students’ learning? What type of evidence would be necessary to make the argument?
Told through stories: Elements for storytelling and persuading a specific audience. Here are the basic elements of storytelling that you can use to think through and craft an effective narrative to a specific audience.
- Audience: For whom is this narrative written? What counts as evidence for the different audiences of the report?
- What kind of story are you telling? (i.e., compliance, improvement, loss, struggle, quest, tragedy, fantasy, etc.) What context is needed for readers to understand the story? 背景是什么?
- Who are the character(s) in your story? (Is there a protagonist in your story ─ someone who is driving the action and/or someone with whom your audience is likely to identify? What are the motivations of the characters?)
- 情节是什么?? (The plot is the causal sequence of events and includes setting and conflict.)
- What evidence do you have to assert your claims?
- Based on the story you crafted, what is the best medium through which to share it? Video, written narrative, shorter visual image pieces, a combination, others?
- If you are using visuals in your narrative – are they appropriate? Do they support the story you are trying to share or detract from them?
- How will you make your target audience(s) aware of the story?
A good story is easy to read, 介绍了一个问题, and shares how the problem was solved, highlighting the role of the institution in addressing the problem. We need context and a story because evidence gives stories substance, but stories give evidence meaning. Our stories can be our context, 我们的历史, 我们的任务, our organizational saga – it is how we see the world and why we do what we do.
An evidence-based story pushes back on assumptions, creates narratives that will resonate with internal and external stakeholders such as students, 教师, 和雇主, and most of all tells an improvement story. 事实上, an evidence-based institutional assessment story is a way to bring together improvement and accountability in a meaningful mechanism.
For more detailed information on this topic, watch the Using Elements of Evidence-Based Storytelling to Tell Your 评估 Story 会议记录.
Dr. Gianina贝克 is currently the Assistant Director at the National Institute for Learning Outcomes 评估, co-located at the University of Illinois Urbana-Champaign and Indiana University-Bloomington. As Assistant Director, Dr. 贝克的研究, 写, and presents on all topics related to the assessment of student learning outcomes on college and university campuses. Her main research interests include student learning outcomes assessment at Minority Serving Institutions and community colleges, access and equity issues for underrepresented administrators and students, 高等教育政策, and assessment of student learning in athletics. 在这个职位之前, she served in roles of Counselor, Director of Institutional Research, and Director of Institutional Effectiveness & Planning, all at Richland Community College in Illinois. 她有博士学位.D. in Educational Organization & Leadership with a Higher Education concentration from the University of Illinois, a M.A. in Human Development Counseling from Saint Louis University, and a B.A. in Psychology from Illinois Wesleyan University.
Want to see how 织 can help you simplify your assessment processes?
Fill out the form below for a personalized demo. You’ll receive advice on your current accreditation system, as well as an overview of how 织 can simplify your workflows, improve collaboration and make your assessment and accreditation process a success.