Who I am, who I am not

My name is Heather Mernitz.  I’d like to start today by telling you a little about me.  My PhD is in biochemistry.  My doctoral training was in lung cancer biology.  While I worked as a project assistant in the late 90s on one of the big NSF curricular reform grants in chemistry and while I have several colleagues and good friends doing excellent work in chem ed, I am not a chem ed researcher.  I am not a trained evaluator.  And despite others’ attempts to bill me as such, I do not consider myself an expert in assessment.  I am just a biochemist who has spent the last 14 years teaching chemistry and assessing student learning at Alverno College.  For the last seven years, I was the Chair of the Council for Student Assessment at Alverno. This makes me a practitioner, just like most of you joining us today.  So, I am not here to tell you about my research or about the latest research findings.  Our curriculum and practices at Alverno are grounded in decades of educational research (see especially Mentokowski and Associates, 2000), but I was not the person who did that research.  I am here to talk to you from the standpoint of a faculty member in the trenches.  What makes me somewhat unique is that I teach chemistry and biochemistry at Alverno College.

Alverno College

Let me tell you a little bit about Alverno.  If you haven’t heard of us, we are a pretty unique place.  We are a small liberal arts college for women in Milwaukee, WI.  If you take a look at our student population, we might seem a little different from other liberal arts colleges.  But what really sets us apart at Alverno is that we have been teaching in a competency-based system for almost 50 years now.  Further, if you look close, you will notice that we are a non-graded curriculum.  These two aspects of our curriculum have challenged Alverno educators to think very carefully about what we assess, how we assess, and what kinds of assessment and feedback practices promote deep and lasting learning.

Slide 3:  Speaks for itself

Alverno Educators

At Alverno, we aren’t just responsible for determining if a student has passed our course and we don’t get to give the student a single letter or number grade that summarizes all of their learning.  Instead, we have to validate the students’ demonstration of specific relevant institutional outcomes at whatever developmental level is appropriate for our course and we need to have clear evidence for our judgments.  Further, we need to define our program and course learning outcomes and provide similar evidence that successful students are demonstrating these outcomes.  Think about that for a second.  I have full confidence that all of you are developing important skills in your students – skills like analytical reasoning, problem solving, working in teams, effective communication, and being an ethical professional.  But do you systematically collect evidence of how students demonstrate these skills in a developmental manner in the context of your discipline?  The fact that we do this is what makes Alverno educators especially conversant in pedagogy and assessment.  We have to learn to do these things well at Alverno because our competency-based, non-graded curriculum requires it. 

Outcomes

That brings me to the outcomes for our time together today.  I’d like to build on some of the material covered in the first webinar in this series by Dr. Marilyne Stains when she discussed assessment of, for, and as learning.  I’ll differentiate these practices and explore the implications of using different types of assessment practices in student learning and skill development in chemistry.  Finally, I’d like to provide you with some examples of assessment practices that combine assessment of, for, and as learning as a means to begin a conversation about how we can best use a variety of assessment practices to help promote student learning within the context of our classrooms.  

9 Principles of Good Practice for Assessing Student Learning

I’d like to start with some principles of good practice for assessing student learning, first published in 1992 by the American Association of Higher Education Assessment Forum and re-examined since then for continued relevance.  I would like to start here and then come back to these principles at the end of our discussion because some of what I cover next you might think is unique to Alverno.  I’d like to argue that it is not, but the main point here is that these principles for good practice in assessment were designed to reach across disciplines and institutions. 

Go through some of the key points highlighted in blue font.

Content Framework vs. Outcomes Framework

When I began teaching at Alverno in 2007, I needed to adjust my thinking and approach to teaching, learning, and assessment.  I had learned and previously taught in what we would call a content framework.  My professors assessed me and I assessed my students primarily to give a grade, which would be averaged in some way to yield a final course grade.  This meant taking mostly (at the time) paper/pencil quizzes and tests, which generally focused on recalling information and specific algorithmic operations.  

When I started teaching at Alverno, I had to move to an outcomes based framework.  Since I wasn’t going to put a grade at the top of the test or paper, I had to think differently about assessment.  Each question I asked was not going to generate a point total for a student based on some determination of correctness vs. incorrectness.  Instead, I had to start thinking about what outcomes I had for the students in my courses, how I could break these outcomes into specific criteria that blended content with skills, and how I could summarize my feedback to students in some combination of rubrics and narrative feedback.  While this was a lot of work, it freed me to think creatively about the best ways for a student to demonstrate assessment criteria and course outcomes.  If I had an outcome about communicating clearly using chemical vocabulary, a paper/pencil quiz wasn’t going to do it.  I might be able to probe some aspects of understanding of chemical terminology, but I am not going to get a full picture of my students’ communication skills through a paper/pencil test alone.

Testing vs. Assessment

When I started at Alverno, I also had to get used to some new terminology.  What I am showing here are not “official” definitions, but a summary of what WE mean at Alverno when we say assessment vs. testing. At Alverno, we use BOTH, but we lean heavily toward assessment.  How we test or assess depends on the course and our goals.  What learning are we trying to measure?  In some courses and at certain points in a learning program, testing may be absolutely appropriate and desired.  A question to ask is whether the type of testing/assessment you are using is appropriate for what you are intending to measure.  And do you have goals for your assessment process that go beyond measuring learning?

Assessment of, for, as Learning

Dr. Stains introduced the concept of Assessment –of,for,and as– Learning in the first webinar of the series.  She defined assessment OF learning as assessing to demonstrate achievement – the high stakes, summative assessment at the end of a unit or a course.  Assessment FOR learning is assessment that is done to give feedback on learning and teaching, and assessment AS learning is assessment that promotes self regulation and critical evaluation.  She presented these second two types of assessment as lower-stakes and formative forms of assessment ongoing throughout the unit and before the exam.  She gave us sound rationale for using assessment FOR learning and provided excellent examples of formative assessment strategies in the chemistry classroom, including peer instruction.  

I use all three types of assessment in my classroom, but at Alverno Assessment OF Learning is usually the lower-stakes, formative work that students do on a continuous basis.  We use computer-based quizzes and paper/pencil quizzes in class to probe what students know, remember, and can apply.  And we bring students into this process.  Scores on these quizzes are meaningful to students and to myself in a developmental manner, but they don’t factor heavily into student success toward demonstrating my course outcomes.  So really, this becomes assessment FOR learning, and it joins all the other strategies that Dr. Stains articulated within the classroom that help promote student learning and provide evidence of student progress toward demonstrating course outcomes.  We don’t use these recall or algorithmic quizzes as a means to grade or rank students, but rather as a means of improving understanding for all students and to prepare them to apply their learning in some relevant context.  

Assessment-as-Learning

What, then, does summative assessment look like?  This is where Alverno educators stress performance-based assessment-AS-learning.  Once a student convinces us that they have a decent grasp of the content and/or algorithmic competency, then we move toward demonstration of conceptual understanding in practice.  I said before that I can count on one hand the number of times I have had to demonstrate my knowledge and abilities in my working career by taking paper/pencil tests.  So what do we have to do as professionals?  We have to communicate through writing literature reviews and original research manuscripts.  We give oral presentations on established theory and new research.  We explain scientific concepts and translate research results to various audiences from laboratory journal clubs to curious neighbors, modulating our message to meet the needs of our listeners.  We write and peer review research papers, book chapters, and grant proposals.  We engage in interdisciplinary debates and apply our disciplinary expertise to solving complex, real-world problems.  Our goal at Alverno is to develop summative assessments that reflect, in important ways, the manner in which they will apply our course outcomes in a professional environment.  Now, I am not as lucky as my Education and Nursing colleagues in that my students follow diverse paths after graduation.  But the principle is still the same – authentic assessment means that my students need to demonstrate my course outcomes through application of knowledge and skills in context.  

Performance-Based Assessment

You can see on this slide some of the types of performance-based assessments that we might use at Alverno.  I want to stop here and say that I did some of these as a part of my fairly traditional undergraduate experience in chemistry.  I think what makes my classroom different is that when I was an undergrad, my grade and success in a chemistry course was mostly determined by my performance on high-stakes paper/pencil tests.  Oral presentations, case studies, and group work were sort of low-stakes “extras” that were added on and contributed minimally to my final grade.  At Alverno, it is the paper/pencil and computer quizzes that are lower stakes preliminary work and we view the application work as the higher stakes work that determines whether students demonstrate course outcomes and pass our classes.  

Assessment OF Learning

So, in my classes, assessment OF learning happens before and during every class.  Because it is used on a lower-stakes, formative manner, this really blends with Assessment-FOR-learning.  And students have responsibility for evaluating their learning before moving on to the larger assessments.  

Did learning happen? We can use in-class quizzes, concept inventories, even old ACS exam questions to have students test themselves and identify gaps and areas for further review.  In some classes, we use Aktiv tools.  In other classes we write our own quizzes.  I haven’t tried this, but I’ve heard of faculty using scratch-off tools to help students receive formative feedback on quiz questions.  

Assessment OF Learning – slide 2

Whatever formative assessments you use, we find it useful and important to allow students time to reflect on their level of understanding in relation to assessment criteria.  Are they ready to engage with a complex, performance-based assessment?

scratch-off tools to help students receive formative feedback on quiz questions.  

Assessment FOR Learning

This is assessment that provides feedback to the learner and teacher.  Many of the things I mentioned on the last slide do this.  But assessment FOR learning doesn’t just have to formative in nature.  Higher stakes, credit-bearing assessments can also provide opportunities for students to learn through making and correcting mistakes, revising their work, or demonstrating outcomes more completely as curriculum progresses in a spiraling manner.

Examples:  Peer feedback on lab notebooks, students revising errors in calculation work, students held responsible for incorporating instructor feedback on a subsequent lab report, or students having to reassess or demonstrate an unmet criteria on a future assessment.

Assessment AS Learning

This is really about becoming metacognitive learners:  Good assessments require students to critically evaluate their work in relation to explicit criteria and reflect on strengths and areas for continued growth.  They can help students integrate and transfer their learning from the classroom to internships, jobs, graduate work, etc.

Here is one examples from an oral presentation that students do in my biochemistry course during the metabolism section.  More examples will follow.

Quality Assessment, Feedback – slides 15-19:  

A favorite book passage of mine comes from Alison Gopnik, a developmental psychologist from Berkeley.  It goes something like this:

• Imagine if we taught baseball the way we teach science.  Until they were 12, children would read about baseball techniques and occasionally hear inspirational stories of the great baseball players.  
• They would answer quizzes about baseball rules.
• Undergraduates might be allowed, under strict supervision, to reproduce famous historic baseball plays.
• But only in graduate school would they, at last, actually get to play the game.

We don’t teach baseball by giving multiple choice tests.  We let kids play the game (in a safe and formative way), we let them make mistakes, we let them learn from their mistakes and their successes…. If we want to teach anything, including science, we need to integrate content with competencies and teach and assess for both.  A person who memorizes the rules of baseball does not necessarily make a good player.  Memorization of discrete facts is sometimes necessary, but not sufficient to make a great scientist, clinician, or engineer.   And students shouldn’t have to wait until the end of their programs, or worse – grad school, to see how content and competencies integrate and to practice using what they know. 

Good assessment is about probing what students know AND IMPORTANTLY WHAT THEY CAN DO WITH THAT KNOWLEDGE, at ALL levels, in relation to learning outcomes and competencies.  

QL156 Example:  I have since found that even for content and criteria where a paper/pencil quiz might work, there is something to be gained by developing assessments that allow students to engage their creativity.  For example, I teach a quantitative literacy course within our gen ed curriculum.  In a unit on probability and odds, I could easily give a paper/pencil test.  But when I think about where in my life I use these skills (remember – this is a gen ed course), it is helping my daughters with their homework.  So my assessment for my students on a section on probability, odds, and counting concepts involves them having to create a video tutorial talking me through their process as they answer a set of questions, and then writing and answering their own original questions.  I have had students write questions about basketball, fashion, and planning a child’s birthday party.  When I had to teach this course online in 2020 and 2021, these videos allowed me to experience the personality of my students.  I gave them no time limits, but for larger classes you could certainly do this. 

Protein jmol tutorials in biochem:  My undergraduate biochem experience involved a lot of memorization and recall quizzes and tests.  My students still do a lot of this on their own to prepare for class, but my assessment projects are much more complex and applied.  Take for examples my semester-long project where students create their own original computer tutorial on the structure and function of a protein of interest to them.

Talk through some examples from 2020-2021

Grant proposals in analytical chem (instrumental?):  

What did we do more/less slides 21-22:  Slides speak for themselves

Assidere

We can’t talk about assessment without talking about feedback. The word “assessment” comes from the Latin verb “assidere,” meaning “to sit with” or “sit down beside”.  This word origin implies that in assessment the teacher sits with.  This reminds me of my time as a volleyball coach when I would sit down with one or more players and watch game tape.  We would dissect our wins and our losses, evaluating players individually and together as a team.  The goal was never to shame or stigmatize, but always to examine a performance as objectively as possible with a eye toward improvement.  Now, it is not always possible to sit down with each student at regular points in a semester and do this kind of intensive examination, but keeping this idea in mind helps me remember that assessment is something that I do WITH and FOR my students and not something I do TO them.  

Feedback slides and examples, slides 27-34:  Slides speak for themselves

9 Principles of Good Practice for Assessing Student Learning

Let’s conclude by returning to these principles.  Given our history with competency-based education and assessment-as-learning, Alverno educators do a lot of consulting work.  We try to always remind our colleagues that our goal is not to proselytize or convert others to the Alverno system.  Instead, we encourage faculty and faculty teams at institutions to examine their own beliefs about the education process and the outcomes they have for their students at the end of a course, program, and degree.  Then, we encourage faculty to carefully and honestly examine the alignment between their goals and their current teaching, learning, and assessment practices.  We still do this here at Alverno and we are still, nearly 50 years later, finding gaps between our principles and our practices.  We are also not immune to the financial and political pressures that all institutions of higher education are feeling.  We can’t always sit down beside each student as we assess their learning, so we have to find efficiencies and strategies to do what we are able to do in the moment and remind ourselves of our aspirational goal. 

Slide 2:  So in looking at these principles of good assessment, I encourage folks to pick ONE area where they would like to focus, and maybe even just one assessment in one class, and consider some realistic strategies within the context of your institution and your student population to move closer to the aspirational goal of using assessment OF, FOR, and AS student learning.