User Generated Education

Education as it should be – passion-based.

Posts Tagged ‘assessment

I Don’t Get Digital Badges

with 14 comments

Digital badges appear to becoming the next, “new” thing in education. What follows is a description of digital badges as described by Digital Media and Learning:

A digital badge is an online record of achievements, the work required, and information about the organization, individual or other entity that issued the badge. Badges make the accomplishments and experiences of individuals, in online and offline spaces, visible to anyone and everyone, including potential employers, teachers, and peer communities.

In addition to representing a wide range of skills, competencies, and achievements, badges can play a critical role in supporting participation in a community, encouraging broader learning goals, and enabling identity and reputation building. For a learner, a sequence of badges can be a path to gaining expertise and new competencies. Badges can capture and display that path, providing information about, and visualizations of, needed skills and competencies. They can acknowledge achievement, and encourage collaboration and teamwork.  Finally, badges can foster kinship and mentorship, encourage persistence, and provide access to ever-higher levels of challenge and reward (http://dmlcompetition.net/Competition/4/badges-about.php).

The proposed benefits of such a system would be a broader and deeper picture of skill sets acquired both in formal and informal settings.

Advocates of this vision for K-12 contend that such badges could help bridge educational experiences that happen in and out of school, as well as provide a way to recognize “soft skills” such as leadership and collaboration. Badges could paint a more granular and meaningful picture of what a student actually knows than a standardized-test score or a letter grade, they say (http://www.edweek.org/dd/articles/2012/06/13/03badges.h05.html).

Badge-diagram-2.2https://wiki.mozilla.org/Badges

The Functions of Badges

Daniel Hickey proposed four functions of badges in Intended Purposes Versus Actual Function of Digital Badges:

  1. Recognizing Learning.   David Wiley has argued cogently that this should be the primary purpose of badges.  If we focus only on purposes, then he may well be right.  His point is that badges are credentials and not assessments.
  2. Assessing Learning.  Nearly every application of digital badges includes some form of assessment.  These assessments have either formative or summative functions and likely have both.
  3. Motivating Learning.  If we use badges to recognize and assess learning, they are likely to impact motivation. So, we might as well harness this crucial function of badges and study these functions carefully while searching for both their positive and negative consequences for motivation.
  4. Evaluating LearningDigital badges have tremendous potential for helping teachers, schools, and programs evaluate and study learning.  At the minimum, just having a system for tracking all of the information included in all of the badges that a program awards might be very valuable.

As good playground and game designers understand, it really does not matter what the intended purposes are of the adult designers.  Children and young people will approach and interact with the apparatus in ways that fit their own development and sensibilities.  Based on past history, such as stickers in class or at home for good behavior; badges in scouts for accomplishments, I have a hunch kids will approach the gathering of badges as a form of rewards.

Badges As Rewards/As a Means for Motivation

It’s a fine line between working for the reward of the badges (think badge of honor or bragging rights) and for demonstrating a skill set. Although digital badges are being touted as a means of documenting one’s achievements, we cannot get away from the notion that learners will work towards getting badges for the rewards.

Many point to research that suggests rewarding students, with a badge for instance, for activities they would have otherwise completed out of personal interest or intellectual curiosity actually decreases their motivation to do those tasks (http://www.edweek.org/dd/articles/2012/06/13/03badges.h05.html).

The problems of using rewards for motivation have been discussed by Alfie Kohn and Daniel Pink.  Alfie Kohn, an outspoken anti-reward proponent, notes that they can actual achieve outcomes in direct opposition of desired results.

Rewards are no more helpful at enhancing achievement than they are at fostering good values. At least two dozen studies have shown that people expecting to receive a reward for completing a task (or for doing it successfully) simply do not perform as well as those who expect nothing (Kohn, 1993). This effect is robust for young children, older children, and adults; for males and females; for rewards of all kinds; and for tasks ranging from memorizing facts to designing collages to solving problems. In general, the more cognitive sophistication and open-ended thinking that is required for a task, the worse people tend to do when they have been led to perform that task for a reward (http://www.alfiekohn.org/teaching/ror.htm).

Daniel Pink discusses a similar occurance in organizational settings:

We’re not mice on treadmills with little carrots being dangled in front of us all the time.  What’s frustrating, or ought to be frustrating, to individuals in companies and shareholders as well is that when we see these carrot-and-stick motivators demonstrably fail before our eyes – when we see them fail in organizations right before our very eyes – our response isn’t to say: “Man, those carrot-and-stick motivators failed again. Let’s try something new.” It’s “Man, those carrot-and-stick motivators failed again. Looks like we need more carrots. Looks like we need sharper sticks.” And it’s taking us down a fundamentally misguided path.

Digital Badges as a Form of Gamification

There is an argument that digital badges will be successful due to their similarity to game mechanics.  Terry Hick in How Gamification Uncovers Nuance In The Learning Process stated that:

A digital trophy system–if well-designed–offers the ability to make transparent not just success and failure, accolades and demerits, but every single step in the learning process that the gamification designer chooses to highlight. Every due date missed, peer collaborated with, sentence revised, story revisited, every step of the scientific process and long-division, every original analogy, tightly-designed thesis statement, or exploration of push-pull factors–every single time these ideas and more can be highlighted for the purposes of assessment, accountability, and student self-awareness.

But comparing the digital badge system to gaming behaviors is a weak one.  Bron Stuckey, Quest Atlantis research fellow, succinctly puts it as “slapping a layer of badges over learning isn’t gamifying learning.”

There are several significant differences:

  • The user is self-motivated.  Gamers choose the games they play and their goals for using that game. They are mainly drive by intrinsic motivators.  The leveling-up becomes the symbols of their personal achievements.  In other words, the leveling up badge of recognition is a result of performance not the primary goal of playing the game (subtle but significant difference).
  • Gaming provides continuous feedback about personal performance.  Although claims have been made that digital badges can provide this type of formative assessment, most appear to be designed as a summative assessment – learners meet a set of pre-established criteria to earn a digital badge.

Who Decides?

Another possible flaw in and potential downfall of this system revolves the difficulties and dilemmas of deciding what the badges represent, how one earns the badges, and how badges will be standardized for recognition of “institutions” of learning and of employment. This lack of consensus about the meaning of badges will create further problems once the learner leaves that learning platform.  What value will the badges have in unrelated institutions?

Other Alternatives

A purpose or rationale given for the use of digital badges is to provide evidence of specific skills sets that goes beyond courses taken and degrees earned.   Randy Nelson of Pixar states that when choosing potential employees, he wants to see the proof of a portfolio not the promise of a resume (http://www.edutopia.org/randy-nelson-school-to-career-video).  Digital badges have the potential to become a type resume on steroids – showing that skill sets have been achieved but not providing specific evidence and artifacts that demonstrate those skill sets.  Given this age of personal websites, blogs, photo and video sharing sites, learners can easily leave a trail of competencies and skills via an aggregate of their achievements via some form of electronic portfolios.  Here is a question to all of those who are considering developing or using digital badges for learning:  If you wanted to evaluate someone for competency (e.g., for progressing to the next level of learning or for employment), would you prefer to see a digital badge or artifacts that provide evidence of that learning?

It is important to hack digital badges to explore potential intended and non-intended consequences of their implementation.  Technology has sadly produced a learning society that easily gets seduced by each brand new shiny toy.

Written by Jackie Gerstein, Ed.D.

March 16, 2013 at 12:13 am

Why do we give tests? What purpose does it serve?

with 4 comments

I absolutely have no idea why educational institutions use tests to presumably measure student learning.   I believe that tests provide an illusion that something has been learned, one that all stakeholders; teachers, administrators, parents, and students, themselves, have bought into.

I do not give tests.  I have taught undergraduate and graduate level college, gifted elementary students, middle school physical science and K-8 PE; and never give paper and pencil tests. One of my missions as an educator is to provide students with transferable life skills. I do not believe that the ability to take tests is one of them.  Would a vast majority of learners say . . .  “Wow, I can’t wait, we get to take tests at school today.”  “I found that test totally engaging.  I was in a state of flow”

Prior to going forward, let me clearly state that I believe that are qualitative differences between assessment, measure, and tests. I think that feedback and assessment are important aspects of the learning cycle, but am unclear how tests, in their traditional form, provide students with feedback that lead to increased personal performance.  Human learning is often complex, multifaceted, and idiosyncratic so to attempt to quantify it, often to a single number, really diminishes and minimizes this incredible experience.

As Cathy Davidson discussed in How Do We Measure What Really Counts In The Classroom?

Education has not yet moved past the standardized assessment, which was invented in 1914. Frederick Kelly, a doctoral student in Kansas, was looking for a mass-produced way to address a teacher shortage caused by World War I. If Ford could mass produce Model T’s, why not come up with a test for “lower order thinking” for the masses of immigrants coming into America just as secondary education was made compulsory and all the female teachers were working in factories while their men went to the European front? Even Kelly was dismayed when his emergency system, which he called the Kansas Silent Reading Test, was retained after the war ended. By 1926, a variation of Kelly’s test was adopted by the College Entrance Examination Board as the Scholastic Aptitude Test (SAT). The rest is history.

But because the No Child Left Behind national law began requiring the standardized tests for all students since 2002, it takes them one to two years to retrain these great students not to think in terms of single-best-answer (multiple choice) options. They have to make them “unlearn” the skill of guessing the best answer from five available ones (a pretty useless skill in the workplace), and begin to “relearn” how to think about what they do or don’t really understand about a situation, who to go to in order to find out, and what they need to do to have the best results. In other words, whether we are 1st or 17th, we’re failing at testing what we really value in the workplace. There is an extreme mismatch between what we value and how we count.

Leon Nevfakh, in a Boston Globe article, discusses the implications of the Harvard cheating scandal in What to test instead

Being successful in today’s world, as we all now recognize, requires more than an ability to think quickly and recall facts on command. And our education system has, however fitfully, moved to address those values. The problem is that our tests still lag behind.

“[Historically], the testing industry, because it was pragmatic, only tested what it was easy to test,” said James Paul Gee, a professor at Arizona State University. “But as a parent, I don’t want you to just test what’s easy to test, I want you to test what’s important to test.”

In terms of developing more authentic assessments, Nevfakh notes:

Using new testing ideas like computer simulations, games, and stealth monitoring, they are trying to take what they believe is a huge and necessary leap—changing the test as we know it from a fixed measurement of what a student can remember on a particular day, to something far more dynamic and informative.

The researchers at the forefront of test design also have a bigger dream, rooted in the idea that tests aren’t just a static part of education, but can actively shape what teachers teach and what students learn. If you can really build smarter, more sophisticated tests, they say, you can change education itself.

Written by Jackie Gerstein, Ed.D.

September 22, 2012 at 5:33 pm

Assessment as a Means for Developing a Sense of Achievement

with 4 comments

They replaced the old spin bikes with some new ones at the health club where I work out.  These new ones have a feedback monitor that provides feedback about effort via the RPM, watts, and gear level,  The spin instructor told us that the recommended watts for a good workout is over 200.  I started my workout as I always do, putting out my typical amount of effort.  The watts indicator hovered between 100 and 125.  Yikes!  I have gone to two workouts using this monitor.  I have reached, huffing and puffing, 200 watts on a few occasions, and attempt to keep it at around 150.   I wasn’t able to reach 200 watts the first time and felt a great sense of achievement upon doing so during my second spin class with the monitor.  Needless to say, these were some of the best spin workouts I have accomplished. I realized that the monitor made my spin performance into a type of game by me providing me with ongoing and continuous feedback and a way to level up.

I made the connection between my experiences on the spin bike and the need for humans to feel a sense of achievement.

Need for achievement (N-Ach) refers to an individual’s desire for significant accomplishment, mastering of skills, control, or high standards. The term was first used by Henry Murray[1] and associated with a range of actions. These include: “intense, prolonged and repeated efforts to accomplish something difficult. To work with singleness of purpose towards a high and distant goal. To have the determination to win”. (http://en.wikipedia.org/wiki/Need_for_achievement)

I began thinking about how all of this applies to the educational setting.  I have a cynical view about assessments, most often in the form of tests, and how they are used at school.  They are often contrived and separate from the learning process, and a measure of a student’s deficiencies.  As such, students do not use information gleamed from the assessment process to improve their performance. As a deficiency model, rather than one that promotes a sense of achievement, students who do not achieve 100% proficiency feel as though they have failed in some way.

Assessment should be a continuous feedback loop, one that is integrated into the learning process, and where the feedback improves the competency of the learner.  Assessments should be used as opportunities to develop competencies and the related sense of achievement.

Sal Khan discusses this problem of testing:

Regardless of whether they can prove proficiency in 70, 80, or 90 percent of the material, they are “passed” to the next class, which builds on 100 percent of what they should have learned. Fast-forward six months, and students are lucky to retain even 10 percent of what was “covered.”

This is a grand exercise in labeling and filtering students with arbitrary grades rather than teaching them. It is a hugely inefficient use of time and resources, but no one wants to notice, because it is the way things have always been done.

Perhaps the worst artifact of this system is that most students end up mastering nothing. What is the 5 percent that even the A student, with a 95 percent, doesn’t know? The question becomes scarier when considering the B or C student. How can they even hope to understand 100 percent of a more advanced class?

Ten years from today, students will be learning at their own pace, with all relevant data being collected on how to optimize their learning and the content itself. Grades and transcripts will be replaced with real-time reports and analytics on what a student actually knows and doesn’t know. (YouTube U. Beats YouSnooze U.)

This is why I believe that game-based learning is becoming popular and being promoted viable means for assessment.

As James Gee notes:

Games don’t separate learning and assessment.  They are giving you feedback all the time about the learning curve you are on.

So what is the difference between a game or a machine giving feedback and a teacher giving a grade? How does all of this relate to intrinsic versus extrinsic motivation?  Is getting feedback from a game extrinsic motivation?  Does the external rewards gained through leveling up in a video game or gaining a badge by completing a series of competencies diminish the sense of accomplishment?

Judy Willis, A Neurologist Makes the Case for the Video Game Model as a Learning Tool:

It may seem counter intuitive to think that children would consider harder work a reward for doing well on a homework problem, test, or physical skill to which they devoted considerable physical or mental energy. Yet, that is just what the video playing brain seeks after experiencing the pleasure of reaching a higher level in the game. A computer game doesn’t hand out cash, toys, or even hugs. The motivation to persevere is the brain seeking another surge of dopamine — the fuel of intrinsic reinforcement.

Good games give players opportunities for experiencing intrinsic reward at frequent intervals, when they apply the effort and practice the specific skills they need to get to the next level. The games do not require mastery of all tasks and the completion of the whole game before giving the brain the feedback for dopamine boosts of satisfaction.

. . . which she further notes, helps students develop competencies and the related sense of achievement.

In the classroom, the video [game] model can be achieved with timely, corrective feedback so students recognize incorrect foundational knowledge and then have opportunities to strengthen the correct new memory circuits through practice and application. However, individualized instruction, assignments, and feedback, that allow students to consistently work at their individualized achievable challenge levels, are time-consuming processes not possible for teachers to consistently provide all students.

The best on-line learning programs for building students’ missing foundational knowledge use student responses to structure learning at individualized achievable challenge levels. These programs also provide timely corrective and progress-acknowledging feedback that allows the students to correct mistakes, build understanding progressively, and recognize their incremental progress.

How can all of these ideas influence how educators provide feedback to learners and opportunities to develop competencies along with the resultant sense of achievement?

Written by Jackie Gerstein, Ed.D.

May 12, 2012 at 3:04 pm

%d bloggers like this: