Eval17 | ProjectsHalfDone.png

This week I’m at the American Evaluation Association‘s annual conference. I’ll be presenting on data visualization with my good friend and colleague from Virginia Tech. I will post an update next week on all the amazing evaluation, assessment, and data visualization awesomeness but for now here’s a recap of my favorite AEA conference moments and why you should join AEA!!

Past Conference Highlights

  • MQP Selfies
  • Seeing my favorite people
  • Meeting Chris Lysy of FreshSpectrum
  • Chris drawing a cartoon I helped inspire
  • Learning from Stephanie Evergreen
  • Going to a speakeasy with Stephanie, Chris and the whole #dataviz crew
  • Statistics coloring books
  • Meeting a ton of new eval friends to keep in touch with throughout the year!

Stay tuned for this year’s highlights and join AEA!


Data Visualization Basics: Color Palettes

Color Palette Basics | ProjectsHalfDone.jpg

In the last few weeks I have gone over a few aspects of data visualization that help craft the foundations of visualizing your assessment data, but in my opinion none are more critical than color. Color takes a plain microsoft excel graph from generic to representative of your organization or institution. It’s the foundation of branding in most organizations, especially when those organizations are universities. If you’re from Virginia as I am, you know the difference between Virginia Tech orange and Tennessee Volunteers orange. Moreover, if you’re a Tennessee fan you know that there is only one correct orange and you will correct everyone in an off-color.

Before we get into the specifics of applying these colors to your graphics, let’s do a bit of an experiment. If you carefully look over your university’s page, you’ll notice the main colors but you will also see some that you do not recognize as your university colors, however those colors are most likely still a part of your universities color palette and were carefully selected to go with the main colors. Let’s look at my institution, Iowa State University.

Here is our main homepage. You clearly see our university red and yellow colors, complimented with some white accents in font and line choices. These are all part of what everyone in the midwest will recognize as ISU colors.

Data Viz Secondary Palettes | ProjectsHalfDone.png

Then if we move over to the library’s home page we see the ISU red banner, but then the tabs here are blue, green, and yellow. Yellow is an ISU color but not THAT yellow…and blue and green aren’t anywhere in a traditional ISU logo.

Data Viz Color Palettes | ProjectsHalfDone.png

What’s that about? Well, those colors are actually still part of the university color scheme, from the “secondary color palette”. All universities I’ve ever interacted with have at least one if not two color palettes. Here you can see ISU’s primary and then secondary color palette.

Primary Colors.png

Secondary Colors.png


Now we can see where those blues and greens come from. And seeing the colors all together really allows us to understand how these colors work together. Someone designed this for us. Our job is halfway done thanks to them!  Try searching your university followed by the words “color palette” and see what you find. I’ve yet to find a major institution without some type of color guidelines easily searchable on the internet. If you can’t find your colors, but your university does have a logo you can still get the colors with a bit of work. Feel free to drop me a line if you want some help or stay tuned as I will cover that in a future blog post!

Along with a general sense of the colors associated with our universities, we can also often find other “brand guidelines” that we are supposed to adhere to when representing our institution. How many of us have messed that up at conferences? For example at ISU we are never to use purple with any of our colors or in any presentations. If you’re at Virginia Tech as I once was, you know better than to use blue with any orange, lest you look like those hoo-vians from up north. (Kidding, UVA) If we are using the microsoft default palette…what are the first two colors to come up? Yes, blue and orange. Blue and orange doesn’t represent ISU and it certainly doesn’t represent Virginia Tech. So why are we putting those basic blue graphs on our assessment reports?? Let’s do a better job at representing our data AND our institutions in our reporting practices!

So go find your color palettes. In my next post we will learn how to get those exact colors into your data visualizations!

Other posts in the Data Visualization Basics Series can be found here:

The analysis of “other”

The analysis of "other" | ProjectsHalfDone.jpg

Today I want to chat a little bit about survey analysis. Recently I’ve heard a few stories that disturbed me. One in particular resulted in what I would consider falsifying data. A large organization had hired an outside consultant to conduct some survey work. In the report from the consultant, the response category of “other” had disappeared from a question and after a careful look at the raw data, the responses of “other” had been split up and lumped into the other existing categories. While some people that responded “other” might have actually provided a response that fit in one of the original categories….they still selected the response of “other”.

Is this cleaning the data or is this mishandling the data?

It’s one thing to clean the data for incomplete responses or responses that did not record properly but it’s another to move categories around after the survey had closed. To me, this indicates poor survey design. This isn’t always the case, but it certainly happens.

Now, not to say that we shouldn’t use the category of “other” or that well designed surveys don’t have this category. Quite the opposite, actually. There are times when you absolutely should be using “other” to allow participants to describe their particular experience or when a program has several parts that may fluctuate and you won’t be able to adjust the survey for last minute additions or issues. “Other” can be a very important and informative category.

So let’s pretend that we’ve created a survey for a local college’s orientation and we do need to include the category of “other”. (Note: The category “other” should not be used when pertaining to gender identity. People aren’t “other”. Don’t make them feel that they are. For more information click here.) We have distributed the survey and collected responses from about 50 participants. One question that used “other” was our question on which part of college orientation students liked best.


Analysis of "other" | ProjectsHalfDone.png

Looking at the data, some students picked “other”. (And a lot of students seemed to enjoy the talk on financial aid which is a little ridiculous but hey, it’s not real data) Our reaction is often to just ignore this category…but let’s stop for a second and look at it. If we look at the data for more than 10 seconds we will actually see that “other” had more responses than one of our actual categories. This…might indicate a flaw in our survey. So let’s look at the actual data. We should have had a space for students to fill in what they meant by “other”. If we didn’t, then we really have failed to represent the people we are surveying.

Cateogry of Other | ProjectsHalfDone.png

From reading the responses we can see that several students enjoyed meeting the faculty members. Maybe we should have had that as a category. We can also see that one student did like the campus tour which was an option but they had other thoughts on it, so they wanted to share those. That information is still important and that response should not be moved to be grouped with “campus tour” in my opinion. The respondent chose “other” and we need to respect their choice. To change the category is going to far with “data cleaning”.

By looking at what falls under the “other” umbrella we can get a better sense of what we need to ask in our future surveys and what a reasonable number of our respondents were saying. This was an oversimplified example, but I think it translates. I’ve seen so many rich and important points come from the category of “other”. This category needs to be examined, reported, and considered in future survey design. Too often “other” is swept under the rug. Don’t let that happen in your survey design or reporting!


Do you overlook “other”? Have you seen other surveys poorly handling this category? What’s the best information you’ve found hidden in an “other” response? Let me know in the comments!


The Language of Evaluation

Language of Evaluation |projectshalfdone.wordpress.com

At a recent staff meeting, I presented my colleagues with a variety of evaluation terms. I was interested to see their reaction to the terms in order to open up a dialogue about evaluative practices in our office. Evaluation culture is something that fascinates me and I thought that perhaps starting at the most basic level, the words we use when talking about evaluation, could give some insight in to how people view evaluation practices in my particular workplace.

I instructed my non-evaluator work-family to write down whatever came to mind regarding the term on the card I had given them. The evaluation terms I had chosen were:

  • Monitoring
  • Assessment
  • Evaluation
  • Performance
  • Judgement

After a few minutes we went around and discussed their thoughts on the words. Not surprisingly, there were several negative connotations surrounding evaluation terms. But let’s focus on the positive for a minute.

The most “friendly” evaluation term seemed to be “Performance”. Responses to this word included “Evaluation”, “Artistic”, “Scale” and “Skills”. I wouldn’t necessarily classify these terms as positive or negative, however this was one of the only cards without the perception that evaluation was a threat.

Regarding “Monitoring” the words that came up were “Fiscal duties” and “Transactions”. Monitoring appears to bring up more financial and budgetary thoughts within my group.

“Assessment” had a lot of writing and a lot of discussion. A few of the concerns that came up regarding this term were academic freedom and accreditation. Within the realm of higher education, this word tends to have more of a threatening connotation than does “Evaluation”. Other terms associated with this word included “Measurement”, “Process”, “Formative/summative”, and “Accountability”.

Assessment  |projectshalfdone.wordpress.com

The first thing written on the card for “Evaluation” was “Sounds better than assessment” which did make me laugh. Other responses included “Not sure if it is the same as assessment” and “May scare some people”. Although this term was decidedly less threatening within the group discussion, there was still a lot of worry and concern surrounding the word.


Somewhat unsurprisingly the term “Judgement” was almost completely negative in perception. Responses included “Criticism”, “Evaluation”, “Unfair” and “Picking sides”. It is easy to see how this term would be threatening or negative to most people, but seeing the exact parallels drawn is really interesting.

After reading over the responses from my colleagues and listening to their thoughts on their language of evaluation I have begun to rethink how I approach conversations with my group. Understanding what each organizations “evaluation language” is can help an evaluator to gain footing with organization members. Perhaps by slowly introducing evaluation terms in a non-threatening way, we can change the perception of some of these more negative terms.

I may even consider doing this activity with a new group of stakeholders when starting a new project or working with a new organization. I think that it would also provide insight for ECB and evaluative thinking work. If nothing else it may be a good icebreaker for new groups. As a field we have to work to show the positives than can come from evaluation, and in many cases we also must demonstrate how evaluation is not a threat. In order to develop a culture of evaluation and a dialogue around evaluative practices it helps if you start off speaking the same “language”, or at least understanding the meaning behind the words that you share.

Have you discussed appropriate evaluation terms in your workplace or with stakeholders? Do you find “Assessment” or “Evaluation” to be more acceptable in your particular field?


The Last 6 Months

Ok Blog’o’sphere. I slacked.

I’m done with that now. So let’s review what has happened since we last spoke. There’s been a lot going on!!



  • Finished the semester
  • Got married!!


  • Collaborated on the evaluation piece for a grant that is still under review. CROSSING MY FINGERS!!
  • Co-taught an online class. Fun!


  • Helped out with the Governor’s School for Agriculture on campus! Got to work with some awesome kids.


  • Worked with Dr. Archibald on Evaluation Capacity Building with Cooperative Extension


  • Back to life…back to reality (Which kind of means things slow down…)
  • EXCEPT WAIT!! Prelims!!!!!!
  • AHHHH!!!


  • Go to AEA for my first time!!
  • Officially a Doctoral Candidate.


  • Check back and find out!!! And hear more about the rest of that stuff!

Open Access in Evaluation

It’s on every graduate student’s mind…publish, publish, publish.

But where?

What happens if you publish in an open access journal? What IS open access?

Well, in the fields of Evaluation and Extension, there isn’t a lot to go on. Other fields have expansive Open Access journals that would add to ones CV tremendously. For Extension the only viable option I have located is the Open Access Journal of Medicinal and Aromatic Plants.  Interesting, but with Agriculture being so broad, I know nothing about this topic…however I am interested in herbal remedy type-things so perhaps I will peruse this in my spare time. But for me, this is not an option for submission. Still fun though!

Then looking at Evaluation there was a bit more to go off of. The one I found most notable was Practical Assessment, Research and Evaluation. Much closer to my interests!! It is peer reviewed and has been around for 15 years. Pretty impressive! The articles are all mainly focused on education and statistical tests. I’m familiar with these facets of assessment but it’s not my main focus, however looking over their policy page I do see that they publish “Issue papers” on controversial topics. With my focus on Evaluation Culture I wouldn’t necessarily call this a controversial topic, but I think I could work something out.

Interesting options for publication.

Do you think it’s easier or harder to get published in an open access journal? Do you have any experiences? Would you try it as a graduate student?

Project Personal Mission Statement

A professor in my department has a personal mission statement that she shares when doing her “elevator speech”. It’s usually what she leads with. Hers is really more of a tag-line almost. It’s “Science literacy for all”. As a recovering high school science teacher, I love it. I am totally on board.

And clearly, this personal mission statement sticks with you when you hear it. It’s how I define this professor in my head when thinking about her research interests and general professional work. I look up to her a lot as we have a lot in common, being the former science teachers in the Agriculture department. (Honestly, I look up to her a lot in all manner of ways. She’s a pretty inspiring person.)

So this has gotten me thinking. Perhaps I should craft a personal mission statement or tag line.

What do you think? Do you have one?? I’d love to hear it!

I’ve also discussed this idea with one of the professors on my committee, Dr. A. He said he had seen similar things in the Evaluation field, and I could use it in my email signature or in the blog, or even on the CV. Hmmm…

Right now, mine is rather lengthy…and mostly in my head. So let’s try to get it written out and then maybe work on shortening it, shall we?

Ok. Here goes.

Evaluation should be used to help you improve. It’s a snapshot, not a snap judgement. Evaluation should not be intimidating or used to threaten. It should be used to focus on how you can grow either as an individual or as an organization. I want to take the negative connotation out of the thought “I’m being evaluated”. It should be helpful, not harmful. Productive, not destructive.

What do we think about that? Ok, enough with the “this…not that”. Too many? Haha.

Ok…let’s try for the short version

 Evaluation is a tool. Not a judgement call.

Yes? Maybe? Does that get my point across or allow questions that I could then elaborate my point?

What parts do you like? Anything you disagree with? I’d love to hear from you!