The analysis of “other”

The analysis of "other" | ProjectsHalfDone.jpg

Today I want to chat a little bit about survey analysis. Recently I’ve heard a few stories that disturbed me. One in particular resulted in what I would consider falsifying data. A large organization had hired an outside consultant to conduct some survey work. In the report from the consultant, the response category of “other” had disappeared from a question and after a careful look at the raw data, the responses of “other” had been split up and lumped into the other existing categories. While some people that responded “other” might have actually provided a response that fit in one of the original categories….they still selected the response of “other”.

Is this cleaning the data or is this mishandling the data?

It’s one thing to clean the data for incomplete responses or responses that did not record properly but it’s another to move categories around after the survey had closed. To me, this indicates poor survey design. This isn’t always the case, but it certainly happens.

Now, not to say that we shouldn’t use the category of “other” or that well designed surveys don’t have this category. Quite the opposite, actually. There are times when you absolutely should be using “other” to allow participants to describe their particular experience or when a program has several parts that may fluctuate and you won’t be able to adjust the survey for last minute additions or issues. “Other” can be a very important and informative category.

So let’s pretend that we’ve created a survey for a local college’s orientation and we do need to include the category of “other”. (Note: The category “other” should not be used when pertaining to gender identity. People aren’t “other”. Don’t make them feel that they are. For more information click here.) We have distributed the survey and collected responses from about 50 participants. One question that used “other” was our question on which part of college orientation students liked best.

 

Analysis of "other" | ProjectsHalfDone.png

Looking at the data, some students picked “other”. (And a lot of students seemed to enjoy the talk on financial aid which is a little ridiculous but hey, it’s not real data) Our reaction is often to just ignore this category…but let’s stop for a second and look at it. If we look at the data for more than 10 seconds we will actually see that “other” had more responses than one of our actual categories. This…might indicate a flaw in our survey. So let’s look at the actual data. We should have had a space for students to fill in what they meant by “other”. If we didn’t, then we really have failed to represent the people we are surveying.

Cateogry of Other | ProjectsHalfDone.png

From reading the responses we can see that several students enjoyed meeting the faculty members. Maybe we should have had that as a category. We can also see that one student did like the campus tour which was an option but they had other thoughts on it, so they wanted to share those. That information is still important and that response should not be moved to be grouped with “campus tour” in my opinion. The respondent chose “other” and we need to respect their choice. To change the category is going to far with “data cleaning”.

By looking at what falls under the “other” umbrella we can get a better sense of what we need to ask in our future surveys and what a reasonable number of our respondents were saying. This was an oversimplified example, but I think it translates. I’ve seen so many rich and important points come from the category of “other”. This category needs to be examined, reported, and considered in future survey design. Too often “other” is swept under the rug. Don’t let that happen in your survey design or reporting!

 

Do you overlook “other”? Have you seen other surveys poorly handling this category? What’s the best information you’ve found hidden in an “other” response? Let me know in the comments!

 

The accountability approach to productivity

In my last post I talked about the importance and impact of scheduling your day, or at least having a general schedule for those days where you’re not rushing from meeting to meeting. Today we are going to talk a little more about schedules, but more specifically about how knowing your schedule can help with productivity.

I’ve tried a lot of things when it comes to increasing productivity. Having a schedule is just one component of that equation. For those of us in academia, especially in the summer, our days are mostly up to us. Sure, meetings happen, but if we aren’t teaching classes or leading programs, we can choose what we work on each day. We get to pick where to spend our time. It’s both wonderful and terrible at the same time. A lot of us got here because we are very good at following the rules, showing up, doing what we are told, and then suddenly, once your PhD is completed, your schedule is your own. What do you do with it? There’s work to be done but…the deadlines are often weeks or even months away. How do you cope?

For me, it’s accountability. I need to have some kind of report or product to show me where I’ve been and what I’ve done with my time. I’ve tried a few different things, paper planners, to-do lists, evernote, but what seems to work best for me is a time tracking app. It’s similar to what an independent consultant would use, or a contractor, or anyone who needs to estimate hours spent on different tasks. There are several out there. Damon over on the Art of Productivity has reviewed several options so I won’t repeat his efforts, but you should check them out (Link: http://artofproductivity.com/top-time-trackers/).

Personally, I like ATracker. I used it on and off during grad school but now that my time is really under my direction it helps me be accountable to myself and know where I’m spending the bulk of it (no shocker that it’s meetings and email…). It took me a little while to remember to use it and I admit I still forget sometimes, especially on days that my schedule gets thrown off, but I can always go back and edit the hours and add in what I missed.

Here, you can see how I’ve spent my morning so far. Again…mostly email and organization.

Accountability and productivity | ProjectsHalfDone.PNG

I like the fact that I can color code my tasks. For me, each of the colors represents a portion of my PRS (research percentage, university service, etc) and allows me to quickly see how much of my time I’m devoting to those activities and then compare it to my PRS document in my annual review.

If I look and yesterday’s breakdown I can see that I spent a little over 70% of my time on Professional Practice (red categories in my world). I can also look at the past 30 days, or entire year. Again, great data to take to that annual review.

The accountability approach to productivity | ProjectsHalfDone.PNG

By having a clear graphic (even though it is a pie chart….ugh) of how I’m spending my day, I feel a sense of accountability. I need to make sure I’m spending my time effectively. It’s not always perfect, but it keeps me on track. If I check the app at the end of the week and see that I did little to nothing in the Research category….I know I need to up my game and work on that next week.

Having some type of accountability helps you stay focused and on track. Accounting for your time can give you a huge push in the direction of increased productivity. I know it’s worked for me. How do you account for your time? Have you used any other apps or trackers? Leave a comment down below!

The accountability approach to productivity | ProjectsHalfDone.jpg

 

The beauty of scheduling your workday

After many life changes in the last year I ended up taking some much needed time off from blogging, social media, and most of the internet to just adjust. Since January of 2016 I’ve gone from PhD graduate to postdoc to job searching to landing the dream job I never dreamed of and moving a thousand miles across the country. With so many changes it was very easy to get off schedule and sure enough, that’s how I’ve been feeling. So that got me thinking about how important and….well…beautiful a good schedule can be!

I’ve always been a planner and a fan of scheduling things. It just works for my brain. On Saturday we do laundry and on Sunday we grocery shop. I like order and organization. So why not schedule my work day in a similar way to how I schedule the rest of my life? Sure, my life-schedule doesn’t always go exactly as planned but the structure is there to give me direction. It’s not a big deal if I don’t stick to it exactly…but the purpose is to help make sure the most important things on my list do get accomplished. I’ve always got clean clothes and food for the week thanks to my schedule….so why not apply that to my workday so I can always have things like productivity and satisfaction?

I find that if I don’t have a decent schedule outlined for the day, I tend to get off-task or spend WAY too much time on one task when two others need attention that day as well.

So how to develop a schedule for a workday that can go in a million different directions? For me, I read several things about scheduling before I even realized I wanted to create a scheduled day. For example, I read somewhere that keeping your email open all day is a huge distraction and it leads to task switching and a reduction in productivity (more on productivity in an upcoming post). The article suggested scheduling email checks at 8:30, 12:00, and 4:30. Or in general, when you get to the office, mid-day, and at the end of the day. Not a bad plan.

In order to create a schedule I outlined the things I needed to do each day. Check email is a must…and I like the idea of scheduling it so it’s less of a distraction. Then I need some organization time, I need to schedule writing time. A lot of my work is project based so “Project time” is a good catch-all. Then there’s always the possibility of meetings. I also need to be able to read for my job so reading time is important. And so is lunch. We can’t forget lunch time.

I schedule mine for 9:30, 1:00 and 4:00. I didn’t like checking email before lunch….as I would often get wrapped up in something from my email and then have to rush to eat my lunch so I could make it to the next task. I don’t want to digest my email when I’m trying to enjoy my lunch. By keeping email off and having a notification from my calendar to check it….email is not forgotten and it’s not a distraction.

After I had email on my calendar I knew writing time was important based on several things I’d read and my own experiences…so I added that from 8:30-9:30 with exceptions for the days that I have 8:00am meetings….which are a lot of days. In my perfect world, no meetings would ever be scheduled until after lunch. I’m a morning person and I’m going to do my best work in the morning. Meetings aren’t (productive) work. Meetings are talking. I can do that anytime with little to no caffeine needed. But we don’t live in my perfect world so I have to build my writing time around the people who have scheduled 8:00am meetings. So some days I get an hour to write, some days it is just a half hour…but any time is better than the large expanses of no writing time scheduled….

Now that we have email and writing down….what other job requirements do I have? For me, I have a lot of projects I work on but they vary. So I scheduled my lunch and project times in one hour increments. In the afternoon I scheduled project work OR meeting time just as a reminder that if the meeting time is up to me, it will be after 1pm.

Then I also need to do a good deal of reading for my job. I can’t write or do research if I don’t read. So I schedule writing time with coffee (BONUS) for 10:00am.

Here’s what my ideal day would look like:

Scheduling | ProjectsHalfDone.png

 

I typed this up in word and set the background to black and took a screenshot. My computer background is black and so this image pops up on my background…to help me remember how to have my best day. I find that when I can stick to this schedule or as close as possible to this schedule, I have a very productive and happy day. Who doesn’t want to make it to 5pm and feel satisfied and productive?

For me, scheduling works. Keeping the email turned off works. Scheduling writing time is how this blog post happened. I hope it’s how more continue to happen.

Do you have your ideal schedule written out? What do you do when meetings or other things get in the way of your plans? What’s one part of your day you try and keep at the same time each day? I’d love to hear your thoughts.

Student Evaluations: The comment section

Congratulations! Most of us have successfully reached the end of another semester. Well done!

Now that things are winding down, those pesky little things called “evaluations” are being sent out, far and wide, in every student inbox across every campus. Soon that feedback will be compiled and land in the inbox or mailbox of faculty everywhere. Half the time it will sit there for a long while. Ignored as we go into the summer holiday or summer research venue. But eventually you have to look at it.

Well you SHOULD look at it. Right?

Yes, of course you should! It’s not just there to sit in judgement of you. It’s there to help you. It’s there for you to respond to as well. No, not directly to those students, but to the next class. And the next. Your response comes in the changes you make, the ways you continue to improve over the course of your career. Yeah, sometimes the feedback is painful. Sometimes it’s funny. Sometimes it could be more constructive, but it’s feedback nonetheless. It’s an assessment. And that data needs to be utilized. Not just by your administration, but by you. How will you respond? How will you use YOUR data?

I suggest ripping off the band-aid. Just sit down with a strong cup of coffee (or a strong drink, I don’t judge) and skim through it. I would be willing to bet that you find at least one compliment.

We know that if given no incentive, typically we get the most satisfied and most dissatisfied individuals that respond to evaluations. They love you or they hate you (unless you gave them credit for filling out the evaluation, in which case you get the good, the bad, and the ambivalent ones that are here for the points). So don’t go in expecting all sunshine and daisies. Expect to get feedback that is less than pleasant (and sometimes downright immature) and know that you have room to improve. We all do. I do. My boss does. My mom does. And you do too. No one is perfect. So go in prepared.

The quantitative feedback is easy. The ratings are right there. High or low, simple. Often, that’s where most of us stop reading. We close the document, and go on our way. But typically there are several more pages in that report.

Yes, the comments section.

That’s where the discomfort starts.

So how do you examine the comments? You don’t have to do an in-depth analysis. Just a quick read-through of each comment is enough. See where the majority of student opinion is falling. Get a blank piece of paper (or spreadsheet, Evernote note, whatever) out and divide it in two columns. Do a quick scan of the comments. Read each one and write the general theme down. If there are lengthy comments then sure, two themes. See what students are liking about your course. Write those things down in one column, just short notes. Then look for things you may want to change based on the less-than-positive feedback. Write those in the second column. Maybe it’s that lecture at the start of the course where students all seemed half-asleep. Maybe it’s the pace of the course. Maybe it’s that one guest speaker who got way off topic. See what changes can be made. Don’t avoid the discomfort by not reading the comments.

quote-discomfort-brings-engagement-and-change-discomfort-means-you-re-doing-something-that-seth-godin-40-40-13

Take all the feedback with a grain of salt. Not just the bad parts, but the good too. You may be perfect in the one student’s eyes, the one who got an A, loves the discipline, knew you from another class and thinks you walk on water. But are you really all that that student makes you out to be? No. And are you really a horrible, terrible, mean, imbecile like the other student said? Also no. Where are you? Probably in between. So be humble and self-reflective. Know that you aren’t perfect but know that by reading and analyzing your feedback, you can make your teaching practice better.

I know it’s hard to read your feedback. Some students are malicious in their responses, but some are highly complimentary. Print those compliments. Save them in a document for a pick-me-up on a bad day. You’re not doing everything wrong. (And….well, if the comments show that you are, you may want to take them a bit more seriously)

But it’s your data. The assessment of your efforts (but not of you as a person, remember that). Either use your data to move you to change….or let your data sit and stagnate. Either way, you can’t make that data not exist. So use it to your advantage.

Who run the world?

 

9d6db7c9eb13027ba753335418fcf66a(Image credit: http://advancedrestorativedentistry-tmj.com/)

Happy Administrative Professionals Day to everyone out there that makes the world run smoother. You book our travel, listen to our whines, hand us the tylenol, get us shiny new toys when we break our old ones, organize our schedules, stand in for us when we can’t show up, and support us in every way possible. Thank you. You’re the reason we can do our jobs. You’re the reason that anything actually gets done. We have an advanced society built on the talents of amazing organizers, planners, coaches, therapists, travel agents, all rolled into one thing…administrative professionals.

Throughout my life I’ve encountered so many administrative professionals who held things together and kept the ship on course. I’ve never forgotten everything you’ve done for me, for the office I worked in, and for the institutions we’ve served, so a big thanks and much love to:
Sandy, my mom’s administrative professional who always put me through to her when I was a little kid or annoying teenager trying to get ahold of mom at work.
Jinky, words cannot express how much I miss you. You were an amazing lady. You kept the halls of MHS filled with laughter despite your battle with cancer. We miss you.
Michelle, Nothing. Would. Happen. Without. You. Miss you tons.
Kim, you are amazing and talented and you crack me up. Thanks for always being my go-to for literally everything I needed.
Teresa, thank you for being a bright start to every one of my days in UAA.
And Melody, you’re invaluable and we couldn’t accomplish all that we do without you guiding the way.

So whatever you do, stop and think about who has helped you, find your administrative professional and thank them. They deserve that and so much more.

b30891d4753570a7391ba0868733171b(Image credit: google)

The Language of Evaluation

Language of Evaluation |projectshalfdone.wordpress.com

At a recent staff meeting, I presented my colleagues with a variety of evaluation terms. I was interested to see their reaction to the terms in order to open up a dialogue about evaluative practices in our office. Evaluation culture is something that fascinates me and I thought that perhaps starting at the most basic level, the words we use when talking about evaluation, could give some insight in to how people view evaluation practices in my particular workplace.

I instructed my non-evaluator work-family to write down whatever came to mind regarding the term on the card I had given them. The evaluation terms I had chosen were:

  • Monitoring
  • Assessment
  • Evaluation
  • Performance
  • Judgement

After a few minutes we went around and discussed their thoughts on the words. Not surprisingly, there were several negative connotations surrounding evaluation terms. But let’s focus on the positive for a minute.

The most “friendly” evaluation term seemed to be “Performance”. Responses to this word included “Evaluation”, “Artistic”, “Scale” and “Skills”. I wouldn’t necessarily classify these terms as positive or negative, however this was one of the only cards without the perception that evaluation was a threat.

Regarding “Monitoring” the words that came up were “Fiscal duties” and “Transactions”. Monitoring appears to bring up more financial and budgetary thoughts within my group.

“Assessment” had a lot of writing and a lot of discussion. A few of the concerns that came up regarding this term were academic freedom and accreditation. Within the realm of higher education, this word tends to have more of a threatening connotation than does “Evaluation”. Other terms associated with this word included “Measurement”, “Process”, “Formative/summative”, and “Accountability”.

Assessment  |projectshalfdone.wordpress.com

The first thing written on the card for “Evaluation” was “Sounds better than assessment” which did make me laugh. Other responses included “Not sure if it is the same as assessment” and “May scare some people”. Although this term was decidedly less threatening within the group discussion, there was still a lot of worry and concern surrounding the word.

Evaluation

Somewhat unsurprisingly the term “Judgement” was almost completely negative in perception. Responses included “Criticism”, “Evaluation”, “Unfair” and “Picking sides”. It is easy to see how this term would be threatening or negative to most people, but seeing the exact parallels drawn is really interesting.

After reading over the responses from my colleagues and listening to their thoughts on their language of evaluation I have begun to rethink how I approach conversations with my group. Understanding what each organizations “evaluation language” is can help an evaluator to gain footing with organization members. Perhaps by slowly introducing evaluation terms in a non-threatening way, we can change the perception of some of these more negative terms.

I may even consider doing this activity with a new group of stakeholders when starting a new project or working with a new organization. I think that it would also provide insight for ECB and evaluative thinking work. If nothing else it may be a good icebreaker for new groups. As a field we have to work to show the positives than can come from evaluation, and in many cases we also must demonstrate how evaluation is not a threat. In order to develop a culture of evaluation and a dialogue around evaluative practices it helps if you start off speaking the same “language”, or at least understanding the meaning behind the words that you share.

Have you discussed appropriate evaluation terms in your workplace or with stakeholders? Do you find “Assessment” or “Evaluation” to be more acceptable in your particular field?

 

What’s Your Googleability?

No, we aren’t talking about your ability to use google to find a local restaurant. I’m talking about how googleable are you?

ProjectsHalfDone | Googleability Blog

Professional friends – When is the last time you searched yourself on google?

As a former high school teacher, I’ve searched my first and last name many times. Why? Because you KNOW your students will. And students? We search you too. Same goes for college students that you teach, and potential employers. These days, it’s highly likely that employers will not only google your name, but also specifically search for you on social media to see how you have been presenting yourself in public.

That’s right, folks. The internet is public. In case you have forgotten.

I try to stress the importance of searching yourself to my students, but it’s worth a mention to colleagues as well. Think about how you would want that perspective employer, colleague, student, person you meet at a conference, etc to see you? What do you NOT want them to see? What are you doing in public (i.e., social media) that you maybe should not be doing? Are you advancing your career with your public image, or are you doing more harm than good?

Personally, I want my blog, my university affiliation, and my LinkedIn to be what people see. I’m proud of those things, I’m comfortable with those things, and I don’t care who sees them. In fact, I would like it if potential employers saw these pages, because I feel like I am demonstrating professionalism and capability in these venues.

Thankfully, when I google, these things do come up. (I always google both with quotes around my name, and without. Taking the quotes out doubles my search results but also makes the results much more inaccurate) 

ProjectsHalfDone | Googleability 1

Scrolling further down, all still things I would be fine with other professionals seeing.

ProjectsHalfDone | Googleability 2

I happen to be one of those (un)lucky individuals with a rather uncommon name. Most of what comes up, is truly me. Or related to me somehow. If you have a name like “John Smith”, well then, unless you are the most famous John Smith, you will probably have tens of thousands of google results, and none will actually be you. For myself, I’ve only got about 10 pages. It’s pretty easy to click through a few and make sure there’s nothing I wouldn’t want anyone to see. The worst thing that comes up is a negative review I left for a restaurant when I was displeased with their service. I’m fine with that.

What wouldn’t I want someone to see? Well, thankfully there isn’t much about my life, even college life, that I wouldn’t want someone to see. Now I’m sure there are posts (opinions) out there that I have written and probably regretted later, but they are few and far between. I’m sure someone somewhere has an embarrassing photo of me, in fact, I know there are some less-than-flattering photos of me running in races, but again, I’m ok with those being public. I don’t love the idea, but I’m proud of my running accomplishments, so the photos are fine too.

Speaking of pictures, DO NOT forget about pictures. I hadn’t thought about this much, but I did a google image search for myself. It was a little odd. Somewhat creepy. But still, good to look through and make sure there isn’t anything that shouldn’t be there.

ProjectsHalfDone | Googleability Images

Mostly, these are pictures of me, and then some pictures of people who clearly are not me. These mostly appear to be professional contacts. I think those are popping up from my university affiliation and from LinkedIn. Which, again, I’m fine with. As for the random picture of garlic….no idea, but that’s fine too.

Now, what didn’t come up in my search was also interesting. My “real” Facebook page did not come up. The one that did appear, was a highly searchable one that I made specifically for interacting with my students. This way, no friends could post things that I wouldn’t want my students to see. I kept all adult humor, politics, and opinions off this page. Also, my personal blog and instagram does not come up. That’s fine. I wouldn’t be upset if anyone saw either of those. I think the reason they do not come up is that I am somewhat careful not to use my name on those sites.

Overall, my google search is pretty accurate. You find my professional web presence, some of my local race results (and to be honest, I need to run more so these come up more. It’s a point of running pride to have a lot of race results pop up), and then some sites that clearly are not related to me, random “phone lookup” sites and the like.

If you’ve never googled yourself, here’s an article that might persuade you to consider it, if my suggestion is not good enough. Also, Google does provide some resources on editing your search results.

What about your search results? How often do you check google for yourself? Are you an employer, do you have tips for potential employees? Leave your thoughts in the comments!