Design researchers are trying way too hard to be post-positivist.

My reflection for this week's Design, Cognition, and Learning class readings; full citations and more detailed notes can be found at my zotero group.

I gotta say this first because I'm fed up with it: what's with design researchers? They're trying so hard to be post-positivist and sound Conventionally Seriously Scholarly. "No, no, we're not hand-wavy people... look, we're counting how many times we wave our hands!" Design is subjective, it's weird, it's not going to look like an experimental physics journal article -- so why try to make design research resemble something it's not? It's like eating a tofu dog. I'd rather have some Damn Good Tofu that celebrates the fact that tofu is not meat.

The article by Dorst and Cross does a little bit of the "tofu... is awesome" thing I'd like to see more of. They were studying creativity, a "fuzzy" subject if there ever was one, and needed a way to rate design for how "creative" they were. So they rounded up some trained evaluators (design faculty) and had them independently score designs, then essentially said "look, even if they -- and we -- can't fully explain why they were rating something as more or less creative, their ratings agreed with each other even if they were doing these evaluations alone, so there's something there, we just don't know how to talk about it well enough yet." Good. More of this, please. Stand your ground. The picture may be fuzzy; so what? Say it's fuzzy, and you know it's fuzzy, then keep working on articulating it better in the next round. Don't try to make it sound like it's clear. Do not use positivist empirical research from physics and chemistry and such as the holy grail; it's the wrong norm for most of this.

Dorst and Cross also talked about one factor they found in all the creative moments -- emotions. Back when I was doing software QA, I read an article about the role of feelings in bug-hunting. It talked about how emotions are our bloodhound scent, how we decide how things are important, even if we think we're doing objective, quantitative things. How do we decide where to apply our technical tools, our instruments of measurement? Feelings.

They also described some "creative pathways" as almost inevitable ones; when exploring a certain sort of problem space, most people will have the same sorts of "aha!" moments. And they will be moments of creative originality for those people, they truly will -- it's just that everyone else will independently come up with the same solution. While the article seemed to imply this was a negative side effect that needed to be taken into account so we could get past the ideas everyone else has and get into truly creative-and-original-to-the-world thoughts, I thought "ooh, that's FANTASTIC." I'm thinking about teaching, about how we can harness the same effect when we want students to make specific complex conceptual jumps, have certain "aha!" moments they can't be led to logically and algorithmically. I'd like to read more about that.

There were also good things to pick up on in the other articles. For instance, "Verbal Protocol Analysis" (VPA) is what we call the method of "have people think out loud as they're doing stuff," and now I can use that as a search term (but why can't we just call it "think out loud"?). Atman's study of first and fourth year engineering students designing a playground used VPA and found that it wasn't really how long they spent on a design that correlated with a higher scoring result, it was how they used their time; older students spent more time scoping out the problem at the start and transitioned more frequently between stages in the design process, which brought up the mental image (for me) of a learner becoming increasingly more fluent improvising in a space, like a jazz musician or a dancer.

There's a difference between knowing the basics and being able to do something with them. As a musician, I'd rather jam with an adventurous player who knows 3 chords but isn't afraid to play with them than a classical pianist who can play dozens of scales fluently but balks at veering from sheet music. You can teach the adventurous player more chords, and they'll play with them; you can teach the adventurous dancer more steps, and they'll play with those -- but how do you teach the ability to play? This is why I think that the improvisational arts (which, for me, includes the study of foreign language conversation) are so important in the design education of engineers. There are developed pedagogies in other places for kicking students out of the nest of follow-the-procedure, for encouraging them to play with what they know; I think more engineering teachers should learn and use them.  Maybe that's why I'm so drawn to the improvisational arts; I am that classical pianist (literally -- except I've forgotten many of my scales by now), I am that dancer, I am the student who'd rather read a book in German than have a conversation in the language because... controlled environment! Script! Correct answers! If I play, I might get it wrong! My seeming fearlessness in improvising (which many people seem to think I'm "good at") is actually a continued series of attempts to counteract a fear that's been a weak spot for me all my life.

The meta-study by Mehalik and Schunn ("so many people have written about the design process, so we sat down and looked at all their papers and summarized the state of that conversation") noted that some steps of the design process seemed to be under-studied. This included (as of 2006) my personal favorite, "reflection on process" -- along with one of my runner-ups, "failure analysis." Metacognition hasn't been explored enough? Very well then; perhaps I can help explore that.