Looking at our own criteria for evaluation
University of Fukui, Japan
SLA research is often defended on the basis that it can be 'generalized' into epistemological claims. However, this is actually one of those crucially unexamined aspects of SLA research. For it to generalize, they would have to specify what population the findings are supposed to generalize to (and they never do). Another related weakness of SLA research is that they are, in effect, running tiny, short-term 'sociological' studies of populations and then trying, rather weakly, to draw out psychological (developmental)/psycholinguistic (acquisitional) implications based on a 'small group' study. This makes for conceptual and interpretive incoherence in most SLA.
However, in the classroom we practitioners often say, we use this or that because it 'works', because it is 'effective'. But what criteria do we base that on? I mean, I would like to think I know what is and what is not effective, but do I really? I know SLA research isn't going to help me.
I think one danger is that teachers slide from activity to activity and too often think that the activity or task that caused management problems is not effective. However, I would suggest that often the problematic activities or tasks point to serious learning issues and instead of abandoning the issue that is coming up, we need to back up, reconsider how we implement the activity or at least try to find alternative ways to address the same learning issue.
For me, effective seems to come from initial difficulties that are then overcome in order to scaffold more complex, more inter-related tasks and activities.
Difficulties in introducing new task types has also been a constant source of frustration with EFL classes here in Japan. First, most students are only familiar with a very small number of task types (e.g., listening to the teacher explain things in Japanese, simple pair work activities, fill in blanks/cloze listening). Second, at the university level, trying to teach in a 90-minute format that meets once a week leads to a lack of continuity and baffles being able to concentrate at something intensely. Adding to the difficulties is the fact that the classes are often large, but comprised of learners with varying levels of language ability, motivation (which itself can be variable in its level, but also in its objectives), needs, wants and interests.
I suspect that veteran teachers do operate with fairly complex but flexible criteria for evaluating what is and what is not effective in their classrooms over the time-frame of courses and cycles of courses. However, it seems questionable that (1) this can be put into academic discourse and doubtful (2) that most teachers will ever get the chance to try.