Why we need Evidence: part.1 'it works!'

Teaching isn't the most rigorous of professions.  It's not glamorous and usually not very well paid. Most of the teachers I meet do it because they love the job, and they love the students.  It's often said that teaching is "an art not a science".  There might be some truth in this. But is evidence unimportant?  I'm going to try to argue 'no' in a series of posts. 

 

It's difficult to prove much of anything in TEFL and there is very little for which  there is solid evidence. However, new techniques and approaches appear all the time and are taken up with vigour by teachers who become convinced that this time they have hit upon the holy grail of teaching -the method to rule them all!  They are sure that this time....this time...they have discovered the method that will turn their barely communicative disinterested students into fluent autonomous learners. Said teacher is convinced of the efficacy of the approach due to the stunning results it produces and the expressions of sheer joy on student faces. This position could be called the "It works- just look at their faces!" position. A few examples are the following:

"Of course we all know Genki English works great because we see it every time on the kids’ faces" (2009 online) Richard Graham, founder of GenkiEnglish, presenting the 'evidence' that his method "really, really works"

"both kids and teachers told us it really works" Video extolling the virtues of Mindfulness training in classrooms.

Teachers using BrainGym continue to this day, despite all the evidence against it, continue to insist that it works.

"In the final analysis, like any other methodology, [neuro-linguistic programming]NLP will work or not for an individual teacher because it is right for them and not because it is scientifically proven or not." (Harris 2002:37)

 

I cannot really say that these doubts have completely disappeared but I can say that ,little by little, I BELIEVE that the magic of NLP can actually come true. I am really conscious that the changes I am experiencing with myself and with my students in class...How do I know is it working? Because I can see it in my students´faces, gestures and attitudes (Esteve online)
 

 

Now some teacher might take me to task here saying "well how can we prove whether a method works or not in any scientific sense?"  This is a fair point and I agree but I have two caveats to add to it.  Firstly, if a method can't be proved to work, then we should resist saying that "it works".  Certainly we should not suggest that students' reactions or the way we feel about it constitute any kind of reliable evidence.  Secondly, though it may be difficult to prove that something works, it's relatively easy to prove that something doesn't work, -or can't work.  for example, NLP claims that you can tell a persons "learner style" by watching their eyes move and listening to the pitch of their voice.  BrainGym claims that children can children can massage their bodies to increase the oxygen supply to their brains.  Both of these claims are demonstrably false. 

  
Carl Sagan's baloney detection kit, is a good place for teachers to start.  In this case, the following principle might be useful:

wherever possible there must be independent confirmation of the facts

This is because people believe what they want to believe.  A teacher who likes a a particular activity/method/approach will find it easy to convince themselves that their students like it, or benefit from it.  Confirmation bias (i.e. recording the hits and forgetting the misses) will do the rest to convince a teacher that their method "really works".

But what does "work" mean here anyway?  If you want to test something then it's a good idea to have a clear idea of what it is you want to test.  Does "work" mean "make the students happy" or "allow me to skive off the lesson" or "make me, the teacher, feel good about myself" or "increases the chances the group of students will become more proficient"?  If you don't know what "works" means then it's meaningless to say that something works. 

no one is impervious to this kind of thinking, which is why we do need evidence that our practices work, or at least, the ability to weed-out those which really do not.  Fifty years ago teachers were making their students listen and repeat and declaring that "it really works!" and 20 years ago communicative language teaching came and that "really worked" too and now Dogme "really works!"  If all these methods work, why do we keep changing them?

 


part 2 is here
for more about GenkiEnglish read this.


NB:  If you want to read a blog which basically says everything I do, except funnier and before me, then check out this one.

references 

Harris, T. 2002. ‘NLP: If it Works, use it … or is there Censorship Around?’ in HLT magazine retrieved September 23 2012 www.hltmag.co.uk/sep02/martsep023.rtf

Graham. R, (2009) Academic Research: Genki English really, really works. In Genki English. Retrieved May 7 2012, from http://genkienglish.net/teaching/academic-research-genki-english-really-really-works

Prediction

Imagine an alternative universe in which a large number of teachers, experts and textbooks promoted a model of teaching which was untrue.  Imagine that despite ample evidence of the flaws of this approach,  in the form of books and journal articles, teachers and publishers just carried on teaching it, -all the time taking students money for this "education".   Unfortunately, many EFL teachers live in that world where reading skills are concerned.  I'm hoping to write more about reading skills, but this entry will only deal with prediction.

Prediction is a popular EAP activity.  One of the key principles of reading listed by Harmer is prediction.  He notes that things like the title can help students to form opinions relating to the work before they begin reading (2007).  Grellet (1990: 56)  adds that, “reading is an activity involving constant guesses that are later rejected or confirmed”.  The British council note that " Prediction is a valuable stage in...reading activities. It mirrors L1 skills use, where predictions form an important base for being able to process language in real time."(online)  Prediction is an idea that comes from one model of reading, "this model of how people read is called the "psycholinguist guessing game model"(Grabe 2009:102). 

Grabe makes two important points about this model of reading.  The first is that it is very popular among "applied linguists" and the second is that "it has been proven wrong in its predictions by accumulating evidence for the past 20 years" (Grabe 2009:102).   Grabe hammers the point firmly home noting that this approach "has no empirical validity and is problematic"(Grabe 2009:103)  adding “One needs only to pick up a newspaper in an unknown language to verify that background knowledge and prediction are severely constrained by the need to know vocabulary and structure.” (1991: 380)  

 20 years of being wrong and we still use it?  Still teachers may not be familiar with articles in obscure second language reading journals and Grabes book only came out in 2009.    If only an article had appeared in something more accessible, like the ELTJ a little earlier, say around 1996.......if only!

In 1996, Amos Paran in an article called "reading in EFL facts and fictions" published in the ELTJ bemoans the use of the "psycholinguistic model" of Reading in EAP courses, noting that it "was never accepted as an important model in the first place "(1996:29) and adds:

                As a final point, it is important to stand back and think how the Goodman and Smith view of reading, with all the reservations LI reading researchers expressed towards it, has been able to hold sway over L2 reading models for such a long time. (1995:33)

A point made perhaps more worrying by fact the paper was first presented at IATEFL in 1992.  Despite this, I can still pick up textbooks, such as Oxford's "well read" which include, in Swan's words, "the standard battery of exercises designed to train students in ‘skimming’, ‘scanning’, ‘predicting’, ‘inferring’ and so forth, that one finds in textbook after textbook" (2008:266)

Swan, beating Paran by 8 years, noted another problem with this model, namely the assumption that non-native speakers lack the ability to predict.  With his usual finesse for cutting through bullshit he writes:

               One of the comprehension skills which we now teach foreigners is that of predicting. It has been observed that native listeners/readers make all sorts of predictions about the nature of what they are about to hear or read, based on their knowledge of the subject, their familiarity with the speaker or writer, and other relevant features. Armed with this linguistic insight (and reluctant to believe that foreigners, too, can predict), we 'train' students in 'predictive strategies'. (For instance, we ask them to guess what is coming next and then let them see if they were right or wrong.) But I would suggest that if a foreigner knows something about the subject matter, and something about the speaker or writer, and if he knows enough of the language, then the foreigner is just as likely as the native speaker to predict what will be said. And if he predicts badly in a real-life comprehension task (classroom tasks are different), it can only be for one of two reasons. Either he lacks essential background knowledge (of the subject matter or the interactional context), or his command of the language is not good enough. In the one case he needs information, in the other he needs language lessons. In neither case does it make sense to talk about having to teach some kind of 'strategy'. (Swan 1985: 8)

Maybe it's time to stop wasting our students' time?

references

British Council (2012) Prediction.  In teachingEnglish. Retrieved July 6 2012, from http://www.teachingenglish.org.uk/knowledge-database/prediction

Grabe, R (2009) reading in a second language Cambridge: Cambridge University Press

Paran, A. (1996). Reading in EFL: Facts and fictions. ELT Journal, 50(1), 25-34

Grabe, W. (1991) Current Developments in Second Language Reading Research TESOL Quarterly, Vol. 25, no 3, 375-406

Grellet, F. (1990) Developing reading skills Cambridge: Cambridge University Press

Harmer, J. (2007a) How to teach English Essex: Pearson Education Limited

Swan, M. (2008) Talking Sense about Learning Strategies RELC Journal 2008 39: 262 261-273

Swan, M (1985) a critical look at the communicative approach 1 (1), ELT Journal 39/1, pp.2-12

MA TESOL /app ling or DELTA? Which to do?

DISCLAIMER: This is not a piece based on evidence but just personal experience.  If you feel there are any factual inaccuracies then please let me know and I'll change them.

 

I recently got an email asking which of these is a better option for an EFL teacher.  Although the person asking probably didn't expect such a long winded reply, it inspired me to put my thoughts down in this blog.  I've wanted to write about the DELTA for a while now but this is not that blog...hopefully it will push me to start writing that blog though...so I'm going to lay out some of the pros and cons of both here and let you make up your own mind depending on your needs and situation. 

Time

This is fairly straight forward.  The DELTA is a 2 month course (with 1 month to write the essay for module 3) and most UK based MAs take a year.  things get more complicated if you want to do part time, or distance learning.  A lot of people choose the DELTA because taking a whole year off work is quite tough for many EFL teachers -in terms of pay, EFL is hardly banking.    

Cost

Most DELTAs are being advertised for around the £2,000-£3,000 mark for a full time course including exam fees. Of course, if you do a full time course it is quite likely that you'll have to travel somewhere or live in another country so you can add the cost of flights, accommodation etc to that.  I did my DELTA by distance with BELL and I think it was around the £3,500 mark.  If you're lucky, your employer may be willing to pay for some/all of the fees.

In 2006 Masters programs cost about the same.   They have subsequently increased in price and I've heard they will shoot up in the near future to match BA courses, -though this could be a rumour.  I luckily did my masters in 2006 for about £3,600.  The same course now costs over £4,600.  On top of this you have the loss of earnings for one year, the accommodation and living costs.  This makes the number of people able to even think about doing an MA much smaller, I imagine.  The part time option for the same course comes in at about £7,000 over two and a half years.  There are, though, scholarships available it seems. 

There is a good chance that if you choose a fairly big university and have decent qualifications, there may be chances to work in the English Language centre on campus.  There are at least 4 people where I currently work who were in that position. 

Order of acquisition

There are a number of MA TESOL courses which offer exceptions for DELTA holders.  That is, if you have a DELTA you can receive credit for a portion of the MA without having to do it. The list ranges from nothing (Unis not on the list) all the way up to 60 credits.  Rather ironically, Cambridge, producers of the DELTA, offers nothing.  It thus makes sense to do the DELTA first IF you are planning on going to one of these universities.  I did the DELTA second and it worked out for me because, by that time I had a full time job in the UK and the institution paid some of the fees.  I did try to start the DELTA abroad but for module 2 you will need a trainer and it's pretty tough trying to find one in Asia. 

Although it's sensible to do the DELTA first, it might be easier (as it was in my case) to take a year off work when you are younger.  If you get a DELTA and then get a well paid job you might be more reluctant to leave it to start an MA with no promise of there being a job at the end of it.  However, the longer you wait to do the MA the more you'll probably get out of it.  That is, you'll probably have a better idea of teaching and more experience to give you a better idea of what it is you want to focus on. 

It's also perhaps worth adding that as module 1 and 3 are exams, you can enter by yourself without actually doing a DELTA course.  So, you don't have to take the course to apply for the exams and if you feel confident you might find this is a good way to save money.  I will add that the exam has some very odd expectations in terms of answers, so make sure you aware of these if you plan on doing this. 

What you will learn

The DELTA course is 3 modules.  The first is an exam in which you will have to define terms like "notional functional" and "unbounded morphemes"  and be able to say who started the "silent method" and what it involves.  Why this is important for a teacher to be able to do is anyone's guess.  The test, which is actually two 90 minutes exams,  does have a few useful sections.  The section in which you have to analyse and correct a student's work seems pretty authentic to me.  Also the section in which you must analyse a test and find its faults is quite useful...though you inevitably start to think about the flaws of the DELTA exam itself.

The second module is the practical part and this is the real meat of the DELTA.  you are assessed over two months and have to produce a huge amount of paper.  There are five lessons (including the experimental) four of which are observed and one of which is observed by a n external candidate.  If you fail that then you fail the whole thing.  you do have a chance to retake this though as I did.  One complaint about this module is that it doesn't explicitly tell you what good teaching is, rather it just seems to allow anything so long as you can justify why you did it.  Another problem is the huge amounts of writing you have to do.  5x 2,500 essays plus a detailed lesson plan each time 500 word post class reflection and a 800 word linking piece between the essay and and the lesson plan.  There is also a personal development essay of about 5000 words, which you cannot fail and which is full of the kind of meaningless pseudo-babble that I personally despise.  "I feel I have developed as a teacher and met my objectives" --ugh!  (edit: I might be being a little harsh here)

The third module is quite interesting.  It is a long essay which is divided into sections and staged quite cleverly so that if you mess up the start you're pretty much done for.  You have to firstly do a needs analysis with a class.  Using the needs analysis you devise an exam for the students to test their abilities and then finally you create a syllabus/15 lesson course around your findings.  It's quite a neat intellectual challenge though I did have some issues with it as well.  The literature on needs analysis is a bit fluffy and lacking any real scientific basis.  It just seems likes opinions dressed up with academic language.  It also seems a bit questionable to me to take time out of lessons to test students for a course that, in many cases, they will not actually ever do.  I wonder how ethical this is?

The DELTA has also recently introduced a 3rd module for managers and those wanting to be a DOS which seems like an interesting move.

Though the DELTA curriculum is standardised,  MA courses are much less so.  It is also worth remembering that two holders of an MA TESOL could have studied completely different things.  For example:

[course A] Methodology/ Second language acquisition/ Intercultural studies/ sociolinguistics /phonology/

[course B] Syllabus design/ testing/ psycholinguistics/ corpus studies/ grammar

Therefore it's probably worth thinking about what you want to study and trying to find a Uni which offers something along those lines.

Difficulty

in short, people can and often do fail or give up the DELTA.  It is very time consuming and I wasn't always convinced I was doing anything other than busy work.  It would take some spectacular skill to manage to fail a master's degree.  Universities are not very good at failing people and short of not submitting work or plagiarising it's a good bet that you will pass.   

 

Opportunities

It's a bit of a risk doing either one or the other because there are some jobs which prefer the DELTA and others, the MA.   There isn't really one choice that will satisfy everyone and as the job market gets more competitive, the number of places asking for, and getting candidates with both is increasing.  The place I work used to require a DELTA or equivalent qualification.  Now they state DELTA essential despite it being essentially an academic department. 

Generally speaking the DELTA will get you further.  The DELTA is the British council's baby and hence they will look favourably on people with it.  The DELTA is also more respected as a 'practical qualification'.  Jobs in EFL in Europe will more often require the DELTA than an Master's.  If your goal is university work in Asia, (particularly Japan where the British council doesn't have a great presence) the DELTA is quite often unheard of.  A search of Gaijin pot (Japan) brought up about 3 jobs which asked for a DELTA (an then they were just listed as 'desirable') and 1 on the TEALIT (Taiwan) site.  A search for Master's degree's brought up slightly more but this time they were listed as essential. 

It is worth noting that a master's degree is not the guarantee of lucrative university work in Asia that it once was.  Almost always the departments will want people with a MA TESOL or applied linguistics and almost always they will require some published papers.  Taiwan is also quite fussy about what kind of master's degree you got and they will want it to be officially stamped by your university notary and then by their 'embassy' in whichever country you are from.  they will also not accept MAs that were done part-time or those which are over 3 years old.  This legislation is apparently an attempt to avoid fake degree certificates. 

 Pro/cons

The DELTA gives you a chance to examine your teaching.  Unfortunately as there is so little actual solid theory in EFL teaching you can't be convinced that what you're being sold is actually worth anything.  OK, so now I know what a notional functional syllabus is, but I'm not sure if I should be teaching one or not.  The module three essay at least gives you the ability to try to set up a course doing a needs analysis and designing a course around the results.  It might not be great but it's perhaps the best we've got a this moment.  For those with an MA though, the theory side of the DELTA might seem a bit superficial.  Getting a DELTA though has some kind of magic aura associated with it.  For English teacher's it's like being a war veteran or a karate black-belt.  You just exude confidence and authority (whether or not you have any is another question...)

I personally preferred the MA, so I'm probably quite biased but the MA allows you to investigate whatever it is you want to investigate. The DELTA essays do allow this as well, to some extent.  In short the DELTA seems to say "this is how is it" whereas the MA says "why is it like this?" I felt I got a lot more out of the MA and though I can't say I became a better teacher by doing it (after all there is no practical element on most courses) it (cheesy cliche) enhanced my world view. 

 Notes

Any questions or correction please comment.  I would love to make this article more general as at the moment I can only go on my own experience. 

some interesting criticisms of the DELTA and a blog from a DELTA tutor, Marisa Constantinides

Here are a few threads discussing the topic in more detail. 

http://www.esl-jobs-forum.com/viewtopic.php?p=8841

http://www.englishforums.com/English/MaInEslOrACeltaTesolDelta/jlgc/post.htm

http://www.eslcafe.com/discussion/dz1/index.cgi?read=1408953984