Findings & Comments

Findings bubble. [Online Image]. (n.d.). Retrieved from http://www.thoughtleader.org.uk/images/findings_bubble.gif.

• Expectations on involvement were too high and response showed only Participant A took the study seriously, but when he found out Participant B was not cooperative, he slowed down, too. This shows levels of interdependence are important in the 8LEM and this kind of mismatches or string personalities can spoil the entire strategy and require a different approach.
• There is a direct relationship between involvement in the 8LEM and the quality of reflection an production.
• Readability formulas should be seen with less prevention, as they might offer further research opportunities in our L2 teaching contexts. 
• Both participants, however, used all the technological tools suggested in the self-access activities and showed very positive attitudes towards them.
• Participant A acknowledged increased awareness on what learning the language entails and the acquisition of better autonomous language learning strategies.
• The implementation of the 8LEM model in itself did not carry significant changes in one of the participants’ involvement and, rather, his negative attitude weakened the active role adopted by the other participant. This was evident in the data shown in the positive 8LEM role match chart where the initial roles, which depend mostly on guidance and exemplification, were carried out entirely, yet as the level of independent demand increased, their degree of autonomous active participation decreased.

In fact, the implementation was really optimistic in its foundations and the initial discussion held with the two participants mainly sought to modulate the negative attitudes previously shown by Participant B, since the 8LEM model requires permanent interaction amongst the members of the educational unit and a close and attentive relationship to monitor self- and peer-progress. In order to grant this desired degree of involvement, the study itself was presented as an innovative perspective to the lessons, and the results show it was working very well at the first stage. In the first lessons, the use of self-access activities which allowed them to browse different sources of information as well as different media (videos, reading texts, avatar-creating software, etc.) promoted highly positive involvement and interaction –though Participant B did not show enough seriousness in his comments and this attitude started to hinder the academic nature of the tasks and by the third module, the self-access guide was the only task they did. This lack of commitment was further worsened by the fact the course was suddenly cancelled two weeks before and the additional strategy of carrying out an exit test, in similar conditions to the ones provided for the entry test, and which aimed to provide internal validity, was impossible.

Limitations and problems

The sky is the limit. [Online Image]. (n.d.) Retrieved from https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQ1tUvzTItrMhWbsEZedITEBO9n7-DN45lnhzouusuTcSrq6zB8p6lUBnf0iRLUbgl4b74ZhZbnkEJb8Bto-ll9iINdkF8YKkggacvAVVqwRnbZr30c_H2jAxgZFePtuInF1bjg5NCM6k/s320/SkysTheLimit.jpg.

• Though the roles Participant B did not play could not have been replaced by the teacher because the results would have been manipulated, Participant A suffered the consequences of his lack of commitment.
• There were also some time limitations which affected implementation, like the abrupt cancellation of the semester, before closing the final module and doing the planned “exit test”, an impromptu essay writing essay in class, like the entry essay done at the early stage of the implementation.
• They were given a copy of the timeline, but it would have been more useful another strategy to control their participation, e.g. a checklist.
• If all the data expected had been collected, the internal validity of the study through triangulation would have been more reliable. For instance, participants had done some PowerPoint presentations but never handed them in for inclusion.
• For further implementation cycles, other data collection instruments should be used so that the analysis is both reliable and plausible.

One of the conclusions which follow from this is the fact that an implementation strategy which relies so heavily on active participation certainly needs monitoring mechanisms which allow peer-assessment and active demand for fulfilment of expectations; on this occasion, they were given a soft copy of the timeline for the implementation process, but it could have been more effective to provide each of them with a printed checklist showing expected roles and dates so that they could have been much more critical of their own roles and on the other participant’s role. It is also worth noting that the role expected from the peer could possibly have been taken by the teacher, but it was a conscious decision to neglect it since, in the end, it was not his role, and the participant should be aware of the negative effect the lack of commitment from the other learner had on his own process and, consequently, they would individually realise their importance as active participants of the model.


Another aspect to consider for future implementation instances is the need to establish constant collection of the participant’s artifacts since these participants refused to provide all the additional tasks they had done (e.g. they both had done analytically interesting PowerPoint presentations which they promised to hand in later, but never actually did), so the data available was limited in the end.

The expected outcome of further implementation of strategies similar to the one reported here can be increased motivation for teachers to carry out more cognitively challenging and interesting tasks for the learners. Consequently, professional skills will have to be permanently fine-tuned to keep up with the challenges posed by more and more critical students and national and international professional guidelines. An immediate result of individual research efforts in educational settings like the one undertaken here –especially if they bring about positive outcomes, is awareness from the other members of the community of the possibilities available to engage in more reflective and professionally developing teaching practices, and the subsequent creation of research groups. In this specific case, the implementation of strategies aimed at promoting better writing skills will lead to an overall increase in the quality of teaching practices, especially based on the demand from learners and the need to keep up with them: as part of the 8LEM, their level of active participation will make teaching more challenging because learners will demand coherence in the guidelines set, better quality in the materials, engagement with their educational community and with society in general as social roles will become stronger.

Data Analysis

This will be a very colourful entry since I will post all the charts I have devised from the data collected before attempting any interpretation. The legends underneath every chart should suffice for interpretation, but questions/comments/suggestions, etc. are always welcome!

Units of Analysis

Research cycle diagram. [Online Image]. (n.d.). Retrieved from http://www.casenex.com/casenet/images/methodology_200px.gif.

As noted in the previous entry, once the case study research methodoogy had been chosen and the componentsof the research design had been broadly stated, it was necessary to pinpoint the specificities of each one of them. However, before introducing the first one, it is necessary to make a brief outline of Leclercq and Poumay's 8LEM model, as one of the units is closely related to it.


Leclercq & Poumay's 8LEM

Leclercq & Poumay devised a comprehensive approach to the teaching and learning of any subject –i.e. domain of learning, which allows individuals to develop skills and to apply them to answer any question or solve any situation. In this model, motivational issues, social interaction and individual roles and performance are included in a systematic sequence resulting in the development of academic literacy and citizenship, as content is not seen detached from the social framework, but rather as an essential component for proper understanding. The core element of the model is the concept of learning/teaching event (LE), defined as “the joined description of paradigms […] of a learner’s activity and a tutor (or teacher or coach)’s activity, these actions being complementary and interdependent, in a learning situation.” (Leclercq & Poumay, 2005, p. 1).

These are the eight learning events:


1. Imitation/Modelling

2. Reception/Transmission

3. Exercising/Guidance

4. Exploration/Documenting

5. Experimentation/Reactivity

6. Creation/Confortation

7. Self-reflexion/Co-reflexion and

8. Debate/Animation.

They are represented in Figure 1 below

Figure 1. Graphical representation of the 8LEM from the students’ perspective

The first unit of analysis was the Positive 8LEM role match: The concept of “positive match” was adapted from security procedures and it is related to checking the actual occurrence of a predicted event. Since the 8LEM framework outlined the roles and tasks learners should do in every stage of each module, I decided to design a chart to keep track of their positive realisation of the intended role. The modules led the learners from mostly passive roles in which they answered self-access activity guides on their own with low levels of stress since they were given an entire week to prepare it and send it for correction before moving onto the second stage, done together in which they presented, debated and analysed situations before they were given an additional week to carry out the more actively productive stages of the model, which involved freer research, experimentation, discussion and further creation and the expected process writing task, done together with their peer and the teacher.

The second unit of analysis was the accomplishment of the process writing cycles: The four 8LEM-framed modules, aimed at the production of argumentative essays, had been designed so that the learners carried out certain tasks in each stage of the modules: write a first draft, peer-review their partners’ paper, send it to their teacher for further review and comments –especially in terms of grammar and vocabulary, and write a final draft. These results are put in charts and represented in charts to find patterns with the first unit of analysis.

The third unit of analysis was the score obtained in three different readability formulas. In an attempt to provide a more reliable basis to the analysis of the texts, I decided to use readability formulas, instead of other tools like rubrics or comparison to established outlines, since they still depend on the justification given by every person who applies the rubric or makes the analysis. I then resorted to the concept of readability, a construct defined as: “[a]ccording to Klare (1963) readability is ‘the ease of understanding or comprehension due to style of writing’. This definition focuses on writing style, in contrast to factors like format, features of organisation and content” (as cited by Anagnostou and Weir, 2006, p. 3). The readability formulas are considered “a way to use vocabulary difficulty and sentence length to predict the difficulty level of a text” (DuBay, 2004, p. 2), and in short, is mathematics applied to reading comprehension, an idea which might look outrageous at first. However, a closer analysis of its background proves the underlying importance of the concept of readability: it is related to research in corpus linguistics, the typical cloze exercises so familiar in tests worldwide is based on its principles, and the concept of graded readers derives from the research started in the 1920s. Though there are more than 200 different formulas available and used in various fields, just a few of them have endured criticism and are considered reliable. For the present study, three readability formulas were used and their selection was made on the basis of proven reliability and appropriateness in terms of intended audiences; they were the Flesch-Reading Ease, the New Dale-Chall, and the SMOG Grade. The application of the first one yields a number representing the degree of difficulty in an inverted scale from 1 (difficult) to 100 (easy), while the core obtained from the second and the third readability formulas represents the number of years of education required to understand the text properly. The results were put in a chart to track changes and they were eventually graphed to identify trends, which were again related to the data collected through the previous units of analysis.

 

References

Anagnostou, N.K. & Weir, G.R.S. (2006). From corpus-based collocation frequencies to readability measure. In: ICT in the Analysis, Teaching and Learning of Languages, Preprints of the ICTATLL Workshop 2006, 21-22 Aug 2006, Glasgow, UK. Available from http://www.cis.strath.ac.uk/cis/research/publications/papers/strath_cis_publication_1539.pdf

DuBay, W. H. (2004). The principles of readability. Costa Mesa, CA: Impact Information. Available from http://www.impact-information.com/impactinfo/readability02.pdf.

Leclercq, D. & Poumay, M. (2005). The 8 learning events model and its principles. Available from http://www.labset.net/media/prod/8LEM.pdf.

An eventual research design

Confined. [Online Image]. (n.d.). Retrieved from http://www.claudiam.com/images/Confined.jpg.

Here I am three months after the last post in this research blog with mixed feelings. I had not been able to restart my data analysis due to external issues and the pressure this delay and the consequences it would bring were overwhelming. Besides, on one of those occasions in which I tried to retake the study, I went back to the data collected and I noticed the learners had not handed in all the additional tasks assigned (e.g. PowerPoint presentations, additional guides), and a careful analysis of the guides and writing tasks collected showed their performance along the process had not been as positive as I had initially considered. Something even more worrying: the information collected in this blog had been more anecdotic than research-related and it did not help much compensate or provide validity to the information I had. As a result I had decided to give up, abandon the project and give myself a second chance in the action research adventure with lighter perspectives in the future.

However, a friendly conversation helped me realise that a real study, in real contexts, with real people is prone to this kind of mismatches between planning and implementation and the value of the research study could be in making sense of the information I saw insufficient. Then, I decided to go back to the data and, having left the study at rest helped me approach things from a different, more relaxed perspective. The first step then, was to identify which research design would fit the general profile of the implementation, and I found the concept of case study appropriate: “an intensive study of the background, current status and environmental interactions of a given social unit: an individual, a group, an institution, or a community” (Brown and Rodgers, 2002, p. 21).


The lighthouse

Case study research book cover. [Online Image]. (2009). Retrieved from http://www.coverbrowser.com/image/design-books/47-2.jpg.

Doing some further research in the field of case study, I found the view of Stake (2000) of case study as “an overall strategy rather than a genre of research” (as cited in Rossman and Rallis, 2003, p. 104) very useful for a novice researcher like myself because it allowed the implementation of principles instead of step-by-step procedures which might fail to fit the gaps I had identified in the data collected. I was then lucky to find an expert in case study research and implementation: Robert K. Yin, who had written several books on its principles, components of research design and approaches to data analysis. He had identified the following matrix for the taxonomy of case study, and it helped me identify a more accurate approach to the specific case I wanted to analyse:

I could then label my study as exploratory single-case study because it is “aimed at defining the questions and hypothesis of a subsequent study (not necessarily a case study) or at determining the feasability of the desired research procedures”. (Yin, 2003a, p.5). Then Yin (2003b) outlined the components of the case study research design and it was my duty to turn them into concrete items for analysis; they were:

1. A study’s questions

2. Its propositions (if any)

3. Its unit(s) of analysis

4. The logic linking the data to the proposition

5. The criteria for interpreting the findings. (Yin, 2003b, p.21)

These components were then identified as follows for this specific study:

1. The study’s main question: What are the effects of attempting an 8LEM-framed process-writing strategy on two graduate learners’ quality of argumentative essays and on their level of active participation in their language learning process? Based on the elaboration Creswell (2007) does of the kinds of subquestions devised by Stake (1995), the following subquestions were developed:

Issue subquestions

• What does the 8LEM explicitly involve in terms of language teaching?

• Which of the 8LEM roles appear to be predominant in this context? Why? How can it be proven?

• What happens if any of the roles is not performed entirely?

Procedural subquestions

• How can process-writing exercises be integrated into the 8LEM?

• Which strategy(ies) can be used to measure essay quality?

2. Its propositions:

- The principles of the 8LEM model, which had been originally designed as a framing strategy for online environments, could be transferred to language teaching modules, taking advantage of the roles and interaction proposed.

- The interaction involved in process writing seems an appropriate companion for this 8LEM attempt.

- The creation of argumentative essays involves a great deal of personalisation, which fosters the development of critical thinking skills and engages learners in lifelong learning processes.

3. Its unit(s) of analysis: established on the basis of three observable elements, which constituted the data collection methods in themselves, as will be shown in detail in the following paragraphs. The first two had to do with the degree of active involvement of the learners in every module and in each one of the writing tasks, and the last one attempted to analyse the quality of the essays written through a variety of tools designed to identify the degree of complexity of a written text. The units of analysis were the positive 8LEM role match, the accomplishment of the process writing cycles and an analysis of the changes observed from the application of three different readability formulas to the essays written by the learners along the modules.

The fourth and fifth components of the case study research design, i.e. linking data to propositions and criteria for interpreting the findings are actually hard to pinpoint, according to Yin (2003b): “these components foreshadow the data analysis steps in case study research, and a research design should lay a solid foundation for this analysis” (p. 26). In this specific case, the trends identified in positive role match, and in the accomplishment of the process writing cycles were then contrasted to the propositions and justified in the additional data collected.

References

Brown, J. D. & Rodgers, T. (2002). Doing second language research: Oxford handbooks for language teachers. Oxford: Oxford University Press.

Creswell, J. W. (2007). Qualitative inquiry & research design: Choosing among five approaches. Thousand Oaks, California: SAGE Publications, Inc.

Rossman, G. R. & Rallis, S. F. (2003). Learning in the field: An introduction to qualitative research. (2nd ed.). Thousand Oaks, CA: SAGE Publications, Inc.

Stake, R. (1995). The art of case study research. Thousand Oaks, California: SAGE Publications, Inc.

Yin, R. K. (2003a). Applications of case study research: Design and methods (3rd. ed.) – Applied social research method series, Vol. 5. Thousand Oaks, California: SAGE Publications, Inc.

Yin, R. K. (2003b). Case study research: Design and methods (3rd. ed.) – Applied social research method series, Vol. 5. Thousand Oaks, California: SAGE Publications, In