Research methodology

"In a very real sense, every piece of research is unique and calls for a unique methodology. We, as the researcher, have to develop it." (Crotty 1998:13-14)

The definitions of key terms such as "concept" (in the Literature review) and "animation" (in the "Explanatory Animation Framework") have helped shape this research methodology to its current form. The third key element of this emerging praxis is my view that conceptual consolidation is a continuum between abstract and concrete following on from Wilensky's "Abstract Meditations on the Concrete" (Wilensky 1991:193-203) which is also discussed in the Literature review.

If conceptual consolidation is a process whereby abstract ideas become increasingly concrete (i.e. grasped), it is easy to conceive that the process of representing these ideas to make an explanatory animation is indeed a vehicle for understanding. This process provides a context for conceptual consolidation on a case-by-case basis. Much of the actual growth comes from discussions with the teacher. “The teacher’s role in this instruction is central because without the constraints and guidance that the teacher provides, students would not encounter focussed workable issues or engage experimental practices to address them.” (Ford & Forman 2006:144) This should come as no surprise as there is nothing magical about the technology being used.

The Storyboard project was also an artistic task as the children were creating imagery throughout the project. One benefit for the children of being in the digital domain is that their work could be easily edited as we were in a state of continual refinement. Sullivan captures this dynamic with his term "create to critique" (Sullivan 2005) which was also enhanced in our context by the simple practice of "saving as" multiple files in case mistakes need to be undone. We shouldn't, however be too quick to correct or remove error. This point is eloquently put by artist Patricia Mullins; "When something doesn't work out, I take another sheet and do it again. If I were to rub out or destroy what went wrong I'd lose a sense of progression, or maybe later it might not look so bad." (Mullins 2009:49)

The technological and conceptual sides of this research became very practical in terms of how the data was generated. The basic functionality of “save as” was at the heart of this method as every new tangent became a new file as a working back-up and/or to facilitate trial-and-error. In this sense it's like a sculpture having special clay that only hardens when the sculptor wants it to.

This research project has been conceptualised with a constructionist mindset. Papert and his MIT colleagues involved their students as participant researchers in the true sense of knowledge construction as the students made artefacts which generated data. In the Storyboard project the primary school participants made the artefacts and generated much of the data as they too were working within a constructionist methodology.

Target audience is one difference between the Storyboard project and constructionism as conceived by the MIT Media Laboratory team. “Project headlight” was featured in Harel and Papert’s “Constructionism” including the “Instructional Software Design Project” (1991:41-84). In their project fourth grade students wrote software to explain fractions which were then tested by third grade students. The ethics approval governing the data collection process for the Storyboard project allows for online publication of the resulting animations so the potential audience is much larger than immediate peers in a local school setting. Whether this is within the spirit of constructionism is open to interpretation. Other initiatives from MIT such as Scratch animation where children’s work is featured online would suggest that Storyboard is an authentic constructionist activity. When children are authoring their own multimedia content they are working from a constructionist methodology.

Participants in the Storyboard project are learning technical animation skills and also developing their content knowledge pertaining to their topic. "Design problems are constructionist: They are about creating situations which are problematic in ways the "problem solver" can handle. Design problems take into account the learner as well as the body of knowledge "out there" which it is valuable for the learner to acquire." (Gargarian 1991:315)

The construction of meaning in a classroom involves both learning and design. The origins of design theories and learning theories have traditionally been seen as emphasising the product (design) or the process (learning). These differences are diminishing as theorists from both sides have found common ground. “Both design theorists and learning theorists now view “construction of meaning” as a core process.” (Kafai & Resnick 1996:4)

This framework first expressed by Herbert Blumer in 1937 informs my theoretical perspective due to the basic idea that symbols possess the meanings that we give them as a community. Even the meaning of iconic images can change over time. “Blumer does not provide a detailed recipe for the conduct of symbolic interactionist research. Rather, he offers some very general guidelines.” (Manning & Smith 2010:39) Blumer believed that symbolic interactionism could be summarised in the following single injunction: “Respect the nature of the empirical world and organize a methodological stance to reflect that respect.” (Blumer 1969:60) Symbolic interactionism is an entire world view and much more than a paradigm for conducting research.

Semiotics is another theoretical perspective which initially seemed appropriate for the Storyboard project as it too is concerned with images and their meaning. Although the Storyboard project uses images to represent meaning, the main emphasis is not on iconography but rather on conceptual consolidation. The conceptual journey of each primary school participant is more revealing than the meaning of any particular imagery that the students might present. Symbolic interactionism has additional resonance with the Storyboard project beyond the obvious emphases on symbols and social interaction. Blumer understood the dynamic nature of concepts and devoted two whole chapters to this in “Symbolic Interactionism.” (1969:153-182) “Through conception objects may be perceived in new relations, which is tantamount to saying that the perceptual world becomes reorganized.” (1969:164-165)

Using practitioner action research as a research methodology steers this project in the qualitative direction by seeking to gain a deeper understanding of animation praxis. Storyboard has been designed as a reflective practitioner case study. My empirical space is the Victorian government primary school sector and my local setting is the inner Melbourne school in which I am employed.

“Practitioner” denotes that the research is in my own institution which is also known as insider action research. The term “insider” could equally be applied to the children involved in this study as it is also their institution. The nature of the children’s explanatory teaching task has many parallels with my research aims as we are all trying to explain conceptual phenomena whist documenting the journey. These similarities and the close proximity between myself and the students is to be embraced rather than discounted. “The most powerful action research studies are those in which the practitioners recount a spiralling change in their own and their participants’ understandings.” (Anderson et al. 2007:42)

Some of the literature suggests that action research is ongoing; "The cyclical process of action research does not come to a natural conclusion, although at some point it is necessary to bring it to a close and publish the outcomes in some form." (Noffke & Somekh 2005:89) The Storyboard project however had fixed dates for commencement and completion due to the limitations of the school calendar.

This research project is both interpretive and interventionist. There are differing views about the nature of naturalistic inquiry and the role of the ethnographic researcher. Clearly the breadth of previous case studies and contexts ranges from observing pre-existing phenomena to examples where the researcher makes a deliberate intervention. "As researcher, I place myself in the system and exert a considerable degree of control over the system throughout the school year." (Vincent 2004:77) This might best be conceptualised as a scale of involvement. Researchers such as Burns (1997) would be at one end advocating that the role of the researcher is "...to observe social processes as they occur naturally without the intervention of researchers." (Burns 1997:301) Participant action research such as Storyboard is at the other end with issues such as "co authorship" (see section "5.8 Issues of validation and trustworthiness").

"Because of the importance of context naturalistic inquiry is often best conceived as case study." (Norris & Walker 2005:133) Although the Storyboard project is a case study, the data is presented and analysed as 8 mini case studies as each participant chose a different topic.

Kress and Mavers recognize that multimodally constituted texts rest on design (2005:174) and that each communication must be weighed on its own merits to determine the most appropriate medium, including consideration of the intended audience. The Storyboard project starts with animation as the medium and then assesses potential conceptual topics depending on how well they can be expounded using this medium.

The weekly operation of the Storyboard sessions is detailed in the lessons plans section. The children's work is presented in a cumulative, weekly format in the data collection section and reviewed as a summary in the data analysis section.

The data generated during the 2010 Storyboard trial has been archived on a separate page as it was foundational in terms of how the 2011 project was conceptualised. At the time the participants and I did not think of the project as a trial. It was relegated to trial status due to the recommendation of my PhD confirmation seminar committee in March 2011. This was not a reflection on the children's work but rather on shortcomings in my own methodology where I had not given sufficient attention to the process of creating an explanatory animation as I was too focused on obtaining the finished animations. In 2010 I placed too much reliance on the director's commentaries as being sufficient to provide the supplementary data about the children decisions. This realisation led me to circumscribe 12 sources of data for the 2011 project. Most of the participants completed their animations and most of them also managed to create an accompanying director's commentary. All of this data including a weekly researcher's reflection is available here.

Action research is known for effecting change and often this change occurs in the thoughts and revised plans of the researcher. There are at least three guiding assumptions which have led me to design this research proposal in its current form. “Unearthing the tacit knowledge we bring to a question can be an important source of data, but it also needs to be critically examined.” (Anderson et al. 2007:130)

Limitations pertaining to this study have been categorised as either practical issues relating to the facilitation of the case study or methodological limitations.

Participants in this study are not free to choose alternative methods of representing their topics. Students enter this project knowing that explanatory animation creation will be their vehicle of knowledge representation.
Participants must choose conceptual topics for their animations as conceptual consolidation is the major theme of the research question for this project.

Concepts may be further classified according to the limitations of our experience in various areas. There are varying levels of certainty involved depending on the concept:

(Table 7: Types of concepts)

Speculation

Partial explanation through active observation

Full explanation of convention or invention

Examples:

Black holes
Life on other planets

Examples:

Biology (cancer research)
Psychology (human behaviour)

Examples:

Musical key signatures (convention)
Internal combustion engine (invention)

5.7.2 Methodological limitations

I am the only researcher directly involved in this project. To increase the reliability of a study it is generally considered to be beneficial to have other researchers involved in data collection and/or analysis. In addition to the obvious practical considerations involved with multiple researchers, there is a rationale for conducting this study in its current form. If the aim was simply for each child to consolidate their knowledge of a topic then it would not matter if others assisted. A third party (researcher) could be involved in helping each child with their conceptual consolidation but it would be harder for me to articulate when and where these changes occurred. The student might be clear as to when this happened but it the responsibility of the researcher to make these observations. This is easier to do with a regular two-way dialogue with each student. Action research gives full consideration of the context, setting and proximity of all stakeholders. “The importance of the idea of proximity is that it brings to the attention the more subtle pleasure and qualities of sharing in each others lives.” (Brohm 2009:8)
The topics chosen by each student were all different so direct comparisons between the students are of limited value. It is also beyond the scope of this project to use the same topics for other children at other times, with or without other researchers. This study is limited to finding out how particular students consolidate their conceptual understanding of particular topics. The only constant throughout the study is the Explanatory Animation Framework (EAF) which is unlikely to evolve much further during the project. This framework could be used by other researchers but the onus would be on them to show that this is useful in their own context. (Lincoln & Guba 1985:298)
How do I position myself as one possible source of information for the children’s topics? This was one of my key areas of interest and exploration as I sought to track how each child developed their own understanding. Theoretical perspectives support and inform research methodologies and symbolic interaction had practical implications at this point. Blumer’s emphasis on socially mediated meaning has guided me thus far. “Rather than remain detached from the research context, researchers are implored to intervene where possible, using interventions as opportunities to examine core theoretical issues and explore learning.” (Barab & Squire 2004:10) Research and teaching have different agendas but my interactions with the children in this case study were conducted as such that I was available for assistance as required.

Knowing when to intervene is a question which confronts teachers on a daily basis. Intervention for researchers is a continuum on which the researcher must tread consciously. A grade 6 girl from the 2010 Storyboard trial was designing an animation about the (musical) bass clef knowing that I had made one on the treble clef a few years earlier. I deliberately avoided showing her my animation as I didn’t want to influence her design approach. After several weeks of this she said “Don’t deprive me!” After viewing my animation she proceeded to use my basic structure but with her own personality and presentation style. Wells notes the same issue as one common to many reflective educators: "I am also very aware of the tension that this creates for those teachers who have a very different vision of the ideal classroom, in which a constructivist view of learning is matched by a Vygotskian interpretation of the dual roles of the teacher as planner of appropriately challenging activities and provider of assistance in students’ zones of proximal development." (Wells 2007:266)

Guba and Lincoln's (1999) criteria of credibility, transferability, dependability and confirmability would seem to be an appropriate place to start this discussion but the reflexive nature of this project leads me into new territory. The integrity of qualitative research continues to grow as researchers articulate and justify their assumptions and methods. Janesick however questions the "...psychometric assumption that the trinity of validity, generalizability, and reliability, all terms from the quantitative paradigm are to be adhered to in research." (Janesick 2000:391) Instead of looking for ways to condense this section by dismissing measures which might not be applicable, I would actually like to expand the criteria and introduce a new issue which is co-authorship.

During the 2010 Storyboard trial there were some students who didn't finish their animations. With a group of 18 students this seemed reasonable. With the 2011 case study it was imperative that all of the animations were completed in time for the debriefing session. The only way this was possible was for me to assist directly as a co author. This commenced during the second last session where I floated the idea and the group was unanimous in approving the suggestion. It still seemed imperative that I could distinguish my work from theirs. At this stage their work was still in PowerPoint format as we had yet to start the video editing process. Whenever I worked on a student's PowerPoint file I saved it as a new name so we could easily undo my contribution if necessary. For example "stringed instruments 081211.ppt" became "stringed instruments 081211b.ppt" (b for Brendan).

Some background to my position on this issue might be helpful. Teaching animation is not my usual role as I am the Performing Arts teacher. Nearly 10 years ago at my previous school, I was working with our Junior Band of 6 to 8 year-olds rehearsing the usual songs such as "Twinkle Twinkle Little Star." I hadn't thought to play along with them as it was their band. By playing along on Bass guitar I was able to guide the rhythm and chord changes which helped unify the sound. Once I accepted that this was more beneficial than waving my arms around out the front (i.e. conducting) I settled into this new format and found that we could successfully attempt a much more interesting repertoire.

Applying the same principle to our Orchestra had even more of an impact. Staff members and even parents throughout the school were encouraged to play in the orchestra to model how each section could and should sound. We had representation for trumpet, saxophone, drums, bass, violin, clarinet and our school principal was our conductor. This became a 50 piece orchestra which won the 2006 Flame Award for the best music program in the State under that year's theme, "Community connections." The staff members would routinely abstain from playing to ensure that the children were keeping up with their own parts. Regardless of how well we played, we were sending another message to the children which was that music is fun and we choose to play because we enjoy doing so. Our involvement was not disguised in any way as an audience could obviously see that we were all making music together.

With our biannual film festivals I would also assist the children as video and audio editing can be a steep learning curve. I am not seeking perfection but these films are screened at the local cinema and the children always appreciated my function as an executive producer to ensure that everything came together on the big screen.

During the last two weeks of the Storyboard project my direct involvement became necessary. The amount of assistance each child received varied depending on how much they had been able to do themselves. In hindsight, I think my direct assistance could have only been avoided if the group size was reduced to 4. The presence of the researcher will always have some impact on the phenomenon being investigated. This is best accounted for by explicitly stating what that involvement actually is. Scaffolding is a well established practice in education but such scaffolding is never invisible. This is not like the dubious practice of student projects which are done at home and have excessive help from parents. My assistance is detailed in the researcher's commentaries on a case-by-case basis.

These issues are further explored in the debriefing session where I asked the children about this. They all appreciated the opportunity to get their work finished. One boy echoed my thoughts that ideally, they would have had time to finish it themselves so as not to compromise their sense of ownership. Co-authorship might seem abhorrent in terms of doctoring or polluting the data but our collaboration did not detract from the spirit of the research question as we were able to embrace our proximity. The same criticism could have been levelled at my willingness to share information with the students when their ability to articulate their voice-over scripts needed assistance. This issue has already been discussed in section "5.7.2 Methodological limitations." Co-authorship did not restrict or hinder the children's learning but rather enhanced it.

Returning to Guba and Lincoln's (1999) criteria of credibility, transferability, dependability and confirmability, the Storyboard project would have serious difficulty in the areas of transferability, dependability and confirmability in terms of substantiating what the children actually know about their topics. Hearing their voice-over scripts is not enough for their proof of content knowledge when it's a script which I have co-authored. It is better for me to concede that any claims for transferability, dependability and confirmability would not be adequate. Rather than see this as a weakness, I see it in a strictly legal sense where I can't make absolute claims about what the children know. Instead I can make claims about what they don't know. This led me to conclude that Explanatory animation creation can be a powerful diagnostic tool. In section 8.3 of the conclusion I wrote: "My co authorship with the students placed limits on the extent to which their content knowledge could be objectivity measured. My involvement however did not taint the clear examples of what the children didn't know. The most insightful data generated throughout this project related to areas where the children's knowledge about their concepts was incomplete or incorrect."

The burden of proof for credibility is one for which I can stand my ground. Guba and Lincoln have articulated 6 criteria (1999:147) to assist in establishing credibility:

Credibility and trustworthiness are basically synonymous as the key requirement appears to be a critical perspective. "The challenge for the arts researcher is to maintain and monitor a creative and critical perspective so as to be able to document and defend the trustworthiness of interpretations made." (2006:29)

Mixed methods, more accurately defined as multiple methods, is usually when qualitative and quantitative methods are used. In the Storyboard project, the multiple methods are all qualitative. Morse & Niehaus (2009) identify two conditions in which a QUAL-qual design is appropriate:

The different levels of analysis occur when the director’s commentary data is:

Creswell & Plano Clark (2007) do not believe that research can be said to be "mixed methods" unless there is a combination of both qualitative and quantitative methods. They prefer the term "multimethods" when the methods are all from the one paradigm (2007:12). Definitions aside, what is important in the Storyboard methodology is the multiple levels of analysis achieving different perspectives on the same phenomena. Many researchers talk about the richness of a data set but I believe that the QUAL-qual approach used in this study takes the richness description further by creating a logical framework for deeper analysis.

There are two factors which provide this richness in the Storyboard project:

(Table 8: Issues of credibility) addressing Guba and Lincoln's criteria (1999:147)
Prolonged engagement	The project ran for 6 months but I have taught these same students for many years as I work with the whole school.
Persistent observation	This is generally an outcome of prolonged engagement.
Peer debriefing	Discussions with colleagues at school and supervisors at University.
Triangulation	12 sources of data (see Data collection)
Referential adequacy materials	In addition to the data presented in this thesis, there are several gigabytes of unedited video and audio footage. When I edited this material I was careful to preserve the original sources as unedited source files.
Member checks	Member checks were ongoing as the children were constantly revising their work. Their revisions are preserved as different date-based files in PowerPoint (.ppt or .pptx) format.