7. Data analysis

“As in all research, analysis of the data is the most difficult as well as the most interesting aspect of the work, but in action research it is an ongoing process which is integral with reflection during data collection.” (Noffke & Somekh 2005:91)

7.1 Introduction to Data analysis

7.2 Discussion of the 12 data sources

7.3 Classification table of the 12 data sources

7.4 Reviews of the 8 participants

7.5 Multiple levels of analysis

• 7.5.1 The qualitative nature of multimodal data
• 7.5.2 Tangents from across the data set

7.1 Introduction to Data analysis

"Analysis" is to break a subject down into the component parts for closer scrutiny. Most of the 12 sources of data in the Storyboard project are already broken down as the children assemble their animations by creating the basic building blocks and forms of multimedia such as images and audio. Interpretation and synthesis are then equally important aspects of this data analysis.

It would be a relatively straightforward task to critique the children's animations by discussing their technical merits and also how well they convey their chosen topic. This however is a secondary outcome of this analysis as the central theme is really conceptual consolidation. The conceptual consolidation of the chosen topics for the primary school students must be analysed student-by-student and topic-by-topic. The children's voices are also evident in this analysis as they critiqued their own work through immersion in the design process.

This study has much overlap between the data collection and the data analysis. This is because most of the data is intrinsically analytical, interpretive and reflective. The following table highlights just how rich these data sources are. It is not coincidental or good fortune that this variety of data is available. Rather, it is the deliberate and rigorous application of incremental improvement inherent in the design process. Some of this data such as the director's commentaries are a unique source of insight previously unused in the fields of educational research or even cognitive science research.

7.2 Discussion of the 12 data sources

In order to improve the reliability of data, triangulation is a general principle in which multiple sources of data are rightly presumed to be better than single sources.  A new metaphor for validating or conceptualising data is that of a crystal.  “What we see depends on our angle of repose – not triangulation but rather crystallization.” (Richardson & St Pierre 2008:478)  Validity is enhanced through an “…infinite variety of shapes, substances, transmutations, multidimensionalities, and angles of approach.” (ibid) 

7.2.1 Prior knowledge videos

There is a prior knowledge video of each student explaining their topic at the commencement of data collection. "The central principle of conceptual change research is the constructivist idea that "old" ideas (or mental structures) are influential in supporting or constraining learning (or development)." (diSessa 2008:35) These "old ideas" are often minimal as most of the students claimed to know nothing or very little about their topics during the first session. It seemed to be sufficient for the students to have an interest in their topic without any assumed or proclaimed expertise.

Initially my rationale for the inclusion of the prior knowledge videos was to show how much each student had learnt. It became more important however to show how and why they developed their conceptual understanding along the way.

7.2.2 The storyboard itself (i.e. PowerPoint file). 

These files were saved weekly using a date suffix (or even more frequently when important revisions occur during a session). These PowerPoint files have not been uploaded to this thesis site although some imagery was uploaded if it was relevant in the weekly discussion about each student. Such imagery is in the data collection section where each child has their own page. Collectively there are hundreds of PowerPoint files as each student created at least one new file for each week of the project.

7.2.3 Voice-over scripts

The voice-over script was a separate slide within the PowerPoint file. As such it could be seen at various stages of development although I found that the children often didn't change these very frequently. The children were instructed to be careful that the content in these voice-over scripts was accurate and to only include information which they were confident about. Maybe this contributed to their reluctance to focus their efforts on the voice-over script.

The children were generally most interested in working on their imagery. Ironically, working on their voice-over script was the most direct method of finishing their animations because until the voice-over script was completed and recorded, no imagery was rendered. All imagery had to be synchronised with the voice-over script audio as the final process in the creation of an animation.

7.2.4 Completed explanatory animations

These completed animations are rendered in “Flash” (.flv) format. Until the component media such as imagery and audio is completed, the children were not encouraged to get into video editing to have the animations ready at various stages. This was simply a practical consideration as the video editing process including synchronisation or imagery and audio is time consuming and therefore, best reserved for the final stage. The ideas containing in the unrendered animations (i.e. the storyboard) are still sufficiently clear and visible for discussion and input along the way.

7.2.5 Director’s commentaries 

These commentaries were created on weekly basis as audio files (.mp3 format).  There will also be a final director’s commentary which will correspond with the completed explanatory animation (.flv format).

The multiple sources of data generated by the children should evidence some sense of alignment or congruence.  A new idea emerging from this research is that the misalignment between the voice-over script and the director’s commentaries is actually the potential for further learning which I am equating with cognitive space.  Baars (1998:86) describes cognitive space as the “workspace of the mind.” Another metaphor I’ve used to conceptualise cognitive space is cognitive growth plates. At first this might appear to resemble scaffolding as one of the boundaries is the current understanding of a student.  The difference is that scaffolding represents potential learning with assistance but cognitive growth plates are the seeds of further learning already existent in the student’s musings about their topic (i.e. their director’s commentary).

To achieve conceptual consolidation the student first had to envision or conceptualise new ideas as evidenced by their director's commentaries. Although the director's commentaries are recorded by the director (i.e. student) they arose from discussions with the teacher/researcher. "Once ideas have been stated, both child and teacher can develop experiences that help the child think about a question." (Gallas 1995:29) The inclusion of the teacher's input as influencing the director's commentary brings this notion closer to scaffolding again but these interactions are in the present tense and not set as future potential for learning.

Director's commentaries are an invaluable source of data unique to this study. Prior to commencing the trials for this project I had assumed that the director’s commentaries would simply be the explanatory animation footage with a different audio track exactly like a traditional director’s commentary found on a DVD.  I would now recommend that the duration of the final director’s commentary should be appropriate for the content regardless of the duration of the original video footage. What we found was:

7.2.6 Researcher’s reflexive journal 

These reflections were made each and every day focussing on interesting developments with various students or additional insights which I was exploring. It reads like a blog except that I decided not to publish it online until then end of each month. "The privacy of the diary makes it easier to disregard consideration of style and punctuation. Self-censorship disturbs the free flow of thoughts; editing comes later if the results are to be published." (Altrichter & Holly 2005:26)

Altrichter and Holly (2005) also noted how research diaries can include both data and interpretation, commentaries and reflection. This promotes ongoing analysis as "preliminary results of analysis can indicate which additional data are necessary to fill in the gaps in a theoretical framework and to evaluate intermediate results." (2005:25) This is also in keeping with action research in general where improvements are made along the way.

7.2.7 Conceptual consolidation rubrics 

The conceptual consolidation rubric is simply a table like a conventional rubric. The shading varies for each student and is included below an an example:

(Figure 3: Conceptual consolidation rubric)

The shaded boxes reflect my professional judgements about each child.  These assessments were reviewed on a weekly basis. As my official teaching responsibility for these students was limited to Performing Arts, my assessment of their animations was not part of their school reports. This also mitigated some ethical concerns as there is a perceived conflict of interest when a teacher has to assess student work which is part of their own research. Assessment of the animations is therefore to provide feedback to the Storyboard students.

7.2.8 Graphic imagery

The graphic imagery can be seen at various stages of development.  In the initial phase of the data collection these images are contained within the PowerPoint file.  When the students are ready to move into the video editing phase they will render these images as separate files (.png format). This imagery is more than a component part of the animation. They are the represention of each child's understanding of their conceptual topic.

The Grade 6 boy who created an animation about tectonic plates in the 2010 Storyboard trial thought that plate shaped objects penetrated the Earth’s crust without realising that the plates are the Earth’s crust.  In conversations he asserted that he understood this but his imagery continued to show plates which were distinct from the crust. 

7.2.9 Lessons plans

The majority of the Storyboard sessions consisted of free time for the students to work on their animations. The initial focus session from me is responsive to needs as they arise so a basic dot-point description of these sessions outlines the actual session development. A retrospective researcher's commentary column documents how the aims of each session were realised or challenged.

7.2.10 Debriefing session

This was a transcribed video recording conducted during the final session. This session was opened up to general discussion which provided additional insights into the process of conceptual consolidation including the children's self-assessments on their projects. The Explanatory Animation Framework (EAF) was also critiqued for further refinement.

7.2.11 Concept maps

Concept maps allow key words and linkages to develop during the sessions. I contributed freely to the development of each child’s concept map as it is still up to the students to determine the appropriate chronology for their storyboard. Having a separate slide for organising this information will also help reduce cognitive load for the children.

7.2.12 Voice-over script / imagery tables

The final stages of the explanatory animation design process involved the creation of a table with the completed voice-over script in the left column with a corresponding column on the right for imagery. This table functioned as a checklist to ensure that we had appropriate imagery throughout the animation. In some ways this table functioned as a traditional storyboard as it locked in the order and chronology of the animation.

7.3 Classification table of the 12 data sources

(Table 8: Classification of the 12 data sources)
Data source		Analytical (able to be subdivided into smaller component parts)	Interpretive (offering or requiring further explanation)	Reflective (reciprocal, i.e. acting on itself)	Moment in time or evolution of thought?	Independent element or component part?
1	Prior knowledge videos
2	The storyboard itself
3	Voice-over script
4	Explanatory animations
5	Director’s commentaries
6	Researcher’s reflexive journal
7	Conceptual consolidation rubric
8	Graphic imagery
9	Lessons plans
10	Debriefing session
11	Concept maps
12	Voice-over script / imagery table

7.4 Reviews of the 8 participants

In addition to guiding the students with their animations, my role was to notice the breakthroughs on a case-by-case basis.  The Storyboard project was not designed to measure these breakthroughs quantitatively but rather to document and interpret them.  The complete picture of each student's work is to be found in the Data collection section which is constructed around a weekly, session-by-session chronology. The following links are to a review of each student which is a more concise summary of the conceptual consolidation for each participant:

7.5 Multiple levels of analysis

In the last column of Table 9: Classification of the 12 data sources, the heading "Independent element or component part?" could also be described as "single or multiple levels of analysis?" It is the use and reuse of this data which constitutes the QUAL-Qual multiple methods approach to data analysis. A basic tenet of cognitive science requires analysis to be on multiple, interdisciplinary levels. Levels and sources are not synonymous.  Perhaps the old analogy for religion has further import here where three blind people grasp an elephant and then proceed to testify to the truth as they had experienced it.  These various perspectives are sources, not levels.  A level must be quantifiably distinct.  For example, a voice-over script is quantifiably different to a journal entry.  

The primary data analysis for this study is provided in the previous section "7.4 Review of the 8 participants." This is because the children's conceptual consolidation needed to be analysed on a case-by-case basis. Looking at this data as a whole, two further discussions can further explicate this analysis under the headings "The qualitative nature of multimodal data" and "Insights from across the data set."

7.5.1 The qualitative nature of multimodal data

The merits and richness of multimodal data has been discussed in section "3.5 Knowledge representation and animation" in the Literature review but there is an obvious link between multimodality and triangulation. In section "7.2 Discussion of the 12 data sources" in this Data analysis page, I have already said that multiple sources of data are rightly presumed to be better than single sources as a rationale for triangulation or indeed "crystalization." (Richardson & St Pierre 2008:478) Multimodal data is intrinsically analytical because it is constructed from component parts. These component parts are then available for triangulation. In the case of animation, the component parts retain their own qualities and attributes even after being merged into the animation medium. The audio doesn't change by being added to the video and the images don't change by being played successively.

7.5.2 Tangents from across the data set

During session 12 I presented a much simplified version of the following table to the Storyboard participants. I didn't actually present a table at all but rather summarised each point verbally during a 90 second monologue during our focus time at the start of the session. The point I made to the children at that time was there although they are focused on their own animations, I am seeing the bigger picture of how the overall story of this project can be illustrated by using examples from issues that have arisen through each of their topics. My reason for listing this table here is to illustrate how the QUAL-qual nature of this project can reuse data in new contexts.

Of course I could have talked about general issues like multimodality, constructionism or the Explanatory Animation Framework for any of the students but these were the examples which seemed the most illustrative or appropriate.

Main menu