Wednesday 21 October 2009

Multimedia as evidence

Core activity 4.1 - How can you create and store evidence of your engagement in the following types of activity on H808?
  • Contributions to online discussion
  • Personal blog postings or comments on others’ blogs
  • Contributions to the course wiki
  • Notes and informal reflections written by hand
  • Examples of formal writing (TMAs, reports, etc.)
  • Extracts from PowerPoint presentations
  • Extracts from audio presentations
  • Extracts or screendumps from websites or video presentations
  • Comments from peers and tutors
  • Extracts from published sources (images, newspaper/magazine stories etc.).
The following map is multimedia evidence in itself, created with Mind42, a free, online, collaborative mind-mapping tool.You can drag the map around to view, zoom using the slider, or select the node level to focus on.

Monday 19 October 2009

Key features of eportfolio systems

Core activity 3.1 involved the comparison by each student of two (or more) eportfolio systems on about six of the features which they considered to be key. By aggregating the features selected by each student we can see which features (in our opinions at least) really are key.

Wordle gives a lovely visual representation of this.
Wordle: H808 Core activity 3.1 - key features of an eportfolio system

This post also demonstrates how a multimedia item may be used as an item of evidence of thinking (and technical skills) for an eportfolio (core activity 4.1).

Sunday 18 October 2009

Personal and professional development planning

Core activity 3.4 required the adaptation of a PDP needs analysis grid for our own competency requirements, making self assessments against the required competencies, and development objective setting. My PDP grid is shared here.

This blog completes the activity with a reflection on what I have learned about my strengths and weaknesses as an elearning practitioner as a result of the activity.

I approached this activity with a little trepidation, well aware of the need to understand personal competencies in order to set development goals, but also conscious that people are often not very good at making accurate self-assessments (Boud & Falchikov 1989, Mitrovic 2001, Ward, Gruppen & Regehr 2002, Eva, Cunnington, Reiter, Keane & Norman 2004). Even after completing the activity, how do I know that my assessments are correct? I suppose one way would be to go down the ePortfolio route, assembling items of evidence against each competency - in fact, would this be a new use of ePortfolios (for assisting students' self-assessment - I'm not sure if I've seen that purpose before).

Given that people (myself!) may not be very good at self-assessment, I was also concerned about trying to make assessments to fit the given levels (Complete novice, Below average, Average, Above average and Expert). These seemed to me subjective, and therefore likely to make assessment all the harder. They also implied a comparison with others, whereas I felt my assessments of my competencies would be valid however they compared to others' abilities. Similarly, if self-assessments are likely to be inaccurate, how can I accurately assess the 'average' of a larger community?

I therefore found myself defining slightly more detail for each competency level, resulting in the following:

Complete novice

Limited knowledge / understanding

Some knowledge / understanding
Someone reasonably intelligent, with at least a general interest in eLearning. Perhaps able to use/do the competency, but not to analyse/critique/develop. Probably a base level for students of H808.

Good knowledge / understanding

As above, but with some experience, further study etc in this area of learning/technology.

High knowledge / understanding (expert)

Perhaps able to be an authority or have higher level study in this area, able to teach this element, have greater awareness of issues, practices, debates and resources.
Given these level definitions, I found it easy to consider how I related to each competency.

I think the most striking outcome for me is the distinction between technical aspects, where I score quite highly, and other teaching and learning aspects, where I am comparatively weak. This is not a huge surprise to me, given my background of Computing and Psychology followed by AI in Education research. I clearly lack experience in working with elearning students, and in developing elearning tasks, and these are aspects where I hope to gain first knowledge (from this and subsequent courses) and then real world experience.

I also found the objective setting element useful, and something I haven't done for a long time. This in itself is probably a weakness in terms of both my learning and my elearning professionalism. Useful to be reminded of the benefits.

References
D Boud and N Falchikov, “Quantitative Studies of Student Self-Assessment in Higher Education: A Critical Analysis of Findings,” Higher Education 18 (1989): 529-549.
A Mitrovic, “Self-Assessment: How Good are Students At It?,” in Workshop on Assessment Methods in Web-Based Learning Environments & Adaptive Hypermedia (presented at the 10th International Conference on Artificial Intelligence in Education, San Antonio, 2001), 2-8.
M Ward, L Gruppen, and G Regehr, “Measuring Self-Assessment: Current State of the Art,” Advances in Health Sciences Education 7 (2002): 63-80.
K. W. Eva et al., “How Can I Know What I Don't Know? Poor Self Assessment in a Well-Defined Domain,” Advances in Health Sciences Education 9 (2004): 211-224.
H851 Practice Guide 7, Reviewing and Improving your Teaching, The Open University, 1998, p.31.

Monday 12 October 2009

Wordle for this blog (4)

Unit 3 (part 1): ePortfolio systems is now done, and so it's time for the weekly Wordle.

Wordle: www.allies-in-elearning.blogspot.com (4)

System appears to be the dominant word this week, closely followed by systems, able, activity, eportfolio, evidence and use. Given the unit title, I'm quite happy with these as the key features, and it's interesting that they remain dominant despite one of the core activities being only a link to a Google Doc, rather than a full post in itself.

The terms activity, evidence, able and use seem likely to be a result of my reflection on the collation of artefacts and creation of evidence within the framework provided for the course - and remind me that I must post to the tutor group about my uncertainties (a bit nervous about that one).

None of these terms seem particularly aligned with the tag cloud of labels for my posts. I may need to keep an eye on this to ensure that I'm using useful tags, or it may be that the tags I've chosen are suitable summary words for the type of content that is showing up in the Wordle.

One more thing I'm curious about, and that's exactly what Wordle looks at when it creates its cloud. I provide the URL for the RSS feed of the blog, but I wonder how far back in the blog's history it goes... all the way I'm assuming, since that's what you get on the feed URL page. I wonder if there's any difference if I only feed it with the blog URL - does it then just do a screen-scrape of the text, or does it find the associated RSS feed?

Recommending an eportfolio system

The task for core activity 3.2 was to write a 500-word report aimed at an audience of managers in an educational institution or organisation of your choice (or your imagination), overviewing some available eportfolio systems and recommending one or more of them. It should provide information to help them decide which of the systems is best suited for their purposes*.

For this report I consider the selection of an eportfolio system for a scenario which draws on some elements from my work but which fictionalises a number of aspects.

The navy of a European country wishes to expand its training programmes using a range of distance and e-learning techniques. Among their requirements are the need for aneportfolio to:
  1. Record student progress on tasks to provide evidence for certification
  2. Record student actions in tasks for formative assessment
  3. Allow students to develop and reflect upon a record of their competencies for use in personal development planning (including identifying future training needs)
  4. Integrate with current systems including a VLE, and potential future systems such as synthetic training environments
  5. Accessible by mobile phone/handheld device
  6. Consider security of data - hosting should be by the client organisation.
The users of this system will be students/trainees (technical operators, such as maintenance technicians, service crews etc), trainers and assessors of these personnel, and senior managers with responsibility for ensuring training is delivered effectively. The trainees are familiar with operating complex computer-based systems, but training time for theeportfolio system should be minimal. Therefore after an initial familiarisation session, functionality should be intuitive and trainees should be able to use the system without assistance. This is also the case for trainers and assessors, but their interaction with theeportfolio system may be expected to be more restricted (focusing on accessing and commenting on presentations of evidence). Therefore, initial training for these roles may be focused accordingly, and should also allow users to rapidly reach a state where they interact with the system without support. The final user group (management) may be less familiar and adept at using new systems, and so views extracting the data required should be available.

The requirements given above lead to a number of possible system, dealt with here by requirement.

Requirement 1. All systems offer a way for students to record progress, although not all are tailored for assessment. eNVQ, Mahara, MyStuff and One File all support assessment or are particularly designed for it.

Requirements 2 and 3. These point to a need to support student reflection, and for others (e.g. tutors) to be able to offer formative feedback via comments.Mahara, MyStuff and PebblePad are particularly tailored to support reflective learning.

Requirement 4. The need to integrate with potential future (i.e. as yet unknown) systems is very open ended. Mahara can integrate with the Moodle LMS, and as it is open source, its code can be inspected to consider its appropriateness for future integration.

Requirement 5. Mobile access is necessary for trainees working in the field, and may be provided by MyStuff and PebblePad, but not currently Mahara.

Requirement 6. Mahara and MyStuff both offer the facility for the client organisation to host eportfolio data within their own secure network.

Given this analysis, an open source solution (Mahara or MyStuff, for example) may be most appropriate for a number of reasons:
  • Specific requirements mean current off-the-shelf solutions all have shortcomings
  • In-house developers are available to deliver a customised solution based on an open source system 
  • Unknown future requirements can be met by subsequent development.

* The full activity requirements were:
Consider the following issues:
  • the primary users of each system (i.e. higher education, schools, PDP, industry)
  • any unique or particularly desirable features from the point of view of the learner, teacher and organisation
  • the most appropriate system for this context, given the type of use you intend.
Conclude the report with a specific recommendation (which can include rejecting all of them).

Wednesday 7 October 2009

A bit of reflection - mainly on evidence of practice-related competencies(!)

I'm enjoying activity 3.1 - considering the key features of eportfolio software. We seem to be moving towards a discussion stage now, which should be really interesting. I really value Madeleine's proactivity in proposing a collaborative effort on the task as I think it has helped us focus on the fact that we are aiming for a discussion, not just 15 different contributions.

I'm also enjoying the activity as I feel I'm able to be proactive myself. Again I've been able to use some of the slightly more technical skills I have to support the group with wiki pages and a collaborative spreadsheet in Google Docs. This has been interesting for me in several ways. Trying to make a large table display in the wiki resulted in a mass of very narrow columns - it would quickly become unreadable as the rows would become enormously high. This reinforced for me the lesson of choosing the right assistive technology for the job... while a wiki was good for collaboration, it wasn't going to be able to display the data. So, next thought was a Google Doc (although I use this a lot, including for collaborative work) it wasn't until I saw Kevin had posted his incredibly comprehensive comparative grid to Google that I realised what a good idea it was). Google Docs also gave the flexibility for our different needs - Kevin's to display to viewers without giving editing rights, and mine to allow full and equal access to all who had the URL.

I hope that I can use activity like this as evidence of technology and communication-related skills and proactivity (categories from our course framework for professional development). I've created my own 4x4 grid to try to start logging how each piect of work relates to the competencies. I've also set up tags in my OneNote notebook - and using its search/aggregation feature I can easily see where I've tagged items against each competency/feature. So, the technology to log my evidence is not the problem. Where I feel I am coming unstuck is in how to identify an item as evidence for a particular competency. I think I can actually relatively easily identify when work relates to the technology, communication or research competencies... so the tricky one for me is practice-related. I really can't get my head around what would demonstrate this in the context of the course. I wonder if an activity like 3.2 (writing a report to educational managers, recommending an eportfolio system) is the type of thing that would count here. My concern again, (as in my blog reflections last weekend), is with the term practice. In my current job I don't work with eportfolios, reflection, or even with students or educational institutions. I'm at the technology end of the educational technologist spectrum - investigating new options, and their applicability to various users and contexts. Perhaps I will actually be able to use this as a practice-related competency... but I feel unsure.

[Quite enjoying this reflection... following where my thoughts go, watching questions bubble up, trying to focus myself on answering them... bit of solution-focused therapy!]

So it seems my concerns are about the practice-related elements, and whether I'll be able to supply appropriate evidence. To reduce these concerns I probably need to better understand what constitutes evidence here - so I need to ask. I guess a tutor group forum post would be appropriate here. I could privately email the tutor, but hopefully I'm not the only one with this confusion... and if that's the case, a forum post would be proactive in itself... and could that also be a practice-related competency (knowing when personal professional development is required, and seeking guidance?)... or am I clutching at straws here? Any comments much appreciated!

[Hoping that this will have got some of the meandering thoughts out of my head, and left me a little bit of room to contemplate the next activity - but it's identified some points I need to come back to.

Finally - thanks to anyone who made it this far through a rambly reflection. It's good to feel my reflective writing is developing though... I'll stop now before I descend into reflecting on reflecting...!

Feature comparison of eportfolio systems

Core activity 3.1 - to produce a comparison grid of the key features of two eportfolio or eportfolio-type applications. Tutor group discussion of which features really are 'key' for an eportfolio system to follow.

Not able to display all table columns here... so please see this published Google Doc.

Sunday 4 October 2009

Wordle for this blog (3)

This week's Unit 2 (part 2) title was Reflection and Learning. Here's this week's Wordle, which shows that, happily, I seem to blogging on the same themes! Eportfolios are still fairly dominant in the image... I'd be interested to know how far into the history of a blog's RSS feed Wordle goes when it creates the image.

Wordle: www.allies-in-elearning.blogspot.com - 20091004 (Wordle 3)

Saturday 3 October 2009

Further reflections on core activity 2.4

This post returns to the task for core activity 2.4 - to post a comment to your blog reflecting on your own attitude to reflection in learning and on the usefulness or otherwise of blogging as a means of doing it. After completing the readings* for the task, I think my feelings about reflecting on learning, and using a blog as a tool for this are relatively unchanged from my previous post.

However, there are a couple of points/feelings/ideas that I noticed while going through the readings.

Firstly, I felt I identified with those students in Crème's work who would prefer a friendly non-judgemental reader than an assessor to view their reflections. My reflections feel personal, and while I'm happy with the purpose and practice of reflecting, it's the sharing for assessment that I feel is more difficult.

I also noted that the type of reflection I have been doing in these blog posts is more akin to the 'progressive journal' rather than ''reflective writing such as that used in professional courses where students have to relate theory to examples of their practice, a distinction noted by Crème. I think that this is probably ok for the current stage of the course, and my treatment of the blog as more of a learning journal. However, I think that I will have to undertake some more specific, directed reflection, in order to experience these techniques, to develop my abilities in them, and (not least) to meet the course requirements.

I have an ongoing concern about meeting all the criteria for the course assessment. There seem to be many, and I feel that there is likely to be a conflict between honesty and personal investment, and the need to meet assessment requirements in a systematic way. Hopefully the OU's careful development of criteria, designed to assess the development of reflective skills which actually be an enabling factor which may allow me to feel confident that I have conducted an appropriately comprehensive reflection.

A couple of other slightly unrelated noticings and reflections...
- I found reading the papers much easier because I had specific information which I wanted to extract from them, rather than just gain a general understanding
- Pointing our activities to blogs and forums means I am seeing others approaching the tasks differently. This is good and bad. It raises anxiety that I am not doing it right, or am interpreting it wrongly if I approach things differently to others. Equally, seeing these differences between our approaches helps reinforce the recognition of our diversity of backgrounds, expertise, interests and interpretations - and that fact that a varitey of contributions are both interesting and valuable.

[Readings were:

Friday 2 October 2009

Reflecting on reflection - towards activity 2.4

Core activity 2.4 involves reading several papers on reflection, considering their relation to the OU's methods of assessing our development of our reflective skills in this course, and asks for a blog post reflecting on your own attitude to reflection in learning and on the usefulness or otherwise of blogging as a means of doing it.

I thought I'd write a few notes on my own attitude to reflection here, before I read the papers, to see if my feelings change after reading.

At the moment I'm not too sure what I think. Reflection feels hard - especially when it's a required task. I feels that it comes more easily when it's spontaneous... such as when I begin by thinking about a technical concept, but unintentionally diverge in asking why, and how and what this means for me. This type of organic, unstructured reflection has certinaly made me think in ways I otherwise wouldn't have, but I'm not sure it's the best way to undertake the reflection for this course - that needs structure, and I think that's one aspect of the 'professionalism' of the course title.

Having just reread the course and assignment guides, and their comments on assessing reflective writing I feel that I will have to reflect in a very specific, directed way in order to achieve in the course. This isn't, in itself, good or bad, but it may have different outcomes to if it were spontaneous.

I've certainly done more reflection on this course, its contents, and my own learning because I've been writing blog posts, so in that sense, for me,blogging has been useful. Had I not been writing the posts, I strongly suspect the reflection would not have occurred to such a degree. However, I do note that I am conscious that there is an audience for a blog post. I don't attempt to write for them (they can always choose not to read if they're not interested), but I do feel that I censor, re-work, or re-phrase my writing because I know people are reading. If my blog were private, which it could be, then the outcomes may be different - although I don't know in what way or to what degree. I may try a few bits of reflective writing not in the public blog, and consider afterwards whether I want to post them, just to see if there's a difference. However there will be a conflict as I want to post as much on the blog as possible, and if I start feeling the need to re-write some work for public viewing then I'm going to run into serious time problems. It might be that this re-working and re-phrasing is actually a further level of reflection in itself, or perhaps it's just editing, during which the piece might lose its initial honesty.

I think I'm particularly concerned about the course's requirements to relate content to your own professional practice. Given that I'm not currently in an e-learning role (and that the course requirements stated that you didn't have to be), this is going to be difficult for me. I can speculate on how I would (hopefully will) apply what I've learnt, but it's difficult without a context to relate it to. I think this adds to the anxiety about doing the 'right' sort of reflection to pass the course, which may not necessarily be the same reflection that is natural to me. However, being prompted to think outside my comfort zone will almost certainly be valuable to my development, so it's welcome in that respect.

[Thanks are due to Hazel, Isabel and Madeleine here... this post was going to be a private one, but their comments in the tutor group forum have helped me realise I am among friends, and safe to post here (ignorring all the billions of internet users who fortunately don't read this!)]

Thursday 1 October 2009

‘e-portfolios’ by George Siemens (2004) - a summary

Core activity 2.3 was to write a summary of the issues raised in en eportfolio paper. I selected ‘e-portfolios’ (Siemens, G. 2004) Available from: http://www.elearnspace.org/Articles/e-portfolios.htm 

This short paper (only 5 pages) provides a balanced and considered overview of eportfolios, including definition, influences, benefits, comments on the creation process, issues, tools, trends and implementation needs.

Siemens makes the argument that portfolio implementations can be viewed as a continuum, that is, that they are driven by the task (either "assessment, PDP, learning portfolio, or group portfolio"). This makes sense: given the many and varied definitions and conceptions of eportfolios, the argument that "the intended task of the portfolio is the ultimate determinant of value" goes some way to explaining why eportfolios are many things to many people. Siemens also argues that, regardless of the format selected, eportfolios should encourage learners to develop the skills to continue building their portfolio as a life-long learning tool. This relates to one of the anticipated outcomes of using an eportfolio identified by this paper, namely that in a knowledge economy the ability to express knowledge effectively improves one's opportunities for employment and access to education. This does however assume that that learners will be motivated to develop a portfolio, and see value in maintaining it beyond compulsory education - something which is far from guaranteed, and which Siemens doesn't really address, except for a single sentence in his conclusion.

Siemens suggests a number of benefits that learners may receive from using an eportfolio to reflect on their experiences. Crucially, he sees eportfolios as a skill for lifelong learning. He suggests that eportfolios support this through (amongst other things) their ability to incorporate artifacts from informal learning and learning through life experiences. Siemens considers that this offers learners personal knowledge management and a history of their development and growth. Perhaps the most powerful benefits to learners identified by Siemens are the use of eportfolios as a planning/goal setting tool, as assistance in making connections between their learning experiences, and in providing the metacognitive elements necessary to assist in planning future learning needs. However, he does not articulate how an eportfolio will achieve these (not insubstantial) aims. My feeling (not yet supported by research evidence) would be that the eportfolio can be a tool in supporting these goals, but that they are processes undertaken by the learner which need support to develop, may require training, and certainly the opportunity to practice. Reflection is often not an easy or natural process, and simply having a tool available will not automatically make the benefits available to learners.

Siemens makes a number of interesting comments on implementation challenges. Unlike some others (refs?), he considers that the "standardization of eportfolios is a potential challenge" and that regulation may stifle creativity and innovation. He also considers that institutions should not be in control of the portfolio, and that learners themselves should retain control over their own portfolios. This is in contrast to arguments such as Jafari (2004) which consider institutional control over alumni eportfolios may be a useful revenue stream!

Siemens notes the need for eportfolios to be embedded into the process of instruction and assessment - that learners require time, training, advice, and promotion of eportfolios by faculty. These are all requirements of culture or attitude - it is not the technology that will make or break an eportfolio implementation. As Siemens concludes, "for many institutions, the challenge is ... to integrate various activities and extend current practices" and to recognise that "effective life-changing use is dependent on the learners themselves seeing the value and benefits".