The Caribbean Examinations Council (CXC) provides curriculum and measurement support to participating countries for reform of their education sectors and to underpin the three major certificates, for which CXC processes more than 270,000 entries each year:

Caribbean Certificate of Secondary Level Competence (CCSLC)

Caribbean Secondary Education Certificate (CSEC)

Caribbean Advanced Proficiency Examination (CAPE)

The project involves providing support over two years to CXC, in particular to the Measurement and Evaluation Division, to help with strengthening the exam system – working to ensure procedures are compatible with international standards, developing staff capacity to operate those procedures and document and publish the processes and associated guidelines. Undertaking the work will begin with an assessment of need of both the system and people. A development strategy will then be developed, which will include training programmes and documentation. The project involves a high level of collaborative working – both with the Chief Executive of CXC and with the Education Section of the Commonwealth Secretariats Social Transformations Programmes Division.

Open Source Schools has evolved from among a community of educators and technologists who want to share their knowledge, experience and insights of using open source software in education. It is both a community of open source practitioners and a website.

The website provides a rich source of information about open source software contributed by members of the community. This includes insights and experiences about the benefits and challenges of using open source set out in case studies, articles and blog posts and through discussion in the forums. There is also a directory of open source applications.

Throughout the two year project a number of face to face events were held enabling the community to grow and members to share their experiences of using open source software.

Post the election and post Becta, Open Source Schools began to make new plans to ensure that the community was able to respond to the changing educational landscape where the benefits offered by open source were becoming ever more important.

An informal steering group, formed from among core community members, agreed to work together on a voluntary basis and a number of tasks were identified. For its part, AlphaPlus offered to continue to host the website and to make some funds available to support the work of the community in these early stages. Currently the steering group is planning a conference for senior leaders in April 2011.

Please visit http://opensourceschools.org.uk/ for more information.

The goal of this Content Ecosystem Phase 1 Pilot was to develop a system (including technical infrastructure, policies and pilot work) which standardizes and simplifies the distribution and discovery of digital learning resources in the UK.





AlphaPlus worked with Schemeta to deliver the index policy, stakeholder engagement and sustainability models.

AlphaPlus was commissioned by LLUK to undertake a feasibility study to explore how personal skills in ICT might be evidenced, identify the implications and potential impact of evidencing ICT skills, and make recommendations as to what might be required in terms of resource planning, quality assurance and the timescales involved.

The need to evidence personal ICT skills is in the context of FE teachers seeking to gain professional formation and subsequently maintain their professional status through ongoing CPD. As part of gaining qualified teacher or associate teacher learning and skills status, it is a key requirement for teachers to evidence personal skills in literacy and numeracy. Lifelong Learning UK (LLUK) was originally funded by BIS to develop a guidance document to inform teachers and employers about what they should do to meet this requirement and what evidence they should provide.

This project built on the prior work by LLUK in relation to Literacy and Numeracy and involved consultation with a wide range of FE stakeholders, including ITT providers, colleges, unions and professional bodies.

Through 2010 and 2011, the Department of Business, Innovation and Skills (BIS) is conducting a major survey of around 7500 households, in order to measure the population’s skills in literacy, numeracy and ICT. This project follows on from previous pilot work, and the preceding survey in 2003 (also conducted by AlphaPlus and BMRB).

·         This project looked at the Applied GCSEs across a range of AOs that occupied the same curriculum space as Creative and Media Diplomas (art and design; theatre studies; performing arts). The result was a briefing document to inform QCDA of the extent of the duplication and the likely effect of removing or keeping each GCSE.



The project maps the content of selected applied GCSEs against the principal learning content of the corresponding Diplomas to establish:





where the qualifications have areas of overlap

what gaps in topic coverage exist between the applied GCSEs and the corresponding Diploma principal learning and the significance of such gaps

where applied GCSEs could be used as they stand to complement the Diploma provision

how easily learners studying the current suite of applied GCSE programmes could progress on to Diploma lines of learning



what, if any, changes would be required in the future to the content of the applied GCSEs (or the Diploma principal learning) to achieve full alignment.

City and Guilds operates a large number of item banks. AlphaPlus was engaged to work with City and Guilds on quality review processes for these banks and the associated management processes.

Following 18 months of Additional and Specialist Learning (ASL) review work for the QCDA Diploma team, AlphaPlus was awarded a further contract to continue this work until September 2010. The aim was to ensure that the three versions of the Diploma ASL catalogue published in 2010 were fit for purpose, contain ASL qualifications which reflect policy requirements and to ensure the best range of good quality and appropriate qualifications were included.

The work consisted of two main strands:

Full technical reviews of qualifications submitted as specialist learning





Barring reviews of qualifications submitted by AOs during the catalogue population phase.

AlphaPlus provided EAL with consultancy support on the development of Functional Skills Sample Assessment material in mathematics, ICT and English.

Becta commissioned AlphaPlus to write a white paper on e-assessment as part of its Emerging Technologies Series. The white paper is published at http://emergingtechnologies.becta.org.uk/upload-dir/downloads/page_documents/research/emerging_technologies/e-assessment_and_innovation.doc

JENNY SMITH

AlphaPlus was commissioned by LLUK to recommend an approach to identifying the underpinning language, literacy, numeracy (LLN) and ICT in the national occupational standards (NOS) for the lifelong learning sector. Following a literature review, AlphaPlus made recommendations for an approach for consultation, which was reported on in mid-March 2010.

The online consultation explored with employers in the LLUK footprint whether the ‘bottom-up’ approach, based on previous work with SSCs, was of value to them and supported their use of NOS.

AlphaPlus is engaged as a contractor on the Independent Evaluation of the Functional Skills Pilot. This pilot is a high profile, large scale and complex project of significance in the context of education strategy and for the future skills base of the working population.



AlphaPlus has been involved in the external evaluation of the pilot since March 2009, undertaking summative quantitative and qualitative analysis and evidence gathering.

Development of key publications related to Functional Skills implementation and the phase 3 Diplomas as part of developing and maintaining the Functional Skills Support Programme web site.

AlphaPlus has written Functional Skills ICT assessments for NCFE (www.ncfe.co.uk) which are designed to meet the exacting criteria for contextualised evaluation of process and outcome in Functional Skills.

AlphaPlus led a partnership that included the British Market Research Bureau (BMRB), BTL Group Ltd and the Centre for Developing and Evaluating Lifelong Learning (CDELL) in the School of Education at the University of Nottingham to consider the best research design options for a new Skills for Life survey taking place in 2010.

The main objectives of the project were to:

Consider how to improve the 2003 survey design in the 2010 research tools, whilst ensuring comparability with the 2003 survey analysis

Revise, develop and pilot research tools for use in the 2010 survey, including literacy and numeracy assessment tools, an ICT assessment tool and a contextual and attitudinal questionnaire

Produce a specification which can be used for commissioning the main-stage survey

AlphaPlus undertook the creation of learner voice material in support of the three general Diplomas. This project involved the use of innovative research techniques to elicit learner attitudes and opinions about Diploma learning styles and content.

AlphaPlus undertook field research and report writing for production of guidance material supporting the use of qualifications and courses to improve maths and English skills of 16-18 year olds.

CACHE requested support from AlphaPlus to advise on the assessment methodology most appropriate to approximately 10 new qualifications.

Becta commissioned AlphaPlus (and partners) to work with Becta, JANET, NEN (http://www.nen.gov.uk/) and other partners in the production of a National Resource Discovery and Delivery Strategy Foundation Study, for England. The study took into consideration the existing landscape covering such parameters as the role of search engines, Web 2.0 applications, the NEN, learning platforms, shibboleth, web services and repositories of content (both commercial and non-commercial).

As part of Becta’s work to improve the functioning of the UK market for Digital Learning Resources, AlphaPlus was contracted to develop technical and business case resources for the DLR market. Substantial research shows that:

DLR consumers (teachers, mainly) are ill-equipped to search for and select DLRs because there is a lack of objective quality criteria and associated purchase support information

Purchasing decisions are often based therefore on factors other than fitness for purpose (eg cost, ease of purchase)

Teachers’ anticipate that they will need to invest substantial time in choosing and then learning how best to use resources. Additionally, many do not feel they have adequate skills to do this efficiently.

The research project developed an improved approach to publishing information about quality standards.

Open Source Schools began as a two year project funded by Becta. Over that period a community of educators and technologists who wanted to share their knowledge, experience and insights of using open source software in education developed. Open Source Schools is now both a community of open source practitioners and a website.

The website provides a rich source of information about open source software contributed by members of the community. This includes insights and experiences about the benefits and challenges of using open source set out in case studies, articles and blog posts and through discussion in the forums. There is also a directory of open source applications.

Throughout the two year project a number of face to face events were held enabling the community to grow and members to share their experiences of using open source software.

Post the election and post Becta, Open Source Schools began to make new plans to ensure that the community was able to respond to the changing educational landscape where the benefits offered by open source were becoming ever more important.

An informal steering group, formed from among core community members, agreed to work together on a voluntary basis and a number of tasks were identified. For its part, AlphaPlus offered to continue to host the website and to make some funds available to support the work of the community in these early stages. Currently the steering group is planning a conference for senior leaders in April 2011.

Please visit http://opensourceschools.org.uk/ for more information.

Remove

e-Assessment is as an essential component of making learning relevant and engaging for 21st Century learners, and as a result, Becta wanted to ensure the effective use of technology in assessment and that barriers to this are understood and enablers and good practice are acknowledged and shared. They engaged AlphaPlus to report on the use of technology for high stakes testing and surveys 20 FE providers to study their technical and organisational readiness to engage in e-assessment.

AlphaPlus was commissioned by the Welsh Assembly Government, Department for Education, Lifelong Learning and Skills (DELLS) to provide support in developing the credit and qualification framework for Wales (CQFW). The two main issues associated with the further development of the Credit and Qualifications Framework for Wales were coping with the proposed rapid expansion into less well trodden educational territory and the sustainability of the framework arrangements once in place. Our work included consideration of:

• The impact and influence of policy and legislation in Wales and the UK

• The attitudes and opinions of learners, teachers, assessors and employers

• The development of new administrative systems and processes

• The use and impact of technology and standards

• Developing technology, standards and practices outside the UK

This three month project was based around an evaluation of progress one year after the main Embedding Project and the creation of 10 new Skills for Life embedding models (6 SSC, 2 AB, the TUC and an employer).

AlphaPlus supported CDELL with field research and looked at practitioner reactions to the pilot activity maths reform agenda.

The EMP project was an independent evaluation of the mathematics pathways pilots, conducted on behalf of QCA. This project evaluates the manageability and impact of a range of new 14?19 mathematics qualifications following the Smith Report.

This project was a study of the Becta Vocabulary Management Service (VMS), and the business case for a sustainable service to support Digital Learning Resource (DLR) providers. The output was a report describing the current technical and operational aspects of the VMS including creation and management of vocabularies, DLR tagging, management and discovery, as well as aspirations and development activity for the service, in addition to looking at similar services elsewhere. The work comprised technical and educational desk research, formal DLR Provider consultation, informal interviews with key stakeholders, and business case analysis.

The aim of the project was to develop “Comparative Assessment Tools”, to be used as unseen tests, so that CCEA could compare performance of a sample of pupils in the current KS3 statutory assessments in English and mathematics, with their performance on the Comparative Tools. The Comparative Tools were designed to meet the requirements of the new draft Levels of Progression for Communication and Using Mathematics Cross- Curricular Skills in the Revised Northern Ireland Curriculum.

The HE@Work consultancy works with blue chip employers to accredit their in-house training schemes to degree level, boosting the recognition of Work Based Learning and learners achievements. A key element in creating personalised learning plans for individuals with significant work based experience is the ‘Accredtation of Prior Experience and Learning’. Whilst a well recognised method in HEIs this is a manual and time consuming activity. AlphaPlus assisted them by investigating means of applying ICT productivity tools to the process for HE@Work.

The UFI Diagnostic tool is used within Learndirect centres to help learners identify their current literacy and numeracy capabilities and work with their tutors to create an effective learning plan for their progression.

An updated tool, was developed with a new algorithm to reduce the time a learner takes to complete the Diagnostic, as well as provide more innovative questions and a revised look and feel to keep the tool looking fresh and modern. Updates to the user interface improved the accessibility of the test to users with a range of visual and navigation needs.

SkillsActive (http://www.skillsactive.com) commissioned BTL Group Ltd to produce a contextualised Skills for Life initial assessment tool to meet the needs of their market. AlphaPlus has contributed to this development with editorial work and tool design. AlphaPlus has also undertaken a centre-based trial of the tool.

Authoring questions for TDA Numeracy Test.

The current suite of initial assessment tools and diagnostic tools used within teacher training are focussed on the school teacher cohort and as such the contexts lack relevance for the FE sector. The tools also do not reflect the diversity of background and career progression that FE teachers typically have. AlphaPlus worked with ACER to develop the framework for an integrated suite of IA, Diagnostic and selfhelp tools in Key skills, starting with ICT.

AlphaPlus undertook a review of high stakes assessment e-portfolios including discussions with Awarding Organisations.

Developing the design for the UFI Adaptive Diagnostic Test.

The UFI Diagnostic tool is used within Learndirect centres to help learners identify their current literacy and numeracy capabilities and work with their tutors to create an effective learning plan for their progression.

An updated tool, was developed with a new algorithm to reduce the time a learner takes to complete the Diagnostic, as well as provide more innovative questions and a revised look and feel to keep the tool looking fresh and modern. Updates to the user interface improved the accessibility of the test to users with a range of visual and navigation needs.

This project, in partnership with the Hamlet Computer Company, was a major three year contract in the Republic of Mozambique to assist the Ministry of Education (MINED) in the reform of the country's examination system.

In pursuit of its efforts towards poverty alleviation, the Government of Mozambique has put the investment in access to, and the quality of, education at the top of its development priorities. One of the areas regarded as fundamental to this is the improvement of the school assessment and examination system.



The overall objective of the project was to improve the assessment and examination system, and AlphaPlus' role was be to work with MINED to:

• improve the content validity of the examinations, including the introduction of multiple choice tests for grades 11 and 12

• train local teachers in best practice for authoring and editing items

• train local teachers in best practice for the assessment of coursework

• eliminate opportunities for fraud within the system

• accelerate the issue of results

• improve the quality and reliability of information about student achievement

• reduce costs for support activities

• support the creation of an Item Bank, for storage and retrieval of questions and the reconfiguration of the tests.

AlphaPlus supported LLUK (www.lluk.org) in developing an implementation strategy for on-screen assessment in core skills for teachers qualifying in the post-compulsory sector (similar to the TDA QTS tests)

Alpha Plus was engaged as a sub-contractors on a QCA commissioned project with the University of Warwick, Centre for Education and Industry, to define the assessment and grading criteria, and then the assessment principles of the Specialised Diplomas.

This project involved research, specification and development of an on-screen Skills for Life diagnostic tool for UFI learndirect centres.

·          The Association of Accounting Technicians (AAT) commissioned AlphaPlus to carry out an independent review of its current assessment and quality assurance procedures focusing on whether AAT’s National Vocational Qualification (NVQ) assessment and quality assurance policies and procedures were fit for purpose and comply with the requirements of the regulatory bodies. The review covered:

Assessments - including examinations, skills tests and other acceptable forms of NVQ assessments undertaken by approved centres, for example, workplace evidence

Centre approval and on-going monitoring.

AlphaPlus worked with SQA to extend the work within the Functional Skills project to apply to Core Skills in Scotland. The resulting on-screen assessments are used as part of an integrated assessment-for-learning approach.

The project involved development and trialling of test specifications and on-screen test instruments for the minimum core curriculum for Initial Teacher Training in the post-16 workforce.

AlphaPlus undertook research into the issues relating to on-screen assessment through the medium of Welsh,

investigating the implications of offering tests on-screen in Welsh for a variety of subject areas, notably Key Skills and Skills for Life, and working with awarding bodies and vendors to determine practical strategies.

AlphaPlus was commissioned by Stephen Austin and nferNelson to investigate and report on the implications of computerised testing for paper-based assessment in the education sector. We were posed a number of specific questions which, between them, suggested to us that we needed to

• conduct a review of the current relationship between what we will call e-assessment and paper-based assessment

• identify the key features of the technological scope offered by e-assessment and paper-based assessment

• look in some detail at the ways in which each of these forms of assessment may be delivered by awarding bodies

• look at the effects of current policy initiatives, reviews and developments that may affect the nature of the assessment that is being provided

• study relevant implications that changes may have on the nature of the assessment and the nature of learning

and, in relation to all of these, attempt to identify the likely changes that may occur in the foreseeable future. This report follows the pattern of these five areas.

The convergence project was a joint project between AlphaPlus Consultancy, BTL and QCA working for DfES. The project examined various aspects of the convergence of Key and Basic Skills, and Skills for Life, covering the detailed curriculum, guidance material, innovative assessments and funding issues behind the take up of the qualifications. Of particular note was the development of a range of innovative on screen assessments, going far beyond the ‘standard’ multiple choice tests adapted from paper tests previously used.

This small project preceded the larger "Convergence Project" and consisted of a survey of the Adult Literacy Standards and Adult Literacy Curriculum codes. Tables were produced to show overlaps between the curriculum and standards in order to establish where the two converged. The work was carried out for the Adult Basic Skills Strategy Unit (ABSSU) of the DfES.

e-learning project – cancelled by UFI

REMOVE

Assessment Tools

This project developed a broad suite of over 50 initial assessment and screening tools for literacy and numeracy up to level 2, in both paper and computer-based format. The computer based tools included innovative adaptive algorithms and made full use of the multimedia capabilities of computer based testing. The project included initial trials of the tools and the tools are now being handed over to the QIA for dissemination and further trialling. The project included significant publishing through subcontractors and consortium management.

Of relevance to DSA is the large quantity of trialling (with hundreds of centres and tens of thousands of learners). The numerical and qualitative data captured from these trials was analysed using a variety of techniques to validate the tools prior to publication. This included adoption of a number of complex methods to support validation of adaptive and rich media tools.

The assessment tools can be viewed and downloaded at http://www.toolslibrary.co.uk/.

Itembank

The itembank project (www.itembank.org.uk) was undertaken as part of this project and is a live ongoing service for learners developed by AlphaPlus and BTL Group ltd. The itembank contains approximately 10,000 assessment questions drawn primarily from live examinations, metatagged against level, subject and topic. The learner can then access the site and request practice assessments to meet their specific needs, for example, downloading a complete practice examination (a “mock exam”), or shorter tests dealing with specific subjects, sub-topics from the curriculum.

The tests can be provided on-screen with electronic marking, or delivered as pdf files with mark schemes for the learner to print out. The site is widely used by learners as part of their formative work both within their education institution and as private study.

We videoed interviews with learners speaking about their experiences with the National Tests

AlphaPlus played a significant part in the early development of the "Move On" project and carried out an investigation (funded by the Adult Basic Skills Strategy Unit (ABSSU) of the DfES to capture the "voices" of adults taking part in the Move On initiative. AlphaPlus interviewed adult learners and their teachers in number of centres in Somerset and produced a report for ABSSU. This work also involved developing a computer-based screening tool and editing the Move On publicity and support materials. (2002)