Česká cesta

What you gain

Measuring efficiency of education is quite time consuming and demanding in terms of organisation; these requirements increase with increased depth of the measurements. However, as a reward you will gain:

• quantifiable data concerning success rate of the realised training, its efficiency and correct choice of the training supplier (i.e. verification of your investment)
• exact overview of what is happening with your employees during the process of education
• data for planning further education of your employees (precondition for further savings based on correctly selected focus of trainings)
• identification of possible bottlenecks in the process of development of employees and impulses for further improvement

Linchpins

• Jan Marek
• Ing. Boris Janča

Method

The results are paramount, i.e. evaluation what changes the programme caused. In this area we follow the approach of Donald L. Kirkpatrick 

What do the individual levels mean and what can we offer to you?

Reaction of participants

We measure immediate reaction of participants to the realised programme, we focus on their satisfaction, impression and mood, personal perception of the benefit of the programme in the given moment.

Even though a positive reaction does not automatically mean good results in the other levels of the measurement, it is an important parameter of the participants’ perception and acceptance of the education.

What we offer:

• Questionnaires your own or created by us and adapted to the needs of the specific group and your needs; the advantages of questionnaires are the broad scope (from technicalities of the course to benefits of the programme) and at the same time easy evaluation and presentation of acquired data. Time consuming nature of this method is the disadvantage.
• Various forms of scales (graphic, spatial…) which are often used by our trainers as a tool for quick evaluation of a specific situation during the programme; the advantage is speed and original form of this method, the disadvantage is the need to record the scale and process the results later.
• Separate evaluation activity placed in the scenario of the training; such an exercise aims at processing and expressing participants’ reaction to the course, for example in the form of artistic activity and visual associations; often these activities also include reflection and sharing among participants; advantage of this method is the non-invasiveness which can bring a completely new perspective, the disadvantage is the time-consuming nature of the activity and the need of subsequent processing of the results.

Learning

In this level we evaluate the scope of change in the knowledge and/or skills of participants of a development programme. The evaluation can happen outside of the training, participants are informed about the results either personally, or in an electronic form.

What we offer:

• Knowledge tests, constructed specifically for individual training topics. The advantage is simplicity and speed. It is necessary to carry out at least two tests for an objective result – one before the training, one after the training. We also recommend carrying out a third test after several months after the training – this determines how effective was the training in terms of long-term memory.
• Evaluation of the participants by the trainer (possibly with involvement of the whole participant group) on the basis of their behaviour in model situations during the course. The evaluation is made easier and more transparent by using a “checklist” (a guide through evaluation of individual parameters of the performance). The disadvantage of this method is the risk of subjectivity.
• Self evaluation (self reflection) - participant evaluates himself/herself in given parameters. The outcomes of such evaluation serve for comparing their self perception with observations of qualified evaluators (trainers). Such comparison can give interesting information about the ability of the participant to develop because one of the basic precondition of self development is the ability to see one’s own strong and weak sides, potential and limitations realistically.

Behaviour

On this level we observe long-term effects of the educational programme on the activities of the participant in their everyday workplace activities and practice of the organisation. The expected situation is that their behaviour has moved nearer to the required state or is identical with the required state. This evaluation can be carried out after several weeks or months after the end of the training.

What we offer:

• Structured evaluation of activities of the employee (participant of the training) in real workplace situations, in everyday operation. It is necessary that parameters of performance and evaluation are clearly stated, structure allows transparency and objectivity. Participation of management is necessary, the trainers can help mainly in the preparation phase when setting the parameters and structure, as evaluators when observing the employee during their functioning and in the final phase during objective and sensitive interpretation of acquired data.
• Self evaluation – self reflection is a valuable tool for development not only during training but mainly in real workplace situations. In combination with evaluation of the superiors it allows to set realistic and specific development goals. The role of the trainer consists mainly in interpretation of the results and formulation of recommendations towards the employees as well as management.
• Evaluation interview - conducting an evaluation interview is usually a role of the manager. The trainer can help by training leading of such an interview; during the interview the trainer can offer a sensitive support for the employee to accept the development impulses.
• 360° feedback ((or 180° feedback) including view on the employee from all sides: from superiors, subordinates, colleagues as well as clients and suppliers (external and/or internal) and self reflection of the employee himself/herself; the advantage of this method is the complexity, the disadvantage are time consuming and organisational demands.
• Mystery techniques (mystery shopping, mystery calling, etc.) allow to test participants in everyday operation of the company, for example with the use of false client (suitable for dealers, employees of call centres, project managers, etc.)
• Development centres (DC) are an independent part of our offer. You will find more detailed info here (odkaz na DC/AC). DC allow to assess the competences of individuals and their development in a complex way. Measuring the entrance level of participants’ skills is an essential part of this method.
• Action plans and real tasks for the period after the training (or between trainings) are formulated at the end of the programme. Success rate and fulfilment are evaluated after specified period of time.

Results

By this level we mean performance, or more precisely its change, which was observed with an employee after absolving a development programme. It can be parameters such as increased production, improved quality, reduction of accidents and their severity, cost reduction, increased sales, etc. They express the return of investment into the given programme.

At this level, it is of course quite difficult to distinguish the influence of the completed training from other influences which affect performance of an employee (market, organisational, political and other changes, family situation and ensuing mental state of the worker, etc.) and may distort the results of the measurement.

What we offer:

• We dot not offer services in the area of financial analysis and evaluation of economic indicators. We can help you mainly in the phase of setting the qualitative evaluation criteria.

It is always necessary to consider which way of evaluation should be used and what should be included. Practice as well as research show that there is no clear and simple causal relation between the results on individual levels. At the same time, each of the levels can have different importance for you. Correct selection of evaluative level must therefore take into consideration specifics of the given situation and be adapted to your needs.

When results are not everything

(CIPP Approach – complex evaluation of training programmes in the phase of preparation and realisation)

Each training aims at good results. However getting them and keeping them in the long term rests also on other factors than efficiency of the programme itself. Therefore we also offer CIPP approach mainly for long-term and repeated projects. This approach was elaborated by Daniel L. Stufflebeam (first in 1971) and from the beginning it was developed to support decisions of the management. This approach evaluates broader context of development projects; evaluations happen not only at the end or after the completion but continuously during the whole preparation and realisation of the project. CIPP approach allows to point out improperly set goals or atmosphere which does not allow to carry through changes efficiently.

CIPP approach consists in four elements:

• Context
• Input
• Process
• Product

What do the individual elements mean?

Context
The goal of context evaluation is an efficient plan of education: it specifies needs which should be fulfilled by the training, helps with defining the goals of the programme, takes interest in already existing programmes and possibilities of their use for satisfying given needs, etc.

Inputs
Evaluation of inputs serves to decide about structure of the training project; it specifies accessible resources, shows possible approaches, finds variants with the highest potential of reaching the specified needs.

Process
Evaluation of the process focuses at implementation of the programme. Possible bottlenecks are identified, conformity with plan is checked and required changes are suggested.

Product
The answer to question whether everything that we have organised together, really functions and brings results. We evaluate what results have been reached, how were the needs satisfied, what needs to be done after the completion of the programme and what should follow, etc. One of the levels of so called Kirkpatrick’s method can be used for evaluation (as described above).

What we offer:

• Depth input analysis with project report – identification or analysis of goals, analysis of input conditions of participants (e.g. AC/DC, psycho diagnostic tests, interviews, etc.)
• Final report from the project (or partial concurrent reports) evaluating individual components, their development and possible changes in the project.

By means of CIPP approach the whole education process is monitored from the initial planning to the completion which allows for flexible reactions and adjustments in any part.