The European Board of Ophthalmology Diploma (EBOD) examination consists of a written part (MCQ-paper), representing 40 percent of the total candidate score, and an oral part (Viva Voces), representing the remaining 60 percent of the total candidate score.
The MCQ-paper contains 52 questions, each with 5 independent true-false items. These questions are carefully selected by the EBO Examination Committee and represent 10 predefined topics (see ‘Structure of the Examination’ link). The MCQ-paper is available in English only.
By this careful procedure of pre-selecting and controlling the MCQ paper, EBO ensures that the EBOD examination remains a test in ophthalmology and not in language. Statistical analysis (using both parametrical, non-parametrical and item-response [Rasch and three-parameter] statistics) of the different language groups allowed EBO to have objective arguments for this statement. In fact, no statistical significant difference between language groups has been observed. The results have been presented for EBOD 2008, at the EBO-symposium “Teaching the teachers” (SFO 2009), and for EBOD 2009, at the EBO-instructional course “How to be successful at the EBO examination?” (SOE 2009).
The pass rate of the EBOD examination has proven stable over the years and when compared to the European examinations of other medical specialties, the pass rate of the EBOD exam is noticeably high (around 90 percent of candidates pass EBOD, compared to approximately 60-70 percent of candidates for other medical specialties).
One of the measures taken to maintain a stable pass rate over the years is the careful pre-selection of the MCQs. This not only applies to the pre-defined topics, but also involves a pre-assessment of the level of difficulty of the MCQ-items. By this pre-assessment, EBO aims to have comparable levels of difficulty within the EBOD examination over the years.
Another measure taken to maintain a stable pass rate of the exam is the application of a norm-referenced pass mark for the MCQ-paper. In order to pass the MCQ-paper, the candidate score should be above or equal to the average MCQ-score of all candidates minus one standard deviation. This type of pass mark is also indicated by many educationalists according to literature on the subject.
The norm-referenced pass marking system offers the advantage to candidates not to jeopardize their chance at passing the examination, in the event that it turns out to be more difficult than pre-assessed by the EBO Examination Committee.
In conclusion, pre-assessment of the EBOD exam by the EBO Examination Committee includes not only language, but also the level of difficulty of the written part of the examination. It can be concluded that by using this strategy, EBO has been able to prove that no bias whatsoever have occurred over the years due to language problems or level of difficulty.
In 2010, negative marking was introduced. The EBO Examination Committee would like to stress once more that application of negative marking for incorrect answers will not result in a decrease of candidates’ chances to be successful at EBOD. On the contrary, by reducing the number of wild guesses, candidates will be able to distinguish themselves better from borderline or minimally competent candidates.
In addition to the pre-assessment measures described above, the EBO Examination Committee relies on both primary and secondary statistical analysis of the MCQ-paper, whereas most other medical specialties only rely on primary statistical analysis for their European examinations.
The primary statistical analysis report is generated by the software tool used for the processing of the examination sheets (Speedwell eSystem, a cloud-based examination management and administration system). This software tool is also used by other European medical specialties boards.
The primary statistical analysis consists of several parameters of which item difficulty (indicated by the percentage of candidates answering correctly) and item discriminative power (not only indicated by the correlation of the item score with the total test score, but also indicated by a graph displaying the performance of the top 20 percent candidates compared to the bottom 20 percent candidates). This primary statistical analysis helps the EBO Examination Committee in identifying MCQs with an optimal balance between item difficulty and item discriminative power (so-called discriminators).
The primary statistical analysis report is generated within minutes by the Speedwell eSystem software tool and is easy to interpret. Because of the reliability of this statistical analysis method and because of the swiftness of the process, the EBO Education Committee is able to display the results of the examination and to organise with absolute confidence the Awards Ceremony the day after the EBOD examination took place. The timeframe even allows for double or triple checking, as well as manual random checking.
The secondary statistical analysis tool has been created in-house by the EBO Examination Committee and consists in an algorithm written in the statistical analysis programme R. This in-house developed software tool not only replicates the methods used by the Speedwell eSystem software (used as another check of the above-mentioned analysis), but also allows a more in-depth (non-parametric and Rasch) statistical analysis to compare the candidates of different countries, the residents versus the specialists etc.
Although available within minutes, the results of this secondary statistical analysis are discussed in detail by the EBO Examination Committee after the EBOD examination. These results do not only help us in proving the validity and stability of our examination (validation of MCQs) but also lead to continuous improvement of our examination (e.g. the introduction of negative marking, which will be beneficial for candidates). In fact, the knowledge drawn from this statistical analysis is very useful for the pre-assessment of future examinations.
In order to consolidate this knowledge, the EBO Examination Committee will use the Speedwell eSystem software, so that questions are not only stored in a secure manner, but also contain all knowledge derived from the questions (statistical analysis results). This is what is meant by validation.
It is not only the written part of the examination which is monitored and analysed with statistical tools. This is also the case for the Viva Voces. Examiners are pre-selected and clearly instructed (with e.g. examples of questions) on how they are supposed to assess. Over the past few years, despite the many different examiners and many different languages (English and native language whenever possible), it has been concluded that the results of candidates are positively correlated. Therefore, EBO is confident to state that candidates do not experience any advantage or disadvantage by being assigned to a specific jury.
The final grade awarded is based on the score for each of 5 sections (MCQ/ written section and 4 viva voce station scores):
The MCQ (written) section pass grade (defined as 6/10) is based on the average score minus one standard deviation. The candidate grade will be converted from their numerical score. This score will account for 40% of the final overall score.
The viva voce (oral) section is composed of 4 interviews. In each of these interviews the candidate must achieve a grade of 6/10 or greater to pass. These interviews (4 viva voce stations) account for 60% (15% each) of the final overall score.
A grade of 6/10 is a pass in all sections of the exam. In some cases, a score below 6 in one of these sections can be compensated for, on condition that all other sections of the exam have been passed(grade above 6) and the final overall grade is 6/10 or greater.
A formula is used to provide a final score between 0 and 1. Candidates must score 6 or greater to pass the examination.
Further detail can be found in Mathysen D, et al. Procedural aspects of the organization of the comprehensive European Board of Ophthalmology Diploma examination. J Educ Eval Health Prof 2016; 13: 27.
Update on Speedwell,CESMA & SOE 2011
ENET European Network for Education of Trainees
All you need to know about the EBO Exam – The Evaluation Process
Presentation at EBO-SFO Teaching the Teachers symposium (2008)
EBOD 2009 Examination: Statistical evaluation of the results (SOE 2009)
Setting pass marks for examinations – CESMA meeting, Brussels, November 28th, 2009
Evaluation of EBOD, Prague 2009
Statistical Evaluation of EBOD 2009
Statistical Evaluation of EBOD 2010
In 2020, the EBO developed a “hybrid” examination, whereby candidates could take the MCQ section on a laptop computer, and the viva voce part was conducted by one examiner on site, and another connected by video conferencing (Zoom). This did not change the scoring methodology or statistical analysis, as the only differences were of format:
In 2021, 2022 & 2023 the EBOD Examination will be fully online. This means a remote-proctoring service will be used (to monitor candidates as they take the exam), and novel methods to assess clinical knowledge will be employed. While answers will still be submitted to (and stored with) the Speedwell eSystem, the analysis may require adjustment. Consequently, the final outcomes will be released 2 weeks after the examination, once the Examination Committee have had time to review and discuss:
Subscribe to our updates & instant alerts | Enter email below
[subscribe2 hide='unsubscribe']
European Board of Ophthalmology
c/o FS Exam Services Ltd.
UEMS AIBSL
c/o Section of Ophthalmology
Rue de l'Industrie 24
BE – 1040 Brussels
BELGIUM
Email: ebo@ebo-online.org
