by UMAP is the Universities Medical Assessment Partnership and is relevant to many medical students from the UK.
It’s essentially the organisation which is writing/ collecting a bank of medical student exam questions for use in ‘high stakes’ examinations, such as medical finals.
UMAP’s description of their activities can be found on their website: On the site they describe themselves as follows:
UMAP’s aim is to improve quality in high stakes written assessments across UK medical schools. UMAP runs best practice item writing sessions where clinical and academic staff come together to learn about item writing techniques and to represent their subject area in the developing question bank.
UMAP QA process
Once written, questions move on to be quality assured at UMAP question review meetings which are convened at our partner schools. Staff members with experience in assessment and who are familiar with UMAP style and technique check each question and amend as necessary to ensure the highest accuracy and conformity to question writing principles. Questions are then ready for use and are listed as part of selection documentation available to our partner medical schools.
Schools are invited to select the items they wish to use and then confirm their selections to us. Schools later return results data in an electronic format which is then analysed and uploaded into the UMAP bank. A summary of this data is displayed within question selection documentation to enable schools to make informed, evidence based, item choices. [Accessed 4.03.09 from http://www.umap.org.uk/about/]
From their site they cover about 14 medical schools and they run an active recruitment of Specialist Registrars to write questions for their bank, most recently to our knowledge in Birmingham at a West Midlands General Internal Medicine training day.
UMAP currently publish on their site that as of October 2008 they have over 2500 questions for use in the these high stakes examinations. They acknowledge that they seem to be lacking in a few key areas, however they are currently working to address this.
What do you need to know about UMAP as a medical student? Well probably not that much. In fact all UMAP really are trying to do is generate sensible MCQ questions that are fair, and are well written. They have a reasonably complicated list of rules that a number of ME’s contributors have been talked through at a number of different times: overall though its not rocket science.
- you shouldn’t be able to answer a question just by using the investigation result or the stem on its own (e.g. a big intro and then showing an ECG with complete heart block etc.)
- The questions are aimed at core FY1 knowledge
- You should be able to guess the likely options for the answer (again straight forward)
In principle, when UMAP look at the answers for any given question, they check that these same answers seem reasonable. They also like the same level of detail for both (for example the answers should all be of similar length).
What about the stem’s themselves: again UMAP make some sensible judgements including avoidance of medical jargon, avoiding using the same words in the questions and the answers etc. They should be readable and comprehensible.
So is there any technique involved? Well yes! Look carefully at results an investigations. Its clear that UMAPs strategy will be not to spoon feed the reader. For example if an important feature is tachycardia the question may read as follows
Ther pulse rate was 124 – Rather than mentioning the tachycardia directly, or showing a picture of an ECG
The same goes for investigation reults: e.g. K+ 3.0 mmol/l (NR 3.5-5.5 mmol/l) rather than ‘hypokalaemia’
All this means is that you have to look carefully at the investigation answers, and then draw conclusions. The rest is guesswork? Not really. Simple mathematics state that you should rule out the maximum number of wrong answers, and maximise your chance of success.
Medical Educator or its wuestion writers have not contributed any questions to the UMAP quesiton bank and have no formal or informal association with the organisation. We’d be interested to hear students comments and opinions about the work done by UMAP.