NTA paper-setting process: From question bank to sealed sets, how exam papers are prepared

The National Testing Agency (NTA) follows a multi-stage process to prepare examination papers, beginning with the creation of a large question bank and ending with the printing, packing and secure storage of multiple question-paper sets.

The process has come under scrutiny after NTA decided to re-conduct the UGC-NET examinations for English, Commerce and Sociology following complaints of “multiple errors” in the three papers held between June 22 and June 30. NTA said the “defects” in question papers were extensive and “cannot be cured merely by dropping questions after the challenge process” and that papers with such defects “do not meet the standards of fairness and error-free examination”.

The re-examinations will be held on September 9, with English from 9am to noon and Commerce from 3pm to 6pm.

Also read | NTA to re-conduct UGC-NET for 3 subjects; 2nd exam cancellation this year after NEET leak

The process begins with item writing, under which subject experts from reputed academic institutions prepare questions and answer options. Experts are not told whether the questions they prepare will eventually be used in an examination.

According to the government's affidavit filed in the Supreme Court on August 4, the question bank contains at least five times the number of questions actually required for an examination.

The questions are then taken up for moderation by another group of subject experts. Moderators select questions for multiple paper sets and check them for syllabus coverage, correctness of answers and broadly uniform difficulty levels.

The process is designed so that no single person has access to the complete question paper that will eventually be administered.

The next stage is vetting. Vetters independently solve the question sets finalised by moderators and their answers are compared with those proposed by the moderators.

Where there is a mismatch, the examination director concerned resolves the issue through a meeting between the moderators and vetters.

The government's affidavit says different stages are ideally handled by different experts, with multiple sets finalised “entirely separately” and with no overlap in either questions or the experts working on them.

After the sets are finalised, they are translated into the languages in which the examination is conducted. Subject experts translate the questions, following which other subject experts carry out back-translation into English to check accuracy and ensure that the context has not been lost.

The government said most NTA examinations are conducted in 13 languages. An “air-gapped” AI-assisted translation environment has also been created. The AI system prepares a first-cut translation, which is then checked by human translators.

The entire question bank is translated into 12 Indian languages, rather than only the questions eventually selected for the examination. Translators therefore receive a much larger pool of questions and cannot identify which ones will ultimately appear.