Assessment design:
writing fair, valid tests

Assessment design is building tests, tasks and evaluations that measure learning or ability accurately and fairly. It includes defining what to measure; aligning items to objectives; choosing formats — multiple choice, scenarios, practical tasks, portfolios, interviews; writing clear items and instructions; creating rubrics; setting pass marks; piloting; analysing item statistics; checking for bias and accessibility; securing tests; and reporting results meaningfully. Assessment designers work in education, edtech, corporate learning, certification bodies, recruitment and psychometrics.

Good assessment matters in many settings. In hiring, Sackett and colleagues' 2022 re-analysis ranked structured interviews — a form of structured, job-related assessment — as the top predictor of job performance.¹ India's National Education Policy 2020 calls for a shift towards competency-based assessment that tests higher-order skills rather than rote memorisation.² AI is reshaping assessment, with 6 in 10 US teachers using AI tools in 2024–25 according to Gallup,³ which raises questions about both generating questions and preventing misuse. Wording matters enormously — Pew Research Center's guidance shows how small changes in question wording change answers.⁴ Assessment work sits alongside instructional design, listed by AmbitionBox at ₹8.2–9.1 lakh in India.⁵

Structured interviews' rank as a predictor of performance (Sackett et al., 2022)

#1

India's NEP calls for competency-based assessment

2020

US teachers using AI tools in 2024–25 (Gallup)

6 in 10

Typical instructional designer pay in India (AmbitionBox)

₹8.2–9.1 lakh

Table: How assessment designers are assessed (editorial summary)

Table: Rewriting a recall question as a scenario (illustrative)

Table: How assessment practice developed (editorial summary)

• An item bank of ten aligned questions
• A rewritten recall question as a scenario
• A rubric for a practical task
• An item analysis of sample data
• A bias and accessibility review
• A verified assessment design certificate

Show measurement impact: "Rebuilt sales certification around scenarios; scores now predict first-quarter performance; created item bank of 200 reviewed questions." Real outcomes only.

"How do you make sure a test measures the right thing?"

Map every item to an objective, review with experts and check results against other evidence.

"What makes a good distractor?"

It's plausible, based on a common mistake, similar in length and style to the correct answer.

"How do you build a fair rubric?"

Clear criteria, distinct levels with examples, and calibration among markers.

"What does a question that everyone gets right tell you?"

It may be too easy or have a giveaway; review its purpose.

"How do you handle AI misuse in assessments?"

Design tasks requiring personal application, use supervised or practical components and clear rules.

• "What decisions are made from these assessments?" — stakes.
• "Is there an item bank?" — maturity.
• "How are items reviewed?" — quality.
• "Is item data available?" — analysis.

Table: Assessment-related roles and typical pay in India (AmbitionBox, October 2026; indicative)

Table: Common assessment tools (editorial guidance)

• Teacher / trainer — classroom and training assessment.
• Item writer — writing questions.
• Assessment designer — test design.
• Psychometrician / assessment lead — measurement quality.
• Head of assessment — strategy.

1. Week 1: Align — Write objectives and ten aligned questions.
2. Week 2: Review — Peer-review items and build a rubric.
3. Week 3: Prove it — Analyse sample results; take the Assessment Design Certification.

Related: Instructional Design, Learning Analytics and Skills-Based Hiring. Guides: instructional design, learning analytics and skills-based hiring.

• Writing items — aligned to objectives.
• Reviews — expert and peer.
• Rubrics — marking guides.
• Data — item performance.
• Updates — retiring weak items.
• Security — protecting tests.

"What's item difficulty?"

The share of candidates who answer an item correctly.

"What's discrimination?"

How well an item separates stronger and weaker candidates.

Table: A first-90-days plan (editorial guidance)

Learn the objectives and review existing items for quality. Write and peer-review new items. By the third month, analyse results and fix or retire weak items.

AI can draft questions, generate variants and mark short answers, which speeds up item creation. It can also produce flawed or biased items and make cheating easier on unsupervised tests. Assessment designers now review AI-drafted items carefully, design tasks that require personal reasoning or live demonstration, and set clear rules for AI use.

List the mistakes learners actually make. Turn each into an option. Keep options similar in length and grammar. Avoid "all of the above" and "none of the above". Check that only one answer is defensibly correct.

Give markers sample responses at each level. Ask them to score independently, then compare and discuss differences. Agree interpretations of the rubric. Repeat periodically to keep marking consistent.

Review items for unfamiliar cultural references, unnecessarily complex language and stereotypes. Compare item performance across groups where data allows. Provide accessibility adjustments. Fair tests measure the skill, not background.

Decide what a minimally competent person should be able to do, then judge how such a person would perform on each item — a common approach is to ask experts to estimate this item by item and combine the estimates. Review the proposed pass mark against real results and the consequences of passing or failing. Document the method so the decision can be explained and repeated.

"Harder tests are better tests." Valid tests are better tests.

"Multiple choice only tests recall." Scenario items test judgement.

"High scores mean success." They might mean an easy test.

Some skills can only be judged by watching people do them — handling a customer call, building a spreadsheet model, writing code or performing a procedure. Start by defining the task as close to real work as possible and the conditions: time, tools and materials allowed. Write a rubric with clear criteria and levels before anyone attempts it, with examples of what each level looks like.

Train assessors and have them practise scoring sample performances together until their scores agree. Pilot the task with a few people to check timing and clarity. Where stakes are high, use two assessors or record performances for review. Practical assessments are harder to run than multiple-choice tests, but they're far harder to game with AI and tell you much more about real ability.

• Items not aligned to objectives.
• Trick questions.
• Giveaway answers.
• No rubric for open tasks.
• Ignoring item data.

Selection validity comes from Sackett et al. (2022); policy context from India's National Education Policy 2020; teacher AI use from Gallup (June 2025); question wording from Pew Research Center; and India pay from AmbitionBox (read October 2026). The sales test scene, question example and history are editorial. Science of Apps publishes the free test linked here. Last reviewed October 2026.

• Validity — measuring what's intended.
• Reliability — consistent results.
• Distractor — a wrong answer option.
• Rubric — a scoring guide.
• Item analysis — reviewing question performance.