By Quick answer: What RAG skills do AI engineering jobs require?
- Core pattern — most enterprise AI assistants retrieve company data before answering.
- Retrieval matters — Anthropic reported failed retrievals cut 67% with contextual retrieval plus re-ranking.
- Position effects — models can miss information in the middle of long contexts.
- What's tested — chunking, embeddings, hybrid search, re-ranking, grounding, evals.
- Proof — a RAG system with retrieval and answer metrics.
Key takeaways
- Most RAG failures are retrieval failures — measure retrieval separately.
- Chunking and metadata often matter more than the embedding model.
- Hybrid search plus re-ranking is a strong default.
- Grounded prompts with citations make answers checkable.
- Freshness, permissions and ingestion quality decide production success.
Picture an AI engineering interview at an insurance company: "Our policy assistant answers 70% of questions correctly. Make it better." The first candidate suggests a bigger model and longer prompts. The second asks for the evaluation data and splits the failures in two. In 22 of the 30 wrong answers, the right passage was never retrieved. Policy tables were mangled at parsing, product codes like "HX-200" weren't matched by vector search, and old policy versions outranked current ones. They propose fixing table parsing, adding keyword search alongside vectors, a re-ranker, a "current version" metadata filter and contextual headers on each chunk. Only after that would they touch the prompt. The second candidate has built RAG systems before.
What RAG engineering involves
Retrieval-augmented generation is the pattern behind most enterprise AI assistants. When a question arrives, the system retrieves relevant information from documents, databases or search, places it in the model's context, and asks the model to answer from it, ideally with citations. The idea was formalised in a 2020 research paper.1 Building it well means mastering ingestion and parsing, chunking, embeddings, vector and keyword search, re-ranking, metadata and permissions, grounded prompting, freshness and evaluation.
Retrieval quality usually decides the outcome. Anthropic reported in 2024 that adding context to each chunk before indexing, combined with keyword search, cut failed retrievals by 49% on its benchmarks, and by 67% with re-ranking added.2 Research on long contexts found models use information at the start and end of their input better than information in the middle,3 so what you retrieve, and where you put it, matters. These skills sit at the centre of AI engineering hiring: LinkedIn reported AI engineering postings in India up 59.5% year on year in April 2026.5
- Fewer failed retrievals with contextual retrieval plus re-ranking (Anthropic, 2024)
- 67%
- Fewer with contextual embeddings plus keyword search alone
- 49%
- Rise in AI engineering postings in India (LinkedIn, Apr 2026)
- 59.5%
- Typical AI engineer pay in India (AmbitionBox, Oct 2026)
- ₹15–16.6 lakh
How RAG skills are tested
| Round | Example | What they check |
|---|---|---|
| System design | "Design a policy Q&A assistant" | Ingestion, retrieval, grounding, evals |
| Debugging | "Answers are wrong — why?" | Separating retrieval from generation |
| Coding | Build a small hybrid retriever | Practical fluency |
| Evaluation | "How do you measure retrieval?" | Recall@k, groundedness |
| Production | "Documents change daily; permissions vary" | Freshness and access control |
Table: How RAG skills are tested (editorial summary)
The skills that matter
Ingestion and parsing
Clean text from PDFs, tables, slides and web pages — errors here poison everything.
Chunking and embeddings
Chunk size and boundaries that keep meaning intact; contextual headers; embedding choice.
Vector and hybrid search
Semantic search plus keyword search for exact terms, codes and names.
Re-ranking and filters
Cross-encoder re-ranking; metadata filters for version, date, product and permissions.
Grounded prompts and citations
Answer only from retrieved text, cite sources, say "not found" when unsupported.
Advanced patterns
Query rewriting, multi-step and agentic retrieval, and GraphRAG for cross-document questions.4
Worked example: a RAG evaluation report
| Metric | Baseline | After fixes |
|---|---|---|
| Retrieval recall@5 (200 labelled questions) | 71% | 92% |
| Answer correctness | 70% | 89% |
| Groundedness (claims supported by retrieved text) | 84% | 97% |
| Citation accuracy | 78% | 95% |
| "Not found" when answer absent (40 questions) | 45% | 90% |
| p95 latency | 3.1 s | 3.6 s |
Table: Example RAG evaluation before and after retrieval fixes (illustrative)
The report separates retrieval from generation, tracks whether the system admits when the answer isn't there, and shows the latency cost of the fixes. Interviewers want to see that kind of reasoning.
How RAG has developed
| Period | What happened | What employers wanted |
|---|---|---|
| 2020 | RAG paper published | Research interest |
| 2023 | Vector databases boom; "chat with your docs" | Basic RAG prototypes |
| 2024 | Hybrid search, re-ranking, contextual retrieval, GraphRAG | Higher retrieval quality |
| 2025 | Agentic retrieval and longer contexts | Multi-step retrieval and evals |
| 2026–2027 | RAG as enterprise default | Freshness, permissions, measured quality |
Table: How RAG has developed (editorial summary)
Proof checklist
- A RAG system over real, messy documents (PDFs, tables)
- A labelled question set with retrieval and answer metrics
- Hybrid search and re-ranking with measured gains
- Metadata filters for versions and permissions
- Grounded prompts with citations and "not found" behaviour
- A verified RAG Fundamentals test result
Your résumé
Numbers and diagnosis: "Raised policy assistant correctness from 70% to 89% by fixing table parsing, adding BM25 hybrid search, re-ranking and version filters; retrieval recall@5 from 71% to 92% on 200 labelled questions."
Interview questions, with how to answer them
"How do you debug a bad RAG answer?"
Check whether the right passage was retrieved. If not, fix ingestion, chunking or search; if yes, fix the prompt or model.
"How big should chunks be?"
Big enough to keep meaning, small enough to be specific — test sizes on your own questions, and add document context to each chunk.
"Why hybrid search?"
Vectors find meaning; keywords find exact matches like IDs, names and error codes. Together they miss less.
"How do you handle permissions?"
Filter retrieval by the user's access rights before anything reaches the model — never rely on the prompt.
"How do you keep the index fresh?"
Incremental ingestion on change events, version metadata, and removing superseded documents.
Could you pass the Retrieval-Augmented Generation (RAG) Fundamentals Certification today?
Hiring managers test judgement with scenarios like this one. Answer it before you open the explanation — the test has 20 of them, 20 minutes, 70% to pass.
A RAG bot gives outdated answers after policy updates. What's most likely?
- The index wasn't refreshed with the new documents
- The model forgot
- Temperature is too high
- Users ask wrongly
Show the answer
A. The index wasn't refreshed with the new documents — RAG is only as current as its index.
- 01Retrieval
- 02Generation and evaluation
- 03Indexing and data pipelines
- 04Advanced RAG patterns
Free, no sign-up. Pass and you become a Retrieval-Augmented Generation (RAG) Fundamentals, with a credential ID employers can verify. Every answer is explained either way.
Where RAG skills pay in India
| Role (India) | Typical pay | RAG focus |
|---|---|---|
| Generative AI engineer | ₹9.2–10.1 lakh | Building RAG features |
| NLP engineer | ₹10.6–12.6 lakh | Retrieval and language models |
| AI engineer | ₹15–16.6 lakh | Production RAG systems |
| AI/ML engineer, 6–9 years | ₹26.6–29.4 lakh | Search and retrieval architecture |
Table: RAG-related roles and typical pay in India (AmbitionBox, October 2026; indicative)6
Career path
- Generative AI engineer — builds RAG features.
- AI / search engineer — owns retrieval quality.
- Senior AI engineer — architecture, evals and freshness at scale.
- Search or knowledge platform lead — shared retrieval infrastructure.
- Solutions architect — designs RAG for clients.
A five-week plan
- Week 1: Ingest — Parse messy PDFs and tables; inspect the output.
- Week 2: Retrieve — Chunk, embed, add keyword search and re-ranking.
- Week 3: Ground — Prompts with citations and "not found" handling.
- Week 4: Evaluate — Labelled questions; retrieval and answer metrics.
- Week 5: Prove it — Publish results and take the RAG Fundamentals Certification.
Related: Advanced Prompt Engineering and LLM Evaluation and Benchmarking. Guides: LLM evaluation, AI agent design and what is RAG?.
How RAG work shows up day to day
- Ingestion fixes — new document types, broken tables.
- Failure analysis — retrieval or generation?
- Index updates — freshness and versioning.
- Eval runs — after every pipeline change.
- Permission reviews — new sources, new access rules.
- Latency and cost tuning — re-ranker size, top-k, caching.
More questions interviewers ask
"Long context or RAG?"
Long context helps for a few documents; RAG scales to large, changing collections, costs less per query and gives citations.
"When would you use GraphRAG?"
For questions that connect entities across many documents, where chunk retrieval misses the relationships.
Mistakes that cost candidates
- Blaming the model for retrieval failures.
- Ignoring parsing quality.
- Vectors only — missing exact matches.
- Permissions enforced in prompts.
- No labelled questions.
Your first 90 days as a RAG or AI engineer
| Period | Focus | What good looks like |
|---|---|---|
| Days 1–30 | Inventory sources, parsers and the current index | You know which documents are missing, stale or badly parsed |
| Days 31–60 | Build a labelled question set; measure retrieval and answers separately | A baseline everyone agrees on |
| Days 61–90 | Fix the biggest retrieval failure class | Recall and answer correctness improve on the same eval |
Table: A first-90-days plan (editorial guidance)
The fastest way to earn credibility on a RAG team is to replace opinions with measurements. Most teams argue about models and prompts; few have a labelled set of real questions with the documents that answer them. Build one with the people who know the content — support agents, lawyers, product experts — and you will immediately see whether failures come from parsing, chunking, search, permissions or generation. Pick the largest category and fix it end to end, then rerun the evaluation. Document the before-and-after numbers and the latency or cost trade-offs. That cycle of measure, fix and remeasure is the core of the job, and showing you can run it is what moves you from implementing features to owning retrieval quality.
How we know
The RAG pattern comes from Lewis et al.'s 2020 paper; retrieval improvements from Anthropic's 2024 contextual retrieval post; long-context behaviour from Liu et al.'s 2023 "Lost in the Middle" paper; GraphRAG from Microsoft Research's 2024 post; hiring growth from LinkedIn's April 2026 India update via IANS; and India pay from AmbitionBox (read October 2026). The evaluation report is illustrative; formats and history are editorial. Science of Apps publishes the free certification linked here. Last reviewed October 2026.
Key terms, defined
- Chunking — splitting documents into passages for indexing.
- Embedding — a numeric representation of text used for semantic search.
- Hybrid search — combining vector and keyword search.
- Re-ranking — rescoring retrieved passages with a more accurate model.
- Recall@k — share of questions where a relevant passage appears in the top k results.
Frequently asked questions
What is retrieval-augmented generation (RAG)?
RAG is a pattern where a system retrieves relevant information — from documents, databases or search — and gives it to a language model to answer from, usually with citations. It lets AI answer questions about private or changing data without retraining the model.
What skills does a RAG engineer need?
Document parsing and chunking, embeddings, vector and keyword search, hybrid retrieval and re-ranking, metadata and permission filters, prompt design for grounding and citations, index freshness, and evaluation of both retrieval and answers.
How do you evaluate a RAG system?
Evaluate retrieval with metrics such as recall@k on questions with known relevant documents, then evaluate answers for correctness, groundedness (supported by retrieved text) and citation accuracy, using human labels and calibrated model graders.
RAG or fine-tuning?
Use RAG when answers depend on specific, private or frequently changing facts and need citations. Fine-tuning suits teaching a model a style, format or behaviour. Many systems combine both, but RAG is usually the first step.
What is hybrid search?
Hybrid search combines semantic vector search with keyword search such as BM25, so the system finds both conceptually similar passages and exact matches like product codes, names and error messages.
What is GraphRAG?
GraphRAG builds a knowledge graph from documents — entities and relationships — and uses it during retrieval, which can help with questions that need connecting information across many documents.
Sources
- Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020)
- Anthropic — Introducing Contextual Retrieval (Sep 2024)
- Liu et al. — Lost in the Middle: How Language Models Use Long Contexts (2023)
- Microsoft Research — GraphRAG: Unlocking LLM discovery on narrative private data (2024)
- IANS — AI engineering hiring in India jumps 59.5% (Apr 2026)
- AmbitionBox — AI Engineer salaries in India (read October 2026)
- Checked October 28, 2026. Figures that change — fees, salaries, prices — are reviewed every six months; if something is out of date, tell us.
Certifications to take next
- RAG Fundamentals Certification — the free 20-minute test on retrieval, generation, indexing and advanced patterns
- What is RAG? — the concept in plain English
- Advanced prompt engineering skills — context engineering for RAG
- Free skill test · 20 minLLM Evaluation and Benchmarking CertificationTests how to evaluate large language models — choosing benchmarks, building task-specific evals, human and LLM-as-judge grading, contamination, statistical significance and safety evals.
- Free skill test · 20 minAI Agent Design and Orchestration CertificationTests how to design AI agents that plan, use tools and work with other agents — choosing between workflows and agents, tool design, the Model Context Protocol, memory, guardrails and human oversight.
- Free certification · AI & Generative AIPrompt Engineering CertificationCertifies that you can get reliable results from large language models — prompt fundamentals, structure and reasoning, prompting for real product work, and evaluation, safety and reliability.
- Free skill test · 10 minGenerative AI Fundamentals CertificationTests the core concepts behind generative AI — models, tokens, context windows, prompts, multimodal inputs, grounding and common limitations — in plain, job-ready terms.
- Free skill test · 15 minAI Image Generation and Art Direction CertificationTests how to direct AI image generation for real work — prompting, composition, style consistency, editing, brand fit, rights and content provenance.
- Free skill test · 20 minAI Agent Evaluation and Reliability CertificationTests how to measure and improve AI agents in production — task success, trajectory evaluation, tool-call accuracy, regression testing, observability and safe failure.
Career guides for related roles
- Career guidesGenerative AI skills employers want in 2027: tokens, context, grounding and the plain-language fundamentals every role now needsGenerative AI skills employers want in 2027: tokens, context windows, grounding, temperature and limits explained, with interview answers and a plan.
- Career guidesHow to get a prompt engineering job in 2027: what the title really pays, the skills employers test and how to prove themHow to get a prompt engineering job in 2027: what the role pays in India, the skills employers test, a portfolio of evals and a four-week plan.
- Career guidesAI hallucination detection skills employers want in 2027: spotting invented facts, citations, quotes, numbers and code — and preventing themAI hallucination detection skills employers want in 2027: spotting fake citations, numbers, quotes and code APIs, and the workflows that prevent them.
- Career guidesHow to get an AI security and privacy job in 2027: data minimisation, vendor terms, prompt injection, leakage, access control and privacy engineeringHow to get an AI security or privacy job in 2027: data minimisation, vendor terms, prompt injection, leakage, access control, DPIAs, pay and a plan.
- Career guidesAI agent evaluation skills employers want in 2027: task success, trajectories, tool-call accuracy, regression testing and safe failureAI agent evaluation skills employers want in 2027: task success, trajectory and tool-call checks, LLM-as-judge, regression tests and red-teaming.
- Career guidesAI skills for data analysts in 2027: SQL and Python with AI, cleaning, exploration, charts and explanation — validated and privacy-safeAI skills data analysts need in 2027: SQL and Python with AI, data cleaning, exploration and charts, validating results, privacy and ML basics.
Get the ideas, not the noise.
Sign up with Google and we’ll send you short UX psychology tips by email — why people tap, stall and leave, and what to do about it. Unsubscribe anytime. Prefer something to keep? Get the free Laws of UX guide as a PDF.
You've read the playbook. Now prove it.
Take the free Retrieval-Augmented Generation (RAG) Fundamentals Certification and earn Retrieval-Augmented Generation (RAG) Fundamentals (SOA-RAG). Free, test-based certifications graded on our server. Pass and you get a founder-signed certificate with a credential ID and QR code employers can verify.