What Does an AI Engineer Do? Role, Skills and Daily Work

What does an AI engineer do? An AI engineer is a software engineer who turns foundation models, such as large language models (LLMs), into products people use. Day to day, that means choosing models, connecting them to company data and tools, testing output quality with evaluations, and running the result as reliable production software. Most AI engineers build on existing models rather than training new ones.

Want the step-by-step path into the role? Read our roadmap on how to become an AI engineer.

Updated November 2026. Job-market figures below are dated and linked to their sources.

What is an AI engineer? Three definitions#

Practitioners, authors and employers all describe the same job: building products on top of foundation models. The ai.engineer field guide (updated August 2026, written by Shawn "swyx" Wang, whose 2023 essay popularized the title) defines an AI engineer as "a software engineer who designs, builds, evaluates, and operates products powered by foundation models."

Chip Huyen's O'Reilly book AI Engineering draws the same line. Its companion repository says AI engineering "focuses on building applications on top of foundation models," while classic machine learning engineering builds on traditional models trained on tabular data.

Microsoft's current certification for the role describes it in employer terms. The Azure AI Apps and Agents Developer Associate page (the AI-103 exam, which replaced the retired AI-102) describes the candidate as "an Azure AI engineer who builds, manages, and deploys agents and AI solutions."

What does an AI engineer do day to day?#

Most of the work falls into six recurring activities. The ai.engineer guide names them: choose and test models, design the AI system, build evaluations, ship production software, operate for reliability, and close the feedback loop.

Choose and test models#

Before writing much code, an AI engineer compares hosted and open models on quality, latency, cost and privacy for a specific task. A support chatbot, a document extractor and a coding assistant can each need a different model. Reruns are common: when a provider ships a new model, the engineer tests whether switching improves results enough to justify the change.

Design the AI system#

A model alone rarely solves a business problem. The engineer combines prompts, retrieval, tools and application code into one system. Two patterns come up constantly:

• Retrieval-augmented generation (RAG). Microsoft's documentation defines RAG as "a pattern that extends LLM capabilities by grounding responses in your proprietary content". In practice, the engineer chunks documents, indexes them for search, and passes only the most relevant passages to the model.
• Tool use and agents. Anthropic's engineering guide Building effective agents separates "workflows," where LLMs and tools follow predefined code paths, from "agents," where the model directs its own tool use. Its advice is to find "the simplest solution possible, and only increasing complexity when needed."

Build evaluations#

Evaluations ("evals") are automated tests for model output. Because LLM answers vary, an AI engineer writes test sets of realistic inputs and grades the outputs, using code checks, human review or a second model as a grader. Anthropic's guide to developing tests walks through defining success criteria first and then building test cases around them. Evals decide whether a prompt change, a new model or a new retrieval setup actually ships.

Ship production software#

AI features live inside normal products, so this part looks like regular software engineering:

• Building APIs and user interfaces around the model.
• Handling authentication, rate limits and errors.
• Requesting structured output (for example JSON that matches a schema) so the rest of the app can use the response.
• Writing logs, tests and deployment pipelines.

Operate for reliability#

Once a feature is live, the engineer watches it. Models, data and user behavior all change, so quality can drift without any code change. Monitoring covers answer quality, safety, latency and cost per request. Cost control matters more here than in most software, because every model call is a metered expense.

Close the feedback loop#

Finally, the engineer reads real traces and failures, decides whether the fix belongs in code, prompts, data or the model choice, and adds the failure to the eval set so it can't quietly come back.

A typical AI engineering task, start to finish#

The example below is illustrative, not drawn from a survey. It shows how one common request moves through all six activities.

Stage	What the AI engineer does	Output
Request	Product asks for a bot that answers questions from the company help center	A written goal and success criteria
Model choice	Tries two or three models on sample questions	A chosen model and a cost estimate
System design	Builds a RAG pipeline over help-center articles	Search index and prompt template
Evals	Writes questions with known answers and grades results	A pass/fail report per change
Ship	Wraps it in an API and chat widget with logging	A deployed feature behind a flag
Operate	Reviews bad answers weekly and adds them to the evals	Fewer repeat failures over time

Do AI engineers train models?#

Usually not. The ai.engineer guide puts it plainly: "Many AI engineers begin with hosted or open foundation models and focus on the system around them." It adds that they may fine-tune, distill or host models when product requirements justify it, while research and ML teams often own more of the training loop.

This is the main difference from a machine learning engineer, who spends more time on data, training and model performance. The same guide calls the distinction "about emphasis, not rank" and notes that the roles overlap substantially. For a side-by-side look at skills, certifications and pay, read our AI engineer vs. ML engineer vs. data scientist comparison.

What skills and tools does an AI engineer use?#

LinkedIn profile data points to LLM application frameworks, retrieval and Python. LinkedIn's Jobs on the Rise 2026 list for the US, published January 2026, ranks AI engineer first among the fastest-growing US roles (based on growth from January 2023 to July 2025) and names LangChain, Retrieval-Augmented Generation (RAG) and PyTorch as its most common skills.

Area	Typical skills and tools
Programming	Python for model, data and eval work; TypeScript for product interfaces
Model access	Hosted model APIs and open-weight models
Retrieval	Embeddings, vector or hybrid search, document chunking
Orchestration	Frameworks such as LangChain, plus tool calling and agent workflows
Quality	Eval datasets, graders, tracing and observability
Production	APIs, cloud deployment, logging, security and cost tracking
Model internals (sometimes)	PyTorch and fine-tuning, when a product needs a custom model

On languages, the ai.engineer guide notes that "Python is common across model, data, and evaluation tooling" while "TypeScript is common for product interfaces and full-stack AI applications," and many teams use both.

To see which of these skills to learn first, and how to tell when you've learned each one, use our AI engineer skills checklist.

Is an AI engineer a coding job?#

Yes. AI engineering is a branch of software engineering, and the work is mainly code, tests and systems rather than research. Microsoft's AI-103 certification page says candidates "should have experience developing apps by using Python." The supporting work around the model (APIs, data pipelines, security, testing) matches the tasks O*NET lists for software developers, such as analyzing user needs and developing testing and validation procedures.

Who does an AI engineer work with?#

AI engineers sit between product teams and infrastructure. Microsoft's role description says the engineer collaborates "with business stakeholders, solution architects, data scientists, DevOps engineers, and cloud security engineers to design, implement, and maintain AI solutions." That usually means:

• Product managers and domain experts define what "a good answer" means.
• Data and platform teams provide clean data and infrastructure.
• Security reviews how the system handles sensitive data and tool access.

Where do AI engineers work, and what do employers expect?#

AI engineer roles cluster in tech and consulting, and most hires arrive with several years of experience. For the AI engineer entry, LinkedIn's 2026 Jobs on the Rise list reports:

• Most common industries: Technology and Internet, IT Services and IT Consulting, and Business Consulting and Services.
• Work arrangement: 26.2% remote and 27.1% hybrid (LinkedIn).
• Experience: a median of 3.7 years of prior experience (LinkedIn).

AI work is also spreading well beyond engineering titles. Indeed's Hiring Lab counted 822 distinct "AI-touched" job titles in the US in the first quarter of 2026, about 1 in 12 of all titles on the platform. In other words, "AI work" now appears in many roles that aren't titled AI engineer.

How much do AI engineers earn?#

There is no official US pay figure for the title, because the Bureau of Labor Statistics does not track "AI engineer" as a separate occupation in its Occupational Outlook Handbook. The closest BLS categories give a reasonable range:

BLS occupation	Median annual wage (May 2025)	Projected growth, 2025–2035
Software developers	$135,980	10% (with QA analysts and testers)
Computer and information research scientists	$140,300	22%

Research scientist roles typically require a master's degree, according to BLS. Application-focused AI engineering roles map more closely to software development. For job-outlook context, see the outlook and salary section of our AI engineer career guide.

Frequently asked questions#

Is prompt engineering the same as AI engineering?#

No. Prompt engineering is one technique within AI engineering. An AI engineer also builds retrieval, tool integrations, evaluations, monitoring and the surrounding product code.

Do you need a computer science degree to be an AI engineer?#

Not always. Research roles often expect an advanced degree; BLS lists a master's degree as the typical entry level for research scientists. Application-focused roles are closer to software development, where shipped projects and engineering skills carry more weight.

Which programming language should an AI engineer learn first?#

Python. It is the most common language across model, data and evaluation tooling. Add TypeScript if you plan to build full-stack AI products.

Why do AI engineer job descriptions vary so much?#

The title is new and employers use it for different scopes. Some postings are close to machine learning engineering with model training, while others are product engineering roles built on model APIs. Read the responsibilities, not just the title.

Next steps#

If this sounds like work you want to do, a practical path is structured practice: build a RAG app, add evals, and deploy it. Compare AI engineering programs side by side, or browse schools and read verified graduate reviews before you enroll. Our methodology explains how listings are reviewed, and our review guidelines explain how reviews are verified.