What it is

Inference is when a trained AI model actually does its job. Training is the learning phase. Inference is the exam. When you type a question into ChatGPT and it generates an answer, that's inference. When your phone recognises your face, that's inference. The model applies everything it learned during training to new inputs it's never seen before. It's the bit that costs companies money every time you use their AI product.

Why it matters for your job

Inference costs are why AI tools have usage limits, why some features are behind paywalls, and why companies care about efficiency. If you're evaluating AI tools for your team or making the case for adopting one, understanding that every AI query costs money helps you make better arguments. It also explains why some AI tools are fast and cheap (small model, simple inference) while others are slow and expensive (massive model, complex inference).

What to do about it

When comparing AI tools, think about inference costs. A tool that's slightly less impressive but runs cheaply and quickly might be far more practical for daily use than the cutting-edge option that costs a fortune per query. This is the kind of practical thinking that gets noticed.