There are different pace layers for getting results out of LLMs.

Slowest: train a new foundation model from scratch.

Also extremely capital intensive!

Medium: fine-tune an existing model

Fast: Prompt engineering

Carefully calibrating the exact structured guidance you give to the LLMs to get it to give reliably good results for a class of problems.

You can do patterns like few-show learning.

Fastest: Context engineering

Precisely what extra background knowledge you pass it to help it answer this question.

People talk about fine-tuning and making new foundation models, but the last two layers are remarkably effective, and wildly cheaper to accomplish.

Now that context windows are so large, you can get very, very far with careful prompt engineering, few shot examples are extremely effective.

There's tons of space to give the models carefully crafted examples!

Don't assume you have to dip to lower levels before you actually have to!