EasyDeepLearn

How does function calling / tool use work in modern LLMs?

medium

Answer

  • Developer declares a set of tool schemas (name, description, JSON-schema parameters).
  • At inference, the model may decide to emit a special toolcall\mathrm{tool}_{\mathrm{call}} token followed by a JSON matching one of the schemas instead of a text reply.
  • The runtime executes the tool, appends the result as a toolresponse\mathrm{tool}_{\mathrm{response}} message, and re-invokes the model.
  • Loop until the model emits a normal message.
  • Foundation for agents, calculators, retrieval, code execution, and every non-trivial LLM app.
Check yourself — multiple choice
  • The LLM runs functions itself
  • Model outputs a JSON toolcall\mathrm{tool}_{\mathrm{call}} matching a schema → runtime executes → appends toolresponse\mathrm{tool}_{\mathrm{response}} → re-invoke LLM in a loop
  • Only regex
  • Impossible in modern LLMs

Function calling: model emits toolcall\mathrm{tool}_{\mathrm{call}} JSON → runtime executes → loop until text response.

#prompting#agents#tools#function-calling

Practise LLMs & GenAI

214 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions