The API: using ChatGPT-style models from your own scripts
Understand how APIs work, what a request and response look like, system and user messages, keys, retries, tokens and cost, and when you really need the API.
In this lesson you will
- Describe the parts of an API call
- Read a request with system and user messages
- Protect keys and handle failures
- Decide when the API is worth it
About 18 minutes
When typing into a chat box is not enough
Zainab runs a small online shop and receives 200 customer messages a day. She does not want to paste each into ChatGPT. She wants a script that reads each message, labels it and files it. That requires talking to the model from code, and the door for that is the API (application programming interface): a way for one program to send a request to another over the internet and get a response back. You do not need to become a programmer to understand it, and this lesson works at the level of ideas with short examples. If you want to learn coding properly, the Python course on this site is the place.
Company names, model names, prices, limits and exact code change often. The examples below show the shape of an API conversation, with placeholders. Always follow OpenAI’s current developer documentation for real code. The two code blocks marked “real output” were run here, but they do not call any AI service.
The conversation, translated into data
Every API call has the same four ingredients:
| Ingredient | What it is | Example |
|---|---|---|
| Endpoint | The web address you send the request to | Given in the documentation |
| Key | A secret that identifies and bills your account | A long string you must never share |
| Request | Your instructions as structured data (usually JSON) | Model, messages, settings |
| Response | The model’s reply as structured data | Text plus usage information |
Anatomy of a request
Chat models receive a list of messages, each with a role: system (standing instructions), user (the person’s input) and assistant (earlier replies). The whole conversation travels with every call, because the model keeps no memory of its own between calls. This script builds such a request and prints it. It does not send anything:
import json
def build_request(system, user, temperature=0.2):
return {
"model": "MODEL-NAME-GOES-HERE",
"temperature": temperature,
"messages": [
{"role": "system", "content": system},
{"role": "user", "content": user},
],
}
req = build_request("You label customer messages as PRAISE, COMPLAINT or QUESTION. Reply with one word.",
"Do you ship to Lahore?")
print(json.dumps(req, indent=2))
{
"model": "MODEL-NAME-GOES-HERE",
"temperature": 0.2,
"messages": [
{
"role": "system",
"content": "You label customer messages as PRAISE, COMPLAINT or QUESTION. Reply with one word."
},
{
"role": "user",
"content": "Do you ship to Lahore?"
}
]
}
Three details to notice. The system message holds the rules that never change, so you write them once. The user message is the only part that changes per customer. And temperature is a setting for randomness: low values give steadier, more repeatable answers (good for labelling), higher values give more varied ones (good for brainstorming). Settings available differ by model, so check the documentation.
Sending it (a pattern, not run here)
With the official library installed and your key stored safely, a call looks roughly like the pattern below. The names are illustrative; copy the current example from the documentation instead of this text.
# PATTERN ONLY - not run here. Check current OpenAI documentation for exact names.
from openai import OpenAI
client = OpenAI() # reads the key from an environment variable, never from the code
response = client.chat.completions.create(**build_request(SYSTEM_TEXT, customer_message))
label = response.choices[0].message.content
print(label)
Pasting your API key into a script, a chat message, a screenshot or a public GitHub repository. Keys are passwords with a credit card attached. Keep them in environment variables or a secrets manager, and rotate any key that leaks.
Real programs expect failure
Networks drop, services are busy, limits are reached. A dependable script retries with growing pauses (called exponential backoff) and gives up politely after a few tries. This is a general pattern that works with any web service, run here against a fake flaky service:
import time, random
calls = {"n": 0}
def flaky_service():
calls["n"] += 1
if calls["n"] < 3:
raise ConnectionError("temporary failure")
return "QUESTION"
def with_retries(fn, tries=4, base_delay=0.01):
for attempt in range(1, tries + 1):
try:
return fn()
except ConnectionError as e:
if attempt == tries:
raise
delay = base_delay * 2 ** (attempt - 1)
print(f"attempt {attempt} failed ({e}); waiting {delay:.2f}s")
time.sleep(delay)
print("Result:", with_retries(flaky_service))
attempt 1 failed (temporary failure); waiting 0.01s
attempt 2 failed (temporary failure); waiting 0.02s
Result: QUESTION
Tokens, cost and limits
- Models process text in pieces called tokens, roughly short chunks of words. You pay (or use allowance) for tokens in and tokens out.
- Longer prompts and longer answers cost more and take longer. Include only the context a request needs.
- Services set rate limits: caps on requests or tokens per minute. Handle them with retries and queues.
- Set a spending limit and alerts in your account before you test with loops. A bug that calls the API ten thousand times is an expensive way to learn.
- Look up current pricing and limits on the provider’s pages, because they change.
From script to reliable tool
- Write the task as a clear system message and test it by hand in the chat first.
- Make the output structured (lesson 22) so the program can read it.
- Validate every response and route failures to a person.
- Handle errors with retries and a limit.
- Log inputs, outputs and errors, but keep personal data out of logs.
- Test with fifty real examples, measure accuracy (lesson 27) and only then use it for real work.
Privacy and policy
Data sent through an API is processed by the provider under its business terms, which can differ from the consumer chat app. Read the data-use and retention terms, and check your own legal obligations before sending customer or employee data. If your workplace has an approved route, use that.
Do you need the API at all?
| Situation | Better choice |
|---|---|
| A few tasks a week | The chat app with a saved prompt |
| A repeated workflow between common apps | No-code automation (lesson 25) |
| Hundreds of items, custom logic, your own app | The API |
| Sensitive data with strict rules | An approved company deployment |
A prompt that does not work well in the chat window will not work better in code. Perfect the prompt by hand, then automate it.
Check what you learned
Five quick questions. Pick an answer and you will see straight away whether it is right and why.
What is an API?
- A type of keyboard
- A chat theme
- A free subscription
- A way for programs to send requests to a service and get responses
It is how code talks to another service.
Why must an API key be kept secret?
- It identifies and bills your account
- It makes text longer
- It is a public username
- It changes the language
Anyone with the key can use your account.
What does a low temperature setting usually give?
- Random answers
- Longer answers
- Steadier, more repeatable answers
- Pictures
Low values reduce variation.
What is exponential backoff?
- Deleting the request
- Using capital letters
- Sending twice at once
- Retrying with growing pauses
Waits grow after each failure.
What should you do before running a loop of calls?
- Run it on everything
- Set spending limits and test on a few items
- Remove error handling
- Share your key
A bug can cause many costly calls.
Summary
- An API call is a structured request and response.
- System messages hold standing rules; user messages vary.
- Guard your key, add retries, set spending limits.
- Prototype in chat before coding.
Questions
Is the API free?
Usually it is billed by usage, separately from chat subscriptions. Check the current pricing and any free allowances.
Do I need to know Python?
Python is the most common choice and the Python course here is a good start. Other languages work too.
Is API data used to train models?
Policies differ by product and change over time. Read the provider's current data-use terms.
