Tokenizer and token counter (GPT, Claude, Gemini, Grok)
Tokens are the basic unit that generative AI models use to compute the length of a text. They are groups of characters, which sometimes align with words, but not always. In particular, it depends on the number of characters and includes punctuation signs or emojis. This is why the token count is usually different from the word count.Use the tool provided below to explore how a specific piece of text would be tokenized and the overall count of words, characters and tokens. Note: Claude, Gemini and Grok only provide token counts, not the actual tokens.
Model:
Text:
Tokenized text:
Text
Token IDs
Use LLMs in bulk in Google Sheets or Excel
GPT for Work enables bulk LLM operations directly in spreadsheets. Process thousands of rows with ChatGPT, Claude, Gemini, and other AI models.
Why token counts matter
Every model provider bills per token, not per word, and every model has a context window measured in tokens. So the token count is the number that decides two things: what a prompt costs, and whether it fits at all. English prose averages about 1.3 tokens per word, but code, JSON, URLs, non-Latin scripts, and emoji are far denser, which is where a guess based on word count goes wrong.
Once you know the count, the API pricing calculator turns it into a cost per call, and into a cost for the whole job if you are about to run the same prompt over a few thousand rows.
What this tokenizer shows you
Paste text and you get the token count next to the word and character counts, so you can see how far apart they are for your kind of content. For OpenAI models the text is also split into its individual tokens, which you can view as highlighted text or as raw token IDs.
Two summaries sit below that. The most expensive words are the ones the encoder breaks into the largest number of tokens, which is usually where trimming pays off. The most common tokens show what repeats in your prompt, which is what to look at when a template is costing more than the content it carries.
Limits to keep in mind
Claude, Gemini, and Grok return a count only. Their tokenizers are not public, so there are no individual tokens to display and no expensive word breakdown for those models. Counts also cover the text you paste and nothing else: the system prompt, tool definitions, images, and the model's own reply all add tokens to a real API call.
How each count is produced
The four providers are not measured the same way, which matters when you compare numbers between them.
| Model provider | Counting method | Where it runs |
|---|---|---|
| OpenAI GPT | o200k_base encoding, the GPT-4o tokenizer | In your browser, no request sent |
| Anthropic Claude | Anthropic's own token counting endpoint | Server request as you type |
| Google Gemini | Gemini's countTokens endpoint | Server request as you type |
| xAI Grok | xAI's tokenize-text endpoint | Server request as you type |
Counts are not comparable across providers. The same sentence can be a different number of tokens for each model, so price out each one on its own tokenizer.
Frequently asked questions
- Is this token counter free?
- Yes. There is no account, no API key, and no limit on how many times you can count. OpenAI counts run entirely in your browser; the Claude, Gemini, and Grok counts go through our server to each provider's counting endpoint.
- How many tokens is one word?
- For ordinary English prose, about 1.3 tokens per word on average. That average breaks down quickly for code, JSON, long URLs, rare proper nouns, non-Latin scripts, and emoji, all of which cost several tokens per word or per character. Paste a real sample rather than relying on the ratio.
- Why do Claude, Gemini, and Grok show a count but no tokens?
- Those providers do not publish their tokenizers. They expose an endpoint that returns how many tokens a text is, but not the split itself, so there is nothing to highlight. Only the OpenAI view shows individual tokens and token IDs.
- Which tokenizer does the OpenAI option use?
- o200k_base, the encoding used by the GPT-4o family. It runs locally in the browser, so nothing you paste is sent anywhere for the OpenAI count.
- How do I estimate the cost of a whole job?
- Count the tokens in one representative prompt, add the tokens you expect in the reply, then multiply by the number of rows or calls. The OpenAI API pricing calculator on this site does that arithmetic per model.
- How do I run the same prompt over thousands of rows?
- GPT for Sheets and GPT for Excel run prompts down a spreadsheet column at up to 1,000 rows per minute, so the prompt you just measured becomes a formula you fill down.
Run the prompt you just measured, in bulk
GPT for Sheets and GPT for Excel take a prompt like the one you pasted and run it down a spreadsheet column, at up to 1,000 rows per minute.
