Coverage for src / harnessutils / tokens / estimator.py: 100%
2 statements
« prev ^ index » next coverage.py v7.13.2, created at 2026-02-12 22:41 -0600
1"""Token estimation using simple heuristic.
3Uses chars/4 approximation which is fast and good enough for
4pruning decisions. Actual token counts from LLM responses are
5used for overflow detection.
6"""
9def estimate_tokens(text: str, chars_per_token: int = 4) -> int:
10 """Estimate token count for text using simple heuristic.
12 Uses chars_per_token ratio (default 4) for fast estimation.
13 This is good enough for pruning decisions. For accurate counts,
14 use actual LLM response token counts.
16 Args:
17 text: Text to estimate tokens for
18 chars_per_token: Characters per token ratio (default: 4)
20 Returns:
21 Estimated token count
22 """
23 return max(0, round(len(text) / chars_per_token))