long running loop when context length exceeded

[jxstanford]

While running a task, it looks like the token count increased to a point that Anthropic failed the 3 of 3 attempts. Then continued to try the same request until the task was manually stopped. I noticed two things:

1. It looks like there was an opportunity to summarize due to summarization being disabled. Summarization appears to be disabled by default due to enable_context_manager defaulting to False in api.py:AgentStartRequest and no value being passed in api.ts:startAgent options. Summarization may have bought some additional space.

2. The second issue is that it just keeps trying the same series of retries until the user intervenes. It may be that it would have stopped after the 150 max_iterations, but that seems like a lot of times to iterate on something that will never succeed (rather than an intermittently failing condition).

backend-1 | 2025-04-27 22:56:37,562 - INFO - Running in local development mode - billing checks are disabled backend-1 | 2025-04-27 22:56:37,896 - INFO - Starting thread execution for thread 5c98b2e9-8e80-468c-8e48-9c5298255ba3 backend-1 | 2025-04-27 22:56:38,108 - INFO - Thread 5c98b2e9-8e80-468c-8e48-9c5298255ba3 token count: 158640/120000 (132.2%) backend-1 | 2025-04-27 22:56:38,108 - INFO - Automatic summarization disabled. Skipping token count check and summarization. backend-1 | backend-1 | Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new backend-1 | LiteLLM.Info: If you need to debug this error, use litellm._turn_on_debug()'.
backend-1 |
backend-1 | 2025-04-27 22:56:38,485 - WARNING - Error on attempt 1/3: litellm.BadRequestError: AnthropicException - b'{"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokens exceed context limit: 174167 + 64000 > 200000, decrease input length or max_tokens and try again"}}'
backend-1 | 2025-04-27 22:56:44,070 - INFO - Request started: GET /api/health from 192.168.65.1 | Query:
backend-1 | 2025-04-27 22:56:44,070 - INFO - Health check endpoint called
backend-1 |
backend-1 | Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
backend-1 | LiteLLM.Info: If you need to debug this error, use litellm._turn_on_debug()'. backend-1 | backend-1 | 2025-04-27 22:56:44,261 - WARNING - Error on attempt 2/3: litellm.BadRequestError: AnthropicException - b'{"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokensexceed context limit: 174167 + 64000 > 200000, decrease input length ormax_tokensand try again"}}' backend-1 | backend-1 | Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new backend-1 | LiteLLM.Info: If you need to debug this error, uselitellm._turn_on_debug()'.
backend-1 |
backend-1 | 2025-04-27 22:56:49,929 - WARNING - Error on attempt 3/3: litellm.BadRequestError: AnthropicException - b'{"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokens exceed context limit: 174167 + 64000 > 200000, decrease input length or max_tokens and try again"}}'
backend-1 | 2025-04-27 22:56:54,935 - ERROR - Failed to make API call after 3 attempts. Last error: litellm.BadRequestError: AnthropicException - b'{"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokens exceed context limit: 174167 + 64000 > 200000, decrease input length or max_tokens and try again"}}'
backend-1 | NoneType: None
backend-1 | 2025-04-27 22:56:54,935 - ERROR - Failed to make LLM API call: Failed to make API call after 3 attempts. Last error: litellm.BadRequestError: AnthropicException - b'{"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokens exceed context limit: 174167 + 64000 > 200000, decrease input length or max_tokens and try again"}}'
backend-1 | Traceback (most recent call last):
backend-1 | File "/app/agentpress/thread_manager.py", line 323, in _run_once
backend-1 | llm_response = await make_llm_api_call(
backend-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
backend-1 | File "/app/services/llm.py", line 324, in make_llm_api_call
backend-1 | raise LLMRetryError(error_msg)
backend-1 | services.llm.LLMRetryError: Failed to make API call after 3 attempts. Last error: litellm.BadRequestError: AnthropicException - b'{"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokens exceed context limit: 174167 + 64000 > 200000, decrease input length or max_tokens and try again"}}'
backend-1 | 2025-04-27 22:56:54,936 - ERROR - Error in run_thread: Failed to make API call after 3 attempts. Last error: litellm.BadRequestError: AnthropicException - b'{"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokens exceed context limit: 174167 + 64000 > 200000, decrease input length or max_tokens and try again"}}'
backend-1 | Traceback (most recent call last):
backend-1 | File "/app/agentpress/thread_manager.py", line 323, in _run_once
backend-1 | llm_response = await make_llm_api_call(
backend-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
backend-1 | File "/app/services/llm.py", line 324, in make_llm_api_call
backend-1 | raise LLMRetryError(error_msg)
backend-1 | services.llm.LLMRetryError: Failed to make API call after 3 attempts. Last error: litellm.BadRequestError: AnthropicException - b'{"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokens exceed context limit: 174167 + 64000 > 200000, decrease input length or max_tokens and try again"}}'
backend-1 | 2025-04-27 22:56:54,940 - INFO - Running in local development mode - billing checks are disabled
backend-1 | 2025-04-27 22:56:55,200 - INFO - Starting thread execution for thread 5c98b2e9-8e80-468c-8e48-9c5298255ba3
backend-1 | 2025-04-27 22:56:55,485 - INFO - Thread 5c98b2e9-8e80-468c-8e48-9c5298255ba3 token count: 158640/120000 (132.2%)
backend-1 | 2025-04-27 22:56:55,485 - INFO - Automatic summarization disabled. Skipping token count check and summarization.
backend-1 |
backend-1 | Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
backend-1 | LiteLLM.Info: If you need to debug this error, use litellm._turn_on_debug()'. backend-1 | backend-1 | 2025-04-27 22:56:55,899 - WARNING - Error on attempt 1/3: litellm.BadRequestError: AnthropicException - b'{"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokensexceed context limit: 174167 + 64000 > 200000, decrease input length ormax_tokensand try again"}}' backend-1 | backend-1 | Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new backend-1 | LiteLLM.Info: If you need to debug this error, uselitellm._turn_on_debug()'.
backend-1 |
backend-1 | 2025-04-27 22:57:04,373 - WARNING - Error on attempt 2/3: litellm.BadRequestError: AnthropicException - b'{"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokens exceed context limit: 174167 + 64000 > 200000, decrease input length or max_tokens and try again"}}'
backend-1 |
backend-1 | Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
backend-1 | LiteLLM.Info: If you need to debug this error, use litellm._turn_on_debug()'. backend-1 | backend-1 | 2025-04-27 22:57:09,853 - WARNING - Error on attempt 3/3: litellm.BadRequestError: AnthropicException - b'{"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokensexceed context limit: 174167 + 64000 > 200000, decrease input length ormax_tokensand try again"}}' backend-1 | 2025-04-27 22:57:14,082 - INFO - Request started: GET /api/health from 192.168.65.1 | Query: backend-1 | 2025-04-27 22:57:14,082 - INFO - Health check endpoint called backend-1 | 2025-04-27 22:57:14,860 - ERROR - Failed to make API call after 3 attempts. Last error: litellm.BadRequestError: AnthropicException - b'{"type":"error","error":{"type":"invalid_request_error","message":"input length andmax_tokensexceed context limit: 174167 + 64000 > 200000, decrease input length ormax_tokensand try again"}}' backend-1 | NoneType: None backend-1 | 2025-04-27 22:57:14,861 - ERROR - Failed to make LLM API call: Failed to make API call after 3 attempts. Last error: litellm.BadRequestError: AnthropicException - b'{"type":"error","error":{"type":"invalid_request_error","message":"input length andmax_tokensexceed context limit: 174167 + 64000 > 200000, decrease input length ormax_tokens and try again"}}' backend-1 | Traceback (most recent call last): backend-1 | File "/app/agentpress/thread_manager.py", line 323, in _run_once backend-1 | llm_response = await make_llm_api_call( backend-1 | ^^^^^^^^^^^^^^^^^^^^^^^^ backend-1 | File "/app/services/llm.py", line 324, in make_llm_api_call backend-1 | raise LLMRetryError(error_msg)