Bug: successful job being marked as failed

[d-walsh]

Input tokens: 11820 | Output tokens: 80
23:00:58 - ACTION
AgentFinishAction(outputs={}, thought='It appears that there is no `test` directory in the project, which is why the test command is failing. Since the README updates did not require any changes to the application code, there are no tests to run.\n\nThe README file has been successfully updated to better represent the actual code and setup process. Since there are no tests to run, I will conclude the task.', action='finish')
23:00:58 - OBSERVATION
AgentStateChangedObservation(content='', agent_state=<AgentState.FINISHED: 'finished'>, observation='agent_state_changed')
23:00:58 - OBSERVATION
NullObservation(content='', observation='null')
23:00:58 - openhands:INFO: resolve_issues.py:131 - ------------------------------
23:00:58 - openhands:INFO: resolve_issues.py:132 - BEGIN Runtime Completion Fn
23:00:58 - openhands:INFO: resolve_issues.py:133 - ------------------------------
23:00:58 - ACTION
**CmdRunAction (source=None)**
COMMAND:
cd /workspace
23:00:58 - openhands:INFO: runtime.py:316 - 
-----------------------------------Container logs:-----------------------------------
    |23:00:58 - openhands:ERROR: client.py:379 - Error getting exit code from bash script
    |INFO:     172.17.0.1:46622 - "POST /execute_action HTTP/1.1" 200 OK
--------------------------------------------------------------------------------
23:00:58 - OBSERVATION
**CmdOutputObservation (source=None, exit code=2)**
cd /workspace
[Python Interpreter: /openhands/poetry/openhands-ai-5O4_aCHf-py3.11/bin/python]
openhands@ef80aeb72581:/workspace $ 
23:00:58 - OBSERVATION
AgentStateChangedObservation(content='', agent_state=<AgentState.STOPPED: 'stopped'>, observation='agent_state_changed')
23:00:58 - openhands:INFO: agent_controller.py:157 - AgentController task was cancelled
ERROR:root:  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/openhands_resolver/resolve_issues.py", line 726, in <module>
    main()
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/openhands_resolver/resolve_issues.py", line 706, in main
    asyncio.run(
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/openhands_resolver/resolve_issues.py", line 562, in resolve_issues
    await asyncio.gather(*[run_with_semaphore(task) for task in tasks])
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/openhands_resolver/resolve_issues.py", line 560, in run_with_semaphore
    return await task
           ^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/openhands_resolver/resolve_issues.py", line 403, in update_progress
    resolved_output = await output
                      ^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/openhands_resolver/resolve_issues.py", line 314, in process_issue
    return_val = await complete_runtime(runtime, base_commit)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/openhands_resolver/resolve_issues.py", line 141, in complete_runtime
    raise RuntimeError(
ERROR:root:<class 'RuntimeError'>: Failed to change directory to /workspace. Observation: **CmdOutputObservation (source=None, exit code=2)**
cd /workspace
[Python Interpreter: /openhands/poetry/openhands-ai-5O4_aCHf-py3.11/bin/python]
openhands@ef80aeb72581:/workspace $ 
  0%|          | 0/1 [03:33<?, ?it/s]
Error: Process completed with exit code 1.

[neubig]

Hmm, this is strange. I'll definitely try to fix this, but in order to do so I'll have to figure out a way to reproduce. Any additional information that you could provide to help me reproduce the error (or any reports from other people) would be appreciated.

[codegefluester]

I am seeing similar behavior where running into rate limits with Claude essentially makes the agent fail, even though it already started with git operations + wrote a changelog file. Running this in GitHub actions.

--------
10:53:54 - OBSERVATION
AgentStateChangedObservation(content='', agent_state=<AgentState.STOPPED: 'stopped'>, observation='agent_state_changed')
ERROR:root:  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/openhands_resolver/resolve_issues.py", line 720, in <module>
    main()
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/openhands_resolver/resolve_issues.py", line 700, in main
    asyncio.run(
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/openhands_resolver/resolve_issues.py", line 556, in resolve_issues
    await asyncio.gather(*[run_with_semaphore(task) for task in tasks])
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/openhands_resolver/resolve_issues.py", line 554, in run_with_semaphore
    return await task
           ^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/openhands_resolver/resolve_issues.py", line 397, in update_progress
    resolved_output = await output
                      ^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/openhands_resolver/resolve_issues.py", line 319, in process_issue
    success, success_explanation = guess_success(issue, state.history, llm_config)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/openhands_resolver/resolve_issues.py", line 231, in guess_success
    response = litellm.completion(
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/litellm/utils.py", line 1069, in wrapper
    raise e
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/litellm/utils.py", line 957, in wrapper
    result = original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/litellm/main.py", line 2955, in completion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2131, in exception_type
    raise e
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 505, in exception_type
    raise RateLimitError(

ERROR:root:<class 'litellm.exceptions.RateLimitError'>: litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"Number of request tokens has exceeded your daily rate limit (https://docs.anthropic.com/en/api/rate-limits); see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}