Fail to restore the commander policy when evaluating with commander.

[arounderor]

Hello, when I try to evaluate with Commander, I got the error below. Looks like it failed to restored the commander policy. Do I need to change some configurations before evaluating with Commander? I use the pre-trained files you provide and carefully read the 'Procedure'.

Traceback (most recent call last):
File "C:\Users\Administrator\Desktop\hhmarl\hhmarl_2D\evaluation.py", line 99, in
policy = Policy.from_checkpoint(check, ["commander_policy"])["commander_policy"] if args.eval_hl else None
File "G:\python_venvs\python310\lib\site-packages\ray\rllib\policy\policy.py", line 335, in from_checkpoint
policies[policy_id] = Policy.from_state(policy_state)
File "G:\python_venvs\python310\lib\site-packages\ray\rllib\policy\policy.py", line 378, in from_state
new_policy = actual_class(
File "G:\python_venvs\python310\lib\site-packages\ray\rllib\algorithms\ppo\ppo_torch_policy.py", line 67, in init
self._initialize_loss_from_dummy_batch()
File "G:\python_venvs\python310\lib\site-packages\ray\rllib\policy\policy.py", line 1405, in _initialize_loss_from_dummy_batch
actions, state_outs, extra_outs = self.compute_actions_from_input_dict(
File "G:\python_venvs\python310\lib\site-packages\ray\rllib\policy\torch_policy_v2.py", line 522, in compute_actions_from_input_dict
return self._compute_action_helper(
File "G:\python_venvs\python310\lib\site-packages\ray\rllib\utils\threading.py", line 24, in wrapper
return func(self, *a, **k)
File "G:\python_venvs\python310\lib\site-packages\ray\rllib\policy\torch_policy_v2.py", line 1141, in _compute_action_helper
dist_inputs, state_out = self.model(input_dict, state_batches, seq_lens)
File "G:\python_venvs\python310\lib\site-packages\ray\rllib\models\modelv2.py", line 259, in call
res = self.forward(restored, state or [], seq_lens)
File "C:\Users\Administrator\Desktop\hhmarl\hhmarl_2D\models\ac_models_hier.py", line 85, in forward
output, new_state = self.forward_rnn(input_dict, state, seq_lens)
File "C:\Users\Administrator\Desktop\hhmarl\hhmarl_2D\models\ac_models_hier.py", line 91, in forward_rnn
x = torch.cat((self.inp1(self._inp1), self.inp2(self._inp2), self.inp3(self._inp3)),dim=1)
File "G:\python_venvs\python310\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "G:\python_venvs\python310\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "G:\python_venvs\python310\lib\site-packages\ray\rllib\models\torch\misc.py", line 169, in forward
return self._model(x)
File "G:\python_venvs\python310\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "G:\python_venvs\python310\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "G:\python_venvs\python310\lib\site-packages\torch\nn\modules\container.py", line 217, in forward
input = module(input)
File "G:\python_venvs\python310\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "G:\python_venvs\python310\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "G:\python_venvs\python310\lib\site-packages\torch\nn\modules\linear.py", line 116, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (32x20 and 10x50)

Process finished with exit code 1
Looking forward to your reply. Thanks for you patience.

[ardian-selmonaj]

It worked in my case, with the same code as on github. I think you altered N_OPP_HL , i.e. the sensing. Check that it is set to N_OPP_HL=2 inside train_hier.py, ac_models_hier.py and env_hier.py. Then it should work.

[arounderor]

I didn't change N_OPP_HL.
I just pulled from github to ensure I have the newest version and added the policies and results dir. Then changed eval configs, all changes are shown as below. The same error occured.

I debugged it and it seems there is problem below. I don't know what's wrong now.