Airsim: dqn_car.py not working

[Andrew Caro Quispe (acq3047)]

Greetings,

I am seeking assistance with regard to the dqn_car.py script found in the AirSim repository. I have encountered an issue while attempting to execute the program, specifically in the VecTransposeImage section. The error message reads, "AssertionError: The observation space must be an image or dictionary observation space." I am uncertain about how to proceed in resolving this issue and would greatly appreciate any guidance or support that can be provided.

Thank you for your time and attention.

import reinforcement_learning.setup_path
import gym

import time

from stable_baselines3 import DQN
from stable_baselines3.common.monitor import Monitor
from stable_baselines3.common.vec_env import DummyVecEnv, VecTransposeImage
from stable_baselines3.common.evaluation import evaluate_policy
from stable_baselines3.common.callbacks import EvalCallback


# Create a DummyVecEnv for main airsim gym env
env = DummyVecEnv(
    [
        lambda: Monitor(
            gym.make(
                "airgym:airsim-car-sample-v0",
                ip_address="127.0.0.1",
                image_shape=(84, 84, 1),
            )
        )
    ]
)
# Wrap env as VecTransposeImage to allow SB to handle frame observations
env = VecTransposeImage(env)

# Initialize RL algorithm type and parameters
model = DQN(
    "CnnPolicy",
    env,
    learning_rate=0.00025,
    verbose=1,
    batch_size=32,
    train_freq=4,
    target_update_interval=10000,
    learning_starts=200000,
    buffer_size=500000,
    max_grad_norm=10,
    exploration_fraction=0.1,
    exploration_final_eps=0.01,
    device="cuda",
    tensorboard_log="./tb_logs/",
)

# Create an evaluation callback with the same env, called every 10000 iterations
callbacks = []
eval_callback = EvalCallback(
    env,
    callback_on_new_best=None,
    n_eval_episodes=5,
    best_model_save_path=".",
    log_path=".",
    eval_freq=10000,
)
callbacks.append(eval_callback)

kwargs = {}
kwargs["callback"] = callbacks

# Train for a certain number of timesteps
model.learn(
    total_timesteps=5e5, tb_log_name="dqn_airsim_car_run_" + str(time.time()), **kwargs
)

# Save policy weights
model.save("dqn_airsim_car_policy")
del model # remove to demonstrate saving and loading

model = DQN.load("dqn_airsim_car_policy")

obs, info = env.reset()
while True:
    action, _states = model.predict(obs, deterministic=True)
    obs, reward, terminated, truncated, info = env.step(action)
    if terminated or truncated:
        obs, info = env.reset()

[skgr07]

I have same problem, too. Is is solved?

[Gavin Chen (Gavin2417)]

Have you solved the error?

[Andrew Caro Quispe (acq3047)]

I wasn't able to solve it using the existing methods, so I ended up using the TensorFlow and Keras libraries to build it from scratch. It seems that there's an incompatibility issue with the algorithm and the gym library, as it instructs you to use "gymnasium," which isn't supported by the algorithm either.

[Gavin Chen (Gavin2417)]

i changed to gymnasium and everythings works, with ppo as well