AirDreamer

AirDreamer studies goal-directed drone flight when the aircraft only has a narrow forward view of the scene and the useful route is not necessarily in that view. In cluttered settings with walls, traps, and narrow gaps, a drone may need to scan the environment, slide sideways while keeping an obstacle in sight, or briefly move away from the goal before it can make progress toward it.

That setting is difficult for several common navigation approaches. Local planners can keep optimizing toward a blocked direction, learned policies can overfit to the shapes they saw during training, and perception pipelines can produce useful intermediate maps while still relying on cost functions tuned for one environment. The paper AirDreamer: Generalist Drone Navigation with World Models addresses this by separating environment understanding from action selection: a world model represents the local scene from onboard depth and state, and a reinforcement learning policy chooses velocity and yaw commands from that representation.

The policy does not receive a pre-built map, global obstacle layout, or hand-written camera-heading rule. The reward is also kept relatively sparse, so the learned behavior is not pushed too strongly toward a designer's preferred heading or toward greedy distance-to-goal progress. That design choice is important because the correct local behavior in these environments can include looking away from the goal, taking a detour, or sacrificing short-term progress to find a viable route.

Paper details

• Paper: AirDreamer: Generalist Drone Navigation with World Models
• Authors: Zian Liu, Andong Yang, Chunkai Yang, Ruidong An, Chao Gao, and Guyue Zhou
• arXiv: 2606.03252v1, posted June 2, 2026
• Category: cs.RO
• Code: The v1 paper says the code will be publicly available, but does not include a repository URL.
• PDF: arxiv.org/pdf/2606.03252

Quick read

AirDreamer uses a Dreamer style world model to encode depth images and low-dimensional drone state into latent memory. A policy chooses body-frame velocity and yaw commands from that latent state. The system runs from onboard depth images and goal location alone, without a pre-built map or global obstacle information.

In XTDrone simulation, AirDreamer reports 59.3 +/- 15.8% success across 150 runs on five maps, a 5.3 percentage-point lead over DepthNav in the paper's table and much higher than NavRL and EgoPlanner under the same local-observation constraint.

On hardware, the authors deploy the policy on a 280 mm quadrotor with a RealSense D455 depth camera, FAST-LIO2 state estimation, PX4 on a Pixhawk 6X Pro, and a Jetson Orin NX Super. The policy runs as a ROS node at 20 Hz. The converted ONNX inference model has 57.9 million parameters and reaches up to 88 Hz on the Jetson.

The real-world demos include a 5 m wall, a 1 m passage, forest layouts, a U-shaped trap, and a C-shaped obstacle the policy did not see during training. In the C-shaped obstacle, the drone first flies about 3 m away from the goal to escape the local optimum, then speeds up after finding the exit. The paper is trying to show non-greedy navigation from partial vision, not just obstacle avoidance.

Why this problem matters

A forward depth camera gives a small drone useful local geometry but not a complete scene. The sides and rear are missing, and the route to the goal may be hidden even when the goal direction itself is known. This makes the short-term signal unreliable: the drone may have to turn its camera before moving, back out of a trap, or follow a wall until an opening becomes visible.

Dense navigation rewards can make that worse because distance-to-goal progress is only a good signal when the direct route is not blocked. Classical local planners can run into the same problem when the local map does not include the eventual route around a large obstacle. AirDreamer is aimed at this partial-observation gap by giving the policy recurrent memory and imagined rollouts instead of asking it to react only to the next visible depth frame.

What the drone sees and controls

The task is goal navigation in unknown, cluttered environments: the drone starts at a random position and yaw, then has to reach a random goal while maintaining altitude and avoiding obstacles.

The observation combines a 48 x 80 depth image with a 15-dimensional state vector. The depth image uses a valid range from 0.6 m to 6.0 m; pixels outside that range are replaced with the maximum range value. The state vector includes goal direction in the body frame, horizontal goal distance, vertical goal offset, body-frame velocity, angular velocity, and attitude.

The action interface is direct. At 20 Hz, the policy outputs a body-frame velocity command and a yaw command; the paper represents yaw as a two-component encoding to avoid angle discontinuities, then decodes it into a yaw setpoint that a low-level velocity controller tracks. AirDreamer is not planning a whole map-level route, but repeatedly choosing where to move and where to look from local state and learned memory.

The world model is the working memory

AirDreamer uses the Dreamer V3 framework. The world model receives the depth image and low-dimensional state, encodes them, and maintains a recurrent latent state.

In the paper's implementation, the recurrent state-space model has:

• a deterministic GRU state with dimension 6144,
• a stochastic latent state of 32 categorical variables with 48 classes each,
• predictors for next latent state, reward, continuation probability, and decoded observations.

The policy is trained mostly on imagined trajectories from that latent model. The actor selects actions from the latent state. The critic estimates return from the same representation. During deployment, live observations are available, so the posterior state is used for inference.

The decoded depth images look plausible for about 20 imagined steps, but prediction fidelity is not the whole point. The policy is trained against an internal model that carries history forward. A single depth image cannot tell the drone what it just passed, what may be hidden behind a wall, or why turning the camera might reveal a better route. The recurrent world model gives the policy a place to keep that context.

Sparse reward is doing real work

The reward design is one of the strongest parts of the paper because it attacks a common failure mode in learned navigation.

AirDreamer uses two terminal rewards and three small auxiliary terms:

The goal reward is 100 and ends the episode. A lethal collision gives a one-time -5 penalty and also ends the episode. The smaller terms stabilize learning: progress toward the goal, obstacle safety within 1 m, and altitude maintenance.

The paper deliberately avoids a reward for camera direction or yaw angle. A hand-written heading objective can teach the drone to stare at the goal, even when the useful behavior is to look sideways, scan a passage, or keep an obstacle in view while moving laterally.

The sparse goal reward also reduces the damage from local progress shaping. If the drone has to move away from the goal to go around a wall, per-step progress can punish the correct behavior. AirDreamer's world model helps propagate the delayed goal signal backward through imagined rollouts, so sparse reward becomes more usable than it would be for a shallower reactive policy.

Training setup

The authors train in OmniDrones, built on Isaac Sim 4.1.0. They use the Hummingbird drone model and a Lee position controller. The environment contains random cuboid obstacles and walls, with U-shaped obstacles included to teach escape from local traps.

The policy executes at 20 Hz and the physical simulation runs at 200 Hz. The training run uses the original Dreamer V3 JAX codebase. The JAX training model has 101.7 million parameters. For deployment, the model is converted to ONNX and reduced to a 57.9 million-parameter inference model.

The paper reports:

Training is substantial, but still within reach for a serious autonomy lab. The deployment target is the more practical point: the final policy runs on edge compute.

Domain randomization is the transfer bridge

AirDreamer uses domain randomization because the simulated drone and real drone do not respond exactly the same way. The randomized parameters include mass, inertia, thrust-to-weight ratio, force-to-moment ratio, drag, and motor spin-up and spin-down behavior.

The ablation result is a useful warning. Disabling domain randomization improves the simulation success rate by about 10%, but the resulting policy cannot be deployed on real drones. A cleaner simulator can make the scoreboard better while making transfer worse.

The paper also scales the model from 101.7M to 180.7M parameters and sees no measurable success improvement. The result points away from raw model size and toward the combination of world model, reward design, yaw control, and randomization.

Benchmark numbers

The paper compares AirDreamer with DepthNav, NavRL, and EgoPlanner. Success is measured in XTDrone across 150 runs over five maps with equal density. Real-world experiments are not used for this table because baseline deployment depends heavily on method-specific flight-controller tuning and open-source availability.

A 59.3% success rate is not a solved navigation problem. The result is useful because the baselines collapse under the same local-observation setting, especially as the maps become more complex.

EgoPlanner is a strong classical local planner in the right context. Here, the local-observation constraint hurts it. NavRL also struggles, partly because its yaw constraints limit paths that are not directly visible. DepthNav is much closer, but still behind AirDreamer in the reported benchmark.

The paper's claim is narrower than "world models beat planning." In cluttered, unseen, partially observed drone navigation, a world-model-based policy can make non-greedy choices that reactive policies and local planners often miss.

Hardware results

The real drone setup is concrete:

• Drone: 280 mm motor-to-motor quadrotor
• Depth: Intel RealSense D455
• Localization: FAST-LIO2
• Flight controller: Pixhawk 6X Pro running PX4 v1.14.3
• Compute: Jetson Orin NX Super with 16 GB memory
• Policy runtime: ROS node at 20 Hz
• Success condition: within 1 m of the goal

The lidar is used for state estimation, not as an input to the navigation policy. That distinction matters. The learned policy is still depth-and-state based, but the full system depends on reliable localization.

The authors run six real-world scenarios. The flight statistics are averaged over three trials per scenario:

The C-shape run is the most useful hardware example because the policy only saw U-shaped traps during training. The real obstacle type is out of distribution, and the drone starts inside the cavity. The policy turns to expand visual coverage, moves away from the goal to escape, and then accelerates after finding a route. Dense direction-following rewards tend to erase exactly that behavior.

What feels new

The ingredients are not isolated breakthroughs. Dreamer style world models, domain randomization, sparse rewards, depth navigation, and sim-to-real deployment are all established ideas.

The paper's contribution is the working composition:

The yaw part is easy to underestimate. A drone's camera is an active sensor, and where the drone looks changes what it can know. If yaw is treated as a nuisance, the policy can miss paths that require scanning. If yaw is treated as an action with no hand-written preference, the policy can learn to look where the task demands.

The reward design shows up directly in the real-world demos. The drone is not just avoiding obstacles in front of it; it is using yaw to change the information available for the next decision.

Limits of the result

AirDreamer is not a complete autonomy stack.

The strongest quantitative comparison is in simulation. The real-world section is qualitative, partly because fair baseline deployment is difficult when each method needs different flight-controller tuning. The hardware evidence should be read as a transfer demonstration, not as a full real-world benchmark against every baseline.

The policy also depends on good state estimation. The paper does not feed lidar into the policy, but the system uses FAST-LIO2 for localization. In field deployment, localization failure would still become navigation failure.

The task is static-obstacle goal navigation. Dynamic obstacles, multi-robot coordination, target tracking, weather, outdoor lighting, and long-range mission planning remain future work. The paper says the framework could extend in those directions, but they are not demonstrated here.

There is also the operational cost of training. A 100M-parameter JAX model trained for nearly 39 hours on two GPUs is reasonable for research, but it is not the same as a lightweight policy that every builder can retrain casually.

Those limits put the result in the right category: AirDreamer is a strong research step for local, depth-based drone navigation under partial observability.

Where this fits with Nimbus and DroneForge

For DroneForge builders, AirDreamer points at a useful product primitive: local navigation that can choose when to look, when to detour, and when to give up short-term goal progress for a better route.

Nimbus workflows already care about the same ingredients: video, telemetry, route planning, object tracking, repeated missions, and real hardware feedback. A world-model policy like AirDreamer would not replace those pieces. It would sit between perception and command generation, turning recent observations and goal state into short-horizon motion.

A practical version might look like this:

goal = droneforge.Goal(x=9.0, y=0.0, z=1.5)

nav = nimbus.navigation.local_policy(
    observations=["depth", "telemetry"],
    objective="reach_goal",
    allow_yaw_control=True,
)

while mission.active:
    command = nav.step(goal=goal)
    drone.send_velocity_command(command)

It would not replace safety layers, geofencing, operator override, or route-level planning. It is a local autonomy primitive with a useful skill: learning when the straight line is a bad idea.

For inspection, search practice, warehouse flight, indoor mapping, and constrained field research, that kind of primitive matters. Many missions fail not because the global objective is unclear, but because the local route requires temporary retreat, sideways sensing, or active camera control.

Bottom line

AirDreamer is worth reading because it treats drone navigation as a partial-observation problem instead of a map problem with a missing map.

The best idea is the combination of world-model memory and sparse reward. The world model gives the policy enough internal structure to reason beyond the current depth frame. The sparse reward avoids forcing a human-preferred heading or greedy progress rule onto situations where those rules are wrong.

The result is a drone policy that can scan, detour, wall-follow, pass through tight gaps, and escape a trap it did not see in training. For drone autonomy, the useful direction is better local judgment from limited sensing, not only faster planning.