Astra: ByteDance's Novel Dual-System Approach to Mobile Robot Navigation

Introduction

As robots become increasingly common in industrial settings, warehouses, and even homes, their ability to navigate complex indoor environments remains a critical bottleneck. Traditional navigation systems often struggle with dynamic layouts, repetitive features, and ambiguous instructions. Addressing these challenges, ByteDance's research team has unveiled Astra, a dual-model architecture that promises to advance general-purpose mobile robot navigation by combining global intelligence with local reflexes.

Challenges in Traditional Robot Navigation

Conventional navigation systems break down the problem into separate rule-based modules: target localization (understanding where to go from natural language or images), self-localization (knowing the robot's position on a map), and path planning (global route generation plus local obstacle avoidance). However, these modules often fail in repetitive environments like warehouses, where artificial landmarks (e.g., QR codes) are required, and in dynamic spaces where obstacles appear suddenly.

While large foundation models have begun to integrate multiple capabilities, the optimal way to structure models for end-to-end navigation—balancing high-level reasoning with low-level control—remained an open question. Astra addresses this by following the System 1/System 2 cognitive paradigm.

Enter Astra: A Hierarchical Solution

Detailed in the paper “Astra: Toward General-Purpose Mobile Robots via Hierarchical Multimodal Learning” (available at astra-mobility.github.io), Astra splits navigation into two complementary sub-models:

• Astra-Global (System 2) handles low-frequency, high-level tasks like self-localization and target localization.
• Astra-Local (System 1) manages high-frequency, reactive tasks such as local path planning and odometry estimation.

This dual architecture allows the robot to reason globally while reacting swiftly locally, mimicking human cognition.

Astra-Global: The Intelligent Brain

Astra-Global functions as a Multimodal Large Language Model (MLLM) that processes both visual and linguistic inputs. Its primary job is to determine precise positions on a map, using a hybrid topological-semantic graph as contextual input. This graph encodes keyframes (nodes) as landmarks and their relationships (edges), enriched with semantic information such as “near the kitchen counter.”

During navigation, Astra-Global takes a query image or text command (e.g., “go to the blue door”) and matches it against the graph to identify the target location and the robot's current location. This global awareness ensures that the robot always knows its position relative to the environment.

An offline mapping process builds the hybrid graph before deployment, as described below.

Offline Mapping and Hybrid Graphs

To construct the topological-semantic graph, the team used temporal downsampling of input video to extract keyframes (nodes V). Edges E connect nodes that are spatially or semantically close. Semantic labels (L) are added via scene understanding models, creating a rich representation that Astra-Global can query.

This approach eliminates the need for manual landmark placement and adapts to diverse environments.

Astra-Local: The Reflexive Body

Astra-Local handles the fast, detailed movements required for local path planning and obstacle avoidance. It processes high-frequency sensor data (e.g., LiDAR, depth cameras) to generate velocity commands every few milliseconds. Unlike traditional local planners that rely on hand-coded rules, Astra-Local learns from demonstrations and real-world interactions, allowing it to navigate around dynamic obstacles like people or moving carts.

The tight integration between Astra-Global and Astra-Local ensures that when the global model identifies a new target, the local model can adjust the route in real time without restarting the entire planning pipeline.

Implications and Future Directions

Astra's dual-model architecture represents a significant step toward general-purpose mobile robots that can operate in unmodified indoor spaces. By separating high-level reasoning from low-level control, ByteDance demonstrates a scalable approach that could be applied to service robots, delivery drones, and autonomous vehicles. Future work may extend the system to outdoor environments or integrate multi-agent coordination.

For more details, the full paper and supplementary materials are available on the project website.