Recent large-scale vision-language models have significantly advanced embodied navigation across diverse paradigms. However, existing navigation agents are typically trained or adapted under static task settings, making it difficult to continually acquire new knowledge in dynamic open-world environments without forgetting previously learned scenarios. Moreover, real-world navigation scenarios are usually determined by multiple coupled factors, such as physical scenes, environmental conditions, and instruction styles, while existing LoRA-based adaptation methods mainly learn task-level matrix adapters and cannot explicitly decouple such hierarchical knowledge. To address these limitations, we formulate a new problem, termed Multi-Factor Lifelong Embodied Navigation (MFLEN), where an agent is required to continually learn from coupled navigation scenarios, decouple factor-level knowledge, and dynamically compose learned knowledge during task-agnostic inference. To solve MFLEN, we propose Tucker Adaptation++ (TuKA++), a generalized high-order tensor adaptation framework that represents multi-factor navigation knowledge through Tucker decomposition. It decouples scene, environment, and instruction-style knowledge into factor-specific expert matrices, and aligns the high-order tensor representation with two-dimensional LLM parameter matrices through tensor contraction. We further design a Factor-wise Knowledge Inheritance and Exploration strategy and a progressively expandable shared Tucker subspace with dynamic zero-padding to support continual learning while structurally preserving old task adapters. Extensive experiments demonstrate that TuKA++ consistently outperforms state-of-the-art baselines.
The TuKA++ training pipeline and the task-agnostic inference procedure.
Task-wise rollouts across the 40-task MFLEN benchmark, in order. Tasks 1–30 are learned sequentially (lifelong learning); Tasks 31–40 are held-out generalization tasks (inference only, not trained).
Walk into the room and turn left. Walk right past the staircase and into the hall to the right of the chair. Walk straight ahead into the large room and stop next to the rug.
Find the lamp. Navigator: Im guessing straight thru the doorway? Oracle: Yes, go through the doorway and turn right to the stairs.
Leave the bedroom, and take a right. Go into the hallway straight across. Enter the sitting room. Stop once you reach the rug.
Find the book. Navigator: Should I go through the door in front of me? Oracle: Yes go through the door and keep going straight past spider man Navigator: Should I go through the door to the left of the window or continue? Oracle: Go straight then head down the stairs on your left Navigator: Should I go left or right at the bottom of the stairs? Oracle: Go right then go right again to the landing of the next set of stairs.
Go around the left corner and find the red ball behind the white line.
Find the sign. Navigator: Should I go to the right or left hallway? Oracle: Go right and keep going straight down the hallway Navigator: should I keep going down this hallway still? Oracle: Just keep going straight till you get to an intersection.
Find the box. Navigator: Do we go left or right? Oracle: Right, down the stairs and towards the kitchen.
Follow the cones forward, then turn left at the blue cone and stop in front of the red ball.
Find the microwave. Navigator: I'd guess I'm looking for a breakroom. The place must be an nightmare if you're new. Where should I be going? Oracle: Definitely! Move down the hall directly in front of you a bit. Navigator: Keep going or did I miss it? Oracle: Nope, it's showing to keep on a going straight. Navigator: Thought it'd be near those chairs. Guess not. Where to? Oracle: Go straight to the hallway, and take a right and head down it. This one is rough lol.
Go to the lounge on level 1 and remove all the items from the top of the fireplace.
Go to the right of the stairs and into the large master bedroom. Wait at the doorway to the bathroom.
Find the yellow cylinder. Navigator: Should I go straight or turn? Oracle: Go straight past the cone, then turn left at the corner; the yellow cylinder is right next to the wall.
Go to the bathroom on level 2 next to the rec room and refold the hand towels to the right of the sink.
Find the toiletpaper. Navigator: Should I proceed inside, or take the path towards the pool? Oracle: You should go straight, turn left at the circle, then go right toward the far end of the building.
Walk straight ahead, pass between the two orange cones, then through the cone-shaped barrels, and stop in front of the red ball.
Move to the hallway next to the kitchen and turn the knob on the sink.
Go to the end of the hallway and bring me the pot next to the doorway.
Head towards the fireplace and pass by it with the fireplace to your right. Stop in front of the sliding glass door.
Exit the storage area, then turn left and move through the bedroom into the hallway. Make a sharp left and move three steps down the stairway.
Find the yellow hemisphere. Navigator: Which way should we go now? Oracle: Go forward, pass the yellow and blue cones on the left, then go forward and stop at the yellow hemisphere.
Go to the lounge on level 1 and on the wall above the stand with the lamp remove the bottom of the two vertically adjacent pictures.
Go to the bathroom on first level and pick up the towel next to the sink.
Walk down the two flights of stairs, stop at the end of the stairs.
Find the towel. Navigator: Whoops! I had to guess this room! Where do I head after I go through the laundry room? Oracle: take a left into the laundry room the a right out of the door leading into the kitchen.
Find the box. Navigator: where do I go from this room? Oracle: go left past the dining room table and up the steps by the front door Navigator: where do i go from the bathroom? Oracle: go forward then right and go down the first set of steps"
Walk into the study and face the fireplace and pick up the trinket on the fireplace shelf.
Walk forward, passing the small ball to the right, then the yellow cone to the left, and continue walking along the cones until you reach the orange ball, then stop.
Move past the table and enter the door with a garbage bin next to it. Stand near the entrance.
Find the nightstand. Navigator: Ok, I see a sort of night stand but I guess I need help. Oracle: Walk around past the dresser, out the door to your right and on to the bottom of the staircase Navigator: Oops I ended up in someone's bedroom. Oracle: Turn around and go out down the hallway until you reach a bathroom.
Past the blue cone and the red ball, find the yellow ball hidden in the grass.
Go to the bathroom that's connected to the bedroom and reach up and wipe the light with a soft cloth.
Turn to the right and go down the stairs. At the bottom of the stairs turn left and proceed out to the patio. Stop right there, you will see the pool in front of you. Wait there.
Find the tv. Navigator: Shall I continue, see for further TV's or head closer? Oracle: Turn to the right and go straight down that hallway until you reach the stairs. Navigator: I unintentionally did not make a movement, just go ahead and repeat your instructions, sorry about that Oracle: Turn to the right and go straight down that hallway until you reach the stairs.
Navigate to the shower located in the bathroom. The bathroom is accessible from the bedroom, which lies at the end of the hallway near the wall with the horse painting. Stop once you are directly in front of the shower.
Go to the bathroom on level one next to the master bedroom and turn off the light inside the shower stall.
Go right past the painting of stones. Continue going past the staircase and turn left at the open doorway on the left. Stop at the laundry machine.
Go to the main room and the large open central area outside the bathroom.
Walk forward to the blue cone, then pass through the right side after reaching it, go between the cones, and stop in front of the blue cone.
Navigator: How do I get to the end of the cone path? Oracle: Pass to the right of the red cone, then follow the cones straight ahead. Navigator: Okay, where do I stop? Oracle: Stop in front of the yellow cone at the end.
Walk to the other side of the corridor and follow the cone-shaped barrels to find the final yellow cylinder.
TuKA++ consistently outperforms state-of-the-art continual-adaptation baselines under the MFLEN setting.
