Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs

Activation steering has emerged as a promising method for efficiently adapting large language models (LLMs) to downstream behaviors. However, most existing steering approaches identify and steer the model from a single static direction for each task or concept, which is inflexible under task variation and insufficient for complex tasks requiring multiple coordinated capabilities. To address this gap, we propose Steer2Adapt, a lightweight framework that enables efficient LLM adaptation by composing steering vectors rather than learning new ones from scratch. In practice, tasks within the same domain (e.g., reasoning or safety) often share a small set of underlying concept dimensions. Steer2Adapt spans these dimensions into a reusable, low-dimensional semantic prior subspace and adapts to new tasks by dynamically discovering a linear combination of basis vectors using only a handful of examples. Experiments across 9 tasks and 3 models in both reasoning and safety domains demonstrate the effectiveness of Steer2Adapt, with an average of 8.2% improvement. Together with our analyses, we establish Steer2Adapt as a data-efficient, stable, and transparent inference-time adaptation method for LLMs.