Congressional AI: A Framework for Task Generalization and Alignment with Expert Language Modes

As foundation models have facilitated rapid adaptation to downstream tasks, a challenge remains in efficiently and flexibly improving their instruction-following capabilities and alignment to human preference distributions. We propose a novel modular architecture, Congressional AI, consisting of parallel trained "experts", such that the top-k relevant experts can be activated during inference. These experts are obtained by fine-tuning LoRA adapters on interpretable data mixtures; for instruction-tuning, each dataset corresponds to a task cluster, while for preference alignment to improve steerability, each dataset represents a group or persona. Our experiments show that instruction-tuning with Congressional AI through low-rank adapter merging is effective via evaluation of cluster-specific adapters across various domains on the MMLU benchmark. These findings demonstrate that Congressional AI is a hardware-efficient and interpretable mixture-of-experts (MoE)-style framework for adapting language models to new tasks and domains, and can be used to further improve both pre-trained and fine-tuned LLMs.