Amazon Nova Models Improve SFT via Self-Distilled Reasoning and Mitigate Catastrophic Forgetting

AWS introduces Self-Distilled Reasoning (SDR) for supervised fine-tuning of Amazon Nova 2 models. The core problem is that SFT datasets lacking reasoning chains cause suppressed reasoning and performance degradation during inference. SDR has three steps: first, use the base reasoning model (e.g., Amazon Nova 2 Lite) to generate reasoning traces for each training sample; second, append these traces to original outputs to form an augmented dataset; third, fine-tune the model in inference mode with supervision on both reasoning and output tokens. Experiments show SDR significantly mitigates catastrophic forgetting compared to pure SFT. For instance, on a math benchmark, pure SFT drops from 70% to 6%, while SDR nearly recovers to 68%-70%, with target performance improving by over 6.5% on average. Unlike model merging, SDR requires no post-hoc interpolation or external teacher, effectively maintaining both target and general performance. For developers using Amazon Nova customization, this means injecting reasoning ability into existing SFT datasets without manual labeling, while improving performance on tasks like tool calling and long-document summarization.