A quantitative research team at a hedge fund wants to adapt a general-purpose large language model to perform better on fund-specific tasks: extracting structured trade rationale from internal research memos, classifying analyst sentiment on earnings calls using the fund's proprietary taxonomy, and summarizing position risk in the fund's house style. The team has 50,000 labeled examples from historical memos. Which fine-tuning approach best balances performance gain, data efficiency, and the risk of catastrophic forgetting on general language capabilities?

Fine-tuning strategy selection for domain-specific LLM adaptation involves three competing considerations: task performance, compute and data efficiency, and model stability, meaning avoidance of catastrophic forgetting. Low-Rank Adaptation (LoRA), introduced by Hu et al. (2021), is the dominant parameter-efficient fine-tuning (PEFT) approach in production settings because it directly addresses all three.

Low-Rank Adaptation injects trainable rank-decomposition matrices, meaning low-rank matrices, into the weight matrices of attention layers. The rank of these injected matrices is far smaller than the original matrix dimension, and only the injected matrices are trained while the original pre-trained weights stay frozen. This has three critical advantages for the hedge fund use case. First, catastrophic forgetting is minimized because the base model's knowledge is preserved; the adapters learn the delta from general to specific without overwriting the general. Second, the trainable parameter count is reduced by 10,000x or more compared with full fine-tuning, making the 50,000-example dataset sufficient without overfitting. Third, the adapters can be merged into base weights at inference time with zero additional latency, which matters for real-time research workflows.

The full-parameter fine-tuning approach on all 50,000 examples using a high learning rate is the most dangerous approach. A high learning rate on 50,000 domain-specific examples will catastrophically overwrite general language capabilities, so the model may become excellent at fund taxonomy but degrade on open-ended reasoning, which is needed for memo summarization. Full fine-tuning with careful learning rate scheduling can work, but it requires far more data and compute.

The classifier-head-on-frozen-embeddings approach is appropriate for single-task classification but cannot handle the multi-task nature of the requirement, which includes extraction, classification, and summarization, and it does not adapt the model's language generation behavior.

The prompt-engineering approach with few-shot examples from the 50,000 examples directly in the context window avoids weight updates but has practical limitations. The 50,000 examples cannot fit in any context window; even at 100K token limits, each example would need to be about two tokens to fit, which is unrealistic. Therefore, that approach cannot leverage the full labeled dataset and will underperform fine-tuned alternatives on specialized tasks.