AI-ENG-H — Model Adaptation - Fine-Tuning, LoRA, Preference Tuning & Distillation

Model adaptation represents the most high-leverage and mathematically invasive layer of the model steering stack. While prompt engineering, in-context examples, context construction, retrieval-augmented generation (RAG), tool interfaces, and output validation enforce constraints at inference time, adaptation alters the underlying parameters or probability distributions of the neural network.1 In accordance with the steering doctrine established in the engineering canon, adaptation must be treated as a diagnosed behavioral intervention rather than a default capability patch.

The governing engineering doctrine dictates: Adapt behavior when the desired behavior is stable, repeated, measurable, and poorly served by lighter steering layers; keep volatile knowledge, permissions, live state, and source-grounded facts outside the weights.

Lighter steering layers operate on the activation space of the model, steering its attention over dynamic context. When the target behavior is highly volatile—such as pricing sheets, inventory counts, user permissions, or live news—encoding this information into parameters through model training guarantees rapid factual obsolescence, weight rot, and security leakage across tenant boundaries.3 Conversely, when the target behavior is structurally stable, highly repeated, and quantifiable—such as generating domain-specific schemas, mapping standardized tool parameters, enforcing syntactic or stylistic invariants, or executing complex multi-step logical operations—adaptation becomes the most computationally efficient and robust intervention.5

The critical architectural boundary lies between knowledge and behavior. Volatile, factual, and permission-sensitive facts belong in context compilation, dynamic retrieval pipelines, or real-time tool lookups. Stable, procedural, stylistic, and formatted behaviors belong in the weights. Attempting to solve knowledge gaps with weight modification leads to memorization pathologies, catastrophic forgetting, and severe out-of-domain regression.4 Conversely, attempting to solve behavioral, stylistic, or formatting gaps solely with prompts bloats the token context window, degrades inference latency, increases cost-per-success, and introduces high-variance failures at the production edge.5

Conceptual Glossary

To establish a structurally durable nomenclature for the model adaptation lifecycle, the following terms are mathematically and operationally defined:

The Adaptation Decision Ladder

A disciplined engineering organization does not default to model training. Before introducing the overhead of weight optimization, practitioners must traverse the Adaptation Decision Ladder. This framework ensures that lighter, more reversible, and cheaper interventions are systematically exhausted before allocating compute resources to parameter modification.

Behavior Gap Diagnostic

To determine the appropriate entry point on the Adaptation Decision Ladder, the system architect must perform a structural root-cause analysis of the system failure.

Adaptation Method Matrix

When adaptation is diagnosed as the required intervention, the choice of method must balance task-specific capability requirements against infrastructure and operational constraints.

SFT vs. Parameter-Efficient Adaptation (LoRA)

Full parameter fine-tuning (FFT) updates all parameters of the network by calculating gradients over a target dataset D = {(x_i, y_i)} using the standard auto-regressive cross-entropy loss:
L_SFT(theta) = -sum_(i=1 to |D|) sum_(t=1 to |y_i|) log P_theta(y_i,t | x_i, y_i,<t)
SFT is highly effective for establishing target styles, domain formatting, and structured task patterns. However, full parameter fine-tuning requires keeping optimizer states (e.g., AdamW’s first and second moments) in GPU memory, which consumes roughly six times the memory of the model weights alone.12
To bypass this memory wall, Low-Rank Adaptation (LoRA) freezes the pre-trained weight matrix W_0 in R^(d_out x d_in) and represents the weight update delta_W through a low-rank decomposition 1:
W_adapted = W_0 + delta_W = W_0 + (alpha / r) * B * A
where B in R^(d_out x r) and A in R^(r x d_in) are trainable parameters, r « min(d_in, d_out) is the rank hyperparameter, and alpha is a constant scaling factor that stabilizes training when adjusting r.1 The matrix A is typically initialized from a random Gaussian distribution N(0, sigma^2), and B is initialized to zero, ensuring that delta_W = 0 at the onset of training, which preserves the model’s pre-trained behavior.1
To optimize memory efficiency further, Quantized Low-Rank Adaptation (QLoRA) introduces three core architectural innovations 16:

1. NormalFloat 4 (NF4): An information-theoretically optimal 4-bit quantization scheme engineered specifically for normally distributed parameters.34 The 16 quantization bins are determined by finding the equiprobable quantiles of the standard normal distribution N(0, 1) 34: q_i = 1/2 * [ Q_x(i / 17) + Q_x((i + 1) / 17) ], for i in {0,…, 15} where Q_x(…) is the quantile function of N(0, 1), and the codebook is rescaled to exactly represent zero.34
2. Double Quantization (DQ): The process of quantizing the quantization constants (scales) themselves. QLoRA quantizes 32-bit floating-point scales per block of 64 weights into 8-bit floats with a block size of 256, reducing the memory footprint from 32 bits per block to roughly 8.12 bits per block.16
3. Paged Optimizers: The dynamic offloading of optimizer states to CPU memory during memory spikes, preventing out-of-memory errors on high-batch training steps.

Empirical research establishes a critical performance-regularization trade-off between full fine-tuning (FFT) and LoRA.

• Learning Capacity: Full fine-tuning learns perturbations with an intrinsic rank that is 10x to 100x greater than standard LoRA configurations (r in {8, 16, 32}).7 Consequently, for complex continued pre-training (CPT) on highly technical domains like mathematics or computer programming, LoRA significantly underperforms full fine-tuning because the low-rank constraint restricts the model’s ability to absorb novel linguistic structures and complex representation shifts.7
• Instruction Following: In standard instruction fine-tuning (IFT) regimes (e.g., 10^5 samples), the performance gap between LoRA and FFT can be largely closed by scaling the rank r to higher dimensions (e.g., r >= 64, alpha = 128) and targeting all linear projections in the transformer architecture—including attention projections (W_q, W_k, W_v, W_o) and feedforward block projections (W_gate, W_up, W_down).1
• Generalization and Forgetting: LoRA acts as a powerful regularizer. Because the base model weights W_0 remain frozen, LoRA prevents the model’s pre-trained representation coordinates from collapsing.7 LoRA significantly mitigates catastrophic forgetting of out-of-domain knowledge compared to full fine-tuning and maintains a higher degree of generation diversity, preventing the token distribution from collapsing into a narrow subset of solutions.7
• Hyperparameter Sensitivity: LoRA is highly sensitive to training hyperparameters. The critical parameters are, in order of decreasing sensitivity: Learning Rate -> Target Modules -> Rank (r) and Alpha (alpha).7

Preference Tuning Architectures

Preference tuning shifts the optimization target from imitating a target corpus to maximizing a comparative utility metric. Rather than asking “what token is most likely to follow?”, preference optimization asks “which candidate completion is structurally, semantically, or safety-wise preferred?”.20 This phase is critical for alignment, style control, toxicity reduction, and complex multi-turn conversational patterns.11
Standard preference tuning assumes a dataset of prompt-response pairs D = {(x, y_w, y_l)}, where y_w represents the preferred (chosen) response and y_l represents the dispreferred (rejected) response.2 Traditionally, this was executed via Reinforcement Learning from Human Feedback (RLHF), which requires:

1. Training a binary classification Reward Model r_phi(x, y) on pairwise preferences using the Bradley-Terry objective 2: L_R(r_phi) = -E_(x, y_w, y_l) ~ D [ log sigma( r_phi(x, y_w) - r_phi(x, y_l) ) ]

2. Optimizing the active policy pi_theta(y

   x) against the frozen reward model r_phi using Actor-Critic PPO, while applying a Kullback-Leibler (KL) divergence penalty against a frozen reference policy pi_ref to prevent policy collapse 2:

Objective(theta) =
  E_{x ~ D, y ~ pi_theta}
  [
    r_phi(x, y)
    - beta * D_KL(
        pi_theta(y | x) || pi_ref(y | x)
      )
  ]

Where:
  pi_theta = active policy being optimized
  pi_ref   = frozen reference policy
  r_phi    = learned reward model
  beta     = KL penalty coefficient preventing policy collapse

PPO requires maintaining four distinct models in memory (Actor pi_theta, Critic, Reward r_phi, Reference pi_ref), leading to high training instability, complex hyperparameter tuning, and massive GPU infrastructure requirements.2
To eliminate this complexity, Direct Preference Optimization (DPO) mathematically reparameterizes the Bradley-Terry reward function directly in terms of the policy’s log-probabilities, bypassing the reward model and reinforcement learning loop entirely.2 The DPO loss is defined as 2:
L_DPO(pi_theta; pi_ref) = -E_(x, y_w, y_l) ~ D [ log sigma( beta * log(pi_theta(y_w | x) / pi_ref(y_w | x)) - beta * log(pi_theta(y_l | x) / pi_ref(y_l | x)) ) ]
where beta controls the strength of the implicit KL penalty.2
While computationally elegant, DPO exhibits distinct empirical vulnerabilities:

1. Likelihood Inflation and Overfitting: DPO maximizes the ratio of the chosen completion likelihood to the rejected completion likelihood.23 If left unregulated, the sigmoid loss function saturates quickly. Once the margin is satisfied, the gradient vanishes, allowing the policy to overfit by driving the absolute log-probabilities of both completions down or memorizing superficial patterns.22
2. Degradation of Reasoning: Standard preference optimization pipelines frequently cause significant regressions on reasoning-heavy, mathematical, or code-generation tasks.11 This occurs because preference loss structures treat the output sequence holistically, punishing step-by-step correct rationalizations if the final token diverges slightly, whereas token-level credit assignment (such as GAE in PPO) is required for rigorous reasoning chains.11

To address DPO’s limitations, alternative frameworks have emerged:

Identity Preference Optimization (IPO)

IPO introduces a quadratic regularization term directly over the implicit reward margin to prevent the policy from saturating and over-optimizing to noisy preference labels.22 The loss is formulated as 22:
L_IPO(pi_theta; pi_ref) = E_(x, y_w, y_l) ~ D [ ( beta * log(pi_theta(y_w | x) / pi_ref(y_w | x)) - beta * log(pi_theta(y_l | x) / pi_ref(y_l | x)) - 1 / (2 * beta) )^2 ]
By replacing the log-sigmoid objective with a squared error deviation from a target margin c = 1 / (2 * beta), IPO prevents logit saturation, regularizes the policy throughout training, and enables convergence without the need for early stopping.20

Kahneman-Tversky Optimization (KTO)

KTO bypasses the requirement for paired preference data.20 Based on the asymmetrical value function of Kahneman & Tversky’s prospect theory, KTO posits that humans perceive utility relative to a reference point and are significantly more averse to losses (dispreferred outcomes) than they are pleased by equivalent gains (preferred outcomes).2 KTO operates on independent binary annotations: whether a completion is desirable (y ~ desirable) or undesirable (y ~ undesirable) for a given prompt.2
The KTO loss function is formulated as 2:
L_KTO(pi_theta; pi_ref) = E_(x, y ~ D) [ w(y) * ( 1 - sigma( tau(x, y; beta) ) ) ]
where tau(x, y; beta) = beta * log(pi_theta(y | x) / pi_ref(y | x)) - z(x; beta) represents the margin-adjusted implicit reward, z(x; beta) is a dynamic reference point tracking the expected reward of the current policy, and w(y) is the asymmetrical weighting factor 2:
w(y) = lambda_D if y is desirable, or lambda_U if y is undesirable
To model human loss aversion, the penalty weight for undesirable outcomes is set higher than the reward weight for desirable outcomes (typically lambda_D = 1.0 and lambda_U = 1.33).37 KTO matches or exceeds DPO performance on task capabilities, excels in handling highly imbalanced datasets, and collects data directly from user telemetry (e.g., thumbs-up / thumbs-down logs).2

Training Dataset Design Framework

A model adaptation run is only as robust as the corpus that defines it. Treating training data as an unstructured folder of examples is a severe systemic failure. A production training dataset must be engineered as a structured, governed database with precise taxonomic diversity and quality controls.

Inter-Rater Agreement Math

To ensure training data label consistency prior to optimization, the annotation pipeline must enforce mathematical calibration steps:

• Cohen’s Kappa (kappa): Used to evaluate the degree of agreement between exactly two fixed raters assigning categorical labels 38: kappa = (P_0 - P_e) / (1 - P_e) where P_0 represents the observed proportion of agreement between the raters, and P_e is the expected proportion of agreement due to random chance based on marginal frequencies.38
• Fleiss’ Kappa (kappa): A generalization of Cohen’s Kappa used when three or more fixed raters (m >= 3) assign categorical labels to a set of items, allowing for random rater sampling per item 38: kappa = (bar_P - bar_P_e) / (1 - bar_P_e) where bar_P represents the mean of the rater agreement proportions calculated subject-by-subject, and bar_P_e represents the expected agreement if all raters made completely random assignments.39

A strict engineering protocol dictates that training data collection must be halted and annotation rubrics recalibrated if the inter-rater agreement score falls below kappa = 0.70.38

Synthetic Data Governance Model

When organic training data is sparse, synthetic data generation is a valid strategy to bootstrap datasets. However, naive synthetic generation pipelines run the risk of model collapse and representation decay.5

Adapter and Artifact Management

In multi-tenant enterprise architectures, training a discrete foundation model checkpoint per customer or task is economically prohibitive and introduces severe cold-start latencies. Parameter-efficient multi-adapter serving decouples base model inference from specialization layers, allowing a single high-capacity base model instance in GPU VRAM to dynamically run hundreds of specialized task adapters.8

S-LoRA and Unified Paging

Traditional adapter serving systems allocate static GPU memory pools for adapter parameters and key-value (KV) caches, resulting in extreme memory fragmentation and low GPU utilization.42 S-LoRA introduces Unified Paging, which manages both the KV cache tensors and the variable-rank adapter weights (A, B matrices) in a single unified memory pool using a paged virtual memory architecture.43
Both adapter parameters and KV cache sequences are broken down into fixed-size pages and stored in non-contiguous physical memory pages on the GPU.43 This allows S-LoRA to dynamically load, swap, and execute thousands of specialized adapters on a single base model instance.43 S-LoRA stores all idle adapters in host system memory (DRAM) and dynamically pages active adapter weights into GPU High Bandwidth Memory (HBM) on a token-by-token basis based on incoming request routing.6

Punica and the SGMV Kernel

When a batch of requests arrives at a multi-adapter server, each request may target a different adapter with a unique rank r and sequence length T.43 Executing these computations using standard Basic Linear Algebra Subprograms (BLAS) libraries would require padding the activation tensors to match the maximum rank and sequence length in the batch, causing severe computational waste and low GPU tensor core utilization.6
Punica resolves this by introducing the Segmented Gather Matrix-Vector Multiplication (SGMV) kernel.45 For a batch of heterogeneous decoding requests, the server executes the computationally heavy base model forward pass X * W_0 in a single batched General Matrix Multiply (GEMM) operation.6 Then, the custom SGMV CUDA kernel executes the low-rank adapter computations on-the-fly 6:
Y = X * W_0 + SGMV(X, A, B, s)
where s is a segment vector defining which rows of the batched activation tensor X map to which target adapter parameters.45
SGMV gathers non-contiguous adapter weights directly from the unified memory pool, performs the segmented low-rank projection X * B and the subsequent expansion X * B * A, and accumulates the residual delta directly into the output tensor Y.43 This eliminates padding overhead and enables highly concurrent, heterogeneous multi-adapter decoding batches with millisecond-level execution penalties.45

vLLM Multi-Adapter Serving Engine Parameters:

# vLLM Multi-Adapter Serving Engine Parameters
# Purpose: serve many LoRA adapters against one frozen base model while bounding GPU memory use.

--enable-lora
# Activates the serving engine's LoRA / adapter manager.

--max-loras <N>
# Maximum number of concurrently active LoRA adapters resident in GPU HBM.

--max-lora-rank <R>
# Maximum supported LoRA rank. Pre-allocates adapter buffers sized to this ceiling.
# Requests requiring rank > R must be rejected or routed to a compatible serving pool.

--max-cpu-loras <N>
# Maximum number of adapters staged in CPU DRAM as an intermediate cache tier.

--lora-modules <adapter_name>=<adapter_path> [...]
# Registers named adapter artifacts and their storage paths for route-time selection.

--fully-sharded-loras
# Optional. Shards adapter computation across tensor-parallel workers for larger adapters.

--max-model-len <TOKENS>
# Bounds context length so KV cache and adapter paging remain within the memory envelope.

--gpu-memory-utilization <0.0-1.0>
# Reserves a safe fraction of GPU memory for model weights, KV cache, adapter pages, and overhead.

Adapter Management Model

To prevent deployment errors, adapter artifacts must be registered alongside precise base model dependencies and structural assumptions.

Adaptation Card Schema

Every adaptation run must generate a structured, immutable Adaptation Card (or Adapter Decision Record) that serves as the metadata package required for production lifecycle handover.18

System Metadata & Registry Coordinates

• Artifact URI: s3://ai-engine-registry/adapters/Llama-3-8B-Code-Extractor-v2.3.safetensors
• Base Checkpoint Hash: sha256:e12a439b83b9c70010f3db1c521852ba7791ccda1b4b1cf93bc51bcf6a17b1bf 18
• Registry Coordinate: production.core-models.llama3-8b.adapters.extractor-v2-3 8
• Owner / Escalation Point: Core AI Platform Infrastructure Team — On-Call Alert ID: AI-INFRA-SEV1

System Objective & Justified Interventions

• Failure Class: True Adaptation Gap.7
• Root-Cause Diagnosis: The baseline model exhibits a 14.2% failure rate when parsing complex nested COBOL records into standardized JSON structures, driven by context bloat and stylistic inconsistencies under few-shot prompting.
• Lighter Steering Alternatives Rejected:
  1. Prompt Engineering: Rejected due to high token overhead (>1,800 tokens per prompt), which increased TTFT by 320 ms and degraded system margins by $0.12 per transaction.5
  2. Structured Outputs (Inference-time constraint): Constrained sampling was retained as an active layer but was insufficient on its own because the base model lacked semantic understanding of COBOL structures, leading to a high rate of logically incorrect field mapping.17

Dataset Lineage & Calibration

• Sourcing Coordinates: dataset-saas-cobol-parser-v2.3 (Sourced via anonymized production transaction logs).18
• Quality Metrics: 42,500 paired sequences, de-duplicated using MinHash LSH at threshold 0.85.5
• Inter-Rater Consensus: Double-blind expert review achieved an overall Fleiss’ Kappa score of kappa = 0.84 across 5 concurrent raters.38
• PII & Secrets Verification: Passed through Microsoft Presidio redaction engine with 100% zero-leak check.4

Training Configuration

• Optimization Framework: Axolotl v0.4.1 running PyTorch 2.3.1 with CUDA 12.1.
• Hyperparameter Settings:
  • Adapter Method: LoRA.7

  • Rank (r): 16

    Alpha (alpha): 32.14

  • Learning Rate: 2 * 10^(-4) with Cosine Decay.7
  • Target Modules: q_proj, v_proj, k_proj, o_proj, gate_proj, up_proj, down_proj.1

  • Batch Size: 128 (Gradient Accumulation Steps: 4)

    Epochs: 3.26

Validation Results & Regression Safety

• Target Task Success Rate: Improved parsing accuracy from 85.8% (baseline) to 98.4% on the held-out validation set.
• General Capability Impact:

  • MMLU Score Delta: -0.4% (baseline: 66.2%

    post-adaptation: 65.8%).4

  • HumanEval Delta: +0.2% (demonstrates general code syntax preservation).31
• Format Adherence Rate: 100% valid JSON syntax over 10,000 simulated test cases.17
• Jailbreak Refusal Preservation: 100% safety preservation when evaluated against the internal Red-Teaming Jailbreak Suite.4

Serving & Deployment Constraints

• Serving Engine: vLLM v0.4.2.8
• HBM Allocation Profile: Pre-allocate dynamic LoRA buffers to –max-lora-rank 16.8
• Co-Existence Restrictions: May stack alongside system-wide safety guardrail adapters, but cannot stack alongside other custom customer-facing parser adapters.
• P95 Latency SLA Limit: 85 ms TTFT at concurrency 32.
• Rollback & Reconsideration Triggers:
  1. Rollback Trigger: Prompt-level p95 latency exceeds 120 ms or runtime syntax exception rate exceeds 0.1% over a 100-request sliding window.8
  2. Reconsideration Condition: If the underlying COBOL-to-JSON database schema expands to support generic mainframe record updates, retrain the adapter or migrate the behavior logic back to dynamic context-space compilation.

Distillation Economics Model

Model distillation is an architectural and economic intervention designed to transition expensive, general-purpose capabilities from a parameter-heavy teacher model into a compact, low-latency student model optimized for a production task envelope.5

Distillation Scaling Laws

The performance of a distilled student model is governed by rigorous scaling laws parameterized by the student size (N_S), distillation token volume (D_S), and the teacher’s capability represented by its cross-entropy loss (L_T).51 Research by Busbridge et al. models the student’s cross-entropy loss as a scaling-law function of student size, distillation token volume, and teacher quality:

L_S(N_S, D_S, L_T) = L_infinity + A * N_S^(-alpha) + B * D_S^(-beta) + phi(L_T, N_S)

The term phi(L_T, N_S) represents the capacity-gap penalty. Under ordinary conditions, a stronger teacher lowers student loss. However, when the teacher’s distribution is too complex for the student’s parameter budget, the student cannot represent the teacher’s behavior faithfully, producing a U-shaped degradation curve rather than monotonic improvement.

The term phi(L_T, N_S) models the Capacity Gap.32 Under standard scaling, a stronger teacher (lower L_T) yields a stronger student (lower L_S).51 However, if the teacher’s capacity is excessively high relative to the student’s parameter limit N_S, a representation mismatch occurs.32 The student model lacks the representational hypothesis space required to model the highly complex, multi-modal logit distributions of the teacher, resulting in optimization instability and an actual degradation of student performance (forming a U-shaped capacity gap curve).32
Furthermore:

• Given infinite compute or token budgets, direct supervised learning from ground truth always matches or outperforms distillation.51
• Distillation is highly compute-efficient under two distinct operational conditions: (1) a pre-trained teacher model already exists, or (2) the training run intends to generate a fleet of diverse student models from a single teacher investment.32
• If only a single student model is required and a teacher model must be trained from scratch first, directly allocating the entire compute budget to supervised pre-training of the student is mathematically superior.51

Distillation Economics Cost Function

To justify distillation, the system architect must evaluate the total lifecycle cost function of the student model relative to the teacher baseline.
C_total(V) = C_generation + C_training + C_evaluation + C_infrastructure + V * T * P_student
C_teacher_baseline(V) = V * T * P_teacher
where V represents the total projected inference request volume, T is the average token footprint per request, P_student is the inference price per token of the student, and P_teacher is the inference price per token of the teacher.5

Distillation Break-Even Volumetric Equation:
V_break_even = (C_generation + C_training + C_evaluation + C_infrastructure) / (T * (P_teacher - P_student))

Volumetric Scenario Comparison

Adaptation Regression and Safety Evaluation Plan

Every model adaptation pipeline must pass through a multi-stage validation suite prior to staging or production deployment to protect against negative transfer, capability regression, and safety decay.

+------------------------------------------------------------------------------------------------+
|                       ADAPTATION REGRESSION AND SAFETY EVALUATION PLAN                         |
+------------------------------------------------------------------------------------------------+
|                                                                                                
|  Goal: prove that an adapted checkpoint or adapter improves the target behavior without        
|  degrading general capability, safety, privacy, schema reliability, or serving stability.      
|                                                                                                
|  [ Adaptation Candidate: SFT checkpoint / LoRA adapter / preference-tuned model / student ]    
|                                      |                                                                                               v                                                         |
|  +------------------------------------------------------------------------------------------+  
|  |  1. Baseline Calibration Sweep                                                           |  
|  |                                                                                          |  
|  |  Run the frozen base model through the same target, safety, schema, retrieval, and tool  |  
|  |  tests. Establish baseline accuracy, latency, refusal behavior, and failure profile.     |  
|  +---------------------------------------------+--------------------------------------------+  
|                                                |                                               
|                                                v                                               
|  +------------------------------------------------------------------------------------------+  
|  |  2. Artifact Compatibility Gate                                                          |  
|  |                                                                                          |  
|  |  Verify base checkpoint hash, tokenizer version, adapter rank, target modules, schema    |  
|  |  config, serving engine version, merge policy, tenant scope, and rollback path.          |  
|  +---------------------------------------------+--------------------------------------------+  
|                                                |                                               
|                                      [ Compatible? ]                                           
|                                      /            \                                            
|                               Yes   /              \   No                                      
|                                    v                v                                          
|                      [ Continue Evaluation ]     [ Reject Artifact / Repair Registry ]         
|                                    |                                                           
|                                    v                                                           
|  +------------------------------------------------------------------------------------------+  
|  |  3. Held-Out Target Behavior Evaluation                                                  |  
|  |                                                                                          |  
|  |  Test unseen target examples. Measure exact-match accuracy, task success rate, schema    |  
|  |  validity, tool-call correctness, output style, and domain-specific acceptance criteria. |  
|  +---------------------------------------------+--------------------------------------------+  
|                                                |                                               
|                                      [ Target Gain >= Threshold? ]                             
|                                      /                      \                                  
|                               Yes   /                        \   No                            
|                                    v                          v                                
|                      [ Continue Evaluation ]        [ Reject / Re-train / Re-scope Dataset ]   
|                                    |                                                           
|                                    v                                                           
|  +------------------------------------------------------------------------------------------+  
|  |  4. Structured Output and Tool-Use Verification                                          |  
|  |                                                                                          |  
|  |  Run high-volume parser tests, tool simulations, constrained decoding checks, sandboxed  |  
|  |  compiler runs, API argument validation, and malformed-input recovery tests.             |  
|  +---------------------------------------------+--------------------------------------------+  
|                                                |                                               
|                                      [ Boundary Valid? ]                                       
|                                      /            \                                            
|                               Yes   /              \   No                                      
|                                    v                v                                          
|                      [ Continue Evaluation ]     [ Enable Constraints / Fix Training Data ]    
|                                    |                                                           
|                                    v                                                           
|  +------------------------------------------------------------------------------------------+  
|  |  5. General Capability Preservation                                                      |  
|  |                                                                                          |  
|  |  Run general reasoning, math, coding, instruction-following, multilingual, and retrieval |  
|  |  tests. Compare against baseline. Maximum tolerated decay: usually <= 2%.                |  
|  +---------------------------------------------+--------------------------------------------+  
|                                                |                                               
|                                      [ Regression Within Limit? ]                              
|                                      /                      \                                  
|                               Yes   /                        \   No                            
|                                    v                          v                                
|                      [ Continue Evaluation ]        [ Reject / Lower Rank / Add Rehearsal Set] 
|                                    |                                                           
|                                    v                                                           
|  +------------------------------------------------------------------------------------------+  
|  |  6. Safety, Refusal, and Adversarial Red-Team Sweep                                      |  
|  |                                                                                          |  
|  |  Test jailbreak resistance, unsafe capability shift, refusal accuracy, benign false      |  
|  |  refusals, prompt injection exposure, and policy-boundary preservation.                  |  
|  +---------------------------------------------+--------------------------------------------+  
|                                                |                                               
|                                      [ Safety Preserved? ]                                     
|                                      /            \                                            
|                               Yes   /              \   No                                      
|                                    v                v                                          
|                      [ Continue Evaluation ]     [ Immediate Rejection / Safety Review ]       
|                                    |                                                           
|                                    v                                                           
|  +------------------------------------------------------------------------------------------+  
|  |  7. Privacy Leakage and Memorization Audit                                               |  
|  |                                                                                          |  
|  |  Probe for memorized training records, PII, secrets, tenant schemas, proprietary code,   |  
|  |  benchmark contamination, and exact-sequence reproduction.                               |  
|  +---------------------------------------------+--------------------------------------------+  
|                                                |                                               
|                                      [ Leakage Detected? ]                                     
|                                      /            \                                            
|                                No   /              \   Yes                                     
|                                    v                v                                          
|                      [ Continue Evaluation ]     [ Reject / Redact / Deduplicate / Retrain ]   
|                                    |                                                           
|                                    v                                                           
|  +------------------------------------------------------------------------------------------+  
|  |  8. Shadow Deployment                                                                    |  
|  |                                                                                          |  
|  |  Mirror live traffic to baseline and adapted artifact. Do not expose users to candidate  |  
|  |  outputs. Log divergence, latency, cost, tool errors, refusals, and operator overrides.  |  
|  +---------------------------------------------+--------------------------------------------+  
|                                                |                                               
|                                      [ Shadow Metrics Pass? ]                                  
|                                      /                    \                                    
|                               Yes   /                      \   No                              
|                                    v                        v                                  
|                      [ Continue to Canary ]        [ Hold Release / Investigate Drift ]        
|                                    |                                                           
|                                    v                                                           
|  +------------------------------------------------------------------------------------------+  
|  |  9. Canary Rollout and Automated Rollback                                                |  
|  |                                                                                          |  
|  |  Roll out progressively: 1% -> 10% -> 50% -> 100%. Roll back automatically on SLA breach,|  
|  |  exception spike, safety incident, schema failure, latency regression, or quality decay. |  
|  +---------------------------------------------+--------------------------------------------+  
|                                                |                                               
|                                                v                                               
|  +------------------------------------------------------------------------------------------+  
|  |  10. Production Registration and Monitoring                                              |  
|  |                                                                                          |  
|  |  Register signed Adaptation Card, dataset hashes, eval results, artifact URI, rollback   |  
|  |  target, monitoring thresholds, and reconsideration conditions.                          |  
|  +------------------------------------------------------------------------------------------+  
|                                                                                                
+------------------------------------------------------------------------------------------------+
| Rule: no adapted model enters production merely because target-task accuracy improved. It must |
| also preserve safety, privacy, general capability, serving stability, and rollback readiness.  |
+------------------------------------------------------------------------------------------------+

1. Baseline Calibration Sweep: Prior to fine-tuning, run the base model through prompt-engineered, context-grounded, and validation scenarios to establish a performance baseline.
2. Held-Out Target Task Evaluation: Assess the candidate on a partitioned, unseen test dataset of target tasks.5 The task success rate must equal or exceed the baseline model.5
3. Syntactic & Structured-Output Validation: Run 10,000 consecutive generations against strict schema parsers.18 The adapter must maintain a 100% valid format adherence rate.18
4. Retrieval-Grounded & Tool-Use Verification: Evaluate the model’s command adherence inside active tool harnesses.47 The model must call target tools with correct parameter types and structures.47
5. General-Capability Preservation Test: Run general capability evaluations (e.g., MMLU for factual density, GSM8K for logical consistency, and HumanEval for code syntax).7 The candidate must display less than a 2% decay across all general benchmarks.4
6. Adversarial and Jailbreak Red-Teaming: Execute automated adversarial prompt sweeps, injection attacks, and safety tests.4 The candidate must not display unsafe capability shifts or bypass established safety boundaries.4
7. Privacy Leakage and Redaction Audit: Run semantic probing to extract potentially memorized private schemas or training data PII. The model must refuse to reveal secure training boundaries.4
8. Shadow Deployment Verification: Route production traffic concurrently to both the baseline model and the adapted model.4 Log output divergence, latency deltas, and token throughput limits without exposing users to adapter errors.
9. Canary Rollout and Automated Rollback: Roll out the adapter to 1% of live traffic, scaling progressively to 10%, 50%, and 100%. Automatically trigger rollback to the base model if the p95 latency exceeds the SLA by >15 ms or if exception rates spike.8

Adaptation Failure Mode Map

Weight optimization carries severe operational and systemic risks that can compromise downstream behavior.

+------------------------------------------------------------------------------------------------+
|                                  ADAPTATION FAILURE MODE MAP                                   | 
+------------------------------------------------------------------------------------------------+
|                                                                                                
|  Weight optimization changes behavior by altering parameters or logit preferences. Every gain  
|  can create an adjacent failure unless regression, safety, and privacy boundaries are tested.  
|                                                                                                
|  [ Adaptation Intervention ]                                                                   
|       |                                                                                        
|       |  SFT | LoRA | QLoRA | Preference Tuning | Domain Adaptation | Distillation             
|       v                                                                                        
|  +------------------------------------------------------------------------------------------+  
|  |                              Training Data and Objective Risks                           |
|  +----------------------+----------------------+----------------------+---------------------+
|                         |                      |                      |                      
|                         v                      v                      v                      
|          +-------------------------+  +-----------------------+  +-----------------------------+
|          | Label-Noise Amplif.     |  | Synthetic Monoculture |  | Benchmark Contamination     |
|          |                         |  |                       |  |                             |
|          | inconsistent labels     |  | repetitive teacher    |  | eval leakage inflates       |
|          | become model behavior   |  | style collapses       |  | apparent performance        |
|          +-----------+-------------+  +-----------+-----------+  +-------------+---------------+
|                      |                            |                            |                
|                      v                            v                            v                
|          [ stricter annotation ]      [ teacher diversity ]       [ contamination sweep ]       
|                                                                                                
|  +------------------------------------------------------------------------------------------+  
|  |                              Parameter and Capability Risks                              |
|  +----------------------+----------------------+----------------------+---------------------+
|                         |                      |                      |                      
|                         v                      v                      v                      
|          +-------------------------+  +-----------------------+  +-----------------------------+
|          | Catastrophic Forgetting |  | Domain Tunnel Vision  |  | Overconfident Hallucination |
|          |                         |  |                       |  |                             |
|          | general reasoning, math |  | narrow task success   |  | false labels become high-   |
|          | or world knowledge drops|  | but brittle transfer  |  | confidence completions      |
|          +-----------+-------------+  +-----------+-----------+  +-------------+---------------+
|                      |                            |                            |                
|                      v                            v                            v                
|          [ rehearsal sets ]          [ prompt diversity tests ]    [ data validation + abstain ]   
|                                                                                                
|  +------------------------------------------------------------------------------------------+  
|  |                              Format, Tool, and Interface Risks                           |
|  +----------------------+----------------------+----------------------+---------------------+
|                         |                      |                      |                      
|                         v                      v                      v                      
|          +-------------------------+  +-----------------------+  +-----------------------------+
|          | Format Regression       |  | Tool-Use Regression   |  | Prompt-Template Dependence  |
|          |                         |  |                       |  |                             |
|          | JSON/XML/schema drift   |  | wrong tool args, API  |  | only works with training    |
|          | after adaptation        |  | misuse, state errors  |  | prompt syntax               |
|          +-----------+-------------+  +-----------+-----------+  +-------------+---------------+
|                      |                            |                            |                
|                      v                            v                            v                
|          [ constrained decoding ]     [ tool harness tests ]       [ template perturbation ]    
|                                                                                                
|  +------------------------------------------------------------------------------------------+  
|  |                              Safety and Privacy Risks                                    |
|  +----------------------+----------------------+----------------------+---------------------+
|                         |                      |                      |                      
|                         v                      v                      v                           
|          +-------------------------+  +-----------------------+  +-----------------------------+  
|          | Refusal Drift           |  | Unsafe Capability     |  | Privacy / Tenant Leakage    |
|          |                         |  | Shift                 |  |                             |
|          | benign requests refused |  | safety barriers decay |  | PII, secrets, tenant data   |
|          | from over-alignment     |  | after tuning          |  | memorized into weights      |
|          +-----------+-------------+  +-----------+-----------+  +-------------+---------------+
|                      |                            |                            |                
|                      v                            v                            v                
|          [ contrastive safety ]       [ red-team retention ]       [ redaction + isolation ]    
|                                                                                                
|  +------------------------------------------------------------------------------------------+  
|  |                              Serving and Artifact Risks                                  |
|  +----------------------+----------------------+----------------------+---------------------+
|                         |                      |                      |                      
|                         v                      v                      v                      
|          +-------------------------+  +-----------------------+  +-----------------------------+
|          | Adapter Incompatibility |  | Stale-Knowledge       |  | Merge / Stack Conflict      |
|          |                         |  | Fossilization         |  |                             |
|          | rank, tokenizer, base   |  | dynamic facts encoded |  | incompatible adapters       |
|          | hash, module mismatch   |  | into weights          |  | combined at runtime         |
|          +-----------+-------------+  +-----------+-----------+  +-------------+---------------+
|                      |                            |                            |                
|                      v                            v                            v                
|          [ registry verification ]    [ keep facts in RAG ]       [ merge/stack policy gate ]   
|                                                                                                
+------------------------------------------------------------------------------------------------+
| Doctrine: adaptation should specialize stable behavior, not smuggle volatile knowledge, private|
| data, or unvalidated preferences into weights. Every adapter needs a rollback path.            |
+------------------------------------------------------------------------------------------------+

Style Overfit

• Target Task Symptom: The model produces extremely verbose, formulaic, or repetitive responses, echoing training length and syntax templates instead of actual task semantics.23
• Root-Cause Mechanism: The cross-entropy loss function overfitted to superficial formatting patterns and sequence length distributions present in the training corpus.4
• Mitigation Protocol: Enforce length penalties during generation; incorporate strict length limits in the training loss; mix in diverse formatting examples.

Memorization

• Target Task Symptom: The model reproduces exact sequences, examples, or training document paragraphs verbatim under minimal prompting, risking private data leakage.4
• Root-Cause Mechanism: High parameter density training on redundant datasets with excessive epochs led to parameter memorization of training instances.26
• Mitigation Protocol: Deduplicate training data using MinHash LSH; apply weight decay; restrict epoch counts; employ DP-SGD.

Catastrophic Forgetting

• Target Task Symptom: The model’s general logical reasoning, world knowledge, mathematical capacity, or multi-task adherence scores collapse post-adaptation.13
• Root-Cause Mechanism: Specialized task gradients overrode the model’s pre-trained deep representation coordinates, destroying general capabilities.3
• Mitigation Protocol: Use Selective Token Masking (STM) to mask high-perplexity tokens 2; mix in 20% general pre-training data 3; use LoRA with tight ranks.7

Format Regression

• Target Task Symptom: The model fails to adhere to system format constraints (e.g., outputs malformed JSON or raw Markdown instead of valid XML).
• Root-Cause Mechanism: Specialized domain training eroded the pre-trained instruction-following and system-prompt constraint layers.4
• Mitigation Protocol: Integrate deterministic structured sampling engines 17; include format-adherence task examples in the training corpus.4

Tool-Use Regression

• Target Task Symptom: The model fails to call tool APIs, mixes up argument parameters, or generates invalid API formats.47
• Root-Cause Mechanism: Representation drift in the token embedding space decayed the model’s pre-trained tool-use execution structures.47
• Mitigation Protocol: Ensure the training dataset contains explicit tool-calling formats and correct parameters; run fine-grained tool tests.47

Refusal Drift

• Target Task Symptom: The model becomes hyper-conservative, refusing benign requests containing completely harmless, adjacent keywords.4
• Root-Cause Mechanism: Unbalanced safety or preference tuning over-penalized safety-adjacent boundaries without contrastive examples.4
• Mitigation Protocol: Mix contrastive pairs into preference tuning showing correct refusal vs. correct execution; align with KTO prospect loss.2

Unsafe Capability Shift

• Target Task Symptom: The model begins generating toxic content, bypasses pre-trained jailbreak constraints, or leaks secure boundaries under adversarial queries.4
• Root-Cause Mechanism: Specialized domain adaptation inadvertently dismantled pre-trained safety barriers or alignment bounds.4
• Mitigation Protocol: Include safety retention sets in training data 4; execute continuous red-teaming checks during candidate validation.21

Overconfident Hallucination

• Target Task Symptom: The model generates highly convincing but structurally incorrect or fabricated facts with high probability.4
• Root-Cause Mechanism: SFT training on noisy data with incorrect labels forced the model’s output probabilities to align with falsehoods.
• Mitigation Protocol: Enforce strict data validation; train the model to output “I do not know” or escalate when uncertainty is high.4

Domain Tunnel Vision

• Target Task Symptom: The model becomes highly capable on narrow target tasks but displays an inability to adapt to slight variations in prompt style or task framing.4
• Root-Cause Mechanism: The model over-optimized to the specific prompt templates and domain language of the training set.
• Mitigation Protocol: Perturb prompt templates dynamically during SFT; train on diverse multi-task datasets.

Prompt-Template Dependence

• Target Task Symptom: The adapted model performs the task correctly only when prompted with the exact syntax used in training; minor changes degrade accuracy.18
• Root-Cause Mechanism: The model coupled its behavioral changes to the surface features of the training templates instead of the underlying concepts.18
• Mitigation Protocol: Enforce prompt template diversity in the training corpus; run zero-shot prompt robustness tests during validation.

Synthetic Monoculture

• Target Task Symptom: The model outputs repetitive, formulaic transitions (e.g., “delve”, “testament”) and displays a collapse of its logit entropy.26
• Root-Cause Mechanism: Continuous training on synthetic data generated by a single model family caused the student’s logit diversity to decay.5
• Mitigation Protocol: Perturb teacher model prompts dynamically 5; mix in at least 20% organic human-written text; run semantic diversity sweeps.26

Label-Noise Amplification

• Target Task Symptom: The model outputs inconsistent syntax, incorrect structures, and conflicting definitions under normal operations.
• Root-Cause Mechanism: The SFT loss function amplified minor, systematic labeling errors present in the training dataset.5
• Mitigation Protocol: Enforce rigorous data validation and rater consensus using Fleiss’ Kappa before training.38

Benchmark Contamination

• Target Task Symptom: The model displays artificially inflated 99% accuracy scores on public benchmarks but fails simple, real-world task variations.26
• Root-Cause Mechanism: Public evaluation benchmarks leaked verbatim or in near-identical form into the pre-tuning or fine-tuning datasets.26
• Mitigation Protocol: Run contamination sweeps using the Kernel Divergence Score (KDS) before training.29

Stale-Knowledge Fossilization

• Target Task Symptom: The model output is structurally correct but references outdated policies, broken URLs, or dynamic facts that have changed in production.4
• Root-Cause Mechanism: Volatile factual knowledge was encoded into the parameters instead of being managed in retrieval or context space.3
• Mitigation Protocol: Keep dynamic facts out of weights; decouple behavioral formatting from information retrieval.

Privacy Leakage

• Target Task Symptom: Under semantic probing or adversarial extraction, the model leaks training-data PII, API tokens, or tenant schemas.4
• Root-Cause Mechanism: Failure to redact sensitive data prior to SFT, allowing the model to memorize PII within its weights.4
• Mitigation Protocol: Enforce strict Presidio-based PII scrubbing; run adversarial extraction checks before staging.

Tenant Bleed

• Target Task Symptom: User B receives custom outputs displaying private data, stylistic quirks, or schemas belonging to User A.3
• Root-Cause Mechanism: Multi-tenant data records were co-mingled in the same physical checkpoint or adapter during fine-tuning.3
• Mitigation Protocol: Restrict multi-tenant training to isolated physical LoRA adapters; serve them on a per-tenant basis.3

Adapter Incompatibility

• Target Task Symptom: Swapping an adapter at runtime causes the model to output structural gibberish, crash, or enter infinite token generation loops.8
• Root-Cause Mechanism: The adapter config (rank, target modules, tokenizer special tokens) mismatched the host base model.8
• Mitigation Protocol: Enforce sha256 base model hash verification; freeze tokenizer versions across the adapter serving pool.18

Evaluation and Observability Guidance

An adapted model in production requires systematic runtime monitoring. The dashboard below details the critical telemetry indicators that track the operational health and performance of adaptation deployments:

Cross-Canon Handoff Map

The model adaptation lifecycle interacts directly with surrounding architectural layers across The AI Engineering Systems Canon:

Doctrinal Principles for Model Adaptation

1. Rule of the Diagnostic: Never train a model to fix a problem that can be resolved with deterministic structured outputs, schema constraints, or harness-level validations. Always name the system failure before choosing the adaptation instrument.
2. Factual Separation: Keep dynamic knowledge, tenant-private data, permissions, and real-time facts in the context space. Modify weights only to specialize style, enforce structures, or master stable procedural transformations.
3. LoRA by Default: For behavioral adaptation and instruction following, prefer parameter-efficient adapters (LoRA/QLoRA) over full fine-tuning. They act as powerful regularizers, preserve pre-trained capabilities, and prevent representational collapse.
4. Data Over Compute: A model adaptation run is only as robust as the curation, balance, and quality of its dataset. Prioritize rigorous annotation rubrics and multi-rater agreement validation over hyperparameter optimization.
5. No Alignment Without Safety: Preference alignment (DPO, IPO, KTO) is an alignment tool, not a reasoning enhancer. When tuning logits to match comparative styles, always run comprehensive regression sweeps to guard against catastrophic reasoning decay and safety regressions.
6. Decoupled Architecture: In multi-tenant environments, never train customer-specific records into a shared base model. Enforce customer isolation by serving dynamically loaded, single-tenant LoRA adapters on a shared read-only base model.

Works cited

1. LoRA: Low-Rank Adaptation of Large Language Models - arXiv, accessed June 8, 2026, https://arxiv.org/html/2106.09685v2
2. KTO: Model Alignment as Prospect Theoretic Optimization - arXiv, accessed June 8, 2026, https://arxiv.org/pdf/2402.01306
3. How are you handling catastrophic forgetting in multi-domain LLM fine-tuning pipelines?, accessed June 8, 2026, https://www.reddit.com/r/mlops/comments/1rnhoa6/how_are_you_handling_catastrophic_forgetting_in/

4. Avoiding Amnesia: Some Practical Guides to Mitigate Catastrophic Forgetting in LLMs Post-training

   by Baicen Xiao

   Medium, accessed June 8, 2026, https://medium.com/@baicenxiao/avoiding-amnesia-some-practical-guides-to-mitigate-catastrophic-forgetting-in-llms-post-training-6a23e4f064cb

5. Teacher-Student Distillation: How It Works and When to Use It - distil labs, accessed June 8, 2026, https://www.distillabs.ai/learn/teacher-student-distillation/

6. Serving Thousands of Concurrent LoRA Adapters

   by Mukul Ranjan - Medium, accessed June 8, 2026, https://medium.com/@mukulranjan/serving-thousands-of-concurrent-lora-adapters-6b407e8df516

7. LoRA Learns Less and Forgets Less - arXiv, accessed June 8, 2026, https://arxiv.org/html/2405.09673v2
8. LoRA Multi-Adapter Serving: Fine-Tune Once, Serve 100 Customers on One GPU - Spheron, accessed June 8, 2026, https://www.spheron.network/blog/lora-multi-adapter-serving-gpu-cloud/

9. Model Distillation and Knowledge Transfer in AI 2026

   Zylos Research, accessed June 8, 2026, https://zylos.ai/research/2026-02-08-model-distillation

10. 𝛼-LoRA: Effective Fine-Tuning via Base Model Rescaling - arXiv, accessed June 8, 2026, https://arxiv.org/html/2510.21345v1
11. All Knowledge You Need about DPO and its Variants - arXiv, accessed June 8, 2026, https://arxiv.org/html/2404.14723v2

12. DPO vs PPO: Which RLHF Algorithm to Use for Production LLM Alignment (2026 Decision Guide)

    Spheron Blog, accessed June 8, 2026, https://www.spheron.network/blog/dpo-vs-ppo-rlhf-algorithm-production-llm-alignment/

13. What is Catastrophic Forgetting? - IBM, accessed June 8, 2026, https://www.ibm.com/think/topics/catastrophic-forgetting
14. How Much is Too Much? Exploring LoRA Rank Trade-offs for Retaining Knowledge and Domain Robustness - arXiv, accessed June 8, 2026, https://arxiv.org/html/2512.15634v1
15. LoRA Adapters: Efficient Fine-Tuning - Emergent Mind, accessed June 8, 2026, https://www.emergentmind.com/topics/low-rank-adaptation-lora-adapters

16. Crafty Patchwork: Parameter-Efficient Fine-Tuning

    Vectors & Verbs, accessed June 8, 2026, https://vectorsandverbs.com/posts/parameter-efficient-fine-tuning/

17. Quantization in Large Language Models(LLMs) - Intelligent Machines, accessed June 8, 2026, https://www.intelligentmachines.blog/post/quantization-in-large-language-models-llms
18. Deploy multi-LoRA adapters on LLMs - Anyscale Docs, accessed June 8, 2026, https://docs.anyscale.com/llm/serving/multi-lora
19. Saliency-Aware Model Merging - arXiv, accessed June 8, 2026, https://arxiv.org/html/2606.00511v1
20. Preference Tuning LLMs with Direct Preference Optimization Methods - Hugging Face, accessed June 8, 2026, https://huggingface.co/blog/pref-tuning
21. A-IPO: Adaptive Intent-driven Preference Optimization - arXiv, accessed June 8, 2026, https://arxiv.org/html/2510.10077v1
22. Identity Preference Optimization (IPO) - Emergent Mind, accessed June 8, 2026, https://www.emergentmind.com/topics/identity-preference-optimization-ipo
23. Reward Modeling and DPO: Learning What “Good” Means - Suvash Sedhain, accessed June 8, 2026, https://mesuvash.github.io/blog/2026/reward-modeling/
24. Vinija’s Notes • LLM Alignment, accessed June 8, 2026, https://vinija.ai/concepts/llm-alignment/

25. Regarding DPO, IPO, and KTO. DPO method

    by Mohammad Reza Esmaeiliyan - Medium, accessed June 8, 2026, https://esmln.medium.com/regarding-dpo-ipo-and-kto-02e94e6958ed

26. When Benchmarks Lie: Why Contamination Breaks LLM Evaluation, accessed June 8, 2026, https://thegrigorian.medium.com/when-benchmarks-lie-why-contamination-breaks-llm-evaluation-1fa335706f32
27. From Teacher to Student: Model Distillation for Cost-Effective LLM Deployment - Marvik.ai, accessed June 8, 2026, https://www.marvik.ai/blog/from-teacher-to-student-model-distillation-for-cost-effective-llm-deployment
28. How to Alleviate Catastrophic Forgetting in LLMs Finetuning? Hierarchical Layer-Wise and Element-Wise Regularization - arXiv, accessed June 8, 2026, https://arxiv.org/html/2501.13669v2
29. How Contaminated Is Your Benchmark? Quantifying Dataset Leakage in Large Language Models with Kernel Divergence - arXiv, accessed June 8, 2026, https://arxiv.org/html/2502.00678v1
30. What Matters for Model Merging at Scale? - OpenReview, accessed June 8, 2026, https://openreview.net/forum?id=HW26XyHp3P
31. LoRA Learns Less and Forgets Less - arXiv, accessed June 8, 2026, https://arxiv.org/html/2405.09673v1
32. Everything You Need to Know about Knowledge Distillation - Hugging Face, accessed June 8, 2026, https://huggingface.co/blog/Kseniase/kd
33. ICML Poster Distillation Scaling Laws, accessed June 8, 2026, https://icml.cc/virtual/2025/poster/46615
34. NormalFloat-4 (NF4): 4-bit Quantization - Emergent Mind, accessed June 8, 2026, https://www.emergentmind.com/topics/normalfloat-4-nf4
35. 4-bit NormalFloat (NF4) Quantization - Emergent Mind, accessed June 8, 2026, https://www.emergentmind.com/topics/4-bit-normalfloat-nf4-quantization
36. RLHF and alternatives: IPO - Argilla, accessed June 8, 2026, https://argilla.io/blog/mantisnlp-rlhf-part-6/

37. Kahneman-Tversky Optimization(KTO): Revolutionizing Language Model Training with Prospect Theory

    by Yatin Arora

    Medium, accessed June 8, 2026, https://medium.com/@SpielmitDaten/kahneman-tversky-optimization-kto-revolutionizing-language-model-training-with-prospect-theory-99f30c50481e

38. A Tutorial on Sample Size Calculation for Inter-rater and Intra-rater Agreement Studies - PMC, accessed June 8, 2026, https://pmc.ncbi.nlm.nih.gov/articles/PMC12935580/

39. Fleiss’ Kappa

    Real Statistics Using Excel, accessed June 8, 2026, https://real-statistics.com/reliability/interrater-reliability/fleiss-kappa/

40. Fleiss’ Kappa: Measuring Agreement Among Multiple Raters - Numiqo, accessed June 8, 2026, https://numiqo.com/tutorial/fleiss-kappa
41. Fleiss’s kappa - Wikipedia, accessed June 8, 2026, https://en.wikipedia.org/wiki/Fleiss%27s_kappa
42. Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management - arXiv, accessed June 8, 2026, https://arxiv.org/html/2505.03756v1
43. 1 Introduction - arXiv, accessed June 8, 2026, https://arxiv.org/html/2311.03285v3
44. S-LoRA: Scalable LoRA Serving - Emergent Mind, accessed June 8, 2026, https://www.emergentmind.com/topics/scalable-serving-s-lora-system
45. Multi-Tenant LoRA Serving - Punica - arXiv, accessed June 8, 2026, https://arxiv.org/pdf/2310.18547
46. [Tracking] Multi-LoRA Serving · Issue #3446 · mlc-ai/mlc-llm - GitHub, accessed June 8, 2026, https://github.com/mlc-ai/mlc-llm/issues/3446
47. punica_wrapper - vLLM Documentation, accessed June 8, 2026, https://docs.vllm.ai/en/latest/api/vllm/lora/punica_wrapper/
48. Clarification: Does vLLM support concurrent decoding with multiple LoRA adapters in online inference?, accessed June 8, 2026, https://discuss.vllm.ai/t/clarification-does-vllm-support-concurrent-decoding-with-multiple-lora-adapters-in-online-inference/1482
49. TIES-Merging: Robust Model Integration - Emergent Mind, accessed June 8, 2026, https://www.emergentmind.com/topics/ties-merging
50. MergeME: Model Merging Techniques for Homogeneous and Heterogeneous MoEs - Amazon Science, accessed June 8, 2026, https://assets.amazon.science/a2/38/e9dbfd2c4a36a651e0cbf20c20aa/mergeme-model-merging-techniques-for-homogeneous-and-heterogeneous-moes.pdf
51. Distillation Scaling Laws - arXiv, accessed June 8, 2026, https://arxiv.org/html/2502.08606v2
52. Distillation Scaling Laws - GitHub, accessed June 8, 2026, https://raw.githubusercontent.com/mlresearch/v267/main/assets/busbridge25a/busbridge25a.pdf
53. [Literature Review] Distillation Scaling Laws - Moonlight, accessed June 8, 2026, https://www.themoonlight.io/en/review/distillation-scaling-laws
54. How Contaminated Is Your Benchmark? Measuring Dataset Leakage in Large Language Models with Kernel Divergence - ICML 2026, accessed June 8, 2026, https://icml.cc/virtual/2025/poster/43619

Attribution

Part of Stunspot’s Guide to AI Systems — The AI Engineering Systems Canon.

Created by Sam “stunspot” Walker / Collaborative Dynamics.

Repository: https://github.com/Stunspot/stunspots-guide-to-ai-systems
Stunspot: https://stunspot.com
Collaborative Dynamics: https://www.collaborative-dynamics.com
Discord: https://discord.gg/stunspot

Licensed under CC BY-NC-SA 4.0 unless otherwise stated.
Commercial use, resale, paid redistribution, inclusion in commercial training products, or incorporation into paid knowledge-base products requires prior written permission.

← Back to Canon Map