Fine-tuning base models directly creates maintenance overhead whenever base foundation models update. Sally v1.0 separates domain reasoning from base foundation model weights by delivering specialization as a lightweight adapter module.
Architecture
Sally v1.0 is a two-stage LoRA fine-tune of Qwen3 14B, using supervised fine-tuning (SFT) followed by Direct Preference Optimization (DPO). The training applies rank-16 adapters across all attention and MLP projections.
Training targets the domain-specific reasoning methods of the a1c.io preventive health knowledge base rather than raw biomarker datasets. The base model handles general language tasks, while the LoRA adapter injects domain logic, clinical safety constraints, and structured output formatting.
Adapter design rationale
Deploying domain knowledge via adapters provides three structural advantages:
- Decoupled model cycles: Base models can be updated or swapped without rebuilding the underlying domain logic.
- Targeted safety boundaries: Hardcoded refusal routes (e.g., routing type 1 diabetes, pregnancy, pediatric cases, eating disorders, or prolonged fasting to human clinicians) remain stable across base model iterations.
- Infrastructure efficiency: A single host model instance can load multiple domain-specific adapters concurrently.
Target domain capabilities
- Glucose-aware nutrition logic and meal sequence modeling.
- Staged eating window strategies based on user readiness metrics.
- Supplement form compatibility and synergy checks.
- Automated clinical triage and referral routing.
- Direct recommendation outputs instead of unranked options.
Benchmark performance
On MMLU-medical, Sally v1.0 scores 80.8, outperforming its Qwen3 14B base (79.0), Qwen 3.5 (80.0), and Med-PaLM 2 540B (78.0).
| Metric | Qwen3 14B base | Sally v1.0 |
|---|---|---|
| MMLU-medical | 79.0 | 80.8 |
| MedQA | 70.0 | 58.0 |
| PubMedQA | 72.0 | 67.0 |
| MedCalc-Bench | 22.0 | 15.0 |
Scores on general medical benchmarks (MedQA, PubMedQA, MedCalc-Bench) decreased relative to the base model. This occurs because the model was specialized for preventive health rather than broad clinical medicine, and because single-letter multiple-choice scoring penalizes conversational recommendation outputs.
On internal held-out preference evaluations measuring adherence to preventive health guidelines, Sally v1.0 achieved 97.6% accuracy with the system prompt in context.
Limitations and future evaluation
Sally v1.0 has not yet been benchmarked against DM-Bench (diabetes management) or FAM-Bench (therapeutic diets), as both focus on diagnosed clinical conditions rather than preventive health. Evaluations on these datasets are pending.
Weights & Artifacts
Model weights are distributed under the Apache 2.0 license:
- Weights: huggingface.co/sallya1c/sally-1.0
- GGUF Quantization: huggingface.co/sallya1c/sally-1.0-GGUF
- Leaderboard & Logs: huggingface.co/spaces/sally-a1c/sally-llm-leaderboard
The GGUF build includes a Modelfile containing the required system prompt for deployment in Ollama, llama.cpp, and LM Studio.
Notice: Sally v1.0 is not a certified medical device and is not designed for diagnostic, dosing, or clinical treatment decisions.