TRL Training on Hugging Face Jobs

Detailed Guide

Read the detailed guide before executing this skill. It retains the complete procedure and reference material. Treat its safety, prerequisites, and validation requirements as mandatory. For focused work, load the relevant sections; for end-to-end work, read the guide completely.

When to Use This Skill

Use this skill when users want to:

• Fine-tune language models on cloud GPUs without local infrastructure
• Train with TRL methods (SFT, DPO, GRPO, etc.)
• Run training jobs on Hugging Face Jobs infrastructure
• Convert trained models to GGUF for local deployment (Ollama, LM Studio, llama.cpp)
• Ensure trained models are permanently saved to the Hub
• Use modern workflows with optimized defaults

When to Use Unsloth

Use Unsloth (references/unsloth.md) instead of standard TRL when:

• Limited GPU memory - Unsloth uses ~60% less VRAM
• Speed matters - Unsloth is ~2x faster
• Training large models (>13B) - memory efficiency is critical
• Training Vision-Language Models (VLMs) - Unsloth has FastVisionModel support

See references/unsloth.md for complete Unsloth documentation and scripts/unsloth_sft_example.py for a production-ready training script.

Prerequisites Checklist

Before starting any training job, verify:

✅ Account & Authentication

• Hugging Face Account with Pro, Team, or Enterprise plan (Jobs require paid plan)
• Authenticated login: Check with hf_whoami()
• HF_TOKEN for Hub Push ⚠️ CRITICAL - Training environment is ephemeral, must push to Hub or ALL training results are lost
• Token must have write permissions
• MUST pass secrets={"HF_TOKEN": "$HF_TOKEN"} in job config to make token available (the $HF_TOKEN syntax references your actual token value)

✅ Dataset Requirements

• Dataset must exist on Hub or be loadable via datasets.load_dataset()
• Format must match training method (SFT: "messages"/text/prompt-completion; DPO: chosen/rejected; GRPO: prompt-only)
• ALWAYS validate unknown datasets before GPU training to prevent format failures (see Dataset Validation section below)
• Size appropriate for hardware (Demo: 50-100 examples on t4-small; Production: 1K-10K+ on a10g-large/a100-large)

⚠️ Critical Settings

• Timeout must exceed expected training time - Default 30min is TOO SHORT for most training. Minimum recommended: 1-2 hours. Job fails and loses all progress if timeout is exceeded.
• Hub push must be enabled - Config: push_to_hub=True, hub_model_id="username/model-name"; Job: secrets={"HF_TOKEN": "$HF_TOKEN"}

Example Training Scripts

Production-ready templates with all best practices:

Load these scripts for correctly:

• scripts/train_sft_example.py - Complete SFT training with Trackio, LoRA, checkpoints
• scripts/train_dpo_example.py - DPO training for preference learning
• scripts/train_grpo_example.py - GRPO training for online RL

These scripts demonstrate proper Hub saving, Trackio integration, checkpoint management, and optimized parameters. Pass their content inline to hf_jobs() or use as templates for custom scripts.

Limitations

• Use this skill only when the task clearly matches its upstream product or API scope.
• Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.
• Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.