Ruoying Li

Frontier LLM Training | GPU Efficiency and Distributed Reliability

• Contributed to a cross-team efficiency program spanning multimodal RL training, pretraining, and evaluation on a fleet of several thousand NVIDIA B200 GPUs; RL-training GPU utilization increased 2.4× (~8% to ~19.5%) in five weeks.
• Designed stall detection, checkpoint-based restart, and automated cleanup for hung training jobs; cut failed-job cleanup from 4–6 hours to ~11 minutes, with auto-termination thresholds validated against 14 historical hangs.
• Fixed distributed evaluation shutdown races using a coordinator/worker acknowledgment handshake; 67 evaluations had rerun 10–19 times (~21,000 wasted GPU-hours). Resolved a separate 202-job restart regression within one day.
• Traced a cross-rank resume deadlock to retry logic around collectives on a message-queue event loop; corrected a recurring failure associated with eight incidents and ~5,750 wasted GPU-hours.
• Led fleet-wide low-utilization investigations, identifying an unreachable reward-model serving pool (~10,000 idle GPU-hours/day), trainer restart loops (~520 idle GPUs in one day), and CPU-only workloads occupying GPU nodes.
• Benchmarked L4 against B200 for audio-codec inference and validated a right-sizing path with no training-throughput regression, freeing eight B200 GPUs per job; classified 51% of 3,300 workloads as migration candidates.
• Reduced crash-loop restart limits from 16 to 10 after identifying 26 workloads with 10+ retries, and increased checkpoint frequency to reduce lost progress during failed training runs.
• Profiled a high-efficiency reference training run to establish reusable job settings; introduced LLM-assisted log forensics across trainer, inference, replay-buffer, and orchestration components for multi-service debugging.

LLM Pretraining | Fault Recovery, Checkpointing and Observability

• Implemented framework-level NaN/Inf and sustained loss-spike detection with automatic checkpoint recovery; validated the end-to-end restart path by injecting NaN loss into a live job.
• Automated hang, stall, and throughput-drop monitoring for distributed pretraining; created shared event logging for goodput and training KPIs, replacing manual checks with standardized efficiency signals.
• Shipped OpenTelemetry-to-Grafana training telemetry; traced a production slowdown to unbounded metric-label cardinality, rolled back promptly, and reintroduced the corrected instrumentation.
• Scaled S3 checkpointing from single-node to multi-node JAX training with Kubernetes workload identity; resolved credential-expiry failures, upgraded TensorStore, and added checkpoint latency profiling.
• Investigated JAX 0.9/Orbax 0.11 checkpoint-resume mismatches with Adam and Muon tensor-level tests; verified bitwise-identical Adam restores and isolated the framework upgrade as the cause.
• Replaced a legacy checkpoint interface across three shared packages with a resume-hook API, validating staged migrations with training test jobs; improved node-level diagnosis of NVLink-related loss spikes.

Amazon Nova Premier | AWS Trainium Training Platform

• Supported on-call operations for 470B-parameter LLM training on 3,000+ Trainium nodes; contributed to the A100-to-Trainium migration, reaching 34B tokens/day at 94% goodput versus 37B tokens/day at 87% on A100.
• Built the production container-image pipeline with accelerator SDK dependency management, automated CI/CD builds, release validation, and runbooks, supporting cluster growth from hundreds to 3,000+ nodes.
• Root-caused a recurring data-loader crash every ~2,000 steps across 512 Trainium workers and introduced spare-node capacity to support faster recovery from failures.
• Transferred restart-time improvements to GPU training, saving ~250 seconds per restart through memory-mapped data-loader indexing (280s to 30s), persistent data loading (

ML Platform | Natural Language Understanding

• Built a Kubernetes-based supervised fine-tuning platform with GPU environments, job metrics, and self-service onboarding for multiple ML teams; introduced scheduler-integrated node isolation to prevent resource contention.
• Led a production EU-to-US training-platform migration across four teams, restoring CI/CD, closing security gaps, and benchmarking A100/L4 replacements for retiring V100 accelerators.
• Directed operational-readiness review and ownership transfer for 10 services and 25 deployment pipelines, including knowledge transfer, runbooks, and on-call shadowing.
• Caught an mTLS migration latency regression before production release; traced the slowdown to per-request authentication-token fetching and eliminated it with token caching.