scott 967ed41239 Revert FP16 autocast — increases TTFA on first request ...

autocast triggers fp16 kernel selection at first call for each tensor
shape. Since the warmup uses short text, real requests re-trigger
selection and are slower net. Keeping FP32 + conditionals cache.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-04-05 20:30:49 -04:00

3.7 KiB

Raw Blame History

View Raw