PR Fixes Qwen3 Text Encoder Crashes in FP8 and GGUF Formats for Z-Image
A pull request (#1904) submitted by developer arifanchan on June 13, 2026, addresses two separate crashes that occur when loading the Qwen3 text encoder in quantized FP8 Safetensors and GGUF formats alongside the Z-Image model. The first issue stemmed from FP8 checkpoints omitting the lm_head.weight key to save disk space, causing a strict key check in offload.py to fail; the fix dynamically aliases the existing embed_tokens weight to that slot without duplicating VRAM usage. The second crash arose from a data type mismatch during attention computation when using GGUF encoders, where the Value tensor was in bfloat16 while Query and Key were in float32, violating PyTorch's SDPA requirements. This was resolved by casting the entire text encoder to a uniform dtype immediately after loading. Both fixes were applied within models/z_image/z_image_main.py through a unified state dictionary preprocessor and a post-load dtype cast.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in