Automatic1111 for Apple metal, 40% speed up sd1.5
This technical deep dive details how to significantly speed up Automatic1111 on Apple Silicon by teaching it to 'speak Metal', achieving up to 40% performance gains. The author meticulously optimizes key components like attention mechanisms and memory handling, all while preserving the popular WebUI's ecosystem. It's a compelling example of iterative, practical engineering solving a common pain point for local AI enthusiasts.
The Lowdown
The author embarked on a quest to optimize Automatic1111, the popular Stable Diffusion WebUI, for Apple Silicon. Noticing that it felt slower than native applications like Draw Things, the goal was to achieve significant speed improvements without sacrificing Automatic1111's extensive ecosystem of checkpoints, LoRAs, and extensions.
Key optimizations and findings included:
- Shape-Selective Metal Flash Attention: A custom Metal kernel was implemented specifically for SD 1.x attention shapes where it proved faster than PyTorch's MPS, with a fallback to PyTorch otherwise.
- Deferred Metal Command Buffer Submission: The Metal kernel was integrated into PyTorch's MPS stream, preventing frequent and costly command buffer commits after every attention call.
- Unified-Memory-Aware Attention Routing: A dynamic system estimates memory cost for native attention, routing operations based on available memory and preventing performance issues under pressure.
- Streaming Online Softmax: The sub-quadratic attention fallback was improved to process K/V chunks iteratively, discarding them after merging, which is memory-efficient like Flash Attention.
- Removal of Obsolete MPS Workarounds: Old defensive code for past PyTorch MPS bugs was removed or made conditional, eliminating unnecessary conversions and copies on newer systems.
- Fused GroupNorm and SiLU Kernel: A single Metal kernel was created to fuse these frequently occurring operations in the SD 1.x UNet, reducing dispatches and intermediate memory writes.
- Negative Guidance Minimum Sigma (NGMS): While not a direct engine optimization, enabling NGMS for low-CFG workflows allowed skipping unconditional guidance, significantly reducing computation.
- Iterative, Practical Approach: Many theoretically promising optimizations, like packed QKV projections or full residual blocks in MPSGraph, were discarded if they didn't yield real-world speed-ups.
The project ultimately achieved substantial speed improvements, such as reducing generation times on an M3 Pro from 8-10 seconds to 3-7 seconds, and on an M1 Mac Mini from 12.8 seconds to 8.7 seconds. The author emphasizes that while native applications like Draw Things have a fundamental advantage by owning the entire execution environment, this project successfully optimized Automatic1111 by targeting the 'seams' where native Apple execution could be integrated without transforming the WebUI into a different application.