Faster binary search: from compiled code to mechanical sympathy
This article dives deep into optimizing binary search for Python's scientific stack, revealing how fine-grained control over CPU mechanics can yield significant speedups. By meticulously eliminating branch mispredictions and leveraging SIMD instructions, the author demonstrates a 6x performance boost. It resonates with HN's appreciation for low-level performance tuning and practical application of computer architecture principles.
The Lowdown
Optimizing computational Python code often starts with algorithm selection, compiled extensions, and parallelism. However, this article, inspired by work on scikit-learn, explores how even greater speeds can be achieved by understanding and working with the CPU's underlying architecture, a concept known as 'mechanical sympathy'. The author demonstrates this by refining a binary search implementation.
- The initial problem involves efficiently bucketing a large array of floating-point numbers into 255 integer ranges, a task scikit-learn performs using binary search on pre-defined boundaries.
- The first optimization targets branch mispredictions, a significant performance killer on modern CPUs. The standard binary search's unpredictable
if/whilestatements lead to substantial CPU instruction pipeline stalls. - By rewriting the binary search to be 'branchless'—using a fixed number of iterations and
select_unpredictablehints—the branch misprediction rate drops to 0%, leading to a 3.5x speedup despite similar CPU instruction counts. - Further refinement involves using Rust's
unsafecode to bypass bounds checks, reducing the overall CPU instructions and branches, and achieving even greater speed. - The final stage leverages auto-vectorization and SIMD instructions. By restructuring the loops and compiling for a modern
x86-64-v3target, the compiler can generate 'Single Instruction, Multiple Data' operations, leading to the ultimate 6x speed improvement. - The article acknowledges trade-offs, such as the need for
unsafecode to be carefully validated and thex86-64-v3target limiting compatibility with older CPUs, suggesting runtime version selection as a solution.
The author concludes by emphasizing that these low-level optimization techniques are learnable, encouraging readers to explore mechanical sympathy further, and subtly promotes his book on Python performance.
The Gossip
Alternative Algorithm Appetites
Commenters explored alternative approaches to the core problem of assigning floats to buckets, questioning if sorting the initial array might be more cache-friendly than binary search's random accesses. Suggestions also included using an Eytzinger layout for the bucket boundaries to improve cache locality, or an interpolation search for smarter initial guesses. The author, @itamarst, weighed in, noting that while interpolation search could reduce mispredictions, memory bandwidth often becomes the real bottleneck in parallel usage, and the current optimizations were sufficient for the motivating project.
Python's Performance Predicament
A familiar Hacker News debate emerged regarding Python's performance. One commenter outright declared Python "terrible and unbearably slow," suggesting that for serious or performant tasks, one should simply switch to a 'better language'. This was met with a practical rebuttal, highlighting that the article *already* leverages compiled languages for speed and that Python's ubiquity in many industries means optimizing its computational parts remains a valid and necessary endeavor.