Everything old is new again: memory optimization

The author posits that the increasing demand for RAM by AI applications is making memory optimization relevant again, even for consumer devices. To explore the potential for improvement, the article presents a case study: a script that parses a text file, splits it into words, counts their occurrences, and prints them in decreasing order of frequency.

• Python Baseline: A concise 30-line Python implementation serves as the starting point, exhibiting a peak memory consumption of 1.3 MB for a small text file.
• Native C++ Version: A C++ equivalent, leveraging the Pystd library, accomplishes the same task in approximately 60 lines (20 core lines). This version employs mmap for file access, validates UTF-8, converts data into UTF-8 views, splits words lazily, and uses a hash table with string views as keys, completely avoiding string object allocations.
• Memory Efficiency: The C++ solution achieves a peak memory usage of roughly 100 kB, representing only 7.7% of the Python version's footprint.
• Further Optimization: The author notes that the C++ runtime itself contributes about 70 kB to memory usage for features like exception handling. By building the code without exception support, the total memory could theoretically be reduced to a mere 21 kB, resulting in a 98.4% memory reduction compared to Python.

While acknowledging that the comparison might seem unfair to Python due to its inherent runtime overhead, the article effectively illustrates the profound memory savings achievable through meticulous native code optimization when general-purpose runtime features are not essential.