ECE’s Tongping Liu and Jin Zhou Acquire U.S. Patent to Address “Cache Misses”
Content
The cache, a hardware component for storing bits and pieces of information in computing, plays an important role in the performance of applications. The key problem is “cache misses,” when desired information cannot be found in the cache, a process that slows down the function. Now Adjunct Associate Professor Tongping Liu of the UMass Amherst Electrical and Computer Engineering (ECE) Department and his former Ph.D. student Jin Zhou have obtained a U.S. patent for their pioneering tool called CachePerf that helps to discover these cache-miss issues and overcome the shortcomings of existing tools.
Liu observes that “Effective utilization of the cache is the key to boost the performance of applications.”
As Liu writes in his patent application, “The cache plays an important role in determining the performance of applications, no matter for sequential or concurrent programs on homogeneous and heterogeneous architecture. Therefore, it is important to locate and differentiate cache misses accurately, but this remains an unresolved issue even after decades of research.”
Liu adds that “Existing tools cannot correctly identify all types of cache misses with significant performance impacts, and then programmers may waste unnecessary effort while achieving no or trivial improvement.”
This is the issue being addressed by Liu with his newly patented CachePerf.
As Liu specifies in his patent application, “This disclosure presents a cache-miss classifier or unified-profiling tool – CachePerf – that can correctly identify different types of cache misses (while imposing reasonable overhead), differentiate issues of allocators from those of applications, and exclude minor issues without much performance impact.”
According to Liu, “The core idea behind CachePerf is a hybrid sampling scheme: It can employ performance-monitoring-unit-based, coarse-grained sampling to filter out few susceptible instructions (with a large number of cache misses) and then can employ the breakpoint-based, fine-grained sampling to collect the memory-access pattern of these instructions.”
In the process, CachePerf imposes only about 14 percent performance overhead and 19 percent memory overhead (for applications with larger footprints), while identifying all types of cache misses correctly. In addition, CachePerf can detect four new issues that cannot be detected by existing tools.
As Liu concludes, “CachePerf offers an indispensable complementary option to existing profilers due to its effectiveness and low overhead.”
Liu directs the MASS Lab (Massachusetts Software Systems Research Lab), which explores software reliability, system security, machine learning and big data, performance of parallel applications, and resource management of operating systems. (April 2026)
Featuring
Tongping Liu
