Poisson Processes and Maximum Likelihood Estimator for Cache Replacement

1. Poisson Processes and Maximum Likelihood Estimator for Cache Replacement A Preliminary Attempt ECEn 670 Semester Project Wei Dang Jacob Frogget

2. Outline • Motivation • Cache replacement algorithms • Poisson processes in webserver-based cache • Poisson processes in microprocessor-based cache • Maximum Likelihood estimator • Predicting procedures • Simulation and evaluation • Results compared to LRU • Future work

3. Motivation • Memory-processor speed gap is getting larger

4. One of the Solutions • Memory hierarchy • L1/L2/L3 • closer to chip, less latency, but smaller size • Cache replacement • Cache is fast but limited in size • conflict • Good replacement policies needed on contention

5. Cache Replacement Algorithms • Cache conflict in a 4-way associative cache Set index

6. Common Cache Replacement Algorithms • Random • Least Recently Used (LRU, most widely used) • Least Frequently Used • FIFO

7. Poisson Processes for webserver cache • Arrival time of queries to a webserver can be modeled as a Poisson Process • Interpretation: the probability of having k queries up to some point of time • Assumption • Arrivals of queries are independent of each other • Not always true but valid for most cases

8. Poisson Processes for microprocessor cache • Independence assumption invalid • References to cache are highly correlated (especially to data cache) • Temporal locality • Spatial locality

9. Poisson Processes for microprocessor cache cont. • One Poisson process for each block within a set Set index An example set from a 4-way associative cache

10. Poisson Processes for microprocessor cache cont. • Correlation between these four random processes One global counter n for each set Set index Local counters to each block

11. Maximum Likelihood Estimator The estimate is the arithmetic mean of

12. Predicting Procedures • Initially each block has • Given a previous calculated for block , the estimated is calculated as: • Probabilities for replacement is then • Replace block with lowest probability • Choose randomly on equal probabilities

13. Simulation and Evaluation • Simulator: MyDLX cache simulator from EE628 • Metrics: miss rate for Instruction-cache and Data-Cache • Various associativities • Five benchmarks • Compared to LRU

14. Results

15. Results

16. Results

17. Results

18. Results

19. Results

20. Results • Results not encouraging • Sometimes 0% miss rate for both algorithms (might be due the inherent characteristics of benchmarks) • Statistical approach worse than LRU for most cases • Getting worse for higher associativity (more blocks to predict)

21. Analysis of deficiencies of our model • Independence model may be inaccurate (even accesses to the same block within a set may not be independent) • Local counter is reset to 0 on eviction (history eliminated)

22. Future work and challenges • More accurate model with more correlation parameters for each Poisson process • Implementation complexity (hardware expensive; LRU is already expensive at high associativity) • May be implemented as a software cache as a supplement to hardware cache.

23. The End Thank you! Questions?