Mantis: Automatic Performance Prediction for Smartphone Applications

1. Mantis: Automatic Performance Prediction for Smartphone Applications Yongin Kwon, Sangmin Lee, Hayoon Yi, Donghyun Kwon, Seungjun Yang, Byung-Gon Chun, Ling Huang, PetrosManiatis, MayurNaik, YunheungPaek USENIX ATC’13

2. Performance Prediction Problem • Predict the execution time of a program on a given input before running it.

3. Two kinds of Approaches • Most existing techniques can be classified into two broad categories. • Domain-specific programs, automatically-extracted features. • General-purpose programs, manually-specified features.

4. Mantis • A new framework to automatically predict the performance of general-purpose byte-code programs on given inputs. • Four components: • Feature instrumentor • Profiler • Performance model generator • Predictor code generator

5. Architecture

6. Feature Instrumentor • Instruments the program to collect the values of feature (f1, … , fM) as per feature schemes. • Feature scheme • Branch counts • Loop counts • Method-call counts • Variable values

7. Examples

8. Profiler • Outputs a data set • ti: the ith observation of execution time. • vi: the ith observation of the vector of M features.

9. Performance Modeling • Performs a sparse nonlinear regression on the feature values and execution time. • Produces a function • is the approximation of execution time • is a subset of • In practice, K << M.

10. Performance Modeling(Cont.) • However, regression with best subset selection is NP-hard. • Find the subset of size K that gives the smallest Residual Sum of Squares(RSS). • Discrete optimization problem.

11. SPORE-FoBa • Sparse POlynomialREgression – FoBa. • A feature from the candidate set is added into the model if and only if adding it makes the RSS decrease a lot. • If the drop is greater than ε. • Remove a feature from the active set if deleting it makes the RSS increase the least. • If the increment is smaller than ε’.

12. Example • Degree-2 polynomial with • Expand (1+x1+x2)2 to get 1, x1, x2, x12, x1x2, x22. • Construct the following function for regression

13. Predictor code generator • Produce a code snippet, called slice, for each chosen features. • Slice: an executable sub-programs that yields the same value v of a feature at a program point p as the given program on all inputs. • Automatically evaluate feature values for each input by executing slices.

14. Example

15. Prototype Toolchain

16. Experiment Setup • Amachine runs Ubuntu 11.10 64-bit with a 3.1GHz quad-core CPU, and 8GB of RAM. • A Galaxy Nexus running Android 4.1.2 with dual-core 1.2Ghz CPU and 1GB RAM. • Six CPU-intensive Android applications. • Each with 1,000 randomly generated inputs. • Train the predictor on 100 inputs.

17. Experimental Results

18. Features and Models

19. Effect of the Number of Training Inputs

20. Compare with Linear Model

21. Prediction Time of Mantis and PE

22. Prediction Error of Mantis and BE

23. Prediction on Different Hardware Platform

24. Prediction under Background Load

25. Offline Stage Processing Time

26. Conclusion • Mantis is a framework that automatically generates program performance predictors. • Combines program slicing and sparse regression in a novel way. • Evaluation shows that the generated predictors estimate execution time accurately and efficiently for smartphone applications.