1 / 92

On Using Tracer Driver for External Dynamic Process Observation

On Using Tracer Driver for External Dynamic Process Observation. Pierre Deransart January 29, 2007 REWERSE WP I3 meeting Dresden. Les unités de recherche. Rocquencourt. Loria. Rhône-Alpes. Irisa. Sophia Antipolis. Futurs. CONTEXT, HISTORY. From DiSCiPl to OADymPPaC

twyla
Download Presentation

On Using Tracer Driver for External Dynamic Process Observation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. On Using Tracer Driver for External Dynamic Process Observation Pierre Deransart January 29, 2007 REWERSE WP I3 meeting Dresden Pierre Deransart Rewerse-WP I3

  2. Les unités de recherche Rocquencourt Loria Rhône-Alpes Irisa Sophia Antipolis Futurs Pierre Deransart Rewerse-WP I3

  3. CONTEXT, HISTORY • From DiSCiPl to OADymPPaC • DiSCiPl (1997-2000): enhance constraint debugging: resulted in prototypes, but still ad-hoc tools for correctness and performance analysis • OADymPPaC (2001-2004) main challenges • Interoperability of tools: make analysis tool development easier --> "standardisation" of the CP platforms and tools parameterisation • Scaling: increase the size of the problems which can be analysed this way (hundreds of variables or constraints) ----> enable using specialized HMI Resulted in powerful analyzers (prototypes and products), but still prototype tools for performance analysis Also still an approach limited to finite domains constraint solving Pierre Deransart Rewerse-WP I3

  4. OADymPPaC Realizations Pierre Deransart Rewerse-WP I3

  5. GENERICITY and INTEROPERABILITY PRINCIPLES Pierre Deransart Rewerse-WP I3

  6. NEW PROBLEMS • OADymPPaC raised unsolved questions: • Trace Semantics (related to its genericity): what is the exact nature of the “Observational Semantics” • Modeling of the Interactions between Observed Process and Analysis Tools • How to keep the whole system efficient (need to evaluate efficiency) • Is the approach generic itself? Can it be applied to new domains (not just constraints) in particular in software engineering Pierre Deransart Rewerse-WP I3

  7. Ways to ANSWER to the PROBLEMS • Trace Semantics (related to its genericity): what is the exact nature of the “Observational Semantics”: model of trace for a family of processes • Modeling of the Interactions between Observed Process and Analysis Tools: Introduction of Tracer Driver and Analyzer Manager • How to keep the whole system efficient (need to evaluate efficiency): Concepts of Virtual full Trace, Actual incremental Trace extraction and rebuilding • Is the approach generic itself? Can it be applied to new domains (not just constraints) in particular in software engineering can the approach (full trace with partial OS, interactive tracer driver) be applied in new domains Pierre Deransart Rewerse-WP I3

  8. RELATIONSHIPS with REWERSE (attempt) • Genericity of traces and trace querying • Semantics (web) mining and observational semantics: looking for combination of traces and models “explaining” the traces • Software Composition vs Trace Composition • Relations action vs event: actions produce (trace) event Pierre Deransart Rewerse-WP I3

  9. HISTORY • Ducassé, Opium: versatile analyzers • Ducassé, Noyer, 1994, JLP: independence of tracer and analyzers • Langevine & Ducassé & Deransart, 2003, ICLP’03: Codeine, First Driven Tracer approach implementation for GNU-prolog • Deransart & all, 2004, OADymPaC project: client/server architecture, trace standardization (XML) for interoperability • Langevine & Deransart & Ducassé, 2004, LNAI 3010: model of « generic trace » for constraint programming • Langevine & Ducassé, 2005, AADEBUG, WLPE: Driven Tracer for FD constraints and experimentation • Now, 2006: « theory » of generic trace and driven tracer = • virtual full trace, actual incremental trace and observational semantics Pierre Deransart Rewerse-WP I3

  10. OBJECTS OF STUDIES • Traces • Interactions and Tracer Driver • Observational Semantics of traces Pierre Deransart Rewerse-WP I3

  11. TRACES Pierre Deransart Rewerse-WP I3

  12. TRACES Pierre Deransart Rewerse-WP I3

  13. « Leave traces, not proofs, only traces give dreams » • René Char • Poète • (1907-1988) Pierre Deransart Rewerse-WP I3

  14. SOME MOTIVATIONS • Dynamic Program Analysis: • typical recent works are based on trace analysis • Ernst & al. 2001: “Dynamically discovering likely program invariants…” from data computed by dynamic execution • Denmat & al. 2005 “Data mining and Cross-checking of Execution Traces…” from a collection of traces • Zaidman & al. 2005 “Applying Webmining Techniquesto Execution Traces…” Pierre Deransart Rewerse-WP I3

  15. WHY TRACES ? • Any phenomenon, any open system leaves traces • Walking (persistent foot traces) • Sedimentation (temporally accumulated traces), fossils • Particles (light): objects only known by their physical or chemical properties • Programs (outputs, observation) • Communication (messages) • Discourse abstract • Human Memory…(persistent and reactive) • Traces are everywhere: we only know processes by their traces Pierre Deransart Rewerse-WP I3

  16. OBSERVING PROCESSES • Everybody watches everyones….. • Everybody receives from everyone • Everybody sends messages to … Pierre Deransart Rewerse-WP I3

  17. We know and define complex objects from the traces they leaveWe know complex programs behaviour by analyzing their tracesCausal analysis is not tractable likely causalityTRACE modeling is the right approach Pierre Deransart Rewerse-WP I3

  18. BUT, what IS a TRACE ???HOW to ANALYSE IT ??? • I want to leave these questions open • ….small demo Pierre Deransart Rewerse-WP I3

  19. What IS a TRACE ??? the answer is:Any possible understandable information related to an observed phenomenon • PRO • Any observing device will find in the trace what it needs • If a process needs instrumentation to produce a trace, this need only to be made once • Analysis tools may be developed independently Pierre Deransart Rewerse-WP I3

  20. CON:- it is not possible to broadcast such a huge information flow (communication slowing down)-then the observed process always computes a huge amount of unused data (construction slowing down) Pierre Deransart Rewerse-WP I3

  21. SPECIFIC ASPECTS • Needs for trace standardization • All levels of granularity accepted • Different kinds of information included (levels of abstraction) • Each analyzer must be able to recognize its relevant information • Use of XML to define possible standards Pierre Deransart Rewerse-WP I3

  22. SEEMS an UNREALISTIC GOAL !!! the answer is:Driven Tracer • SPECS • selects in the trace what the observer needs • keeps tracer listening to the analyzers (dialog between server and clients) • Leave the tracer to distribute and broadcast each specific trace • Allow workload repartition between tracer and analysers Pierre Deransart Rewerse-WP I3

  23. VIRTUAL FULL TRACE (VFT definition) • unbounded sequence of trace events of the form • et: (t, at, St+1) • et: unique identifier of the event • t: chrono. Time of the trace • at: kind of event, an identifier characterizing the kind of actions from which the process changed from St to St+1 • St = p1,t..., pn,t : values of parameters at chrono t. St is the full current state Pierre Deransart Rewerse-WP I3

  24. ACTUAL TRACE • unbounded sequence of actual events of the forme’t: (t, at, At) • such that, there exists a function f, such that for each evente’t = f(et, St). • At denotes a set of attribute values. The actual trace is the trace emitted by a tracer, which can actually be “visible”. • The virtual full trace is a particular case of actual trace where f is the identity, i.e.the attributes of At are the kind of event at and the parameter values St+1. Pierre Deransart Rewerse-WP I3

  25. Different kinds of attributes • f(p) = p, ex “CPU time” • f(p) = (p), current parameter abstraction,ex “depth of the proof tree” • attrt = f(pt+1, St), incremental attribute, ex “values removed from the domain of a variable” • P: parameter(s) Pierre Deransart Rewerse-WP I3

  26. INCREMENTAL TRACE(full actual) • An actual trace is incremental if the attributes describe only the changes affecting the current state, i.e. for every t • et = F(e’t,subpart(St)) • F is the “Observational Semantics” (OS) • It verifies also: (if all attributes incremental) • St+1 = F(At,St) Pierre Deransart Rewerse-WP I3

  27. OBSERVATIONAL SEMANTICS • Trace: • et: (t, at, St+1) • There exists a function f, such that for each event • e’t = (t, at, At)= f(et, St) • This is trace extraction : At = f(St, St+1) • The “Observational Semantics” (OS) is a kind of partial “explanation”): • St+1= F(At,St) (full actual, otherwise ) • (F used for rebuilding) Pierre Deransart Rewerse-WP I3

  28. EXAMPLE (actual trace) • 1 1 Call: '$call$'(bench(2)) • 2 2 Call: bench(2) • 3 3 Call: 2>0 • 3 3 Exit: 2>0 • 4 3 Call: _182 is 2-1 • 4 3 Exit: 1 is 2-1 • 5 3 Call: bench(1) • .... Pierre Deransart Rewerse-WP I3

  29. Pierre Deransart Rewerse-WP I3

  30. Gentra4CP format <post chrono="8" depth="1" cident="c2" /> <solved chrono="9" depth="1" cident="c2" /> <new-variable chrono="10" depth="1" vident="v3" vinternal="_#47" vname="V3"> <vardomain min="1" max="4" size="4"> <range from="1" to="4" /> </vardomain> </new-variable> <new-constraint chrono="11" depth="1" cident="c3" cinternal="fd_domain(v3,1,4)" orig="user"> <state> <constraint cident="c3" cinternal="fd_domain(v3,1,4)" status="just_declared" orig="user"> <variables>v3</variables> </constraint> </state> </new-constraint> <post chrono="12" depth="1" cident="c3" /> <solved chrono="13" depth="1" cident="c3" /> <new-variable chrono="14" depth="1" vident="v4" vinternal="_#69" vname="V4"> <vardomain min="1" max="4" size="4"> <range from="1" to="4" /> </vardomain> </new-variable> Pierre Deransart Rewerse-WP I3

  31. Messages Linux File 01 17 16:53:37 kok ker: Additional sense indicates Medium not present 01 17 16:53:37 kok ker: sdb : block size assumed to be 512 bytes, disk size 1GB. 01 17 16:53:37 kok ker: /dev/scsi/host1/bus0/target0/lun0: I/O error: dev 08:10, sector 0 01 17 16:53:37 kok ker: I/O error: dev 08:10, sector 0 01 17 16:53:37 kok ker: unable to read partition table 01 17 16:53:37 kok ker: usb.c: registered new driver usbdevfs 01 17 16:53:37 kok ker: usb.c: registered new driver hub 01 17 16:53:37 kok ker: usb-uhci.c: $Revision: 1.275 $ time 09:50:48 Aug 17 2005 01 17 16:53:37 kok ker: usb-uhci.c: High bandwidth mode enabled 01 17 16:53:37 kok ker: usb-uhci.c: USB UHCI at I/O 0x2440, IRQ 19 01 17 16:53:37 kok ker: usb-uhci.c: Detected 2 ports 01 17 16:53:37 kok ker: usb.c: new USB bus registered, assigned bus number 1 01 17 16:53:37 kok ker: hub.c: USB hub found 01 17 16:53:37 kok ker: hub.c: 2 ports detected 01 17 16:53:37 kok ker: usb-uhci.c: USB UHCI at I/O 0x2460, IRQ 23 01 17 16:53:37 kok ker: usb-uhci.c: Detected 2 ports 01 17 16:53:37 kok ker: usb.c: new USB bus registered, assigned bus number 2 01 17 16:53:37 kok ker: hub.c: USB hub found 01 17 16:53:37 kok ker: hub.c: 2 ports detected 01 17 16:53:37 kok ker: usb-uhci.c: v1.275:USB Universal Host Controller Interface driver Pierre Deransart Rewerse-WP I3

  32. Actual Trace Requirements • The actual trace may likely be the full virtual trace (attributes are the parameters) • It may be discontinuous, then the full current state shall be available at any time on demand • It should likely incorporate complex objects built from the VFT • It should cover large application domains and many observation methods • It should have a clear semantics (OS): the OS allows to rebuild original states from the trace (assuming that the full current state is known at some moment from which the trace is continuous) Pierre Deransart Rewerse-WP I3

  33. INTERACTIONS and TRACER DRIVER Pierre Deransart Rewerse-WP I3

  34. Classical Approach (general broadcasting) Tracer Observing Process Virtual Full Trace Filter/ Builder Actual Full Trace Observed Process Pierre Deransart Rewerse-WP I3

  35. Classical Approach (general broadcasting) Virtual Full Trace Filter/ Builder Actual Full Trace Tracer Observing Process Observed Process Pierre Deransart Rewerse-WP I3

  36. Driven Tracer Approach (requested trace only) Observing Process Observed Process Virtual Full Trace Builder Requested V. Trace Actual Requested Trace Tracer Filter/ Manager Driver Pierre Deransart Rewerse-WP I3

  37. Workload Analysis • Program (without connections with tracer) T_prog • Tracer (parameters and attributes computation) T_core + T_extract • Driver (dialog and attributes selection) T_cond • Broadcasting (all traces encoding) T_encode-and-com ==================================================== • Trace decoding T_decode • Trace filtering T_ filter • Trace rebuilding T_rebuild • Analyzer execution T_ana Pierre Deransart Rewerse-WP I3

  38. Workload Analysis • Process • --------------- • | | • T_prog + T_core1 • Tracer/Driver • ----------------------------------------------- • | | • T_core2 + T_cond + T_extract + T_encode-and-com • ========================================================= • Analyzer/Builder/Manager • ------------------------------------------ • | | • T_decode + T_filter + T_rebuild + T_ana Pierre Deransart Rewerse-WP I3

  39. Workload balance ================================================== Analyzer --------- | | T_ana • Process and Tracer • --------------- • | | • T_prog + T_core Manager --------- | | T_filter Driver --------- | | T_cond Builder --------- | | T_rebuild Tracer --------- | | T_extract Tracer ----------------- | | T_ encode-and-com Manager --------- | | T_decode Pierre Deransart Rewerse-WP I3

  40. Workload Evaluation: processes ============== Analyzer --------- | | T_ana • Process and Tracer • --------------- • | | • T_prog + T_core • A (sophisticated) analyzer may be extremely slow • With large virtual full traces T_core may become important but is thus balanced by T_ana (example: explanations in Gnu-Prolog and graph visualization) • It is also possible to extend the full trace with complex objects built on demand (genericity of the VFT is not affected, only T_extract) Pierre Deransart Rewerse-WP I3

  41. Workload Evaluation: Driver/Manager Manager --------- | | T_filter Driver --------- | | T_cond ======== • This is the key point of the Driver approach: the manager filters a large broadcasted full actual trace or alternatively the driver filters the trace events (and selects few events wrt the full trace) before the actual trace is extracted. • This makes the approach efficient because the selection consists of 2 steps: • on the kind of events (assumes a finite relatively small number of kind of events) • other criteria may be considered during extraction step, but applied on relatively few events Pierre Deransart Rewerse-WP I3

  42. Workload Evaluation: extraction/rebuilding Builder --------- | | T_rebuild Tracer --------- | | T_extract ========= • Attributes extraction with complex criteria may be relatively slow, but only relatively few events are subject to this operation • State reconstruction by the Builder may be slower (the virtual state may not be explicitly constructed by the process when it may be in the Builder). • The Builder must also prepare the data for the analyzer (ex: build table models for visualization tool) Pierre Deransart Rewerse-WP I3

  43. Workload Evaluation: encode/comm/decode Manager --------- | | T_decode Tracer ----------------- | | T_ encode-and-com ========= • The actual trace may be encoded with different methods in order to optimize the broadcasted data flow: compression, binary conversion … etc. But the implies an equivalent work from the Manager. • This is not specific to this approach. It just shows that verbose XML may be used. There are proposals to compress XML (e.g. http://www.w3.org/TR/wbxml) Pierre Deransart Rewerse-WP I3

  44. Evaluation of workload balance (Ad-hoc Approach) Analyzer --------- | | T_ana ============================================= + + • Process and Tracer • --------------- • | | • T_prog + T_core Manager --------- | | T_filter Driver --------- | | T_cond 0 + Builder --------- | | T_rebuild Tracer --------- | | T_extract + + Tracer ----------------- | | T_ encode-et-com Manager --------- | | T_decode + Pierre Deransart Rewerse-WP I3

  45. Evaluation of workload balance (Tracer Driver Approach) ================================================== Analyzer --------- | | T_ana + + • Process and Tracer • --------------- • | | • T_prog + T_core Manager --------- | | T_filter Driver --------- | | T_cond + 0 Builder --------- | | T_rebuild Tracer --------- | | T_extract + Tracer ----------------- | | T_ encode-et-com Manager --------- | | T_decode Pierre Deransart Rewerse-WP I3

  46. Summary • Ad-hoc Approach • The full actual trace is treated, resulting in excessive tracer extraction, coding and communication times. • The full actual trace must be decoded and filtered by the Manager. • This imposes to reduce the size of the trace at the source, restricting reusability of the trace by different tools. • Tracer Driver Approach • There is no limit of the trace size (except extra computation costs due to core computations preparing extraction of a huge virtual trace). The tracer may be designed for a large collection of potential uses. • Additionally to the tracer a driver must be implemented. But the additional driving time can be low and extraction, coding and communication times may be very reduced (provided that, even if several analyzers are active, only a small part of the virtual full trace is used). Pierre Deransart Rewerse-WP I3

  47. DRIVER/MANAGER REQUIREMENTS • Driver (server) • Analyze trace queries • Store and manage requested trace patterns • Manage the dialog between server (tracer) and clients (analyzers) • Tell the tracer what to extract and to broadcast • Manager (client) • Select the trace events for its analyzer (verify trace adequacy) • Elaborate and send queries • Manage the dialog between server (tracer) and clients (analyzers) Pierre Deransart Rewerse-WP I3

  48. Language of Interactions • Basic instructions between driver (server) and mediator (client). A persistent set of conditions, compiled as a finite automaton, serves for the current trace event selection. • Manager commands (sent to the driver) • current (in sync mode): request to send particular attributes • resume (start async mode) • update (sync or async mode): modifies the event selection conditions • interrupt (emergency signal to stop – may be issued by any external process): pipe is broken and some trace events may be lost. • Driver commands (sent to the manager in trace events) • breakpoint (start sync mode): tracer is waiting for commands • provide (in sync mode): level of the actual trace • complement (in sync mode): additional info or current state as requested by current Pierre Deransart Rewerse-WP I3

  49. OBSERVATIONALSEMANTICS Pierre Deransart Rewerse-WP I3

  50. OBSERVATIONAL SEMANTICS • Trace: • et: (t, at, St+1) • There exists a function f, such that for each event • e’t = (t, at, At)= f(et, St) • The attributes describe only the changes affecting the current state, i.e. for every t • et = F(e’t,subpart(St)) • F is the “Observational Semantics” (OS) • One may just consider the relevant elements: • Trace extraction: At = f(St, St+1) • Trace rebuiling: St+1 = F(At,St) Pierre Deransart Rewerse-WP I3

More Related