1 / 9

Design and Analysis of Experiments for Research Projects

Design and Analysis of Experiments for Research Projects. Rong Zheng. Types of Experiments. Simulation Numerical, event-driven, flow-level, etc. Emulation ( Testbed ) experiments. Which one to pick?. Deciding factors. Nature of the project:

talia
Download Presentation

Design and Analysis of Experiments for Research Projects

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Design and Analysis of Experiments for Research Projects RongZheng

  2. Types of Experiments • Simulation • Numerical, event-driven, flow-level, etc. • Emulation • (Testbed) experiments Which one to pick?

  3. Deciding factors • Nature of the project: • E.g., Energy-efficient activity tracking using mobile phones • E.g, Influence maximization on online social networks • E.g., structure health monitoring

  4. An example of hybrid approaches Abstracted Real Sensing Physical Environments Signal Processing Communication Actuation Sensor node Our solution

  5. Deciding factor • Scale, controllability, repeatability and fidelity • Simulation wins on the first 3 • Test experiments win on fidelity (but not always) • Emulation comes in between • However, if you are doing systems research, the expectation is to have testbed experiments complimented by simulations

  6. When do you design the experiments • From the onset of the project! • What datasets? • What hardware/software? • Which user group? • What metrics? • What to compare against?

  7. Analyzing the results • Know what to expect • Don’t bring completely out-of-the-line results to your advisor • Understand what you didn’t expect • Verify the implementation is correct • Unit test (isolation), sanity check • Find an explanation • In CS, there is always an explanation to everything • When you are done with the experiments, you are still far from being done • Unexpected results can be quite useful in improving your solutions

  8. An Example GNU Radio Signal Generator USRP2 Pre - Amp Trans Amp 440mVpp 4.161Vpp 238.05Vpp Concrete Channel Oscilloscope 1.081Vpp Gain = -22.23 dB USRP Daughterboard File Sink 2.5mVpp

  9. Presenting your results • Statistics • Please give me error bars or confidence intervals • Is the observation really statistically significant? • Obama vsRomeneyGullop poll 47% vs 49% • Both sides of the story: false positive and false negative • Experimental settings please! • 3D graphs should be banned • Up and down curves a bad sign • Colors on black-white printout? Fonts? • Embedded font issues • Tell what you observe and provide insight gained • Don’t just run out of stream when you reach the evaluation section of your paper

More Related