Indexing and Retrieving Dynamic Brain Images

Kickoff Meeting March 28, 2003 SCILS, Rutgers Paul B. Kantor, PI Indexing and Retrieving Dynamic Brain Images

Overall Model • A “brain event” is characterized by an “activation” a(t) which is an (imaginary?) pointer that moves through the brain as neurons act sequentially in some “mental process”. • The set of points {t,a(t)} characterizes the “mental process”

Model (2) • The set of points can be conceived of as a graph in 4-dimensional space with coordinates x,t. • The graph can be concisely represented by graph indexing techniques • Those indexing techniques can be used to support retrieval of related “mental processes” even if the associated circumstances are not apparently similar.

Experimental-Analytic Paradigm • Do {until funds are exhausted} • Experiments produce data • Analysts reduce it to graphs • Study of graphs suggests new experiments • loop • Funds presently: • Rutgers ISTC Pilot funds 25K • NSF 2. Millions/3years • McDonnell Foundation – 500K/ • NIH R01 (planning)

Roles of Key Personnel • J. Cohen - design, conduct experiments • C. Hanson - design, conduct expts; manage data • S. Hanson - co-PI design expts; fusion time resolution • P. Kantor - co-PI design indexing and retrieval • D. Silver - reduce activations to centroids • S. Dickinson - index and retrieve graphs • L. Shepp - improve time resolution -- physics level • B. Bly - RUMBA software

Additional Key Personnel • Graduate Students • Ulukbek Ibraev -- graph finding and indexing • Xiaosong Yuan -- correlation and image analysis • Arnav Sheth -- diffusion of signal around activation • Yaroslav Halchenko—Fusion EEG/fMRI • Adi Zaimi—Experimental Data Collection and Design, Simulations, Modeling • Other Personnel • Donovan Rebbechi—Programming, RUMBA architecture • Mike Edwards—Programming, Archive Maintenance • Barak Pearlmutter—Theory and Algorithm Development

Threats to the Undertaking A. Time resolution of fMRI is very poor B. Hemodynamic response wrt to Neural spike firing is poorly understood C. Signals may leap across brain on long neurons D. All brain activation may be connectionist/distributed E. All other threats (Tell the Wigner story) • Today’s focus: • Improving time resolution & Dynamic Activity Modeling

Improving time resolution • New basis functions for selected regions (today) • Shepp et al • Fusion (at a later meeting)- Hanson et al • Ingenious spacing of stimuli and collection • Kantor (Later)

Dynamic brain activity modeling(BOLD) Wavelets (today) --Daubechies Cluster following (today?) --Kantor & Silver Skeletonization (at a later meeting) --Dickinson

TOOLS and ARCHIVE RUMBA tools (at another meeting) --Ben Bly Archive—RUMBA sharing --Donovan Rebbechi AIR- RUMBA --Donovan Rebbechi

Data Collection & Cognitive Theory Continuous fMRI paradigms: Event Perception (at later meeting) --Catherine Hanson Similarity based fMRI paradigms: Flanker task (at a later meeting) --Jonathan Cohen Allegra User Group--(set up by Cohen—www. Fill in)

Administrivia • Although we are all hard at work • we have just completed the contract Rutgers-Princeton • the NSF would like to know what we are accomplishing, to decide whether to give us the next part of the funding • Materials so far submitted are at the web site: http://scils.rutgers.edu/~kantor/SECRET/brain/AnnualReport/draft.html • Please send material to me. • kantor@scils.rutgers.edu

Indexing and Retrieving Dynamic Brain Images