Dynamics of networks

1. Jure Leskovec Computer Science Department Cornell University / Stanford University Dynamics of networks

2. Today: Rich data • Large on-line systems have detailed records of human activity • On-line communities: • Facebook (64 million users, billion dollar business) • MySpace (300 million users) • Communication: • Instant Messenger (~1 billion users) • News and Social media: • Blogging (250 million blogs world-wide, presidential candidates run blogs) • On-line worlds: • World of Warcraft (internal economy 1 billion USD) • Second Life (GDP of 700 million USD in ‘07)

3. Rich data as networks c) Social networks b) Internet (AS) a) World wide web d) Communication e) Citations f) Protein interactions

4. Networks: What do we know? • We know lots about the network structure: • Properties: Scale free [Barabasi-Albert ’99, Faloutsos3 ‘99], 6-degrees of separation [Milgram ’67], Small-diameter [Albert-Jeong-Barabasi ‘99],Bipartite cores [Kumar et al. ’99], Network motifs [Milo et al. ‘02], Communities [Newman ‘99], Hubs and authorities [Page et al. ’98, Kleinberg ‘99] • Models: Preferential attachment [Barabasi ’99], Small-world [Watts-Strogatz ‘98], Copying model [Kleinberg el al. ’99], Heuristically optimized tradeoffs [Fabrikant et al. ‘02],Latent Space Models [Raftery et al. ‘02], Searchability[Kleinberg ‘00],Bowtie [Broder et al. ‘00], Exponential Random Graphs [Frank-Strauss ‘86], Transit-stub [Zegura ‘97], Jellyfish [Tauro et al. ‘01] We know much less about processes and dynamics of networks

5. This talk: Network dynamics • Network Dynamics: • Network evolution • How network structure changes as the network grows and evolves? • Diffusion and cascading behavior • How do rumors and diseases spread over networks?

6. This talk: Scale matters • We need massive network data for the patterns to emerge: • MSN Messenger network[WWW ’08](the largest social network ever analyzed) • 240M people, 255B messages, 4.5 TB data • Product recommendations [EC ‘06] • 4M people, 16M recommendations • Blogosphere[work in progress] • 60M posts, 120M links

7. This talk: The structure

8. This talk: The structure

9. Background: Network models • Empirical findings on real graphs led to new network models • Such models make assumptions/predictions about other network properties • What about network evolution? Model Explains log prob. Power-law degree distribution Preferential attachment log degree

10. [w/ Kleinberg-Faloutsos, KDD ’05] Q1) Network evolution Internet • Prior models and intuition say that the network diameter slowly grows (like log N, log log N) • Diameter shrinks over time • as the network grows the distances between the nodes slowly decrease diameter size of the graph Citations diameter time

11. [w/ Kleinberg-Faloutsos, KDD ’05] Q1) Network evolution Internet • Networks are denser over time • Densification Power Law: a … densification exponent (1 ≤ a ≤ 2) • What is the relation between the number of nodes and the edges over time? • Prior work assumes: constant average degree over time E(t) a=1.2 Is shrinking diameter just a consequence of densification? N(t) Citations E(t) a=1.6 N(t)

12. Q1) Diameter of a densifyingGnp Erdos-Renyi random graph diameter Densifying random graph has increasing diameter Densification exponent a =1.3  There is more to shrinking diameter than just densification size of the graph

13. Q1) Diameter of a rewired network • Compare diameter of a: • True network (red) • Random network with the same degree distribution (blue) Citations diameter diameter size of the graph Densification + scale free structure give shrinking diameter

14. [w/ Backstrom-Kumar-Tomkins, KDD ’08] Q2) Edge attachment • We directly observe atomic events of network evolution(and not only network snapshots) and so on for millions… We observe evolution at finest scale • Test individual edge attachment • Directly observe events leading to network properties • Compare network models by likelihood (and not by just summary network statistics)

15. [w/ Backstrom-Kumar-Tomkins, KDD ’08] Setting: Edge-by-edge evolution • Network datasets • Full temporal information from the first edge onwards • LinkedIn (N=7m, E=30m), Flickr (N=600k, E=3m), Delicious (N=200k, E=430k), Answers (N=600k, E=2m) • How are edge destinations selected? Where do new edges attach? • Are edges more likely to attach to high degree nodes? • Are edges more likely to attach to nodes that are close?

16. [w/ Backstrom-Kumar-Tomkins, KDD ’08] Edge attachment degree bias • Are edges more likely to connect to higher degree nodes? PA Gnp Flickr

17. [w/ Backstrom-Kumar-Tomkins, KDD ’08] But, edges also attach locally • Just before the edge (u,w) is placed how many hops is between u and w? w Fraction of triad closing edges u PA v Gnp Flickr Real edges are local. Most of them close triangles!

18. Q3) Generating realistic graphs • Want to generate realistic networks: • Why synthetic graphs? • Anomaly detection, Simulations, Predictions, Null-model, Sharing privacy sensitive graphs, … • Q:Which network properties do we care about? • Q:What is a good model and how do we fit it? Given a real network Generate a synthetic network Compare graphs properties, e.g., degree distribution

19. [w/ Chakrabarti-Kleinberg-Faloutsos, PKDD ’05] The model: Kronecker graphs Edge probability Edge probability • We prove Kronecker graphs mimic real graphs: • Power-law degree distribution, Densification, Shrinking/stabilizing diameter, Spectral properties pij (3x3) (9x9) (27x27) Initiator Kronecker product of graph adjacency matrices (actually, there is also a nice social interpretation of the model) Given a real graph. How to estimate the initiator G1?

20. Kronecker graphs: Interpretation • Initiator matrix is a similarity matrix • Node u is described with k binary attributes u1, u2 ,…, uk • Probability of link between nodes u,v: P(u,v) = ∏ G1[ui, vi] v This is exactly Kronecker product a a b b u P(u,v) = b∙d∙c∙b c d c d

21. [w/ Faloutsos, ICML ’07] Kronecker graphs: Estimation • Maximum likelihood estimation • Naïve estimation takes O(N!N2): • N!for different node labelings: • Our solution: Metropolis sampling: N! (big) const • N2 for traversing graph adjacency matrix • Our solution: Kronecker product (E << N2):N2E • Do stochastic gradient descent P( | ) Kronecker arg max We estimate the model in O(E)

22. [w/ Faloutsos, ICML ’07] Estimation: Epinions (N=76k, E=510k) • We search the space of ~101,000,000 permutations • Fitting takes 2 hours • Real and Kronecker are very close Degree distribution Path lengths “Network” values probability # reachable pairs network value node degree rank number of hops

23. [w/ Dasgupta-Lang-Mahoney, WWW ’08] Kronecker & Network structure • Fitting Epinions we obtained G1 = • What does this tell about the network structure? 0.5 edges Core 0.9 edges Periphery0.1 edges Core-periphery (jellyfish, octopus) 0.5 edges

24. Small vs. Large networks • Small and large networks are fundamentally different G1 = G1 = Scientific collaborations (N=397, E=914) Collaboration network (N=4,158, E=13,422)

25. This talk: The structure   

26. Diffusion and Cascades • Behavior that cascades from node to node like an epidemic • News, opinions, rumors • Word-of-mouth in marketing • Infectious diseases • As activations spread through the network they leave a trace – a cascade Cascade (propagation graph) Network

27. [w/ Adamic-Huberman, EC ’06] 10% credit 10% off Setting 1: Viral marketing • People send and receive product recommendations, purchase products • Data:Large online retailer:4 million people, 16 million recommendations, 500k products

28. [w/ Glance-Hurst et al., SDM ’07] Setting 2: Blogosphere • Bloggers write posts and refer (link) to other posts and the information propagates • Data:10.5 million posts, 16 million links

29. [w/ Kleinberg-Singh, PAKDD ’06] Q1) What do cascades look like? • Are they stars? Chains? Trees? • Information cascades (blogosphere): propagation • Viral marketing (DVD recommendations): (ordered by frequency) • Viral marketing cascades are more social: • Collisions (no summarizers) • Richer non-tree structures

30. Q2) Human adoption curves • Prob. of adoption depends on the number of friends who have adopted [Bass ‘69, Shelling ’78] • What is the shape? • Distinction has consequences for models and algorithms To find the answer we need lots of data Prob. of adoption Prob. of adoption k = number of friends adopting k = number of friends adopting Diminishing returns? Critical mass?

31. [w/ Adamic-Huberman, EC ’06] Q2) Adoption curve: Validation DVD recommendations (8.2 million observations) Probability of purchasing # recommendations received Adoption curve follows the diminishing returns. Can we exploit this? Later similar findings were made for group membership [Backstrom-Huttenlocher-Kleinberg ‘06], and probability of communication [Kossinets-Watts ’06]

32. Cascade & outbreak detection • Blogs – information epidemics • Which are the influential/infectious blogs? • Viral marketing • Who are the trendsetters? • Influential people? • Disease spreading • Where to place monitoring stations to detect epidemics?

33. [w/ Krause-Guestrin et al., KDD ’07] The problem: Detecting cascades How to quickly detect epidemics as they spread? c1 c3 c2

34. [w/ Krause-Guestrin et al., KDD ’07] Two parts to the problem • Cost: • Cost of monitoring is node dependent • Reward: • Minimize the number of affected nodes: • If A are the monitored nodes, let R(A) denote the number of nodes we save We also consider other rewards: • Minimize time to detection • Maximize number of detected outbreaks A R(A) ( )

35. Optimization problem • Given: • Graph G(V,E), budget M • Data on how cascades C1, …, Ci,…,CKspread over time • Select a set of nodes A maximizing the reward subject to cost(A) ≤ M • Solving the problem exactly is NP-hard • Max-cover [Khuller et al. ’99] Reward for detecting cascade i

36. [w/ Krause-Guestrin et al., KDD ’07] Problem structure: Submodularity New monitored node: • Theorem: Reward function R is submodular(diminishing returns, think of it as “concavity”) S1 S1 S’ S’ R(A  {u}) – R(A) ≥ R(B  {u}) – R(B) S2 S3 Adding S’helps a lot Adding S’helps very little Gain of adding a node to a small set Gain of adding a node to a large set S2 S4 Placement A={S1, S2} Placement B={S1, S2, S3, S4} A B

37. [w/ Krause-Guestrin et al., KDD ’07] Solution: CELF Algorithm • We develop CELFalgorithm: • Two independent runs of a modified greedy • Solution set A’: ignore cost, greedily optimize reward • Solution set A’’: greedily optimize reward/cost ratio • Pick best of the two: arg max(R(A’), R(A’’)) • Theorem: If R is submodular then CELF is near optimal: • CELF achieves ½(1-1/e) factor approximation a d b c a c d b e Current solution: {a, c} Current solution: {a} Current solution: {} Marginal reward

38. Blogs: Information epidemics • Question: Which blogs should one read to be most up to date? • Idea:Select blogs to cover the blogosphere. • Each dot is a blog • Proximity is based on the number of common cascades

39. [w/ Krause-Guestrin et al., KDD ’07] Blogs: Information epidemics • Which blogs should one read to catch big stories? For more info see our website: www.blogcascade.org CELF Reward (higher is better) In-links (used by Technorati) Out-links # posts Random Number of selected blogs (sensors)

40. [w/ Krause et al., J. of Water Resource Planning] Same problem: Water Network • Given: • a real city water distribution network • data on how contaminants spread over time • Place sensors (to save lives) • Problem posed by the US Environmental Protection Agency c1 S S c2

41. [w/ Ostfeld et al., J. of Water Resource Planning] Water network: Results CELF • Our approach performed best at the Battle of Water Sensor Networks competition Degree Random Population saved (higher is better) Population Flow Number of placed sensors

42. Conclusions and reflections • Why are networks the way they are? • Only recently have basic properties been observed on a large scale • Confirms social science intuitions; calls others into question. • Benefits of working with large data • What patterns do we observe in massive networks? • What microscopic mechanisms cause them? Social network of the whole world?

43. [w/ Horvitz, WWW ’08, Nature ‘08] Planetary look on a small-world • Small-world experiment [Milgram ‘67] • People send letters from Nebraska to Boston • How many steps does it take? • Messenger social network – largest network analyzed • 240M people, 30B conversations, 4.5TB data MSN Messenger network Milgram’s small world experiment Number of steps between pairs of people (i.e., hops + 1)

44. Reflections on Diffusion • Predictive modeling of information diffusion • When, where and what information will create a cascade? • Where should one tap the network to get the effect they want? Social Media Marketing • How do news and information spread • New ranking and influence measures • Sentiment analysis from cascade structure • How to introduce incentives?

45. What’s next?