Scaling the real-time web with ASP.NET SignalR

2. Scaling the real-time web with ASP.NET SignalR Damian Edwards @damianedwards Senior Program Manager Scaling the real-time web with ASP.NET SignalR 3-502

3. This is not an introduction to SignalR talk…See www.asp.net/signalr for getting started information

4. Scaling traditional vs. real-time web

5. The same… but different • Scaling real-time web traffic shares many considerations with traditional web traffic, e.g. CPU, bandwidth, memory • Application scenarios have huge impact on scaling patterns • Big difference is in concurrency, supporting many long running idle and active connections vs. short requests • Different SignalR transports have different overheads

6. General things to watch out for • Blocking calls, e.g. block I/O • Never ever block a Hub method, it jams up the pipes • Use 4.5 and async where available • Sending large messages • Memory leaks caused by misunderstanding SignalR object lifetime, e.g. Hub instances • Session – don’t use it from SignalR, at all, ever (no support) • Use Hub state, cookies, browser storage, database, etc. instead

7. Remember the secret of scale:“Have your app do as little as possible. If you do nothing, you can scale infinitely.” – Scott Hanselman

8. Understanding SignalR load patterns

9. SignalR core architecture: pub/sub 1. Message serialized, saved to cache associated with signal, topic is marked for delivery, publish call returns publisher message bus message cache 3. Worker sends message to client as bytes over a transport 2. Worker is scheduled for a signal, selects a waiting subscriber, retrieves message from cache worker worker worker client client client client client

10. Pattern 1. Server broadcast • Low rate broadcast of the same payload to all clients • One message bus send maps to many users (fan out) • More clients don’t increase message bus traffic • e.g. application wide alerts, simple stock ticker

11. Pattern 2. Server push • Low rate broadcast of the unique payload to each client • One message bus send maps to one user (no fan out) • More clients means more message bus traffic • e.g. unique stock watch list, job monitoring

12. Pattern 3. User event driven • Broadcast on client actions • One message bus send maps to many users (fan out) • More clients means more message bus traffic • e.g. chat application, document collaboration

13. Pattern 4. High frequency real-time • Fixed high rate broadcast from servers and clients • We don’t recommend >25Hz • One message bus send maps to one user (no fan out) • More clients means more message bus traffic • e.g. real-time gaming, shootr.signalr.net

14. Load patterns summary • Server broadcast – low rate, message to all clients • Server push – low rate, message to unique clients • User event driven – broadcast on client action • High frequency – fixed high rate, unique messages

15. Demo: Tools for scale testing

16. Tools for scale testing summary • Performance counters • Microsoft.AspNet.SignalR.Utils, signalr.exe -ipc • crank.exe for generating SignalR load • Load Test Harness for basic endpoint hosting • stress.exe for running isolated scenarios in-proc • IIS settings for adjusting concurrency limits

17. Scaling to multiple servers

18. Scale-out issues: message delivery How do messages from one server get to the other servers in my web farm? ?

19. Scale-out issues: client transience When is a client disconnected from my app? server 1 server 2 How do I avoid duplicate & missed messages as I move from server to server? client

20. Scale-out issues: client distribution What happens if “foo” clients get many more messages than others? server 2 server 1 client “foo” client “foo” client “bar” client “bar” client “foo” client “foo” client “foo,bar” client “foo”

21. Plug-in scale-out providers • SQL Server, Redis & Windows Azure Service Bus • Incredibly simple setup (NuGet package + 1 line of code) • Great for the server broadcast load pattern • Limited for other scenarios • Every message goes to every server, so as traffic increases you’re limited by how fast any one web server can pull messages off of the backplane • Backplanes are *much* slower than single-server performance; throughput != scale

22. Scale-out provider architecture backplane Messages sharded over multiple backplane streams web nodes clients

23. Custom scale-out: Common server • Good for document collaboration, white-boarding • Clients working on same subject go to the same server • Application knows which subject is on each server • Similar to the idea of a real-time gaming lobby and then connecting to a game server • Use groups to segregate traffic, e.g. a group per document • Persist all state changes to your app state • Scale up to a dedicated server per collaboration subject

24. Custom scale-out: Server affinity foo server 2 server 1 Connect for “foo”! Sure, connect to http://server2 I want to work on document “foo” client client

25. Custom scale-out: Specific server • Good for the server push scenario; unique payloads • Connect client to a specific server & save that mapping • Publish messages for client to mapped server via endpoint

26. Custom scale-out: Specific server Client #1 is on server 2 Which server is client #1 on? Message for client #1 Client #1 on server 2 server 2 server 1 Connect! Sure, connect to http://server2 I want to connect client #1

27. Custom scale-out: Filtered message bus • Good for the server push scenario; unique payloads • Web servers connected to a bus with a filtered subscription • Update the filter on each connection connect • When message arrives do a local broadcast

28. Custom scale-out: Filtered message bus backplane web nodes clients “foo” “foo”

29. Custom scale-out: Server transition • Good for real-time gaming when coupled with an in-game experience, e.g. hyperspace to another quadrant (move to another server) • Single-box equiv. performance (very high) • Can’t send messages between clients on different servers without more custom work

30. Custom scale-out: Hybrid • Mix and match the various patterns • e.g. Combine the plug-in scale-out providers with server affinity approach for fault-tolerant document collaboration, where user first arrives at lobby, chooses document/room, then is connected to the cluster currently serving that document/room, which itself is configured with one of the plug-in scale-our providers • e.g. Use server transition for the high-frequency aspect of the app and custom server-to-server delivery for in-app messaging or alerts

31. We’re working on providing better guidance & samples in this space

32. 2.0 scale & performance improvements • First of all, 1.1 included many performance improvements so use that • Support for pre-serialization of messages for when you want to send data that is already JSON formatted • Support for single-serialization when sending to multiple signals

33. Resources • www.asp.net/signalr • www.campusmvp.net/signalr-ebook • github.com/signalr/signalr • twitter.com/damianedwards

34. Required Slide *delete this box when your slide is finalized Your MS Tag will be inserted here during the final scrub. Evaluate this session • Scan this QR codeto evaluate this session and be automatically entered in a drawing to win a prize!