1 / 28

Servers and Threads

Servers and Threads. Jeff Chase Duke University. Processes and threads. stack. main thread. virtual address space. other threads (optional). +…. +. STOP. Each process has a virtual address space (VAS): a private name space for the virtual memory it uses.

bairn
Download Presentation

Servers and Threads

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Servers and Threads Jeff Chase Duke University

  2. Processes and threads stack main thread virtual address space other threads (optional) +… + STOP Each process has a virtual address space (VAS): a private name space for the virtual memory it uses. The VAS is both a “sandbox” and a “lockbox”: it limits what the process can see/do, and protects its data from others. wait From now on, we suppose that a process could have additional threads. We are not concerned with how to implement them, but we presume that they can all make system calls and block independently. Each process has a thread bound to the VAS, with stacks (user and kernel). If we say a process does something, we really mean its thread does it. The kernel can suspend/restart the thread wherever and whenever it wants.

  3. Threads: a familiar metaphor 1 2 3 Page links and back button navigate a “stack” of pages in each tab. Each tab has its own stack. One tab is active at any given time. You create/destroy tabs as needed. You switch between tabs at your whim. Similarly, each thread has a separate stack. The OS switches between threads at its whim. One thread is active per CPU core at any given time. time 

  4. Threads • A thread is a stream of control. • defined by CPU register context (PC, SP, …) • Note: process “context” is thread context plus protected registers defining current VAS, e.g., ASID or “page table base register(s)”. • Generally “context” is the register values and referenced memory state (stack, page tables) • Multiple threads can execute independently: • They can run in parallel on multiple CPUs... • physical concurrency • …or arbitrarily interleaved on a single CPU. • logical concurrency • Each thread must have its own stack.

  5. Two threads sharing a CPU concept reality context switch

  6. Two threads: closer look stack stack address space “on deck” and ready to run 0 common runtime program x code library running thread data R0 CPU (core) Rn y x PC y SP registers high

  7. Thread context switch stack switch out switch in address space 0 common runtime program x code library data R0 1. save registers CPU (core) Rn y x PC y SP registers 2. load registers stack high

  8. Thread states and transitions exit exited running EXIT STOP The kernel process/thread scheduler governs these transitions. wait sleep blocked ready wakeup wait, STOP, read, write, listen, receive, etc. Sleep and wakeup are internal primitives. Wakeup adds a thread to the scheduler’s ready pool: a set of threads in the ready state.

  9. CPU Scheduling 101 The OS scheduler makes a sequence of “moves”. • Next move: if a CPU core is idle, pick a ready thread t from the ready pool and dispatch it (run it). • Scheduler’s choice is “nondeterministic” • Scheduler’s choice determines interleaving of execution blocked threads If timer expires, or wait/yield/terminate ready pool Wakeup GetNextToRun SWITCH()

  10. Event-driven programming • Some of the goals of threads can be met by using an event-driven programming model. • An event-driven program executes a sequence of events. The program consists of a set of handlers for those events. • e.g., Unix signals • The program executes sequentially (no concurrency). But the interleaving of handler executions is determined by the event order. • Pure event-driven programming can simplify management of inherently concurrent activities. • E.g., I/O, user interaction, children, client requests • Some of these needs can be met using either threads or event-driven programming. But often we need both.

  11. Event-driven programming vs. threads • Often we can choose among event-driven or threaded structures. • So it has been common for academics and developers to argue the relative merits of “event-driven programming vs. threads”. • But they are not mutually exclusive. • Anyway, we need both: to get real parallelism on real systems (e.g., multicore), we need some kind of threads underneath anyway. • We often use event-driven programming built above threads and/or combined with threads in a hybrid model. • For example, each thread may be event-driven, or multiple threads may rendezvous on a shared event queue. • We illustrate the continuum by looking first at Android and then at concurrency management in servers (e.g., the Apache Web server).

  12. Android app: main event loop • The main thread of an Android app is called the Activity Thread. • It receives a sequence of events and invokes their handlers. • Also called the “UI thread” because it receives all User Interface events. • screen taps, clicks, swipes, etc. • All UI calls must be made by the UI thread: the UI lib is not thread-safe. • MS-Windows apps are similar. • The UI thread must not block! • If it blocks, then the app becomes unresponsive to user input: bad. 1 2 3

  13. Android event loop: a closer look • The main thread delivers UI events and intents to Activity components. • It also delivers events (broadcast intents) to Receiver components. • Handlers defined for these components must not block. • The handlers execute serially in event arrival order. • Note: Service and ContentProvider components receive invocations from other apps (i.e., they are servers). • These invocations run on different threads…more on that later. main event loop Activity Activity UI clicks and intents Receiver Dispatch events by invoking component-defined handlers.

  14. Event-driven programming • This “design pattern” is called event-driven (event-based) programming. • In its pure form the thread never blocks, except to wait for the next event, whatever it is. • We can think of the program as a set of handlers: the system upcalls a handler to dispatch each event. • Note: here we are using the term “event” to refer to any notification: • arriving input • asynchronous I/O completion • subscribed events • child stop/exit, “signals”, etc. events Dispatch events by invoking handlers (upcalls).

  15. Android event classes: some details • Android defines a set of classes for event-driven programming in conjunction with threads. • A thread may have at most one Looper bound to a MessageQueue. • Each Looper has exactly one thread and exactly one MessageQueue. • The Looper has an interface to register Handlers. • There may be any number of Handlers registered per Looper. • These classes are used for the UI thread, but have other uses as well. Looper Message MessageQueue Handler [These Android details are provided for completeness.]

  16. Android: adding services (simplified) main/UI thread binder thread pool main event loop Activity Service Activity Provider UI clicks and intents incoming binder messages Receiver Service

  17. Pool of event-driven threads • Android Binder receives a sequence of events (intents) in each process. • They include incoming intents on provider and service components. • Handlers for these intents may block. Therefore the app lib uses a pool of threads to invoke the Handlers for these incoming events. • Many Android apps don’t have these kinds of components: those apps can use a simple event-driven programming model and don’t need to know about threads at all. • But apps having these component types use a different design pattern: pool of event-driven threads. • This pattern is also common in multi-threaded servers, which poll socket descriptors listening for new requests. Let’s take a look.

  18. Multi-threaded RPC server [OpenGroup, late 1980s]

  19. Ideal event poll API Poll() • Delivers: returns exactly one event (message or notification), in its entirety, ready for service (dispatch). • Idles: Blocks iffthere is no event ready for dispatch. • Consumes: returns each posted event at most once. • Combines: any of many kinds of events (a poll set) may be returned through a single call to poll. • Synchronizes: may be shared by multiple processes or threads ( handlers are thread-safe as well).

  20. A look ahead • Various systems use various combinations of threaded/blocking and event-driven models. • Unix made some choices, and then more choices. • These choices failed for networked servers, which require effective concurrent handling of requests. • They failed because they violate each of the five properties for “ideal” event handling. • There is a large body of work addressing the resulting problems. Servers mostly work now. • More about server performance and Unix/Linux later. • The Android Binder model is closer to the ideal.

  21. Classic Unix • Single-threaded processes • Blocking system calls • Synchronous I/O: calling process blocks until each I/O request is “complete”. • Each blocking call waits for only a single kind of a event on a single object. • Process or file descriptor (e.g., file or socket) • Add signals when that model does not work. • With sockets: add select system call to monitor I/O on sets of sockets or other file descriptors. • select was slow for large poll sets. Now we have various variants: poll, epoll, pollet, kqueue. None are ideal.

  22. Inside your Web server Server operations create socket(s) bind to port number(s) listen to advertise port wait for client to arrive on port (select/poll/epoll of ports) accept client connection read or recv request write or send response close client socket Server application (Apache, Tomcat/Java, etc) accept queue packet queues listen queue disk queue

  23. Accept loop while (1) { int acceptsock = accept(sock, NULL, NULL); char *input = (char *)malloc(1024*sizeof (char)); recv(acceptsock, input, 1024, 0); int is_html = 0; char *contents = handle(input,&is_html); free(input); …send response… close(acceptsock); } If a server is listening on only one port/socket (“listener”), then it can skip the select/poll/epoll.

  24. Handling a request Accept Client Connection Read HTTP Request Header may block waiting on disk I/O may block waiting on network Find File Send HTTP Response Header Read File Send Data Want to be able to process requests concurrently.

  25. Web server (serial process) • Option 1: could handle requests serially • Easy to program, but painfully slow (why?) WS Client 1 Client 2 R1 arrives Receive R1 Disk request 1a R2 arrives 1a completes R1 completes Receive R2

  26. Web server (event-driven) • Option 2: use asynchronous I/O • Fast, but hard to program (why?) Disk WS Client 1 Client 2 R1 arrives Receive R1 Disk request 1a Start 1a R2 arrives Receive R2 Finish 1a 1a completes R1 completes

  27. Web server (multi-process) • Option 3: assign one thread per request • Where is each request’s state stored? Client 1 WS1 Client 2 WS2 R1 arrives Receive R1 Disk request 1a R2 arrives Receive R2 1a completes R1 completes

  28. Concurrency and pipelining CPU DISK Before NET CPU DISK After NET

More Related