A Concurrent Lock-Free Priority Queue for Multi-Thread Systems

1. A Concurrent Lock-Free Priority Queue for Multi-Thread Systems Bart Verzijlenberg October 16, 2007

2. Agenda • Introduction • Skip-List Overview • Issues with Concurrency • Lock-Free Priority Queues • Atomic Operations • Performance • Conclusion • Questions

3. Introduction • Priority queues are fundamental data structures • Used in a wide variety of applications • The efficient design of priority queues has been extensively researched • In order to use priority queues in a concurrent environment some type of locking is often used. • Some concurrent algorithms simply have a lock at the start of a sequential algorithm. • This locking causes blocking and may lead to starvation and deadlocks. • All are bad for system performance

4. Introduction contd. • Priority Queues have two general operations • DeleteMin • Insert • This algorithm uses the Skip-List data structure • Extends the Skip-List for concurrent use • The Skip-List is sorted on the priority of the nodes • To avoid the pitfalls of locking, this algorithm is lock-free

5. Skip-List Overview • Multi-layer linked list • Probabilistic alternative to balanced trees • Each node of height H has H pointers • Each pointer i points to the next node with a height of at least i • Probabilistic time complexity log(N) for N nodes • 50% of the nodes have height 1 • 25% of the nodes have height 2 • 12.5% of the nodes have height 3 • etc

6. Skip-List Overview

7. Insert in a Skip-List Inserting 17 in the list

8. Issues with Concurrency • When a node is inserted or deleted we need to change multiple pointers • Both in the node, and in other nodes • Need to make these changes consistently • But not necessarily all at once

9. N = Get 2->NextNode (N==4) Set 1->NextNode = N Set 2->NextNode = 3 Losing a Node Deleting Node 2 Insert Node 3

10. Lock-Free Priority Queues • In most 32-bit systems, the address of a 32-bit value is evenly divisible by 4 • Set the 0th bit to 1 using CAS to indicate the node is being deleted • Any concurrent operation will be notified of the delete when their CAS operation fails

11. Insert A Node • Find where the Node should be inserted • Simply do a search of the Queue • When coming across a node being deleted, help the deletion and then continue searching • Ensure no duplicate keys • Try to insert the Node • Use a CAS operation • Starting at the bottom, set the pointers in the node. • Help delete the Node if needed • The new Node may already be getting deleted

12. Delete a Node • Find the node to remove • May not be the head of the queue due to deletions • Try to set the delete mark on the Node • While searching, if a Node is being deleted we help the deletion • Set the delete mark on every pointer in the node • Remove the Node, starting at the top level

13. Atomic Operations • Some existing algorithms require atomic operations not found on most systems. • We need only three • Test And Set (TAS) • Fetch And Add (FAA) • Compare And Swap (CAA) • All are available on most common systems

14. Performance • Faster than other concurrent priority queues for systems with 3 or more threads.

15. Conclusion • This algorithm provides a fast and lock free priority queue • Furthermore, it is also a fast lock-free concurrent linked list (Skip-List) • Currently assumes availability of pointer arithmetic. • This needs to be changed for implementation in Java

16. Questions