Download
1 / 110
1.18k likes | 1.82k Views
Robotic grasping is a complex field. hand design: high level (number of fingers, kinematic structure, etc.) and low-level (mechanism design, motors, materials, etc.); hand control algorithms: high level (find an appropriate posture for a given task) and low-level (execute the desired posture); inf
E N D
1. Robotic Grasp Planning Peter Allen
Department of Computer Science
Columbia University
2. Robotic grasping is a complex field hand design: high level (number of fingers, kinematic structure, etc.) and low-level (mechanism design, motors, materials, etc.);
hand control algorithms: high level (find an appropriate posture for a given task) and low-level (execute the desired posture);
information from sensors (tactile, vision, range sensing, etc.);
any pre-existing knowledge of objects shape, semantics and tasks (e.g. a cup is likely to be found on a table, should not be held upside-down, etc.);
...and more!
3. Human Grasping vs. Robotic Grasping Human performance provides both a benchmark to compare against, and a working example that we can attempt to learn from. However, it has proven very elusive to replicate:
the human hand is a very complex piece of equipment, with amazing capabilities;
humans benefit from an unmatched combination of visual and tactile sensing;
human continuously practice grasping and manipulation, the amount of data they are exposed to dwarfs anything tried so far in robotics;
are we setting the bar too high\
Speed stacking!
4. Robotic Manipulation Process allowing a robot to make physical changes to the world around it.
Includes moving objects, joining objects, reshaping objects, etc.
Moving objects can be done by grasping, pushing, carrying, dropping, throwing, etc.
Task accomplished by a manipulator with some sort of end-effecter.
5. Grippers vs. Hands Structured environments
Reliable
Simple
Low cost
Unstructured environments
Adaptable
Complex
Expensive
7. Simulation for Grasp Planning Integrated grasp analysis
Grasp quality, weak point, force optimization
Perform many grasps quickly
Faster than using a real arm and hand
Build a library of saved grasps
Recall grasp when object is encountered again
8. GraspIt!: A Tool for Grasping Research Library of hands and objects
Intuitive user interface
Visualize grasp wrench space
Quality measures evaluate grasp
Dynamic simulation
Grasp Planning
9. Library of hands and objects
Intuitive user interface
Visualize grasp wrench space
Quality measures evaluate grasp
Dynamic simulation
Grasp Planning GraspIt: A Tool for Grasping Research
10. GraspIt Components
12. Hand Kinematics
13. Hand/Arm Library
14. Grasp Analysis Occurs when a contact is formed or broken
Computes space of forces and torques that can be applied by the grasp
Quality measures numerically evaluate grasp
Provides a means to evaluate grasps
Compare grasps of one hand, one object
Compare grasps of many hands, one object
Compare grasps of many hands, across a task specified object set.
15. Wrench Spaces In 3-space, a wrench is a 6D vector composed of a force and a torque:
The space of wrenches that may need to be applied during a task is the task wrench space.
The space of wrenches that can be applied by a grasp is the grasp wrench space.
A possible quality measure:
16. Special Types of Grasps A force-closure grasp completely restrains the object.
Origin is contained within grasp wrench space.
A manipulable grasp can impart arbitrary velocities on the object without breaking contact.
17. Friction Cones
18. 2D Example
19. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
20. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
21. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
22. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
23. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
24. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
25. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
26. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
27. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
28. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
29. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
30. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
31. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
32. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
33. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
34. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
35. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
36. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
37. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
38. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
39. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
40. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
41. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
42. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
43. Contact Wrenches Each force acts at a position on object.
Compute the corresponding wrench with respect to objects center of gravity:
l - torque scale factor,
44. Objective: find total space of wrenches that can be applied by a grasp of unit magnitude.
Grasp vector:
Define with norm:
Sum magnitude of contact normal forces is 1.
Compute grasp wrench space using qhull: Grasp Wrench Space
45. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
46. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
47. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
48. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
49. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
50. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
51. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
52. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
53. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
54. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
55. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
56. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
57. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
58. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
59. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
60. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
61. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
62. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
63. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
64. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
65. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
66. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
67. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
68. Two Measures of Quality Assume task wrench space is unknown.
Estimate with wrench space ball - good grasps will resist all wrenches equally well.
Previously proposed measures of quality:
Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space.
Volume, v, of unit grasp wrench space.
69. Grasp Wrench Space Projections To visualize the 6D grasp wrench space project it to 3-space by fixing three coordinates.
A useful choice:
See the forces that can be applied without applying a net torque (or vice versa).
70. Visualizing the Results
71. Comparing Grasps
72. Comparing Grasps
73. Comparing Grasps
74. Simulating Dynamics
75. Simulating Dynamics
76. Hand Posture Subspaces A grasp can be a considered a point in a high-dimensional hand configuration space
To find a grasp, need to search this space: expensive, intractable
77. Eigengrasps Can be seen as generalization of grasp taxonomy [Napier 56, Cutkosky 89, Iberall 97, etc.]
78. Grasp Planning using Eigengrasps Low-dimensionality basis for grasping
Derived from human user studies and mapped to robotic hands
Practical implications
dimensionality reduction
Completeness: can it cover the space of good grasps?
Search eigengrasp subspace for good grasps
use simulated annealing as search algorithm
stochastic nature allows escape from local minima
no analytical gradient is necessary
Planner works independent of hand kinematics
79. Grasp planning using Eigengrasps Apply Eigengrasp concept to a range of hand models:
From simple gripper (4 DOF) to human hand (20 DOF)
For human hand, use available data
For robotic hands, attempt to define grasp subspaces similar to the one obtained in the case of the human hand
direct transfer possible for anthropomorphic models
81. Grasp planning using Eigengrasps Energy function formulation attempts to bring pre-specified contact locations on the palm in contact with the object
Simulated annealing search is performed over 8 variables
6 for wrist position / orientation
2 eigengrasp amplitudes
Our implementation: 100K iterations, ~150 seconds
83. Grasp planning using Eigengrasps
85. Grasp planning using Eigengrasps
86. Grasp planning using Eigengrasps We need the final posture for stable (form-closure) grasps
Completeness: perfect grasp postures can not be found in EG space
Can come very close with simple heuristic: close all fingers!
Perform form-closure tests in parallel in a multi-threaded environment
takes advantage of multi-core architecture
Form-closure in 23 out of 30 cases
87. Interactive Online Grasping Dexterous hand prosthetics
human control combined with artificial hardware and algorithms
benefits from results in both human and robotic grasping
but also open new interface problems!
Task: achieve a stable grasp using a dexterous prosthetic
3D tooltip positioning under neural control [Taylor et al. 02]
finger posture has proven more difficult to specify
EMG [Zecca et al. 02], neural control [Taylor et al. 03]
low-dimensional, noisy
Approach:
reduce the dimensionality of the searched posture space
use on-line operator input to simplify hand position
Matei Ciocarlie and Peter K. Allen, Hand Posture Subspaces for Dexterous Robotic Grasping, The International Journal of Robotics Research, Jul 2009; vol. 28: pp. 851 - 867.
88. Neural Control of Prosthetic Hands
89. Interactive Dexterous Grasping Operator
specifies approach direction
has no control over finger posture
Automated grasp planner
knows object shape and position
records position input
searches for a stable grasp according to user approach direction
sets finger posture accordingly
Operator
changes approach direction, or
completes grasp (binary close all fingers command)
90. Interactive Grasp Planning Hand posture: 2 variables (eigengrasp amplitudes)
Hand position:
user not expected to fully specify final position
affects interaction, can not handle noise
3 variables to re-parameterize hand approach:
d, ? and f define a conical search space
Total: 5 variables
loops of 2000 Simulated Annealing iterations
continuously update base hand position
search does not get stuck if one loop fails
best pre-grasps tested for form-closure
91. Interactive Grasp Planning Examples
92. Interactive Grasp Planning Examples
93. Interactive Grasp Planning Examples
94. Is Grasping Indexable?
Many previous attempts to taxonomize grasps
Is there a finite set of grasps we can pre-compute?
If so, can we build an indexable database of grasps?
Given a new object to grasp, can we find a similar grasp
Some Problems:
Lots of objects to grasp
Lots of DOF in a hand (~20 + 6 in human hand)
Lots of different robotic hands
Intractable? But maybe not.
95. The Columbia Grasp Database We have built a large scale grasp database.
High quality form closure grasps for:
Multiple hands
Thousands of objects
Hundreds of thousands of grasps
We can use the grasp database for:
Data-driven grasp planning
Rigorous benchmarking of grasp planners
96. Finding Stable Grasps Problem: Collecting grasp data from humans is expensive
Human studies do not scale to large datasets
How can we collect data on robotic hands?
Solution: Automate the data collection
Use a powerful offline grasp planner as the source
Works for human or robotic hands, inexpensive
Caveat: Database quality depends on the quality of this planner
Advantages of Eigengrasps
Stochastic, so can give many grasps for one object
Gives form-closure grasps
97. Objects to Grasp We reused the 3D models from the Princeton Shape Benchmark*
Well known academic dataset of 1,814 models
Encourage collaboration with shape researchers
All models resized to graspable sizes
We provide grasps at 4 scales
because grasping is scale dependent
.75, 1.0, 1.25 and 1.5 times the size of each model
7,256 3D models in all
*Shilane et al., SMI 2004
98. Robotic Hands We provide grasps for 4 hands
Human hand model (20 DOF)
Barrett Hand (4 DOF + disengaging clutch)
Barrett Hand with rubber coating
Willow Garage gripper
More hands to come!
99. The Completed Database Building the database
Took 1 month on 6 multicore computers.
Resulted in ~250,000 distinct form closure grasps
For each grasp we saved:
Grasp and pre-grasp joint angles and position
Ferrari-Canny volume and epsilon metrics
Available for download now!
grasping.cs.columbia.edu
101. Data Driven Grasp Planning Given a new 3D model to grasp
Find nearest geometric neighbors in database
Initial implementation with Zernike descriptors
Collect pre-grasps from neighbor models
Evaluate candidates in GraspIt! simulator
103. CGDB: Benchmarking That seemed to work, but can we quantify it?
Historically not easily!
Evaluating a grasp planner is very difficult
No consistent benchmarks for comparison
Our proposal: Benchmark on this database
Simulate planned grasps on each model
Report aggregate statistics
Easily repeatable, no expensive hardware required
Quality metrics are only possible in simulation
104. Benchmarking We benchmarked two planners
Eigengrasp Planner and Database Planner
We benchmarked two hands
Human and Barrett
We report the average quality of the first 25 grasps from each planner over 1814 models (single scale)
105. Planner Benchmarks Data driven vs computationally drivenData driven vs computationally driven
106. Results Consistent results over both hands
For first grasp
Database planners quality matches Eigengrasps
But is much faster (not shown in the chart)
1 minute vs. 10-15 minutes
For subsequent grasps
Database planner produces many more good grasps
107. Grasping a Sensed Object Data from a NextEngine laser scanner
noisy, missing and occluded regions
Do geometric match into database, test these grasps
Works for human or robotic hands
109. GraspIt! Simulation Engine focused on robotic grasping
Can quickly test thousands of potential grasps, using arbitrary hand geometries and object models.
Includes dynamics, visualization, GWS calculation
Easy to add new hand designs
Eigengrasps: Reduced subspace for grasping
On-line grasp planning: Add user input to eigengrasps to reduce search space further
Applied to a number of robotic and human hand models, using both real and simulated environments
Columbia Grasp Database: indexed set of grasps across large model and hand space
110. Acknowledgements Andy Miller: Original GraspIt! design and implementation
Matei Ciocarlie: Eigengrasps, online planner, soft finger contacts
Corey Goldfeder: Columbia Grasp Data Base
NSF and NIH
