1 / 10

Integrating Linux Technology with Condor

Red Hat is investing in integrating key technologies with Condor to create a new generation of compute platforms in Linux. This includes real-time capabilities, virtualization, messaging, and stateless execution. Red Hat will be working closely with the Condor team to provide native scheduling support in Linux.

rodarte
Download Presentation

Integrating Linux Technology with Condor

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Integrating Linux Technology with Condor Kim van der Riet Principal Software Engineer

  2. Background Red Hat is investing in a set of defining technologies, actively being developed in the emerging technologies group and in conjunction with partners. These key projects include among others... Real-time (provides pre-emption and latency determinism) Virtulization (provides scale, VM, and ability to provide environment with Job) Messaging (built on AMQP) Stateless (ability to execute from read only image/device) ....and will work with the Condor team to provide native scheduling support in Linux. The combination of these technologies in a standard Linux distribution will provide a new generation of compute platforms. 2

  3. Creating a stronger Linux compute platform Many large IT shops have requirement to schedule work to a compute platform. The work on Grids range from financial services grids, build / test environments, scientific analysis, batch processing, data mining, to name a few. Red Hat believes that having the key facilities included with and integrated into Linux will provide economy of scale and the ability to more efficiently manage compute workloads. Examples include: Condor scheduler support for libvirt can provide: Seamless delivery of an environment with a Job Stop, start, location transparency for execution Condor integration with messaging (AMQP) in the OS can provide: Management of jobs together with the OS resources Standard interfaces for scheduling/reaping data on the grid Condor integration with d-cache and messaging allows parallel scheduled job execution (for things like rate-quote in FS). Stateless provides the ability to remotely host execution environments. Working together we (UW-Madison, RH, community, 3rd party vendors) can provide a stronger, more effective compute platform. 3

  4. What will Red Hat be doing? Red Hat will be investing into the Condor project locally in Madison WI, in addition to driving work required in upstream and related projects. This work will include: Engineering on Condor features & infrastructure Should result in tighter integration with related technologies Tighter kernel integration Information transfer between the Condor team and Red Hat engineers working on things like Messaging, Virtualization, etc. Creating and packaging Condor components for Linux distributions Support for Condor packaged in RH distributions All work goes back to upstream communities, so this partnership will benefit all. Shameless plug: If you want to be involved, Red Hat is hiring... 4

  5. Some questions answered Will Condor on RHEL / Fedora / Linux have an advantage? Yes, tighter integration with the OS and working with both Red Hat and other upstream communities should provide benefits. Will Condor become Linux only? No, Condor will remain a cross-platform project. However, there may be unique advantages to running it on Linux Is Red Hat going to provide technical support for Condor? At what level? At what cost? Yes, as a result of this work, support will be provided. No specifics on the cost can be provided at this time, however. Is Condor going to be packaged as part of the Red Hat distribution? Yes, it will become part of the distributions much the same way that related projects do. 5

  6. Some questions answered (2) What is going to be the relationship with regards to the source code? Is there going to be a fork in the code, aka a UW Condor and a Red Hat Condor? Red Hat will co-develop with the Condor team. Work will be performed upstream, and all efforts will be made NOT to fork any code. What kind of users/customers is Red Hat targeting with this relationship with Condor? High Performance/Throughput market? Data centers? Finance, Scientific, Large-scale build/test, but not limited to specific markets. Can I expect to see Red Hat engineers responding to questions/posts on Condor mailing lists/newsgroups? What will be the relationship the existing Condor user community? Yes. Red Hat strives to play an active roll within the communities with which it works. 6

  7. Some questions answered (3) Why is Red Hat doing this? What is the benefit for Red Hat? Conversely, what would be the benefit to the existing Condor community? Red Hat sees compute as a strategic technology. Red Hat customers will see a more powerful platform when running Linux. The Condor team will have access to Red Hat's deep technical expertise. What is the time frame of the commitment? Is it for X years? Or indefinite? The relationship is not limited by time, but we anticipate extensive development for at least 3 years. How much effort (manpower) is Red Hat targeting for this effort? How much for support activities vs. development activities? Red Hat expects to bring at least 12 full time engineering resources onto the project, mostly located in Madison. Red Hat will not disclose resource effort for support/productization. 7

  8. Some questions answered (4) What will Red Hat's development activities focus on? Why did you decide on these activities? The initial focus of the work will be: Creating a distribution of Condor specifically for Linux. Integration with Linux Virtualization. Integration with Messaging (RHM / AMQP). Development of d-cache. These are areas where users are spending a lot of effort to create compute platforms today. What does this relationship mean for Condor on Windows? on Mac OS X? Nothing, this work will not directly benefit these platforms, but neither will it hurt them. What are the implications of this relationship upon the Linux kernel? It creates information flow between the teams, it also makes it possible for to Condor to utilize specialized kernel features. Perhaps most interesting is that it opens the door for VM-specific or virtualization models designed specifically for Condor and/or Grid computing in general. 8

  9. Resources Contact information Carl Trieloff, Red Hat Emerging Technologies - cctrieloff@redhat.com Kim van der Riet, Red Hat Emerging Technologies - kim.vdriet@redhat.com If you want to join the team Go to www.redhat.com/careers and post a copy of your resume. E-mail one of the above contacts. Related Projects AMQP - www.amqp.org RHM / Qpid - incubator.apache.org/qpid Libvirt - www.libvirt.org REHL - www.redhat.com Fedora - fedora.redhat.com Red Hat emerging technologies (ET) projects - et.redhat.com 9

More Related