Finding Core Number On Queue and LSF

  • Thread starter Thread starter Soaring Crane
  • Start date Start date
  • Tags Tags
    Core Queue
Click For Summary
SUMMARY

The discussion centers on determining the maximum number of cores that can be utilized on a specific queue in IBM's Load Sharing Facility (LSF) using the bqueues command. Users noted that LSF operates on the concept of "processors" rather than directly on core counts, as it manages jobs across various hardware configurations. The configuration of LSF allows for flexibility in resource allocation, but it requires careful management through shell scripts to enforce resource limits effectively.

PREREQUISITES
  • Understanding of IBM LSF job scheduling
  • Familiarity with the bqueues command
  • Knowledge of processor and core distinctions in computing
  • Basic shell scripting skills for resource management
NEXT STEPS
  • Explore IBM LSF documentation for detailed command usage
  • Learn about configuring processor slots in LSF
  • Research shell scripting techniques for resource allocation
  • Investigate best practices for managing job queues in LSF
USEFUL FOR

This discussion is beneficial for system administrators, LSF users, and developers involved in job scheduling and resource management within high-performance computing environments.

Soaring Crane
Messages
461
Reaction score
0
I’m trying to find out the largest number of cores that can be used on a specific queue in LSF with the bqueues command. I’ve used the option –l after reading the manual, but I don’t see any information about cores listed. Any suggestions?

Thanks.
 
Technology news on Phys.org
It's a while since I used LSF, but IIRC, LSF works with more general concepts than "number of cores". Remember one LSF job queue might be starting jobs on several different computers with different hardware specifications, and also one LSF job might be running in parallel on several computers (not just several cores of one computer).

LSF has the concept of the number of "processors" that a each computer has, and the number of "processor slots" that each job requests. It's up to the humans who configure LSF how to map those onto the actual hardware available. For example the number of "processors" might be used as measure of different CPU speeds on different single-processor computers, not necessarily the number of physical cores.

IIRC LSF doesn't know anything about how a job uses multiprocessinig or multi-threading once it is running. If you want to control that (e.g. run four job streams with 4, 2, 1, 1 cores available on a single 8-core machine) either you have to trust your users not to abuse the system, or you write shell scripts to set the approporiate environment variables or whatever, to tell and/or enforce what resources a program can use when it runs. That still means trusting the software developers to do it right - and of course the users and developers may be the same people, in sone situations.
 

Similar threads

  • · Replies 8 ·
Replies
8
Views
2K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 10 ·
Replies
10
Views
3K
Replies
6
Views
2K
  • · Replies 43 ·
2
Replies
43
Views
7K
  • · Replies 9 ·
Replies
9
Views
2K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 1 ·
Replies
1
Views
3K
Replies
3
Views
2K
  • · Replies 29 ·
Replies
29
Views
4K