Fleets workloads

Automatic scaling

With a fleet, you can take advantage of large scale parallelism to complete large, compute-intensive tasks. Code Engine automatically scales the worker nodes in your fleet to meet resource requirements most efficiently. You can let Code Engine determine the profiles of the workers that are deployed, or you can choose a specific profile, such as a GPU type. In either case, Code Engine automatically scales the number of workers for the greatest level of efficiency.

The number of workers deployed is based on the number of tasks to complete, the resources required for an instance of your code, and the maximum number of concurrent instances you want to run at a time. By adjusting these settings, you can scale your fleet to complete as few as 1 or as many as several million tasks.

For example, imagine you have 4 tasks to complete. Each instance of code to complete a task requires 1 vCPU and 2 GB memory, for a total requirement of 4 vCPU and 8GB memory. If you set the maximum number of concurrent instances to 2, Code Engine can deploy 1 worker node with 2 vCPU and 4 GB memory to run 2 instances at one time. If you set the maximum number of concurrent instances to 4, Code Engine can deploy 2 of the same worker nodes to run all 4 instances at the same time across both workers. In the second scenario, the fleet finishes more quickly but requires more workers to be deployed. Keep in mind that there is an infinite number of combinations for automatic worker node deployment, and that worker nodes of various profiles might be deployed.

Automatic scaling is not available for GPUs. To deploy GPUs for a fleet, you must specify the type of GPU worker to deploy.

Task and instance specification

When deploying a fleet, users can specify the number of tasks to complete, the number of instances to run at a time, the order in which tasks are completed. Fleets can work on many tasks in parallel by starting multiple instances concurrently. All instances are created with the same amount of vCPU and memory as per the fleet’s specification. Additionally users can create a task specification file to provide specific commands and arguments for each task or to create custom task indexes.

Task storage in COS

You can use a COS bucket to provide the tasks for your fleet by specifying a persistent data store and bucket subpath when you create the fleet. Each object in the COS bucket represents a single task.

Isolation

Unlike batch jobs, fleets provide single-tenant isolation. Your fleet workers are not shared with other users.

Retries

Each fleet instance runs to completion. However, in the event of an error, Code Engine restarts the instance. You can limit the maximum number of retries to avoid restarting failed instances.

Status

A fleet is in the Running status when at least one worker node is provisioned. When the fleet is no longer running, it is in the Succeeded status if all tasks completed successfully, or Failed if one or more tasks failed. You can also cancel a fleet.

For more information, see Understanding the status of your fleet.

Worker profile

You can optionally choose the profile for your fleet worker. If you do not specify a profile, Code Engine will pick a profile for you.

Code Engine fleets support the following profile families:

Some profiles may not be available in all VPC network zones. You can use the following CLI command to list the profiles in a region.

ibmcloud is instance-profiles

• If you want to know zonal availability and capacity of certain profiles, such as H100 and L40s, open a support case and select "Virtual Private Cloud (VPC)" as the topic.
• If you need a certain VPC profile which is not supported by fleets today, open a support case and select "Code Engine" as the topic.