Community hub
0 subscribers7 pages, 0 posts
Recent from talks
All channels
Be the first to start a discussion here.
Be the first to start a discussion here.
Be the first to start a discussion here.
Be the first to start a discussion here.
Contribute something
Welcome to the community hub built to collect knowledge and have discussions related to Cluster manager.
Nothing was collected or created yet.
Cluster manager
View on Wikipediafrom Wikipedia
Within cluster and parallel computing, a cluster manager is usually backend graphical user interface (GUI) or command-line interface (CLI) software that runs on a set of cluster nodes that it manages (in some cases it runs on a different server or cluster of management servers). The cluster manager works together with a cluster management agent. These agents run on each node of the cluster to manage and configure services, a set of services, or to manage and configure the complete cluster server itself (see supercomputing.) In some cases the cluster manager is mostly used to dispatch work for the cluster (or cloud) to perform. In this last case a subset of the cluster manager can be a remote desktop application that is used not for configuration but just to send work and get back work results from a cluster. In other cases the cluster is more related to availability and load balancing than to computational or specific service clusters.
See also
[edit]Further reading
[edit]Cluster management
[edit]- Adaptive Control of Extreme-scale Stream Processing Systems Proceedings of the 26th IEEE International Conference on Distributed Computing Systems.
- Design, implementation, and evaluation of the linear road benchmark on the stream processing core Proceedings of the 2006 ACM SIGMOD international conference on Management of data.
- Parallel Job Scheduling A Status Report (2004) 10th Workshop on Job Scheduling Strategies for Parallel Processing, New-York, NY, June 2004.
- Condor-G: A Computation Management Agent for Multi-Institutional Grids Springer Journal Cluster Computing Volume 5, Number 3 / July, 2002
- From clusters to the fabric: the job management perspective Cluster Computing, 2003. Proceedings. 2003 IEEE International Conference on
- An Overview of the Galaxy Management Framework for Scalable Enterprise Cluster Computing IEEE International Conference on Cluster Computing (Cluster'00), 2000.
- Performance and Interoperability Issues in Incorporating Cluster Management Systems within a Wide-Area Network-Computing Environment ACM/IEEE Supercomputing 2000: High Performance Networking and Computing.
- DIRAC: a scalable lightweight architecture for high throughput computing Grid Computing, 2004. Proceedings. Fifth IEEE/ACM International Workshop on
- AgentTeamwork: Coordinating grid-computing jobs with mobile agents Springer Journal Applied Intelligence Volume 25, Number 2 / October, 2006
- Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center UC Berkeley Tech Report, May, 2010
Autonomic computing
[edit]- The Laundromat Model for Autonomic Cluster Computing Autonomic Computing, 2006. ICAC '06. IEEE International Conference on.
- Distributed Stream Management using Utility-Driven Self-Adaptive Middleware Proceedings of the Second International Conference on Automatic Computing (2005).
Fault tolerance
[edit]- Fault-tolerance in the Borealis distributed stream processing system Proceedings of the 2005 ACM SIGMOD international conference on Management of data.
- A Global-State-Triggered Fault Injector for Distributed System Evaluation IEEE Transactions On Parallel And Distributed Systems / July, 2004
- Job-Site Level Fault Tolerance for Cluster and Grid environments IEEE International Conference on Cluster Computing (Cluster 2005)
- Fault Injection in Distributed Java Applications Parallel and Distributed Processing Symposium, 2006. IPDPS 2006. 20th International
- Load balancing and fault tolerance in workstation clusters migrating groups of communicating processes ACM SIGOPS Operating Systems Review, October 1995.
Background
[edit]Cluster manager
View on Grokipediafrom Grokipedia
A cluster manager is orchestration software that automatically manages the machines and applications within a data center cluster, coordinating resources across interconnected nodes to function as a unified system.[1][2] It typically operates in distributed environments, such as high-performance computing (HPC) setups or cloud infrastructures, to optimize resource allocation, ensure scalability, and maintain high availability by monitoring node health and handling failures proactively.[1]
Key responsibilities of a cluster manager include job scheduling using algorithms like FIFO or fair sharing, load balancing to distribute workloads evenly, and fault tolerance mechanisms such as automatic restarts of failed tasks or resource reallocation.[2] Core components often encompass a master controller for centralized decision-making, worker nodes for execution, and coordination services like ZooKeeper for synchronization across the cluster.[2] Architectures vary, including master-worker models for simplicity or multi-master designs for greater resilience in large-scale deployments.[1]
Prominent examples of cluster managers demonstrate their evolution and impact: Google's Borg system, which manages hundreds of thousands of jobs across clusters for efficient resource utilization and cost savings; Apache Mesos, an open-source framework enabling fine-grained sharing of CPU, memory, and storage among diverse frameworks; and Kubernetes, a widely adopted container orchestration platform inspired by Borg that automates deployment, scaling, and operations of application instances.[3][4][1] These systems have become essential in modern computing, supporting everything from big data processing with Hadoop YARN to scientific simulations via SLURM, thereby reducing operational overhead and enabling elastic scaling in dynamic environments.[2]
This pseudocode illustrates a simple periodic reporting loop, as implemented in systems like Kubernetes where kubelet status updates serve as heartbeats to the API server.[18] Such protocols collectively support resilient node coordination without overwhelming network resources.
The architecture of these modules is often conceptualized in layers: the control plane, encompassing the master and metadata components for decision-making and state orchestration; and the data plane, comprising agent modules for task execution and resource enforcement on worker nodes. This separation enhances modularity, allowing independent scaling of control logic from workload processing. These core modules collectively enable efficient job scheduling by providing the master with accurate, timely data from agents and stores.
Here, CPU is quantified in millicores (e.g., "250m" for 0.25 cores), and memory in bytes (e.g., "64Mi" for 64 mebibytes), allowing the manager to schedule and enforce allocations via underlying mechanisms like cgroups. Storage and network abstractions follow similar patterns, using plugins to expose persistent volumes and virtual network interfaces as configurable resources.[23]
These abstraction layers enable multi-tenancy by isolating tenant workloads on shared infrastructure, supporting dynamic allocation that adjusts resources in real-time based on demand. This results in enhanced efficiency, with high resource utilization rates through optimized sharing and reduced overhead, compared to lower rates in non-abstracted setups.[24]
This method optimizes resource usage by prioritizing larger tasks, though advanced variants incorporate multi-resource alignment via dot products for heterogeneous demands.[28] Allocation must also consider constraints like affinity rules, which prefer co-locating related tasks on the same node to reduce communication overhead, and anti-affinity rules, which spread tasks across nodes to enhance fault tolerance and load balancing.[29]
In heterogeneous clusters, where nodes vary in capabilities such as CPU types or accelerators, node labeling enables targeted allocation; for instance, labels like "nvidia.com/gpu=a100" tag specialized GPU nodes, allowing schedulers to direct compute-intensive workloads accordingly.[30][31]
Key performance metrics for scheduling include latency, such as under 150 milliseconds for over 80% of decisions in clusters of up to 400 nodes as in evaluations of systems like Tarcil, and throughput, measured as jobs processed per second, which can reach near-ideal levels (e.g., 97% of optimal) in high-load scenarios.[32] These metrics guide policy tuning, with integration to monitoring systems enabling real-time adjustments for dynamic loads.[32]
This replication ensures linearizability and fault tolerance, with the leader handling all mutations while followers replicate passively.[39]
Sharding and partitioning techniques further enhance scalability by distributing metadata and control plane data across multiple nodes or sub-clusters, preventing single points of bottleneck in the central store. In systems like Kubernetes, where etcd serves as the metadata backend, sharding involves splitting the key-value store into logical partitions managed by separate etcd clusters, allowing parallel access and reducing latency for operations like object watches and listings in large environments. This distribution ensures that metadata queries scale with the number of shards, supporting higher throughput without overwhelming a monolithic database.[41]
Performance benchmarks demonstrate the practical limits of these mechanisms; for instance, Kubernetes officially recommends clusters of up to 5,000 nodes and 150,000 pods to avoid control plane overload, with etcd storage capped at around 8 GB for optimal consistency. Advanced configurations, such as those using sharded etcd or edge extensions like KubeEdge, have been tested to handle over 10,000 nodes and up to 100,000 edge devices, maintaining sub-second response times for scheduling and replication under high load.[42][43]
Overview and Fundamentals
Definition and Scope
A cluster manager is specialized software designed to coordinate a collection of networked computers, known as nodes, enabling them to operate collectively as a unified pool of computational resources in distributed computing environments.[5] It automates essential tasks such as workload distribution across nodes, resource allocation to optimize utilization, and failure recovery mechanisms to ensure system resilience, thereby abstracting the complexities of managing individual machines.[2] This coordination allows applications to scale beyond the capabilities of a single node while maintaining efficiency and reliability.[5] The scope of cluster managers encompasses a wide range of distributed systems applications, including high-availability setups that provide fault tolerance through redundancy and rapid recovery, big data processing frameworks that handle massive parallel computations, and container orchestration systems for deploying and managing lightweight, isolated workloads.[5] Cluster sizes supported by these managers vary significantly, from small configurations involving tens of nodes for departmental computing to large-scale deployments spanning thousands or even tens of thousands of machines in data centers, as demonstrated in production environments managing hundreds of thousands of concurrent jobs.[5] These systems have evolved from foundational paradigms in grid computing, adapting to modern demands for dynamic resource sharing.[3] Cluster managers presuppose foundational knowledge of distributed systems principles, such as node interconnectivity, shared state management, and basic clustering concepts, without requiring expertise in specific hardware configurations. In contrast to load balancers, which primarily focus on distributing incoming network traffic across servers to prevent overload, cluster managers provide comprehensive oversight of the entire cluster lifecycle, including job scheduling, monitoring, and proactive fault detection beyond mere traffic routing. This broader functionality ensures holistic resource optimization and high availability in complex, multi-node environments.[6]Historical Development
The origins of cluster manager technology trace back to the early 1990s in high-performance computing (HPC), driven by the need to coordinate resources across multiple commodity computers. In 1994, NASA researchers Thomas Sterling and Donald Becker developed the first Beowulf cluster at Goddard Space Flight Center, comprising 16 Intel 486 DX4 processors interconnected via Ethernet, marking a pivotal shift toward affordable, scalable parallel computing using off-the-shelf hardware.[7] This innovation democratized HPC by enabling cost-effective supercomputing alternatives to proprietary systems. Concurrently, the Portable Batch System (PBS), initiated in 1991 at NASA Ames Research Center as an open-source job scheduling tool, provided essential workload management for distributing batch jobs across clusters, building on earlier systems like the 1986 Network Queueing System (NQS).[8] PBS became a cornerstone for Beowulf environments, facilitating resource allocation and queueing in early distributed setups.[9] By the early 2000s, NASA's continued adoption of cluster managers like PBS expanded their application in aerospace simulations.[10] Beowulf-derived systems were used for large-scale computations in Earth and space sciences, including climate modeling and projects supporting space missions.[11][12] The 2000s saw further evolution amid the rise of big data, culminating in Apache Hadoop's Yet Another Resource Negotiator (YARN) framework, released with Hadoop 2.0 on October 16, 2013, which decoupled resource management from job execution to support diverse workloads beyond MapReduce.[13] Internally, Google's Borg system, developed over the preceding decade and detailed in a 2015 paper, managed hundreds of thousands of jobs across clusters, emphasizing fault tolerance and efficient scheduling; its principles later inspired open-source alternatives. The 2010s marked a transformative phase influenced by cloud computing's explosive growth post-2010, which accelerated the shift from batch-oriented processing to real-time orchestration for dynamic, distributed applications.[14] Containerization emerged as a key driver, with Docker Swarm announced on December 4, 2014, to enable native clustering of Docker containers for simplified deployment and scaling.[15] That same year, Kubernetes originated from Google's internal efforts, with its first commit on June 6, 2014, evolving into a CNCF-hosted project by March 2016 to orchestrate containerized workloads at scale.[16] These developments reflected broader demands for elasticity and resilience in cloud-native environments, solidifying cluster managers' role in modern distributed systems.Architecture and Components
Core Modules
Cluster managers are built around several essential software modules that enable centralized orchestration, local execution, and consistent state management across distributed nodes. These modules form the foundational architecture, separating concerns between decision-making and operational execution while ensuring reliable communication and data persistence. The master node module serves as the centralized control point, coordinating cluster-wide operations and maintaining an authoritative view of the system state. It typically includes an API server that provides a programmatic interface for querying and updating cluster resources, such as deploying workloads or querying node availability. In Kubernetes, for instance, the kube-apiserver component exposes the Kubernetes API, validates requests, and interacts with other control plane elements to manage cluster state.[17] This module often runs on dedicated master nodes to isolate it from workload execution, enhancing reliability in large-scale deployments. Agent modules, deployed on worker nodes, handle local resource management and execution of assigned tasks. These agents monitor local hardware, enforce policies, and report back to the master for global awareness. A key function is sending periodic heartbeats—status updates that include resource utilization, health metrics, and availability—to prevent node isolation. In Kubernetes, the kubelet agent on each worker node registers the node with the API server, reports capacity (e.g., CPU and memory), and updates node status at configurable intervals, such as every 10 seconds by default, to signal liveness and facilitate resource allocation decisions.[18] These modules ensure that the master receives real-time data from the cluster periphery, enabling responsive management without direct intervention on every node. Metadata stores are critical for preserving a consistent, fault-tolerant representation of the cluster state, including node registrations, resource allocations, and configuration details. These stores are typically implemented as distributed key-value databases that support atomic operations and replication. etcd, a widely used example, functions as a consistent backend for cluster metadata, storing all data in a hierarchical structure and providing linearizable reads and writes for up-to-date views.[19] By maintaining this shared state, metadata stores allow the master to recover from failures and ensure all nodes operate from synchronized information. Communication protocols underpin inter-module interactions, enabling discovery, coordination, and failure detection in dynamic environments. Gossip protocols, which involve nodes periodically exchanging state information with random peers, promote decentralized dissemination of membership changes and status updates, scaling well for large clusters. In Docker Swarm, nodes use a gossip-based mechanism to propagate cluster topology and heartbeat data peer-to-peer, reducing reliance on a central point for routine coordination.[20] Complementing this, consensus protocols like Raft ensure agreement on critical state changes, particularly in metadata stores; Raft elects a leader among nodes to coordinate log replication and handle failures through heartbeats and elections, guaranteeing consistency even if minority nodes fail. A basic heartbeat mechanism, common in agent-to-master reporting, can be expressed in pseudocode as follows, where agents periodically transmit status to detect and respond to issues:algorithm BasicHeartbeatAgent:
initialize heartbeat_interval, timeout
while node_active:
wait(heartbeat_interval)
local_status ← collect_resources_and_health()
send(local_status) to master
if no_acknowledge within timeout:
trigger_local_recovery_or_alert()
algorithm BasicHeartbeatAgent:
initialize heartbeat_interval, timeout
while node_active:
wait(heartbeat_interval)
local_status ← collect_resources_and_health()
send(local_status) to master
if no_acknowledge within timeout:
trigger_local_recovery_or_alert()
Resource Abstraction Layers
Cluster managers employ resource abstraction layers to virtualize physical hardware components, presenting them as logical, pluggable entities that can be dynamically allocated across the cluster. These layers typically abstract CPU, memory, storage, and network resources through modular plugins, enabling isolation and efficient sharing among workloads. For instance, in Linux-based systems, control groups (cgroups) serve as a foundational mechanism for isolating processes and enforcing resource limits on CPU time, memory usage, input/output operations, and network bandwidth, preventing interference between concurrent tasks.[21][22] Virtualization techniques within these abstraction layers leverage container runtimes to encapsulate applications with their dependencies while sharing the host kernel, providing lightweight isolation compared to full virtual machines. Basic integration with container technologies, such as Docker, allows cluster managers to deploy and manage containerized workloads as uniform units, abstracting underlying hardware variations. For virtual machine orchestration, these layers extend support to hypervisor-based environments, enabling the provisioning of VM instances atop the cluster infrastructure without exposing low-level hardware details to users. This approach facilitates seamless resource pooling and migration across nodes.[23] Resource modeling in cluster managers often relies on declarative descriptors, such as YAML files, to specify resource requests (minimum guarantees) and limits (maximum allowances) for workloads. A simple example for a pod-like specification might include:yaml
resources:
requests:
[memory](/page/Memory): "64Mi"
cpu: "250m"
limits:
[memory](/page/Memory): "128Mi"
cpu: "500m"
resources:
requests:
[memory](/page/Memory): "64Mi"
cpu: "250m"
limits:
[memory](/page/Memory): "128Mi"
cpu: "500m"
Primary Functions
Job Scheduling and Allocation
Job scheduling in cluster managers involves determining the order and placement of workloads across available nodes to optimize resource utilization and meet performance goals. Common scheduling policies include First-In-First-Out (FIFO), which processes jobs in the order of their arrival without considering size or priority, leading to simple but potentially inefficient handling of mixed workloads where small jobs may be delayed by large ones.[25] Fair-share scheduling, in contrast, allocates resources proportionally among users or jobs to ensure equitable access, mitigating issues like resource monopolization by long-running tasks while allowing small jobs to complete faster.[26] Priority-based scheduling assigns weights to jobs based on factors such as user importance or deadlines, enabling higher-priority tasks to preempt or overtake lower ones for improved responsiveness in diverse environments.[27] Allocation strategies focus on mapping scheduled jobs to specific nodes while respecting resource constraints. Bin packing techniques treat nodes as bins and tasks as items with multi-dimensional requirements (e.g., CPU, memory), aiming to minimize fragmentation and maximize packing density. A basic bin-packing algorithm for task placement, such as the first-fit heuristic, scans nodes in order and assigns a task to the first node with sufficient remaining capacity; for better efficiency, tasks can be sorted by decreasing resource demand before placement (First-Fit Decreasing).[28] The following pseudocode illustrates a simplified First-Fit Decreasing bin-packing approach for task placement:Sort tasks by total [resource](/page/Resource) demand (e.g., CPU + [memory](/page/Memory)) in decreasing order
For each task in sorted list:
For each node in cluster:
If node has sufficient [resources](/page/Resource) for task:
Assign task to node
Update node [resources](/page/Resource)
Break
If no suitable node found:
Queue task or reject
Sort tasks by total [resource](/page/Resource) demand (e.g., CPU + [memory](/page/Memory)) in decreasing order
For each task in sorted list:
For each node in cluster:
If node has sufficient [resources](/page/Resource) for task:
Assign task to node
Update node [resources](/page/Resource)
Break
If no suitable node found:
Queue task or reject
Monitoring and Fault Detection
Cluster managers employ monitoring mechanisms to continuously observe the health of nodes, resources, and overall system performance, ensuring timely detection of issues that could impact reliability. These systems integrate with specialized tools for metrics collection, focusing on key indicators such as CPU and memory utilization, network latency, and node responsiveness to maintain operational stability. A prominent approach involves integration with monitoring frameworks like Prometheus, which scrapes and stores time-series data from cluster components via exporters embedded in nodes or services. For instance, Prometheus collects metrics on resource usage—such as CPU load thresholds triggering alerts—and node liveness through periodic probes, enabling cluster managers to visualize and query cluster state in real-time. This integration allows for multidimensional data modeling, where labels like node ID or job type facilitate targeted analysis without overwhelming storage. Fault detection in cluster managers primarily relies on heartbeat protocols, where nodes periodically send status messages to a central coordinator or peers to confirm availability. If a heartbeat is not received within a predefined timeout, the system flags the node as potentially failed, balancing sensitivity to real failures against tolerance for network delays. Complementary probe-based checks, such as active pings or API calls to verify service endpoints, supplement heartbeats by providing on-demand validation of node functionality. These methods ensure robust detection in dynamic environments, with periodic heartbeats to minimize latency in identification. Event logging plays a crucial role in capturing anomalies during monitoring, generating structured records that include timestamps, affected components, and error codes for post-analysis. Logs classify failures into categories like transient faults, which are temporary and self-resolving (e.g., brief network glitches), versus permanent faults requiring intervention (e.g., hardware breakdowns), aiding in root-cause diagnosis without manual inspection. This logging enables auditing of detection events, such as heartbeat timeouts, and supports querying for patterns in large-scale deployments.[33][34] Proactive measures enhance fault detection through automated health checks that preemptively assess node and resource viability, such as disk space verification or connection tests at regular intervals. These checks trigger alerts or remediation signals upon detecting deviations, like memory leaks exceeding capacity thresholds, allowing the cluster manager to initiate recovery processes integrated with scheduling for resource reallocation. Such mechanisms prioritize early intervention to sustain cluster uptime.[35][36]Advanced Features
Scalability Mechanisms
Cluster managers employ horizontal scaling to accommodate growing workloads by dynamically adding nodes to the cluster, often through mechanisms that integrate with resource provisioning systems to adjust capacity in real-time. This approach allows the system to distribute tasks across more resources without interrupting ongoing operations, ensuring high availability and elasticity. For even larger environments, federation techniques enable the coordination of multiple independent clusters, treating them as a unified whole to handle distributed scaling needs across geographically dispersed setups.[37][38] To maintain coordination in large-scale deployments, cluster managers rely on consensus algorithms such as Raft and Paxos for leader election and state consistency. Raft, introduced as an understandable alternative to Paxos, decomposes consensus into leader election, log replication, and safety mechanisms, making it suitable for implementing fault-tolerant coordination in clusters with dozens to thousands of nodes.[39] In Raft, leader election occurs when no valid leader exists; a follower increments its term and requests votes from other nodes, becoming leader if it secures a majority. Paxos, the foundational algorithm, achieves consensus through phases involving proposers, acceptors, and learners to agree on a single value despite failures.[40] These algorithms underpin state machine replication, where the leader serializes client commands into a log, replicates it to followers, and commits entries once acknowledged by a quorum, ensuring all replicas apply the same sequence of operations. State machine replication in Raft can be outlined in pseudocode as follows, focusing on the leader's replication process:Upon receiving a client command:
- Append the command to the leader's log as a new entry
- Replicate the new entry to all followers via AppendEntries RPCs
For each AppendEntries response from a follower:
- If a majority of followers acknowledge the entry (match prevLogIndex and prevLogTerm, and log entry matches):
- Commit the entry in the leader's log
- Apply the committed entry to the [state machine](/page/State_machine_replication)
- Send the committed entry to the client
- If not a [majority](/page/Majority):
- Retry replication or step down if term is stale
Upon receiving a client command:
- Append the command to the leader's log as a new entry
- Replicate the new entry to all followers via AppendEntries RPCs
For each AppendEntries response from a follower:
- If a majority of followers acknowledge the entry (match prevLogIndex and prevLogTerm, and log entry matches):
- Commit the entry in the leader's log
- Apply the committed entry to the [state machine](/page/State_machine_replication)
- Send the committed entry to the client
- If not a [majority](/page/Majority):
- Retry replication or step down if term is stale
Integration with Cloud Environments
Cluster managers integrate with major cloud providers through specialized APIs that enable dynamic provisioning of virtual machines and other resources, allowing clusters to scale elastically based on workload demands. For instance, in Kubernetes, the Cloud Controller Manager (CCM) serves as the primary interface, leveraging provider-specific plugins to interact with APIs such as AWS EC2 Auto Scaling, Azure Virtual Machine Scale Sets, and Google Cloud Compute Engine instances. This integration facilitates automated node provisioning, where the cluster manager requests new VMs when resource utilization exceeds thresholds, and deprovisions them during low demand, ensuring efficient resource allocation without manual intervention.[44] Support for hybrid and multi-cloud environments is achieved through infrastructure-as-code (IaC) tools like Terraform, which abstract underlying provider differences and enable consistent deployment workflows across clouds. A typical workflow involves defining cluster resources—such as node pools, networking, and storage—in declarative HCL configuration files; for example, provisioning a Kubernetes cluster on AWS EKS might specify VPC subnets and IAM roles, while an equivalent Azure AKS deployment configures resource groups and virtual networks, and a GCP GKE setup handles zones and preemptible VMs, all applied via Terraform'sterraform apply command for idempotent orchestration. This approach minimizes vendor lock-in and supports hybrid setups by combining on-premises resources with public cloud instances in a single configuration.[45][46][47]
Serverless extensions allow cluster managers to handle bursty workloads by integrating with functions-as-a-service (FaaS) platforms, offloading short-lived tasks to event-driven execution models. In Kubernetes, Knative provides this capability through its Serving component, which deploys functions as serverless applications that scale automatically using the Knative Pod Autoscaler (KPA); for bursty traffic, KPA monitors concurrency and scales pods from zero to handle spikes, then scales down to minimize idle resources, integrating seamlessly with the cluster's scheduler for resource isolation. This enables cost-effective processing of intermittent jobs, such as data processing pipelines or API backends, without maintaining persistent infrastructure.[48]
Cost optimization within cloud-integrated cluster managers often involves strategic use of spot instances and reserved capacity to balance performance and expenses. Spot instances, which provide access to unused cloud capacity at discounts up to 90%, are managed by the cluster autoscaler to run non-critical workloads, with mechanisms to gracefully handle interruptions by rescheduling pods across available nodes. Reserved instances or savings plans, committed for 1- or 3-year terms, secure lower rates for steady-state workloads and are applied at the instance level within the cluster, allowing managers like Amazon EKS to optimize procurement based on historical usage patterns for predictable savings of up to 72%.[49][50][51]