An increasing number of companies are utilizing large language models that necessitate access to GPUs. Nvidia’s GPUs, which are the most widely adopted, are often expensive and in short supply. When you only require these costly resources for a single task, committing to a long-term instance from a cloud provider may not be a practical choice.
To address this issue, AWS has introduced Amazon Elastic Compute Cloud (EC2) Capacity Blocks for ML. This offering allows customers to purchase access to GPUs for a specified duration, typically for tasks related to artificial intelligence, such as training machine learning models or conducting experiments with existing models.
Channy Yun, in a blog post announcing this new feature, described it as an innovative way to schedule GPU instances, allowing users to reserve the number of instances they need for a future date for the exact amount of time required.
Customers can access Nvidia H100 Tensor Core GPU instances in cluster sizes ranging from one to 64 instances, with each instance equipped with 8 GPUs. Reservations can be made for up to 14 days, in one-day increments, and can be scheduled up to eight weeks in advance. Once the specified time frame expires, the instances will automatically shut down.
This new product empowers users to sign up for a specific number of instances for a defined time period, similar to reserving a hotel room for a specific number of days. Customers will have a clear understanding of the job’s duration, the number of GPUs they will utilize, and the upfront cost, providing cost predictability.
For Amazon, this approach allows them to efficiently allocate these sought-after resources in a somewhat auction-like environment, ensuring revenue, assuming customers utilize the service. The price for accessing these resources will be dynamically adjusted based on supply and demand.
As users sign up for the service, it displays the total cost for the selected timeframe and resources. Users can adjust this as needed to align with their resource requirements and budgets before making the purchase.
This new feature is now available to customers, starting today, in the AWS US East (Ohio) region.
