Speaker
Gergely Daroczi
Description
Selecting the right cloud instance type for model training or other compute-intensive R workloads is often a guesswork:
- Unclear resource requirements, e.g. "How much RAM do I need to train this hierarchical model?" or "Can my script scale to multiple CPU cores or even GPUs?"
- Pricing and hardware specs exist, but are fragmented across vendors and hard to compare, especially when real workload performance matters.
We introduce Spare Cores, an open-source EU-funded project dedicated to help data scientists optimize their cloud resource utilization via
- Resource Tracker: an open-source lightweight tool to auto-monitor the CPU, GPU, memory, VRAM, network traffic, disk etc. utilization of the server and running processes to build baselines, utilization profiles and early recommendations for server configurations and placements;
- Navigator: an open dataset covering thousands of cloud instance types across small providers to hyperscalers with standardized performance and cost-efficiency metrics we benchmarked and collected over the past years.
This talk focuses on how such instance choice becomes a data task in R: filter by workload, hardware, compliance, or budget, then rank candidates by price-performance to identify the most efficient option for your workload.
If you used AI tools or services to support the preparation of this submission, please state the name and reason for using each of them.
No AI tools/services were used.
Additional Material or Paper
https://sparecores.com/talks
| Keywords: Please list up to 5 keywords to help us find the right session for your contribution. | cloud, performance, benchmarks, cost efficiency |
|---|---|
| Virtual Option | This submission is for onsite presentation only |
| Video Recording | Video sharing is fine |
| The author(s) agree(s) to take responsibility and be accountable for the contents of the submission and is/are authorized to present it. | Confirm |