Speaker
Gergely Daroczi
Description
Selecting the right cloud instance type for model training or other compute-intensive R workloads is often a guesswork:
- Unclear resource requirements, e.g. "How much RAM do I need to train this hierarchical model?" or "Can my script scale to multiple CPU cores or even GPUs?"
- Pricing and hardware specs exist, but are fragmented across vendors and hard to compare, especially when real workload performance matters.
We introduce Spare Cores, an open-source EU-funded project dedicated to help data scientists optimize their cloud resource utilization via
- Resource Tracker: an open-source lightweight tool to auto-monitor the CPU, GPU, memory, VRAM, network traffic, disk etc. utilization of the server and running processes to build baselines, utilization profiles and early recommendations for server configurations and placements;
- Navigator: an open dataset covering thousands of cloud instance types across small providers to hyperscalers with standardized performance and cost-efficiency metrics we benchmarked and collected over the past years.
This talk focuses on how such instance choice becomes a data task in R: filter by workload, hardware, compliance, or budget, then rank candidates by price-performance to identify the most efficient option for your workload.
Additional Material or Paper
https://sparecores.com/talks
If you used AI tools or services to support the preparation of this submission, please state the name and reason for using each of them.
No AI tools/services were used.
| Keywords: Please list up to 5 keywords to help us find the right session for your contribution. | cloud, performance, benchmarks, cost efficiency |
|---|---|
| Virtual Option | This submission is for onsite presentation only |
| Video Recording | Video sharing is fine |
| The author(s) agree(s) to take responsibility and be accountable for the contents of the submission and is/are authorized to present it. | Confirm |