Analysis of different GPUs for skibidi server
(A100, A6000, H100, V100, RTX4090)
| - |
A100 (40GB) |
A100 (80GB) |
A6000 |
H100 |
V100 |
4090 |
| Price [USD] |
~ 10000 |
~ 18000 |
~ 4500 |
~ 30000 |
~ 4000 |
~ 2200 |
| CUDA Cores |
6912 |
6912 |
10752 |
14592 |
5120 |
16384 |
| Tensor Cores |
432 (gen 3) |
432 (gen 3) |
336 (gen 3) |
456 |
640 |
512 (gen 4) |
| Memory [GB] |
40 |
80 |
48 |
80 |
32 |
24 |
| Mem Bandwidth [GB/s] |
1555 |
1940 |
768 |
2040 |
900 |
1008 |
| TFLOPS (FP16) |
78 |
78 |
38.7 |
204.9 |
28.3 |
82.6 |
| TFLOPS (FP32) |
19.5 |
19.5 |
38.7 |
51.22 |
14.1 |
82.6 |
| Power Consumption [W] |
250 |
300 |
250 |
350 |
250 |
450 |
Potential Strategies:
2 A6000 ~ $9000 for 96GB vRAM
- Positives: Relatively cheap, low power (500W)
- Drawbacks: Slightly lower memory bandwidth
3 V100 ~ $12000 for 96GB v RAM
- Positives: Higher mem bandwidth across 3 cards
- Drawbacks: Higher cost
4 RTX4090 $9000 for 96GB vRAM
- Positives: (Initially) cheapest option, Very high FLOPS, High memory bandwidth split across 4 cards
- Drawbacks: Extremely high power draw (1800W)
Single 80GB A100 ~ $18000 for 80GB vRAM
- Positives: Better performance than multi-GPU setups, Lowest power draw (300W)
- Drawbacks: Very expensive, High replacement cost
One Strong GPU vs. Multiple Weaker GPUs
Although it is true that in general a single strong GPU will perform better than multiple weaker GPUs even if the sum vRAM is equivalent, this question is not really relevant unless going with the H100 or 80GB A100 because nothing else has enough memory on its own. Since there will already be more than one GPU, that extra overhead of managing multiple GPUs is going to be present, so the difference between 2 or 3 or 4 GPUs won’t be significant.
Power Consumption
Assuming the cost of power to be between 10 and 15 cents/kWh (just guessing based on data from 2018, but this should be an underestimation and overestimation), the cost associated with operating the server will be mostly based on the GPU setup. Likely the chosen CPU will be the same between all different GPU scenarios. The server power supply could be as low as 850W for the single A100 build which would probably draw about 1kW depending on efficiency which would cost between $900 and $1300 per year in electricity. If using 4090’s, the draw from the GPUs alone would be 1800W, meaning that the server PSU would probably need to supply 2500W, which would draw about 3200W depending on efficiency. This would cost between $2800 and $4200 per year in electricity. Thus, if the server is going to be in use 24/7 for multiple years, the difference between the A100 and 4090 setup would be between $1900 and $2900 yearly. Assuming all other costs are equal, this would mean that after 5 years the A100 would be cheaper than the 4090 setup with power costs. These are scratch calculations though assuming a lot of things and the actual difference may be greater or drastically less.
***************************************
***************************************