I was thinking about how apps scale and what ends up costing the most CPU time or HDD space.
I know this question is a vague question and depends on the type of app but if anyone has experience with applications at scale please share your experience on what ends up costing the most in new cloud/serverless applications.
This is highly dependent you what your infrastructure looks like.
Note that your question could be considered off-topic or too general as formulated, but given the general interest in such issues I will give an answer.
1st of all, you are only considering 2 of the 3 main resources, the 3rd being the RAM. I say this because in a today's VM environment with multiple VMs per host it is more likely to run out of RAM rather than CPU.
Therefore, if your infrastructure is likely to extend and applications to be developed you will have to have an initial planning. In that planning you must consider all aspects regarding CPU, RAM and HDD resources needed and even network bandwidth requirements. In each case you will have to make sure you are good for years without over-spending. If you can add RAM with minimal down-time, in the case of CPUs you're kind-of stuck to what you initially get as for the HDDs it depends on the storage you use: did you use all bays from start, do you use external storage or both.
In my initial plan, I got 5 dell servers, with 3 system drives + 5 large 8+TB drives for storage. Then I spread around 20 VM servers on the 5 physical ones. I was still at less than half on all resources. During the next years, we added more servers which I distributed according to free resources: if a server needed a lot of processing power, I deployed it on a host with a lot of un-allocated CPU cores. If it required a lot of RAM, I deployed it on a host with lots of free RAM. In a similar manner, a log server I deployed on a host with a lot of available space. I added at least 10 more and still no resource is even close to a warning. All this ended up fine due to good planning. I also considered that if any of the physical servers fails, the others have enough resources to accommodate the VMs from the others. Then we just got a backup server for this purpose, therefore not having this restrain anymore.
So as a conclusion: you need a good start plan.
