Start small
Define throughput/latency requirements BEFORE anything else, always let some room for scaling
-> conclude on a default rate-limit
Conduct proper smoke/load/stress tests using locust/k6 whatever
Upgrade Hardware consequently
Re-run the tests every day and after every upgrade to ensure quality is not impacted
Questions for product:
What's the expected usage of a client
What are the most important endpoints