We have deployed two primary polling engines, each with an additional poller and one with an additional web-front-end as well. primary reason was total segmentation and control. We have a mixed breed of physical servers and virtuals, however, DB is preferred as physical.
We currently monitor just over the 30K elements in total, so we had to expand. The next step is for us to tie the two primary instances together with EOC to get a consolidated view across all our client environments. We also might be looking at several additional polling engines in the next year or two!
Your design will depends on a few things:
1) Total number of elements you monitor (regardless of license level) - each polling engine can handle around 10K - 11K elements (element is a node, interface or volume) - depending of course on your polling interval and CPU and memory requirements
2) Each primary polling engine can have up to 9 additional polling engines below it with a max element count of around 100K
3) Your design should take consideration the geography of your devices and polling details - for instance, if you want latency stats between your Head Office and remote branches, the polling engine should reside in your Head office so that the latency reflect the local path(s).
4) It is not recommended to have an additional polling engine on a remote site with your DB at a different site - rather opt then for two primary pollers and use EOC to merge the two together. Your stats/reports will be more accurate and your design will be certified/supported by Solarwinds. They do not recommend the additional remote poller solution.
5) If it is a hosted (MSP) environment, additional network issues such as NAT/Overlapping IP addresses are a real factor. Having separate primary engines overcomes the need for firewalls or NAT which could get very complicated when troubleshooting.
6) With EOC you should cater for at least 1 Mbps bandwidth between the EOC instance and the Primary polling engines
Hope this info helps you!