1. We see in your demo that you used Master node, but Is it mandatory to use master node in arbiter zone?
Ans - Any node based on label
3. Can user use rack insted of zone while labelling nodes? Probably UI won't detect rack label though
Ans- works as per rook .. If we use CRs (CLI), any label can work but UI might only be needing the zone label

5. Will osds from one zone only talk with mon of same zone? or can also talk with other osd&mon

In docs : "Two zones will each run all OCS pods, two mons run in each zone since the OSDs can only connect to the mon in their own zone"

Q In a normal on-prem environment, mons in one rack are able to talk to OSDs of another rack. isn’t it? so in this arbiter cluster, even though we have latency, aren’t the mons and osds on different zones able to connect/talk to each other?
Ans - As per the ceph fix, The OSds wont try to connect to mons of other zones.
if both mons in a zone are down, ceph treats that zone as down, even the OSDs. the other zone keeps working.

OSd can talk to another osd of 2nd zone, but not MON of another zone

3b. ) What happens when whole zone is down?
Ans - if arbiter zone is down, other zones will continue to work
https://github.com/rook/rook/blob/master/cluster/examples/kubernetes/ceph/cluster-stretched.yaml#L33-L37

4. Two datacenters are available in an on-prem deployment.
How do we plan to achieve this? Is it a must-have rule?
-Ans - better to test with some latency

5. Arbiter mode verification ?
Ans - if one mon is down or whole zone is down - can we read/write data to other zone. Whether volumes are accessible

6. Will the mon on arbiter zone failover to another node in the same zone, in case the other node has the required label ? or will it always come up only on that particular node?
Ans - Mon should be able to failover to another zone of the arbiter zone. right now it does not work

7. In what situations data would become read-only ? if 2 osds of same zone go down together, would the other zone continue servicing data?

After deployment, if I add label to a free node in arbiter zone, that will be used for mon failover ?

Ans - @neha if you add a node to the arbiter zone, yes it will be considered for the failover

thanks.. even post deployment as a day 2 activity
right ?
Travis Nielsen9:06 PM
that's right
also, we don't have a BZ yet for the mon failover that's not working, feel free to open one thanks

talk to ceph qe