If we restore a cluster and a previously created tenant was not included in the backup, then the tenant will be marked in an error state on the management cluster. It is then up to the operator to resolve the error, generally by deleting the tenant and recreating it if needed.
There is, however, the possibility that we restored a backup that was older than we wanted, and a newer backup would have the tenant. If we tried to restore the newer backup, it would not leave the previously missing tenant in a fully usable state.
We need to have a way to deal with this case. One option is to allow us to clear the error state of a tenant, and that can be performed before (or maybe even after) the second restore.
Test plan:
Joshua test
100K ensemble: 20230613-225414-yajin-439d13ef3c6b3afd fail=0
Similar to `tenant configure`, this PR adds `ignore_capacity_limit` as an optional argument to `tenant create`.
This allows the user of fdbcli to create a new tenant on an **assigned** cluster, ignoring the tenant group capacity
on that specific cluster.
When creating a tenant with `ignore_capacity_limit`.
- If the user does not specify `assigned_cluster`, this is an error.
- If the user specifies `assigned_cluster`,
- user does not specify `tenant_group`, then the new tenant will be an ungrouped tenant on the `assigned_cluster` ignoring the capacity limit
- user specifies `tenant_group`,
- if `tenant_group` does not exist, then the new tenant will be created on the assigned cluster and the tenant group will be implicitly created.
- if `tenant_group` already exists, then additional check will make sure the tenant_group's cluster matches what the user specifies.
Test plan:
Simulation and metacluster_fdbcli_tests.py
---------
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
* fix metacluster get segfault
* update fdbcli tenant list function to take tenant group filter, support JSON, and report tenant IDs
* code review changes
* code formatting
* additional code review changes
* account for empty tenant groups
* reformat error catching in fdbcli command
* refactor json output and address code review comments
* add back mistakenly removed hint
* keep hints after 4th token
* add to tenant management workload
* fix compile error
* fix test range
* add more asserts to metacluster case
* nest test condition inside if block
* adjust tenant test layout
* refactor some test files
* reorganize test workload logic
