Project

General

Profile

Support #11633

Reduce manual c3p0 tuning burden across cluster scale changes

Added by Eduard Soltan 5 days ago. Updated 4 days ago.

Status:
New
Priority:
Normal
Assignee:
Target version:
-
Start date:
Due date:
% Done:

0%

billable:
No
vendor_id:
GCD
case_num:
version_reported:
version_resolved:
reviewer:
production:
No
env_name:
topics:

History

#2 Updated by Eduard Soltan 5 days ago

One idea could be:

Configure a single cluster-wide connection budget per physical database instead, and let each node derive its own share of that budget automatically from how many peers currently exist.

Shape of the design:
- Each node's pool ceiling becomes a derived value, not a configured one: total budget ÷ current member count, recomputed whenever membership changes (a node joins or leaves) rather than fixed at startup.
- A floor guards the derived value so a temporarily small cluster (e.g., mid rolling-deploy, with only one node briefly up) doesn't get starved down to an unworkably small pool.
- The recompute needs to be debounced against membership flapping — a node's derived share shouldn't visibly change on every transient blip, only on a stable, sustained topology change.
- The mechanism for actually changing a live node's pool ceiling already exists conceptually in how pools are configured today — this design reuses that same idea, just triggered by a membership event instead of only at process startup, and understanding that shrinking a live pool is gradual (existing checked-out connections aren't forcibly reclaimed; the ceiling just stops future growth and lets the pool drain toward the new target).

#3 Updated by Greg Shah 5 days ago

I like this a lot. In order to handle load balancing and the natural variability of the number of tenant sessions per server, it will likely mean that the per-tenant pool size will need to be larger than the current single-server value by some factor. I would hope that factor could be smaller than the single_server_tenant_connections * n (where n is the number of FWD servers).

#4 Updated by Constantin Asofiei 5 days ago

max_connections is 'per cluster'. This needs to consider the max_connections for that cluster, and group these values per host:port (or other ways to identify the psql cluster).

From the cluster, it can be read via show max_connections. And after this we can infer what c3p0 maxPoolSize can be for that cluster, considering the number of databases in the postgresql cluster, the number of agents/MSA sessions, batch processes, etc. I think we may be able to move further with auto-configuring this.

#5 Updated by Eduard Soltan 4 days ago

  • Assignee set to Eduard Soltan

Also available in: Atom PDF