clickhouse-rate-limits

v2026.09.24

Configure ClickHouse query concurrency, memory quotas, and connection limits. Use when hitting "too many simultaneous queries", managing concurrent users, or tuning server-side resource limits so an app never starves the cluster. Trigger with "clickhouse rate limit", "clickhouse concurrency", "clickhouse quota", "too many simultaneous queries", "clickhouse connection limit".

GitHub
安装命令
npx skhub add jeremylongshore/clickhouse-rate-limits
Markdown
SKILL.md

ClickHouse Rate Limits & Concurrency

Overview

ClickHouse has no REST API rate limits like a SaaS product. Instead it enforces server-side concurrency limits, memory quotas, and per-user settings that control resource usage. This skill configures those server-side limits and pairs them with client-side controls so an application stays within them under load.

Prerequisites

  • ClickHouse admin access (or Cloud console) to create quotas and settings profiles.
  • The @clickhouse/client Node package for the client-side patterns.
  • A rough target for peak concurrent queries and per-query memory.

Instructions

Work top-down: cap resources at the server, then make the client respect the cap.

Step 1: Know the server-side limits

The defaults you tune most often:

SettingDefaultControls
max_concurrent_queries100Queries running simultaneously
max_connections4096Max TCP/HTTP connections
max_memory_usage~10GBPer-query memory
max_execution_time0 (unlimited)Per-query timeout (seconds)

ClickHouse Cloud's management API (not the query interface) is separately limited to 10 requests per 10 seconds. Full table in references/implementation.md.

Step 2: Cap resources per user (essential skeleton)

Bind a quota and a settings profile to each application user:

CREATE SETTINGS PROFILE IF NOT EXISTS app_profile
    SETTINGS
        max_memory_usage = 5000000000,       -- 5GB per query
        max_execution_time = 30,             -- 30s timeout
        max_concurrent_queries_for_user = 10 -- 10 parallel queries
    TO app_user;

The full quota (CREATE QUOTA … FOR INTERVAL 1 HOUR MAX …) plus verification queries are in references/implementation.md.

Step 3: Make the client respect the cap

Four client-side patterns keep the app inside the server limits — connection pooling, an app-level concurrency queue (p-queue), retry-with-backoff on TOO_MANY_SIMULTANEOUS_QUERIES, and insert buffering to avoid TOO_MANY_PARTS. Each is a drop-in TypeScript snippet in references/implementation.md, with the concurrency queue as the smallest starting point:

import PQueue from 'p-queue';
const queryQueue = new PQueue({ concurrency: 5, timeout: 30_000, throwOnTimeout: true });
const rateLimitedQuery = <T>(sql: string) =>
  queryQueue.add(async () => (await client.query({ query: sql, format: 'JSONEachRow' })).json<T>());

Step 4: Monitor and verify

Watch live concurrency and confirm limits bind with the queries in references/examples.md (system.processes, system.metrics, system.query_log, SHOW QUOTAS).

Output

Applying this skill produces:

  • A ClickHouse settings profile and quota bound to the app user, capping per-query memory, timeout, and per-user concurrency.
  • Client-side guardrails — a bounded connection pool, a concurrency queue, a retry wrapper, and an insert buffer — so the app cannot exceed the server cap.
  • Monitoring queries that report current running queries, queue depth, and historical peak concurrency, plus a check that the quota is applied.

Error Handling

ErrorCodeSolution
TOO_MANY_SIMULTANEOUS_QUERIES202Reduce client concurrency or raise max_concurrent_queries; retry with backoff
MEMORY_LIMIT_EXCEEDED241Lower max_threads, add query filters, reduce max_memory_usage scope
TIMEOUT_EXCEEDED159Increase max_execution_time or optimize the query
TOO_MANY_PARTS252Batch inserts via the insert buffer, wait for merges

The retry wrapper in references/implementation.md treats codes 202, 159, and network errors as retryable.

Examples

Three worked end-to-end scenarios live in references/examples.md:

  1. Cap a reporting service at 5 concurrent queries — wrap every dashboard tile's query in the p-queue limiter.
  2. Survive a concurrency spike — queryWithRetry absorbs code-202 bursts with exponential backoff + jitter instead of returning 500s.
  3. High-throughput ingest without TOO_MANY_PARTS — InsertBuffer batches a firehose into a few large inserts.

Resources

Next Steps

For security hardening (users, roles, TLS), see the clickhouse-security-basics skill. For query-level performance work, see clickhouse-performance-tuning.

发现
标签

此技能尚未发布标签。

版本
最新版本元数据

版本

v2026.09.24

发布时间

2026年9月24日

分类

未分类

许可证

MIT

源路径

skills/.curated/clickhouse-rate-limits

默认分支

main

最新提交

e5a6c3b

Tree SHA

c2dc8e8