2892 字
14 分钟
PostgreSQL 完全指南 2026:SQL 进阶 + 索引优化 + 分区表 + 高可用方案
PostgreSQL 是功能最丰富的开源关系型数据库,在 2026 年持续保持开发者最喜爱数据库榜首(Stack Overflow 调查连续六年)。从个人项目到超大规模企业系统,PostgreSQL 都能胜任。
本文深入覆盖 PostgreSQL 的进阶使用,帮助你从”能用”升级到”用好”。
快速决策表
| 场景 | 推荐方案 |
|---|---|
| 需要高可用自动故障转移 | Patroni + etcd |
| 连接数过多(>200 并发) | PgBouncer(transaction 模式) |
| 大表查询慢 | 检查 EXPLAIN ANALYZE + 添加合适索引 |
| 日志/时序大表(>1亿行) | RANGE 分区 + BRIN 索引 |
| 存储半结构化数据 | JSONB 列 + GIN 索引 |
| Python 异步访问 | psycopg3 或 SQLAlchemy 2.0 async |
一、SQL 进阶:CTE、窗口函数、JSONB
1.1 CTE(公用表表达式)
-- 基础 CTE:提高可读性,避免子查询嵌套WITH active_users AS ( SELECT id, name, email, created_at FROM users WHERE last_login > NOW() - INTERVAL '30 days' AND status = 'active'),user_orders AS ( SELECT user_id, COUNT(*) AS order_count, SUM(amount) AS total_amount FROM orders WHERE created_at > NOW() - INTERVAL '30 days' GROUP BY user_id)SELECT u.name, u.email, COALESCE(o.order_count, 0) AS orders_30d, COALESCE(o.total_amount, 0) AS revenue_30dFROM active_users uLEFT JOIN user_orders o ON u.id = o.user_idORDER BY revenue_30d DESC;
-- 递归 CTE:遍历树形结构(组织架构、分类目录等)WITH RECURSIVE org_tree AS ( -- 锚点:从 CEO 开始 SELECT id, name, manager_id, 0 AS depth, ARRAY[name] AS path FROM employees WHERE manager_id IS NULL
UNION ALL
-- 递归:逐层获取下属 SELECT e.id, e.name, e.manager_id, t.depth + 1, t.path || e.name FROM employees e INNER JOIN org_tree t ON e.manager_id = t.id WHERE t.depth < 10 -- 防止无限递归)SELECT depth, path, nameFROM org_treeORDER BY path;1.2 窗口函数
-- 常用窗口函数示例(员工薪资分析)SELECT dept, name, salary, -- 部门内排名(相同薪资并列,有跳号) RANK() OVER (PARTITION BY dept ORDER BY salary DESC) AS dept_rank, -- 无跳号排名 DENSE_RANK() OVER (PARTITION BY dept ORDER BY salary DESC) AS dept_dense_rank, -- 行号(按薪资排序的全局行号) ROW_NUMBER() OVER (ORDER BY salary DESC) AS global_row, -- 部门平均薪资(保留所有行) ROUND(AVG(salary) OVER (PARTITION BY dept), 2) AS dept_avg, -- 当前行薪资 vs 部门平均的百分比 ROUND(salary / AVG(salary) OVER (PARTITION BY dept) * 100, 1) AS pct_of_avg, -- 前一位员工薪资(LAG) LAG(salary, 1, 0) OVER (PARTITION BY dept ORDER BY salary DESC) AS prev_salary, -- 后一位员工薪资(LEAD) LEAD(salary, 1, 0) OVER (PARTITION BY dept ORDER BY salary DESC) AS next_salary, -- 累计薪资(滚动求和) SUM(salary) OVER (PARTITION BY dept ORDER BY salary DESC ROWS UNBOUNDED PRECEDING) AS running_totalFROM employees;
-- 实用示例:找每个分类下最新的 5 篇文章(TOP N per group)WITH ranked AS ( SELECT *, ROW_NUMBER() OVER (PARTITION BY category ORDER BY published_at DESC) AS rn FROM articles WHERE status = 'published')SELECT id, title, category, published_atFROM rankedWHERE rn <= 5;1.3 JSONB 操作
-- 创建带 JSONB 列的表CREATE TABLE events ( id BIGSERIAL PRIMARY KEY, event_type TEXT NOT NULL, payload JSONB NOT NULL, created_at TIMESTAMPTZ DEFAULT NOW());
-- 插入数据INSERT INTO events (event_type, payload) VALUES('user.login', '{"user_id": 42, "ip": "1.2.3.4", "device": "mobile"}'),('order.placed', '{"order_id": 101, "items": [{"id": 5, "qty": 2}], "total": 99.9}');
-- ── 查询操作符 ───────────────────────────────────────────────-- -> 返回 JSON 对象(JSON 类型)-- ->> 返回文本(TEXT 类型)SELECT payload -> 'user_id' AS user_id_json, payload ->> 'user_id' AS user_id_text, payload -> 'device' AS deviceFROM events WHERE event_type = 'user.login';
-- 多层路径SELECT payload #>> '{items,0,id}' AS first_item_idFROM events WHERE event_type = 'order.placed';
-- @> 包含操作符(检查 JSON 包含关系)SELECT * FROM eventsWHERE payload @> '{"device": "mobile"}';
-- ? 键存在性检查SELECT * FROM events WHERE payload ? 'ip';
-- jsonb_array_elements:展开数组SELECT e.id, item->>'id' AS item_id, item->>'qty' AS qtyFROM events e, jsonb_array_elements(payload->'items') AS itemWHERE event_type = 'order.placed';
-- 更新 JSONB 字段UPDATE eventsSET payload = payload || '{"processed": true}' -- 合并更新WHERE id = 1;
UPDATE eventsSET payload = payload - 'ip' -- 删除字段WHERE event_type = 'user.login';
-- ── GIN 索引(JSONB 全路径查询加速)────────────────────────CREATE INDEX idx_events_payload ON events USING GIN (payload);-- 或只索引特定路径(更小更快)CREATE INDEX idx_events_user_id ON events USING GIN ((payload -> 'user_id'));二、索引优化
2.1 索引类型速查
| 类型 | 适用场景 | 特点 |
|---|---|---|
| B-Tree | 等值 / 范围查询(默认) | 99% 场景,支持排序 |
| Hash | 仅等值查询 | 等值查询略快,不支持范围 |
| GIN | 数组 / JSONB / 全文搜索 | 体积大,写入慢,查询快 |
| GiST | 几何 / 空间 / 全文搜索 | 支持邻近查询 |
| BRIN | 时序 / 自然有序的大表 | 体积极小(B-Tree 的 1%) |
| SP-GiST | IP 范围 / 非均匀数据 | 分区GiST |
-- 普通 B-TreeCREATE INDEX idx_users_email ON users (email);
-- 复合索引(列顺序重要:高选择性列在前,常用过滤条件在前)CREATE INDEX idx_orders_user_status ON orders (user_id, status, created_at);
-- 部分索引(只索引满足条件的行,体积小效果好)CREATE INDEX idx_orders_pending ON orders (created_at)WHERE status = 'pending';
-- 表达式索引CREATE INDEX idx_users_lower_email ON users (LOWER(email));-- 对应查询必须使用 LOWER(email) 才能命中SELECT * FROM users WHERE LOWER(email) = 'alice@example.com';
-- GIN 全文搜索索引ALTER TABLE articles ADD COLUMN search_vec TSVECTOR GENERATED ALWAYS AS (to_tsvector('chinese', title || ' ' || content)) STORED;CREATE INDEX idx_articles_fts ON articles USING GIN (search_vec);
-- 全文搜索查询SELECT title FROM articlesWHERE search_vec @@ to_tsquery('chinese', '数据库 & 优化')ORDER BY ts_rank(search_vec, to_tsquery('chinese', '数据库 & 优化')) DESC;
-- BRIN 索引(时序大表,极省空间)CREATE INDEX idx_logs_created_brin ON logs USING BRIN (created_at)WITH (pages_per_range = 128);2.2 慢查询诊断
-- 开启慢查询日志(postgresql.conf)-- log_min_duration_statement = 1000 -- 超过1秒的查询写日志
-- 查看当前正在执行的查询SELECT pid, now() - pg_stat_activity.query_start AS duration, query, stateFROM pg_stat_activityWHERE state != 'idle' AND query_start < NOW() - INTERVAL '5 seconds'ORDER BY duration DESC;
-- 查看缺少索引的慢查询(pg_stat_statements 扩展)SELECT query, calls, ROUND(total_exec_time / calls, 2) AS avg_ms, ROUND(total_exec_time, 2) AS total_ms, rowsFROM pg_stat_statementsORDER BY avg_ms DESCLIMIT 20;
-- EXPLAIN ANALYZE(实际执行计划 + 耗时)EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT)SELECT u.name, COUNT(o.id)FROM users uLEFT JOIN orders o ON u.id = o.user_idWHERE u.created_at > '2026-01-01'GROUP BY u.idORDER BY COUNT(o.id) DESCLIMIT 10;EXPLAIN 输出关键指标:
Seq Scan(全表扫描)→ 考虑添加索引rows=100000(估算行数)vs 实际actual rows=10→ 统计信息过时,运行ANALYZEBuffers: shared hit=X read=Y→ Y 很大说明频繁读磁盘(缓冲区不足)
-- 重置统计信息ANALYZE users; -- 更新单表统计ANALYZE; -- 更新所有表统计
-- 查看表索引使用情况(找无用索引)SELECT schemaname, tablename, indexname, idx_scan, -- 索引被使用的次数(0 = 从未使用!) idx_tup_read, idx_tup_fetchFROM pg_stat_user_indexesORDER BY idx_scan ASC;三、声明式分区表
-- 创建按月分区的日志表(RANGE 分区)CREATE TABLE logs ( id BIGSERIAL, level TEXT NOT NULL, message TEXT NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()) PARTITION BY RANGE (created_at);
-- 创建分区(按月)CREATE TABLE logs_2026_07 PARTITION OF logs FOR VALUES FROM ('2026-07-01') TO ('2026-08-01');CREATE TABLE logs_2026_08 PARTITION OF logs FOR VALUES FROM ('2026-08-01') TO ('2026-09-01');
-- 每个分区可以独立建索引CREATE INDEX ON logs_2026_08 (level, created_at);
-- 查询时自动分区剪枝(只扫描相关分区)EXPLAIN SELECT * FROM logsWHERE created_at BETWEEN '2026-08-01' AND '2026-08-31';-- 输出:只扫描 logs_2026_08,忽略其他分区
-- LIST 分区(按枚举值)CREATE TABLE users_partitioned ( id BIGSERIAL, region TEXT NOT NULL, name TEXT NOT NULL) PARTITION BY LIST (region);
CREATE TABLE users_cn PARTITION OF users_partitioned FOR VALUES IN ('CN', 'TW', 'HK');CREATE TABLE users_us PARTITION OF users_partitioned FOR VALUES IN ('US', 'CA');CREATE TABLE users_eu PARTITION OF users_partitioned FOR VALUES IN ('DE', 'FR', 'GB');
-- 自动创建分区(pg_partman 扩展)-- CREATE EXTENSION pg_partman;-- SELECT partman.create_parent('public.logs', 'created_at', 'native', 'monthly');
-- 删除旧分区(比 DELETE 快,且立即释放磁盘空间)DROP TABLE logs_2025_01; -- 直接删除整个月的数据,毫秒级四、高可用:Patroni + etcd
架构: Primary(读写) ├── Standby 1(流复制,只读) └── Standby 2(流复制,只读)
Patroni(在每个节点运行):监控健康 + 自动故障转移etcd(3节点):分布式共识,选举领导者HAProxy / PgBouncer:客户端连接入口# patroni.yml(每个节点配置)scope: postgres-clusternamespace: /db/name: pg-node-1 # 每个节点不同
restapi: listen: 0.0.0.0:8008 connect_address: 192.168.1.10:8008
etcd3: hosts: 192.168.1.20:2379,192.168.1.21:2379,192.168.1.22:2379
bootstrap: dcs: ttl: 30 loop_wait: 10 retry_timeout: 10 maximum_lag_on_failover: 1048576 # 1MB 最大允许延迟 initdb: - encoding: UTF8 - data-checksums
postgresql: listen: 0.0.0.0:5432 connect_address: 192.168.1.10:5432 data_dir: /var/lib/postgresql/data authentication: replication: username: replicator password: rep_password superuser: username: postgres password: pg_password parameters: max_connections: 200 shared_buffers: 2GB # 建议为总内存的 25% work_mem: 16MB # 每个排序/哈希操作的内存 maintenance_work_mem: 256MB wal_level: replica max_wal_senders: 10 max_replication_slots: 10 hot_standby: on# 启动 Patronipatroni /etc/patroni.yml
# 查看集群状态patronictl -c /etc/patroni.yml list# ┌─────────────────────────────────────────────────────────────┐# │ Cluster: postgres-cluster (7094693036467678238) │# ├────────────┬────────────────────┬────────┬─────┬───────────┤# │ Member │ Host │ Role │ Lag │ State │# ├────────────┼────────────────────┼────────┼─────┼───────────┤# │ pg-node-1 │ 192.168.1.10:5432 │ Leader │ │ running │# │ pg-node-2 │ 192.168.1.11:5432 │ Replica│ 0 │ streaming │# │ pg-node-3 │ 192.168.1.12:5432 │ Replica│ 0 │ streaming │# └────────────┴────────────────────┴────────┴─────┴───────────┘
# 手动切换主节点(计划维护)patronictl -c /etc/patroni.yml switchover postgres-cluster五、PgBouncer 连接池
PostgreSQL 每个连接消耗约 5-10MB 内存,高并发时需要连接池:
[databases]# 格式:别名 = host=... port=... dbname=...myapp = host=127.0.0.1 port=5432 dbname=myapp_db
[pgbouncer]listen_addr = 0.0.0.0listen_port = 5432 # 客户端连接此端口
# 连接池模式(重要!)# session - 会话级(一个客户端独占一个服务器连接,类似无连接池)# transaction - 事务级(推荐!每个事务结束后连接归还池)# statement - 语句级(不支持事务,较少用)pool_mode = transaction
# 连接池大小max_client_conn = 1000 # 最大客户端连接数default_pool_size = 25 # 每个数据库的服务器连接数min_pool_size = 5reserve_pool_size = 5
# 认证auth_type = md5auth_file = /etc/pgbouncer/userlist.txt
# 超时server_idle_timeout = 600client_idle_timeout = 0# 查看 PgBouncer 状态psql -h 127.0.0.1 -p 6432 -U pgbouncer pgbouncer -c "SHOW POOLS;"psql -h 127.0.0.1 -p 6432 -U pgbouncer pgbouncer -c "SHOW STATS;"六、Python 异步访问(psycopg3 + SQLAlchemy 2.0)
6.1 psycopg3 原生异步
import asyncioimport psycopgfrom psycopg.rows import dict_row
async def main(): # 连接字符串 conn_str = "postgresql://user:password@localhost/mydb"
async with await psycopg.AsyncConnection.connect( conn_str, row_factory=dict_row ) as conn: # 查询 async with conn.cursor() as cur: await cur.execute( "SELECT id, name, email FROM users WHERE status = %s LIMIT %s", ("active", 10) ) users = await cur.fetchall() for user in users: print(user["name"], user["email"])
# 事务 async with conn.transaction(): await conn.execute( "INSERT INTO orders (user_id, amount) VALUES (%s, %s)", (42, 99.9) ) await conn.execute( "UPDATE users SET order_count = order_count + 1 WHERE id = %s", (42,) )
# 异步连接池(生产推荐)from psycopg_pool import AsyncConnectionPool
pool = AsyncConnectionPool( conn_str, min_size=5, max_size=20, kwargs={"row_factory": dict_row},)
async def get_user(user_id: int) -> dict | None: async with pool.connection() as conn: result = await conn.fetchone( "SELECT * FROM users WHERE id = %s", (user_id,) ) return result6.2 SQLAlchemy 2.0 异步 ORM
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine, async_sessionmakerfrom sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationshipfrom sqlalchemy import String, Integer, ForeignKey, selectfrom datetime import datetime
# 引擎(异步)engine = create_async_engine( "postgresql+psycopg://user:password@localhost/mydb", pool_size=20, max_overflow=10, echo=False,)
AsyncSessionFactory = async_sessionmaker(engine, expire_on_commit=False)
# ── ORM 模型(SQLAlchemy 2.0 风格)────────────────────────────class Base(DeclarativeBase): pass
class User(Base): __tablename__ = "users"
id: Mapped[int] = mapped_column(primary_key=True) name: Mapped[str] = mapped_column(String(100)) email: Mapped[str] = mapped_column(String(200), unique=True) created_at: Mapped[datetime] = mapped_column(default=datetime.utcnow) orders: Mapped[list["Order"]] = relationship(back_populates="user")
class Order(Base): __tablename__ = "orders"
id: Mapped[int] = mapped_column(primary_key=True) user_id: Mapped[int] = mapped_column(ForeignKey("users.id")) amount: Mapped[float] user: Mapped[User] = relationship(back_populates="orders")
# ── 异步 CRUD ─────────────────────────────────────────────────async def get_users_with_orders(min_amount: float) -> list[User]: async with AsyncSessionFactory() as session: stmt = ( select(User) .join(User.orders) .where(Order.amount >= min_amount) .order_by(User.name) ) result = await session.execute(stmt) return result.scalars().unique().all()
async def create_user_with_order(name: str, email: str, amount: float) -> User: async with AsyncSessionFactory() as session: async with session.begin(): # 自动提交/回滚 user = User(name=name, email=email) session.add(user) await session.flush() # 获取 user.id(未提交)
order = Order(user_id=user.id, amount=amount) session.add(order) # session.begin() 退出时自动 COMMIT return user七、常用运维命令
-- 查看数据库大小SELECT pg_database.datname, pg_size_pretty(pg_database_size(pg_database.datname)) AS sizeFROM pg_databaseORDER BY pg_database_size(pg_database.datname) DESC;
-- 查看表大小(含索引)SELECT relname AS table, pg_size_pretty(pg_total_relation_size(oid)) AS total_size, pg_size_pretty(pg_relation_size(oid)) AS table_size, pg_size_pretty(pg_indexes_size(oid)) AS index_sizeFROM pg_classWHERE relkind = 'r' AND relname NOT LIKE 'pg_%'ORDER BY pg_total_relation_size(oid) DESCLIMIT 20;
-- 查看表膨胀(dead tuples 过多需要 VACUUM)SELECT relname, n_dead_tup, n_live_tup, ROUND(n_dead_tup::numeric / NULLIF(n_live_tup + n_dead_tup, 0) * 100, 2) AS dead_pct, last_vacuum, last_autovacuumFROM pg_stat_user_tablesORDER BY n_dead_tup DESCLIMIT 10;
-- 手动清理(通常 autovacuum 自动处理)VACUUM ANALYZE users; -- 回收死元组 + 更新统计信息VACUUM FULL users; -- 完全重写表(排他锁!仅在严重膨胀时用)
-- 查看锁等待SELECT pid, pg_blocking_pids(pid) AS blocked_by, query, state, wait_event_type, wait_eventFROM pg_stat_activityWHERE cardinality(pg_blocking_pids(pid)) > 0;
-- 终止长时间运行的查询SELECT pg_terminate_backend(pid)FROM pg_stat_activityWHERE query_start < NOW() - INTERVAL '10 minutes' AND state = 'active';相关文章:
- Redis 完全指南 2026:数据结构 + 缓存设计 + 持久化
- Docker 完全指南 2026:Compose 多服务编排与生产部署
- Kubernetes 入门完全指南 2026:核心概念 + kubectl + Helm
- FastAPI 完全指南 2026:从零构建高性能异步 Python API
- Nginx 进阶完全指南 2026:反向代理 + HTTPS + 负载均衡
本文基于 PostgreSQL 16.x / psycopg 3.2 / SQLAlchemy 2.0.x 验证。Patroni 3.x 配置格式与 2.x 有细微差异,请以官方文档为准。
PostgreSQL 完全指南 2026:SQL 进阶 + 索引优化 + 分区表 + 高可用方案
https://971918.xyz/posts/docs/postgresql-complete-guide/