GreenPlum新建实例后需要做哪些基本优化操作
OS配置
内核参数
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 | sed -i s/SELINUX=enforcing/SELINUX=disabled/g /etc/selinux/config setenforce 0 ll /lib64/security/pam_limits.so echo "session required /lib64/security/pam_limits.so" >> /etc/pam.d/login cat >> /etc/security/limits.conf <<"EOF" * soft nofile 655350 * hard nofile 655350 * soft nproc 655350 * hard nproc 655350 gpadmin soft priority -20 EOF sed -i 's/4096/655350/' /etc/security/limits.d/20-nproc.conf cat /etc/security/limits.d/20-nproc.conf ulimit -HSn 65535 cat >> /etc/sysctl.conf <<"EOF" fs.file-max=9000000 fs.inotify.max_user_instances = 1000000 fs.inotify.max_user_watches = 1000000 kernel.pid_max=4194304 kernel.shmmax = 4398046511104 kernel.shmmni = 4096 kernel.shmall = 4000000000 kernel.sem = 32000 1024000000 500 32000 vm.overcommit_memory=2 vm.overcommit_ratio=95 net.ipv4.ip_forward=1 vm.swappiness=20 vm.dirty_background_bytes = 0 vm.dirty_background_ratio = 5 vm.dirty_bytes = 0 vm.dirty_expire_centisecs = 600 vm.dirty_ratio = 10 vm.dirty_writeback_centisecs = 100 vm.vfs_cache_pressure = 500 vm.min_free_kbytes = 2097152 EOF sysctl -p |
进程优先级
1 | echo 'gpadmin soft priority -20' >> /etc/security/limits.conf |
配置磁盘预读
在含有数据目录的设备上,blockdev预读尺寸应该被设置为16384。
1 2 3 4 | /sbin/blockdev --getra /dev/sdb /sbin/blockdev --setra 16384 /dev/sdb echo '/sbin/blockdev --setra 16384 /dev/sdb' >> /etc/rc.local |
加入/etc/rc.local文件中。
磁盘挂载参数
1 | mount -o rw,nodev,noatime,nobarrier,inode64 /dev/sdb /data |
禁用透明大页
不建议对数据库工作负载使用 THP (Oracle、MySQL、PostgreSQL、MongoDB均建议关闭THP),因为THP在运行时动态分配内存,而运行时的内存分配会有延迟,对于数据库的管理来说并不友好,会导致数据库性能抖动,所以建议关闭THP。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | cat /sys/kernel/mm/transparent_hugepage/defrag cat /sys/kernel/mm/transparent_hugepage/enabled echo never > /sys/kernel/mm/transparent_hugepage/enabled echo never > /sys/kernel/mm/transparent_hugepage/defrag cat >> /etc/rc.local <<"EOF" if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo never > /sys/kernel/mm/transparent_hugepage/enabled fi if test -f /sys/kernel/mm/transparent_hugepage/defrag; then echo never > /sys/kernel/mm/transparent_hugepage/defrag fi EOF chmod +x /etc/rc.d/rc.local |
参考:https://www.xmmup.com/pgshujukupeizhidaye.html
配置足够的swap
若是内存不足,发生自动切换,会报:“FTS: cannot establish libpq connection (content=0, dbid=11): could not fork new process for connection: Cannot allocate memory”或“FATAL: Out of memory. Failed on request of size 144 bytes. (context 'GPORCAmemory pool') ”或“ATAL: the database system is in recovery mode”,若没有swap内存配置,会发生OOM,特别严重时会导致segment自动故障切换。
1 2 3 4 5 6 7 8 9 10 11 12 13 | -- 主机设置,大小一般为内存的1.5倍 dd if=/dev/zero of=/root/swapfile bs=1G count=100 chmod -R 0600 /root/swapfile mkswap /root/swapfile swapon /root/swapfile echo '/root/swapfile swap swap defaults 0 0' >> /etc/fstab swapon -s |
安装
segment数规划
参数配置
注意:standby master的值需要单独修改postgresql.conf文件
计算gp_vmem_protect_limit 的值:https://greenplum.org/calc/ ,其中,“Primary Segments Per Server”为所有的实例个数,包括p和m。该参数限制每个Instance上所有语句可以使⽤的内存总量的上限值(以MB为单位)。 如果查询导致超出此限制,则不会分配内存,查询将失败,配置合理可以有效避免OOM的发生。gp_vmem_protect_limit没有统计到共享内存,仍旧有操作系统OOM的风险,在计算时需要排除掉该部分内存的空间。
注意: 该参数若配置过大,会导致发生OOM后,实例自动切换;若配置过小,则会导致查询报错OOM(ERROR: Canceling query because of high VMEM usage. Used: 29MB, available 1MB, red zone: 90MB (runaway_cleaner.c:202) 或 Vmem limit reached, failed to allocate或gpcc报错:Warning: Updates stalled due to heavy GPDB workload or segment failure (system, diskusage, alert))或DETAIL: Vmem limit reached, failed to allocate 12582912 bytes from tracker, which has 0 MB available 但并不会发生实例切换。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | -- 以下参数需要重启(注意:standby master的值需要单独修改postgresql.conf文件) -- gp_vmem_protect_limit和gp_resqueue_priority_cpucores_per_segment需要计算 -- max_prepared_transactions需要和max_connections的master配置一样 gpconfig -c gp_vmem_protect_limit -v 16658 -m 102400 gpconfig -c shared_buffers -v 1GB -m 8GB gpconfig -c max_connections -v 3000 -m 1000 gpconfig -c max_prepared_transactions -v 1000 gpconfig -c track_activity_query_size -v 102400 -m 102400 gpconfig -c gp_resqueue_priority_cpucores_per_segment -v 4 -m 32 gpconfig -c gp_enable_global_deadlock_detector -v on gpconfig -c gp_segworker_relative_priority -v 0 -m 0 --skipvalidation gpconfig -c runaway_detector_activation_percent -v 98 gpconfig -c gp_vmem_protect_segworker_cache_limit -v 200 gpstop -M fast -ra -- 查询 gpconfig -s gp_vmem_protect_limit gpconfig -s shared_buffers gpconfig -s max_connections gpconfig -s max_prepared_transactions gpconfig -s track_activity_query_size gpconfig -s gp_resqueue_priority_cpucores_per_segment gpconfig -s gp_enable_global_deadlock_detector gpconfig -s gp_segworker_relative_priority |
定时清理主库日志或只记录DDL语句日志
不需要重启库!!!
1 2 3 4 5 6 7 8 9 10 | gpconfig -s log_statement gpconfig -c log_statement -v ddl --masteronly gpstop -u gpconfig -s log_statement -- 定时清理 0 1 * * * find /opt/greenplum/data/master/gpseg-1/pg_log/ -name "gpdb*.csv" -ctime +7 -exec rm -rf {} \; systemctl status crond |
参考:https://www.xmmup.com/peizhicrontabdingshiqingligreenplumrizhiwenjian.html
其它参数调优(不需要重启库)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 | gpconfig -c work_mem -v 2GB gpconfig -c temp_buffers -v 4GB gpconfig -c maintenance_work_mem -v 2GB gpconfig -c effective_cache_size -v 60GB -m 100GB gpconfig -c statement_mem -v 2GB gpconfig -c max_statement_mem -v 6GB gpconfig -c gp_workfile_compression -v on gpconfig -c gp_autostats_mode -v on_change gpconfig -c gp_autostats_on_change_threshold -v 100000 gpconfig -c gp_vmem_idle_resource_timeout -v 10s gpstop -u -- 查询 gpconfig -s work_mem gpconfig -s statement_mem gpconfig -s temp_buffers gpconfig -s maintenance_work_mem gpconfig -s effective_cache_size gpconfig -s max_statement_mem gpconfig -s gp_workfile_compression gpconfig -s gp_autostats_mode gpconfig -s gp_autostats_on_change_threshold |
注意:
1、参数statement_mem在6.24.4版本中测试的有问题,不能配置该参数,否则会导致GP不能启动。但在6.25.1中测试又正常了。
2、参数work_mem在后期版本中可能会被废弃,不建议配置。WARNING "work_mem": setting is deprecated, and may be removed in a future release.
远程访问
1 2 3 4 | echo "host all all all md5" >> /opt/greenplum/data/master/gpseg-1/pg_hba.conf -- 使改动生效切不中断服务 gpstop -u |
定时收集统计信息
方法1:vacuumdb
缺点:不能并行,执行较慢
优点:占用资源少