合 内核参数kernel.shmall配置太小导致ORA-27102: out of memory和Linux-x86_64 Error: 28: No space left on device
Tags: Oracle故障处理No space left on device内核参数kernel.shmallORA-27102out of memory
现象:启动数据库报错
线下IDC的服务器,RHEL 6.5,使用华为云工具SMS整机迁移到云端,云端集群的内存从60GB升级到190GB:
线下IDC的配置:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | [root@lhrdbs ~]# free -m total used free shared buffers cached Mem: 64556 64451 105 0 9 21062 -/+ buffers/cache: 43378 21178 Swap: 67583 4098 63485 [root@lhrdbs ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 6.5 (Santiago) [root@lhrdbs ~]# cat /etc/security/limits.conf | grep oracle oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536 #oracle - memlock unlimited oracle soft memlock 60397977 oracle hard memlock 60397977 [root@lhrdbs ~]# more /etc/sysctl.conf | grep shm kernel.shmmax = 34359738368 kernel.shmall = 16777216 kernel.shmmni = 4096 [root@lhrdbs ~]# more /etc/sysctl.conf | grep nr_hugepages vm.nr_hugepages = 20485 |
内存升级后:
1 2 3 4 5 | [root@lhrdbs ~]# free -m total used free shared buffers cached Mem: 193837 42558 151279 0 25 210 -/+ buffers/cache: 42322 151515 Swap: 67583 0 67583 |
客户要求配置大页,并优化SGA的参数。
我按照文档 https://www.dbaup.com/oracleshujukupeizhidaye.html 来配置大页后,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | # 该参数只会影响共享内存段的个数 cat >> /etc/security/limits.conf <<"EOF" * soft memlock -1 * hard memlock -1 EOF [oracle@lhrdbs ~]$ ulimit -l unlimited echo -e 'vm.nr_hugepages=76800' >> /etc/sysctl.conf echo 76800 > /proc/sys/vm/nr_hugepages grep -i HugePage /proc/meminfo alter system set sga_max_size=143360M scope=spfile; [oracle@lhrdbs ~]$ grep -i HugePage /proc/meminfo AnonHugePages: 0 kB HugePages_Total: 76800 HugePages_Free: 76800 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB [root@lhrdbs ~]# free -m total used free shared buffers cached Mem: 193837 163461 30376 0 30 7391 -/+ buffers/cache: 156039 37797 Swap: 67583 0 67583 |
在启动数据库时报错了:
1 2 3 4 5 6 7 8 | SQL> alter system set sga_max_size=143360M scope=spfile; System altered. SQL> startup force ORA-27102: out of memory Linux-x86_64 Error: 28: No space left on device Additional information: 1 |
分析
检查内存:
1 2 3 4 5 | [oracle@lhrdbs ~]$ free -m total used free shared buffers cached Mem: 193837 155248 38589 0 24 276 -/+ buffers/cache: 154947 38890 Swap: 67583 0 67583 |
检查共享内存:
1 2 3 4 5 6 7 8 9 10 11 | [oracle@lhrdbs ~]$ ipcs -a ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status ------ Semaphore Arrays -------- key semid owner perms nsems ------ Message Queues -------- key msqid owner perms used-bytes messages |
都没问题,那问题就在/etc/sysctl.conf中,有关kernel.shmmax的参数值:
1 2 3 4 | [root@lhrdbs ~]# more /etc/sysctl.conf | grep shm kernel.shmmax = 34359738368 kernel.shmall = 16777216 kernel.shmmni = 4096 |
参数kernel.shmall配置太小了!!!
其中,16777216页*4kb/1024/1024=64GB
,只能分配64GB的共享内存段,所以启动会报错。
解决
只修改kernel.shmall=4398046511104
即可,但是为了性能,建议修改如下:
1 2 3 | vi /etc/sysctl.conf kernel.shmall = 1073741824 kernel.shmmax = 4398046511104 |
修改完成后,执行如下命令立即生效:
1 | sysctl -p |
然后重启数据库即可。