Rootop 服务器运维与web架构

2020-11-12
发表者 Venus
java.lang.OutOfMemoryError: unable to create new native thread已关闭评论

java.lang.OutOfMemoryError: unable to create new native thread

日志监控系统报警,java日志不停报错:

07:02:44.168 [http-nio-8080-exec-3] ERROR com.xxx.common.base.SLExceptionHandler - Handler dispatch failed; nested exception is java.lang.OutOfMemoryError: unable to create new native thread

看提示是OutOfMemoryError发生了OOM,也就是内存溢出,无法创建新线程,但是通过zabbix监控这台机器内存还有1.7G左右,并没有用完。

想重启服务,发现ssh也无法连接了,提示 reset by peer,只好登陆控制台强制重启。
进系统后发现 /var/log/secure 日志

Nov 11 07:10:25 node1 sshd[1043]: error: fork: Cannot allocate memory

竟然也提示无法分配内存,搜索资料说跟最大进程数有关,但未优化的系统默认是1024,不应该产生这么多进程,而且进程数多了zabbix有报警。
还有一条错误是 /var/log/messages中:

Nov 11 07:24:45 node1 chronyd[566]: Fatal error : pthread_create() failed

pthread_create 是系统创建线程的方法,提示失败,那是不是线程数量超过系统限制才无法创建。但是系统已经重启,已经看不到当时的在运行的线程数。

# 查看系统限制最大进程数
[root@localhost ~]# cat /proc/sys/kernel/pid_max 
32768
# 查看系统限制最大线程数
[root@localhost ~]# cat /proc/sys/kernel/threads-max 
30101

可以看到最大线程数是30101。
想起来前阵子有个服务占内存高,不停的在增长也导致服务器挂过一次,后来加了内存监控报警,小于1G就报警,重启。这次还没小于1G就挂了。
就查此模块的线程数,发现每分钟都在增加。算了下24小时大约会生成4000+个线程。

所以线程数满了后,也错误的提示了内存分配的错误,导致排错方向错误。
开发修改代码后,线程数不再增长了。

注意:系统也会对某个用户进行最大运行进程限制(ulimit -u)

2020-11-04
发表者 Venus
python元组只有一个元素已关闭评论

python元组只有一个元素

print( ('aaa') )

返回字符串 aaa

print( ('aaa',) )

返回一个元组 (‘aaa’,)

# 列表包元组

print( [('aaa')] )

返回列表 [‘aaa’]

print( [('aaa',)] )

返回列表包元组 [(‘aaa’,)]

所以要实现单个元素的元组,就需要在元素后面加一个逗号。

2020-11-04
发表者 Venus
docker搭建以太坊公链已关闭评论

docker搭建以太坊公链

# 拉取镜像

[root@ecs-5731 ~]# docker pull ethereum/client-go:v1.9.23
[root@ecs-5731 ~]# docker run -dit --name geth -p 30303:30303 -p 8545:8545 -p 30303:30303/udp -v /etc/localtime:/etc/localtime -v /home/geth:/root/.ethereum ethereum/client-go:v1.9.23 --rpc --rpcaddr "0.0.0.0"

进入容器并进入geth控制台

/ # geth attach /root/.ethereum/geth.ipc 
> eth.syncing # 同步状态
false

当返回false时,则同步完成。
> eth.blockNumber # 同步过程中通过 eth.blockNumber 查看当前区块号的话会显示为0
11188232
> net.peerCount # 连接了多少个节点进行同步
14

在同步过程中,查看同步状态,currentBlock和highestBlock 差100多个区块,这段时间停留了很久,大约2天才能完全同步完。

同步完成后,eth.syncing会返回false,查看容器日志会打印Imported new chain segment字符信息。

[root@ecs-5731 ~]# docker logs -f geth
INFO [11-04|11:26:48.079] Imported new chain segment               blocks=1     txs=142  mgas=12.446 elapsed=101.256ms   mgasps=122.916 number=11188241 hash="88c31d…4cb3c7" dirty=648.96MiB

当前块和高度块差100多资料:https://www.jianshu.com/p/c172b807514f
注意:8545端口不要对外网开放或限制访问来源。

2020-10-30
发表者 Venus
设置history查看历史命令执行时间已关闭评论

设置history查看历史命令执行时间

用户家目录下的 .bash_profile 和 .bashrc 都可以设置参数,实现用户登陆后加载环境信息或者要自动执行的命令。

# .bash_profile会调用.bashrc脚本

[root@localhost ~]# cat .bash_profile 
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
	. ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

export PATH

# .bashrc会调用/etc/bashrc

[root@localhost ~]# cat .bashrc 
# .bashrc

# User specific aliases and functions

alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
alias vi=vim

# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi

# 调用 /etc/profile.d/*.sh 下所有sh文件

[root@localhost ~]# cat /etc/bashrc 
# /etc/bashrc
略过部分
    for i in /etc/profile.d/*.sh; do
        if [ -r "$i" ]; then
            if [ "$PS1" ]; then
                . "$i"
            else
                . "$i" >/dev/null
            fi
        fi
    done
略过部分

# sh脚本

[root@localhost ~]# ll /etc/profile.d/
total 60
-rw-r--r--. 1 root root  771 Nov  6  2016 256term.csh
-rw-r--r--. 1 root root  841 Nov  6  2016 256term.sh
-rw-r--r--. 1 root root  196 Apr 29  2015 colorgrep.csh
-rw-r--r--. 1 root root  201 Apr 29  2015 colorgrep.sh
-rw-r--r--. 1 root root 1741 Nov  5  2016 colorls.csh
-rw-r--r--. 1 root root 1606 Nov  5  2016 colorls.sh
-rw-r--r--  1 root root   62 Oct 30 15:35 history.sh
-rw-r--r--. 1 root root 1706 Nov  6  2016 lang.csh
-rw-r--r--. 1 root root 2703 Nov  6  2016 lang.sh
-rw-r--r--. 1 root root  123 Jul 31  2015 less.csh
-rw-r--r--. 1 root root  121 Jul 31  2015 less.sh
-rw-r--r--. 1 root root  105 Dec 22  2016 vim.csh
-rw-r--r--. 1 root root  269 Dec 22  2016 vim.sh
-rw-r--r--. 1 root root  164 Jan 28  2014 which2.csh
-rw-r--r--. 1 root root  169 Jan 28  2014 which2.sh

所以就把记录历史命令时间功能的变量放到此目录下。

[root@localhost ~]# cd /etc/profile.d/
[root@localhost profile.d]# cat history.sh 
export HISTSIZE=10000
export HISTTIMEFORMAT="%F %T `whoami` "

效果

 

2020-09-18
发表者 Venus
mysql中explain分析sql执行type字段值的含义已关闭评论

mysql中explain分析sql执行type字段值的含义

# mysql5.7中type的类型达到了14种之多,这里只记录和理解最重要且经常遇见的六种类型,它们分别是all、index、range、ref、eq_ref、const。
# 从左到右,它们的效率依次是增强的。撇开sql的具体应用环境以及其他因素,应当尽量优化你的sql语句,使它的type尽量靠右
# 资料来自:https://blog.csdn.net/dennis211/article/details/78170079

创建2张表测试

CREATE TABLE `user` (
   `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
   `name` varchar(100) DEFAULT NULL,
   `version` int(10) DEFAULT NULL,
   PRIMARY KEY (`id`)
 ) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8

insert into `user` (`id`, `name`, `version`) values('1','aaaaaaa','11');
insert into `user` (`id`, `name`, `version`) values('2','bbbbbb','10');

CREATE TABLE `info` (
   `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
   `name` varchar(100) DEFAULT NULL,
   `age` int(10) DEFAULT NULL,
   `num` int(10) DEFAULT NULL,
   PRIMARY KEY (`id`),
   UNIQUE KEY `index_num` (`num`),
   KEY `index_name` (`name`)
 ) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8
 
insert into `info` (`id`, `name`, `age`, `num`) values('1','user1','1','11');
insert into `info` (`id`, `name`, `age`, `num`) values('2','user2','2','12');
insert into `info` (`id`, `name`, `age`, `num`) values('3','user3','3','13');
insert into `info` (`id`, `name`, `age`, `num`) values('4','user4','4','14');
insert into `info` (`id`, `name`, `age`, `num`) values('5','user5','5','15');

测试:

# type = all ,age字段无索引,则进行全表扫描
mysql -uroot -p"root" -e 'use aaa;EXPLAIN SELECT * FROM info WHERE age = 1;'
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | info  | NULL       | ALL  | NULL          | NULL | NULL    | NULL |    5 |    20.00 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+

# type = index 且 Extra = NULL 时,为走索引的全表扫描。这样的索引,在排序时比type = all的排序速度快。
mysql -uroot -p"root" -e 'use aaa;EXPLAIN SELECT * FROM info ORDER BY id DESC;'
+----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-------+
| id | select_type | table | partitions | type  | possible_keys | key     | key_len | ref  | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-------+
|  1 | SIMPLE      | info  | NULL       | index | NULL          | PRIMARY | 4       | NULL |    5 |   100.00 | NULL  |
+----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-------+

# type = index 且 Extra = Using index 则为索引覆盖,从索引直接获取了数据,不用查表。name字段是普通索引
mysql -uroot -p"root" -e 'use aaa;EXPLAIN SELECT `name` FROM info;'
+----+-------------+-------+------------+-------+---------------+------------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type  | possible_keys | key        | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-------+------------+-------+---------------+------------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | info  | NULL       | index | NULL          | index_name | 303     | NULL |    5 |   100.00 | Using index |
+----+-------------+-------+------------+-------+---------------+------------+---------+------+------+----------+-------------+

# type = range , id自增主键,范围扫描
mysql -uroot -p"root" -e 'use aaa;EXPLAIN SELECT * FROM info WHERE id <= 100;'
+----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type  | possible_keys | key     | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | info  | NULL       | range | PRIMARY       | PRIMARY | 4       | NULL |    5 |   100.00 | Using where |
+----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-------------+

# type = ref , name字段普通索引,sql使用了索引,但该索引列的值并不唯一,有重复
mysql -uroot -p"root" -e 'use aaa;EXPLAIN SELECT * FROM info WHERE `name` = "user1";'
+----+-------------+-------+------------+------+---------------+------------+---------+-------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key        | key_len | ref   | rows | filtered | Extra |
+----+-------------+-------+------------+------+---------------+------------+---------+-------+------+----------+-------+
|  1 | SIMPLE      | info  | NULL       | ref  | index_name    | index_name | 303     | const |    1 |   100.00 | NULL  |
+----+-------------+-------+------------+------+---------------+------------+---------+-------+------+----------+-------+

# type = eq_ref , 字段索引类型唯一,但单表测试结果是const,下面换联合查询可以。
mysql -uroot -p"root" -e 'use aaa;EXPLAIN SELECT * FROM info WHERE num = 11;'
+----+-------------+-------+------------+-------+---------------+-----------+---------+-------+------+----------+-------+
| id | select_type | table | partitions | type  | possible_keys | key       | key_len | ref   | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+-----------+---------+-------+------+----------+-------+
|  1 | SIMPLE      | info  | NULL       | const | index_num     | index_num | 5       | const |    1 |   100.00 | NULL  |
+----+-------------+-------+------------+-------+---------------+-----------+---------+-------+------+----------+-------+

mysql -uroot -p"root" -e 'use aaa;EXPLAIN 
SELECT i.* 
FROM `user` AS u
LEFT JOIN info AS i 
ON u.id = i.id;'
+----+-------------+-------+------------+--------+---------------+---------+---------+----------+------+----------+-------------+
| id | select_type | table | partitions | type   | possible_keys | key     | key_len | ref      | rows | filtered | Extra       |
+----+-------------+-------+------------+--------+---------------+---------+---------+----------+------+----------+-------------+
|  1 | SIMPLE      | u     | NULL       | index  | NULL          | PRIMARY | 4       | NULL     |    2 |   100.00 | Using index |
|  1 | SIMPLE      | i     | NULL       | eq_ref | PRIMARY       | PRIMARY | 4       | aaa.u.id |    1 |   100.00 | NULL        |
+----+-------------+-------+------------+--------+---------------+---------+---------+----------+------+----------+-------------+

# const , id自增主键,精确定位
mysql -uroot -p"root" -e 'use aaa;EXPLAIN SELECT * FROM info WHERE id = 1;'
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
| id | select_type | table | partitions | type  | possible_keys | key     | key_len | ref   | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
|  1 | SIMPLE      | info  | NULL       | const | PRIMARY       | PRIMARY | 4       | const |    1 |   100.00 | NULL  |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+