普元 EOS Microservices 5.0 Governor中查询不到日志信息，logstash报错：[logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})

【问题】

EOS Microservices 5.0 Governor中查询不到日志信息，检查logstash日志，有报错信息：

Retrying individual bulk actions that failed or were rejected by the previous bulk request. {:count=>1}

[2018-09-11T20:08:57,162][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})

【解答】

问题原因： elasticsearch的索引变为只读状态了，无法正常写入数据导致governor里查询不到日志

项目中解决方法：

在linux上执行：curl -XPUT -H 'Content-Type: application/json' http://es服务器:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'

上述命令执行后，governor里可以查询到日志了

其中 _all 可以更改为自己在创建 Eleastisearch 索引时候的name，用来修改单个索引只读状态，用 _all 表示修改了所有的索引

此异常相关其他参考内容：https://blog.csdn.net/jingyoushui/article/details/100690329