In my case I was using Ubuntu 16.04 with Fail2ban installed as if fail2ban takes the ips from nginx access log, Which has repeated cloudflare ips listed on the file (As if Cloudflare used as reverse proxy). Fail2Ban started block Cloudflare ips
The solution would be logging the original ips of the request on the access file instead of Cloudflare ip
First check your Nginx has "ngx_http_realip_module"
nginx -V
If Enabled add the ips below to a file under conf.d
set_real_ip_from 103.21.244.0/22; set_real_ip_from 103.22.200.0/22; set_real_ip_from 103.31.4.0/22; set_real_ip_from 104.16.0.0/12; set_real_ip_from 108.162.192.0/18; set_real_ip_from 131.0.72.0/22; set_real_ip_from 141.101.64.0/18; set_real_ip_from 162.158.0.0/15; set_real_ip_from 172.64.0.0/13; set_real_ip_from 173.245.48.0/20; set_real_ip_from 188.114.96.0/20; set_real_ip_from 190.93.240.0/20; set_real_ip_from 197.234.240.0/22; set_real_ip_from 198.41.128.0/17; set_real_ip_from 199.27.128.0/21; set_real_ip_from 2400:cb00::/32; set_real_ip_from 2606:4700::/32; set_real_ip_from 2803:f800::/32; set_real_ip_from 2405:b500::/32; set_real_ip_from 2405:8100::/32; set_real_ip_from 2c0f:f248::/32; set_real_ip_from 2a06:98c0::/29; # use any of the following two real_ip_header CF-Connecting-IP; #real_ip_header X-Forwarded-For;
CloudFlare ip might change so please check this link