• Mirror: Increased bandwidth now required?


    I wanted to throw this out there, I have been running a mirror for a long while now (since joining the project) and previously I had not breached 1.5TB of usage a month. However, starting in Oct the mirror has been eating approximately 100G/day.

    Has anyone else noticed an uptick in bandwidth consumption on the mirrors. Its not a huge deal but I had to add an extra TB of bandwidth to my VPS last month.

    Is Antergos just getting more popular or is something else going on?

  • @stratus_ss said i

    Is Antergos just getting more popular or is something else going on?

    I don t really know if sth is going on but I keep monitoring Distrowatch daily. Antergos has been growing really fast lately. No 11 in the past month!

    https://distrowatch.com/table.php?distribution=antergos

    1.Antergos Linux KDE plasma / Gnome 2.Ubuntu 17.04 64bit Unity
    Intel Core2 Duo CPU P8400 2.26GHz‖ RAM 3908 MiB ‖ Dell Inc. 0F328M - Dell Inc. Latitude E6500
    Intel Mobile 4 Series Chipset Integrated Graphics [8086:2a42] {i915

  • @stratus_ss ive never seen the message board busier and like @anarch said we been moving up the distrowatch fast.

    for sure getting more popular.

  • @stratus_ss Indeed, we are getting more popular. However, 100G/day is a bit much I think. I’ll look into it ASAP!

  • I have had some time to start analyzing at least some of the traffic. This guy is a major offender

    70.162.95.10, it has made 9000 gets for antergos/x86_64/antergos.db.tar.gz in the month of October
    73.189.70.135 has made 5000 gets for the same file.
    73.189.105.88 has made 6800 gets for the same file between Oct 16 and the 29th
    66.8.186.100 4500 gets, same file
    67.180.245.72 4500 gets, same file

    These offenders have generated 3900 Megs of traffic alone (that file is only 134 in size thankfully).

    What the heck is going on here, this seems absurdly high considering that other IPs don’t have this same level. Some of these guys are making gets less than 5 minutes apart. Others are exactly 5 minutes, this seems like a very aggressive setting somewhere

  • just as a curious person, what is/was the standard average?
    Cheers

    Antergos (default OS) - WIN10 (abandoned)
    I3wm - Mate desktop
    AMD - A4 7300 Radeon graphics
    16 GB ram
    HD 1 TB
    Linux newbie since 06/2016

  • @fernandomaroto said in Mirror: Increased bandwidth now required?:

    just as a curious person, what is/was the standard average?
    Cheers

    Of the 19,000 unique IPs which has more than 10 hits, only 24 of them go above 900 hits in October. I had a total number of 1,182,595 gets which approximates to gets ~75 per unique IP.

    Now this includes the torrent traffic, the individual packages, the isos etc

    If I exclude the the 24 that go above 900 hits I have a total number of gets of: 822,102
    This approximates to ~52 hits per unique IPs

  • @stratus_ss All of those IPs are on global blacklists. I think they are probably bots. I havent had time to analyze the traffic yet myself but I actually notice a significant drop in overall bandwidth usage on our build server in October. It is a pretty significant drop (1TB lower than normal) so that is weird. ANyway, considering your findings and the fact that they are blacklisted, I think its okay to block those IPs.

  • Thanks for the reply. I went ahead and banned the top 20 offenders (anything over 2500 gets)

    For reference:

    	deny from 153.125.237.207
    	deny from 177.227.241.217
    	deny from 187.186.223.39
    	deny from 188.27.33.208
    	deny from 189.212.248.6
    	deny from 190.161.231.215
    	deny from 190.37.204.136
    	deny from 200.52.56.77
    	deny from 50.136.12.94
    	deny from 62.147.202.200
    	deny from 66.8.186.100
    	deny from 66.8.186.100
    	deny from 67.180.245.72
    	deny from 67.180.245.72 
    	deny from 70.162.95.10
    	deny from 72.199.100.8
    	deny from 73.189.105.88
    	deny from 73.189.70.135
    	deny from 73.189.70.135
    

    Though I doubt this is going to bring my bandwidth down significantly

bandwidth1 hosting1 mirror2 Posts 9Views 368
Log in to reply