• Ranking Antergos mirrors


    Hi,

    just wrote a bash script that ranks mirrors in file /etc/pacman.d/antergos-mirrorlist. The link to it is:
    https://pastebin.com/Xd0AK5zf and it is available for a week. @Keegan was keen on creating a GUI for it, but naturally it is not yet available.

    The script goes through the servers in the original /etc/pacman.d/antergos-mirrorlist file and creates a ranked list into that file. The original file is stored into /etc/pacman.d/antergos-mirrorlist.startingpoint for later use.

    User may give options that affect what the script does. User may exclude countries (-x) or specific servers (-xs), and choose (-p) the protocols (http and/or https) for the servers.

    As the file behind the link states, it currently does not handle antergos-mirrorlist.pacnew at all. Do you think it should?

    Feedback and suggestions are appreciated!

  • @manuel said in Ranking Antergos mirrors:

    it currently does not handle antergos-mirrorlist.pacnew at all. Do you think it should?

    That would be a great feature, but if it’s too much work, don’t worry about it:grinning:. I’ll start taking a look at it right away.

    Keep trying, never give up. In the end, you will find that it was all worth it

    My ISO:
    https://rebornos.wordpress.com
    Linux Basics: https://linuxbasicssite.wordpress.com

  • @Keegan
    It is also simple to manually take antergos-mirrorlist.pacnew into account just by copying it onto antergos-mirrorlist.startingpoint. And not very hard to do it on in the script as well, but the problem may be that the format of .pacnew could be different.

  • Whatever you think is best then! It is your creation ;).

    Keep trying, never give up. In the end, you will find that it was all worth it

    My ISO:
    https://rebornos.wordpress.com
    Linux Basics: https://linuxbasicssite.wordpress.com

  • @Keegan
    I’ll look into it. If the script does not delete the “original” files, it should be safe since we have “backups”.
    EDIT: ok, it was simple to do. At least it works for me. ;)
    But for the time being the pastebin version above should be ok, probably changes in .pacnew don’t happen very often.

  • reflector-antergos_v1.sh: line 31: $'\r': command not found
    reflector-antergos_v1.sh: line 34: $'\r': command not found
    reflector-antergos_v1.sh: line 36: $'\r': command not found
    reflector-antergos_v1.sh: line 38: $'\r': command not found
    reflector-antergos_v1.sh: line 40: syntax error near unexpected token $'{\r''
    reflector-antergos_v1.sh: line 40: {
    

    This is the output on running it inside terminal…

    [updates once a week] = [90% less problems]
    http://gofccyourself.com
    my-blog#k
    how to add system logs
    i3 GNOME

  • Here’s an output of running it:

    reflector-antergos -x=China,Russia,USA,Bulgaria,Japan -xs=sourceforge,unizar -p=https,http
    Usage: /usr/local/bin/reflector-antergos [<repo> <arch>] [options]
    where
    <repo>        antergos (default: antergos)
    <arch>        x86_64   (default: x86_64
    options:
      -x=X,Y,Z    Exclude repos from countries; note that
                  word 'Automated' can also be used.
      -xs=X,Y,Z   exclude server urls that contain any of strings X, Y, or Z
      -p=X,Y      Repo protocols in preference order (currently only
                  https and http are supported)
      -q          quiet, don't show progress info
    
    Example: reflector-antergos -x=China,Russia,USA,Automated -xs sourceforge -p=https
    
    Ranking https servers ...
    
    Automated Mirror Selection
       https://mirrors.antergos.com/antergos/x86_64 ... done
    Bulgaria
    Canada
       https://wynsrv.antergos.info/antergos/antergos/x86_64 ... done
    China
    Czech Republic
       https://mirrors.nic.cz/antergos/antergos/x86_64 ... done
    Denmark
       https://mirrors.dotsrc.org/antergos/antergos/x86_64 ... done
    France
       https://cinnarch.polymorf.fr/antergos/x86_64 ... done
       https://eu.mirrors.coltondrg.com/antergos/antergos/x86_64 ... done
    England
       https://www.mirrorservice.org/sites/repo.antergos.com/antergos/x86_64 ... done
    Germany
       https://mirror.de.leaseweb.net/antergos/antergos/x86_64 ... done
       https://antergos-mirror-a.alpix.eu/antergos/x86_64 ... done
       https://mirror.alpix.eu/antergos/antergos/x86_64 ... done
    Greece
       https://ftp.cc.uoc.gr/mirrors/linux/antergos/antergos/x86_64 ... done
    Japan
    Netherlands
       https://mirror.nl.leaseweb.net/antergos/antergos/x86_64 ... done
       https://ftp1.nluug.nl/os/Linux/distr/antergos/antergos/x86_64 ... done
       https://ftp2.nluug.nl/os/Linux/distr/antergos/antergos/x86_64 ... done
    Portugal
       https://glua.ua.pt/pub/antergos/antergos/x86_64 ... done
    Russia
    Spain
    Sweden
       https://ftp.acc.umu.se/mirror/antergos.com/antergos/x86_64 ... done
    USA
    
    Ranking http servers ...
    
    Automated Mirror Selection
       http://mirrors.antergos.com/antergos/x86_64 ... done
    Bulgaria
    Canada
       http://wynsrv.antergos.info/antergos/antergos/x86_64 ... done
    China
    Czech Republic
       http://mirrors.nic.cz/antergos/antergos/x86_64 ... done
    Denmark
       http://mirrors.dotsrc.org/antergos/antergos/x86_64 ... done
    France
       http://cinnarch.polymorf.fr/antergos/x86_64 ... done
       http://eu.mirrors.coltondrg.com/antergos/antergos/x86_64 ... done
    England
       http://www.mirrorservice.org/sites/repo.antergos.com/antergos/x86_64 ... done
    Germany
       http://mirror.de.leaseweb.net/antergos/antergos/x86_64 ... done
       http://antergos-mirror-a.alpix.eu/antergos/x86_64 ... done
       http://mirror.alpix.eu/antergos/antergos/x86_64 ... done
    Greece
       http://ftp.cc.uoc.gr/mirrors/linux/antergos/antergos/x86_64 ... done
    Japan
    Netherlands
       http://mirror.nl.leaseweb.net/antergos/antergos/x86_64 ... done
       http://ftp1.nluug.nl/os/Linux/distr/antergos/antergos/x86_64 ... done
       http://ftp2.nluug.nl/os/Linux/distr/antergos/antergos/x86_64 ... done
    Portugal
       http://glua.ua.pt/pub/antergos/antergos/x86_64 ... done
    Russia
    Spain
    Sweden
       http://ftp.acc.umu.se/mirror/antergos.com/antergos/x86_64 ... done
    USA
    
    Ranking done.
    
    Install new antergos-mirrorlist as root:
    Password:
    
  • @joekamprad
    Somehow the text file is in DOS format (DOS line endings). It can be converted easily with dos2unix.
    But I’ll also see if I can upload it in unix format.

  • Nice work @manuel!
    Here’s a few suggestions.

    1. Help output pattern

    You have quite a few calls to usage() only for the purpose of echoing a string. It’s not a big deal. But may make some users uncomfortable with the rest of your code. You should find ways to optimize that.

    One pattern for help display in scripts is a function making use of a heredoc string:

    showhelp() {
        cat <<EOF
    Usage: $0 [<repo> <arch>] [options]
    
        <repo>        antergos (default: antergos)
        <arch>        x86_64   (default: $(uname -m)
    
    Options:
    
      -x=X,Y,Z    Exclude repos from countries; note that
                  word 'Automated' can also be used.
      -xs=X,Y,Z   exclude server urls that contain any of strings X, Y, or Z
      -p=X,Y      Repo protocols in preference order (currently only https
                  and http are supported)
      -q          quiet, don't show progress info
    
    Example:
    
        $(basename $0) -x=China,Russia,USA,Automated -xs sourceforge -p=https
    EOF
    }
    

    You can now call this function from somewhere else in your code.

    2. Consistent options

    The presence of -xs can confuse some users. A usual convention is for shell programs to make use of getopts, which you don’t in your code but that’s fine. However, the principal motivation is for program options and arguments to follow the POSIX Utility Syntax Guidelines that in layman terms state, among other things, that a single letter options is prefixed with a single dash (-) and more than that requires a double dash (--).

    Double-dash options are usually aliases of single-dash options (for example -S and --sync on pacman) or unique options without a single dash (for instance, --force).

    So in your case, I would suggest you actually replaced -xs with -s. However for a better feeling of consistency, you could also replace the -x option with -c. So, c for country and s for server.

    3. Local variables
    You don’t need to qualify your variables with local unless they could override the caller variables. From a preliminary look at your script, I don’t see anywhere where this could happen.

  • @Krugar
    Hi,
    thank you for your comments!

    1. did you mean ‘echo’ instead of usage? There is only one call to Usage().

    2. option naming is sometimes a bit difficult. And true, -xs can be confusing, although you can see similar option naming in bigger programs, too. I’ll look into this and make them more consistent.

    3. It is a habit of mine to declare variables as local unless they need to be global (probably stems from my C/C++ background). Of course sh on some old systems don’t even know ‘local’ variables, but most Linux distros use bash anyway, even with the program named ‘sh’.

    Cheers!

  • Also problems here

    [[email protected] ~]$ sh ~/Downloads/reflector-antergos_v1.sh
    /home/liveuser/Downloads/reflector-antergos_v1.sh: line 31: $'\r': command not found
    /home/liveuser/Downloads/reflector-antergos_v1.sh: line 34: $'\r': command not found
    /home/liveuser/Downloads/reflector-antergos_v1.sh: line 36: $'\r': command not found
    /home/liveuser/Downloads/reflector-antergos_v1.sh: line 38: $'\r': command not found
    /home/liveuser/Downloads/reflector-antergos_v1.sh: line 40: syntax error near unexpected token$’{\r’’'home/liveuser/Downloads/reflector-antergos_v1.sh: line 40: {

    Why empty lines give errors? lol

    Antergos (default OS) - WIN10 (abandoned)
    I3wm - Mate desktop
    AMD - A4 7300 Radeon graphics
    16 GB ram
    HD 1 TB
    Linux newbie since 06/2016

  • @fernandomaroto
    Seems to be the same old problem, the file has DOS line endings.
    Can be fixed with dos2unix command.

    I’ll put a new version out with some other fixes hopefully in a couple of days.
    Cheers!

  • Hi all!

    Now there’s a new version out! Check https://pastebin.com/Xipe1ETk
    It should fix some problems, e.g.

    1. use Unix line endings!
    2. option names are modified, use the --help option
    3. works somewhat more reliably than the first version
    4. it is shorter (due to structural changes)

    Try e.g. these 3 commands:

    antergos-mirrorlist --help
    
    antergos-mirrorlist --exclude-countries=Automated,China,USA,Russia --exclude-mirrors=sourceforge --protocols=https,http
    
    antergos-mirrorlist
    

    and look at the results in /etc/pacman.d/antergos-mirrorlist. You should see a new file /etc/pacman.d/antergos-mirrorlist.startingpoint (don’t delete it!).

    Note that in the end of a run it asks you to give the root password because it puts the new file into /etc/pacman.d/.

    Cheers!

  • works seems a smart tool ;)

    #Server = http://antergos-mirror-a.alpix.eu/$repo/$arch                                    # downloadtime:    -1 status: server_issued_error                      Germany
    Server = http://mirrors.ustc.edu.cn/antergos/$repo/$arch                                  # downloadtime: 21.71 status: ok                                       China
    #Server = http://sourceforge.net/projects/antergos/files/mirror/$repo/$arch                # downloadtime:  2.73 status: outdated                                 Automated_Mirrors
    

    Would be nice to get the option to watch the new mirrorlist before saving it …

    [updates once a week] = [90% less problems]
    http://gofccyourself.com
    my-blog#k
    how to add system logs
    i3 GNOME

  • @joekamprad
    That’s a good idea, I’ll work on it.
    Cheers!

  • @manuel this is like it looks inside fetchmirrors:

    Would you like to view new mirrorlist? [Y/n]: y
    
    # Server list generated by rankmirrors on 2017-09-19
    # Germany
    Server = http://mirror.metalgamer.eu/archlinux/$repo/os/$arch
    Server = http://mirror.f4st.host/archlinux/$repo/os/$arch
    Server = https://mirror.f4st.host/archlinux/$repo/os/$arch
    Server = http://archlinux.mirror.iphh.net/$repo/os/$arch
    Server = http://ftp.gwdg.de/pub/linux/archlinux/$repo/os/$arch
    Server = http://mirror.fra10.de.leaseweb.net/archlinux/$repo/os/$arch
    
    Would you like to install the new mirrorlist backing up existing? [Y/n]: y
    New mirrorlist installed - Old mirrorlist backed up to /etc/pacman.d/mirrorlist.bak
    
    

    [updates once a week] = [90% less problems]
    http://gofccyourself.com
    my-blog#k
    how to add system logs
    i3 GNOME

  • @manuel said in Ranking Antergos mirrors:

    Now there’s a new version out! Check https://pastebin.com/Xipe1ETk

    Checking it out right now! It soounds amazing:grinning:.

    Keep trying, never give up. In the end, you will find that it was all worth it

    My ISO:
    https://rebornos.wordpress.com
    Linux Basics: https://linuxbasicssite.wordpress.com

  • @joekamprad and others!

    Ok, here’s the change Joe requested. You now have an option (–editor=X) to give your favourite editor in order to see the new generated mirrorlist before installing it. Also, the EDITOR environment variable is supported if you don’t use option --editor.

    And the old mirrorlist is backed up to /etc/pacman.d/antergos-mirrorlist.bak.

    Download the beast from:
    https://pastebin.com/R7h1WPc3

    Cheers!

    EDIT:
    you may use the --editor option like this:

     --editor="emacs -fw"
    

    so that editor options can also be given.

  • @manuel awesome!
    I will give it a try later!

    [updates once a week] = [90% less problems]
    http://gofccyourself.com
    my-blog#k
    how to add system logs
    i3 GNOME

  • @manuel said in Ranking Antergos mirrors:

    I would like to have this onto Antergos servers:envelope_with_arrow:

    #Server = http://antergos-mirror-a.alpix.eu/$repo/$arch # downloadtime: -1 status: server_issued_error Germany

    [updates once a week] = [90% less problems]
    http://gofccyourself.com
    my-blog#k
    how to add system logs
    i3 GNOME

ranking4 mirrors9 Posts 28Views 988
Log in to reply