wiki.allensmith.net

Personal "Rough Notes & Useful Links

User Tools

Site Tools


kb:linux:squiddebproxy

12.04 LTS Linux: Squid-Deb-Proxy Server & Client

Original Source: Best way to cache apt downloads on a LAN? … the answer by Jorge Castro

The below text from the original answer has been lightly edited only to make these instructions specific to 12.04LTS based Linux, and to update how the structure of the configuration files has changed. Also– as an example– my own list of addtional .deb sources has been added to this.

Why squid-deb-proxy?

  • No editing of files on the client side.
  • Use zeroconf so that clients were “zero config”
  • Use an existing, solid proxy solution instead of writing a new tool.
  • Easy to set up for a typical Linux administrator.

Server Config

On the machine you want to act as a server install the tool with:

sudo apt-get install squid-deb-proxy

Now start the service bits:

sudo start squid-deb-proxy

This will install the proxy server (which listens to port 8000 by default) and the tools needed for the server to advertise itself on your network via zeroconf.

Client Config

On each of the computers that you want to use the cache (the clients, and the server itself so it can use the cache too), you need to install the client side tool that let's apt look for the server automatically, have them click here:

or via command line:

sudo apt-get install squid-deb-proxy-client

Optional: For maximum efficiency you should set one machine to automatically download updates, so that when your other machines need it it's already in the cache.

Caching 3rd Party Sources

By default the cache is set up to only cache official Ubuntu repositories. To add more you need to add them to the list of sources at:
/etc/squid-deb-proxy/mirror-dstdomain.acl.d/10-default
This is where you can add ppa.launchpad.net, or other services you might use. After making changes to this file, you must run sudo restart squid-deb-proxy in order for the changes to be effective.

My own file of additional sources is as follows:

10-default
# /etc/squid-deb-proxy/mirror-dstdomain.acl.d/10-default
#
# network destinations that are allowed by this cache
 
# linux distro archives
.linuxmint.com
uberstudent.net
archive.ubuntustudio.org
repo.linuxliteos.com
packages.medibuntu.org
# Medibuntu repository no longer exits ... permit Internet DNS error
 
# launchpad personal package archives
ppa.launchpad.net
 
# system & development package archives
.mate-desktop.org
download.bitdefender.com
download.virtualbox.org
dlc-cdn.sun.com
download.opensuse.org
nginx.org
dl.hhvm.com
 
# additional mirror domains
mega.nz
mega.co.nz
linux.dropbox.com
deb.torproject.org
dl.google.com
deb.opera.com
 
repo.steampowered.com
archive.getdeb.net
mirrors.dotsrc.org

Note: If a source starts with a dot (.), then all sub-domains for that source are included. If a source does not start with a dot, then only the sub-domain listed is included in the .deb sources.

Confirming it Works

First tail the log on the server so you can look at it:

sudo tail -F /var/log/squid-deb-proxy/access.log

Then run an update on any machine that has the client installed; the log should start to scroll with entries like this:

1307310795.647     32 192.168.1.106 TCP_MISS/302 768 GET http://us.archive.ubuntu.com/ubuntu/dists/natty-proposed/universe/i18n/Translation-en.xz - DIRECT/141.210.26.10 text/html
1307310795.683     34 192.168.1.106 TCP_MISS/302 752 GET http://us.archive.ubuntu.com/ubuntu/dists/natty/main/i18n/Translation-en_US.lzma - DIRECT/141.210.26.10 text/html
1307310795.716     32 192.168.1.106 TCP_MISS/302 746 GET http://us.archive.ubuntu.com/ubuntu/dists/natty/main/i18n/Translation-en.lzma - DIRECT/141.210.26.10 text/html
1307310795.750     32 192.168.1.106 TCP_MISS/302 764 GET http://us.archive.ubuntu.com/ubuntu/dists/natty/multiverse/i18n/Translation-en_US.lzma - DIRECT/141.210.26.10 text/html
1307310795.784     32 192.168.1.106 TCP_MISS/302 758 GET http://us.archive.ubuntu.com/ubuntu/dists/natty/multiverse/i18n/Translation-en.lzma - DIRECT/141.210.26.10 text/html
1307310795.817     32 192.168.1.106 TCP_MISS/404 657 GET http://us.archive.ubuntu.com/dists/natty-proposed/multiverse/i18n/Translation-en_US.xz - DIRECT/141.210.26.10 text/html

Which means the clients see the cache but are missing it, which is expected since it hasn't cached anything yet. Each subsequent run should show up as TCP_HIT.

You can find the squid cache files themselves in: /var/cache/squid-deb-proxy

Using it

From then on all the machines on your network will check the cache before hitting the outside network to fetch packages. If there are new packages available then the first machine will download it from the net, after that subsequent requests for that package will come from the server to the clients.

TODO
We still need to enable apt to just use an advertised cache on the network out of the box and by default so you don't need to install the client piece. We also need to fix the bug that 403's deb's not in the mirror list.

Original Source: Best way to cache apt downloads on a LAN? … the answer by Jorge Castro

kb/linux/squiddebproxy.txt · Last modified: 2016/01/13 20:35 (external edit)