cache_ef­fec­tive_user proxy
cache_ef­fec­tive_group proxy

http_port 3128
http_ac­cess al­low all

acl lo­cal_ips dst 127.0.0.0/8 192.168.0.0/24 10.10.0.0/16
acl lo­cal_servers dst­do­main lo­cal­host my.lo­cal.do­main
nev­er_di­rect al­low all
al­ways_di­rect al­low lo­cal_ips
al­ways_di­rect al­low lo­cal_servers

acl http-on­ly port 80
cache_peer first­Proxy.site.com par­ent 3128 3190 no-query proxy-on­ly login=User­name:Pass­word name=prx1
cache_peer sec­ond­Proxy.site.com par­ent 3128 3190 no-query proxy-on­ly login=User­name:Pass­word de­fault name=prx2

# This makes on­ly HTTP load bal­anced
cache_peer_ac­cess prx1 deny all
cache_peer_ac­cess prx1 al­low http-on­ly
cache_peer_ac­cess prx2 al­low all

# Re­in­force­ment of di­rect rules
cache_peer_ac­cess prx1 deny lo­cal_ips
cache_peer_ac­cess prx1 deny lo­cal_servers
cache_peer_ac­cess prx2 deny lo­cal_ips
cache_peer_ac­cess prx2 deny lo­cal_servers

# Some time­outs
con­nect_time­out 8 sec­onds
peer_con­nect_time­out 3 sec­onds

hier­ar­chy_sto­plist cgi-bin ?

cache_mem 64 MB
max­i­mum_ob­ject_size_in_mem­o­ry 64 KB

cache_re­place­ment_pol­i­cy heap GDSF
cache_dir aufs /var/spool/squid3 6000 16 256  # Make sure to check this path
max­i­mum_ob­ject_size 16384 KB

ac­cess_log /var/log/squid3/ac­cess.log squid  # Make sure to check this path

shut­down_life­time 3 sec­onds

de­bug_op­tions ALL,1

One of the prob­lems with us­ing hy­brid Win­dows and Lin­ux en­vi­ron­ments is that one needs to watch close­ly for filesys­tem and file anoma­lies and in­con­sis­ten­cies. Dif­fer­ing end-of-line mark­ers, for ex­am­ple, cause many prob­lems when shar­ing files be­tween the two op­er­at­ing sys­tems. One par­tic­u­lar prob­lem I’ve run in­to is that of hav­ing du­pli­cate files, or in other words, mul­ti­ple files with the same file­name. This can hap­pen if, say, you copy a di­rec­to­ry some­where in Win­dows, then switch to Lin­ux and use a tool such as rsync to copy that same di­rec­to­ry over again. If the cap­i­tal­iza­tion is dif­fer­ent, Lin­ux will not re­place the old files, be­cause Lin­ux, un­like Win­dows, is case-sen­si­tive. This will even hap­pen, and is tech­ni­cal­ly ac­cept­able, on NTFS filesys­tems.

The so­lu­tion I’ve come up with is this sim­ple script, called Re­moveDu­pli­cates.py. Ob­vi­ous­ly, you need Python in­stalled to run it, but it has no ad­di­tion­al de­pen­den­cies. Sim­ply run it in the di­rec­to­ry you wish to clean, and it should do the rest. Note that you shouldn’t use this for en­tire filesys­tems (yet), be­cause it will use ridicu­lous amounts of mem­o­ry if it is given a high num­ber of files. Down­load it here!

P.S. Al­so, I can­not guar­an­tee that this tool will work as in­tend­ed or will be bug-free. Use wise­ly.

So with that said, our con­crete goal for the mo­ment is to be­come rel­a­tive­ly flu­ent in HTML and CSS. We won’t learn too much, just enough to un­der­stand how it all works and how to make it work for us. The­se two lan­guages will en­able us to cre­ate de­cent-look­ing web pages and will al­so provide the pre­sen­ta­tion lay­er of our later web de­vel­op­ment work.

Stay tuned for more in­for­ma­tion. I find this all rather ex­cit­ing, and I hope you guys do too!

Up­date: use the new ver­sion of this con­fig file - it works bet­ter.

Here’s why this so­lu­tion is op­ti­mal, or at least bet­ter than other cur­rent­ly-avail­able so­lu­tions. Con­sid­er one method of forc­ing pro­grams to use the proxy (un­der Lin­ux), which is set­ting the http_proxy and ftp_proxy en­vi­ron­ment vari­ables. Pro­grams that sup­port proxy servers and that sup­port proxy au­then­ti­ca­tion (be­cause my school us­es an au­then­ti­cat­ing proxy), no­tably apt-get, wget, and lynx, will use those en­vi­ron­ment vari­ables to con­nect through the proxy server. Pro­grams that don’t, or those that don’t read the en­vi­ron­ment vari­ables (biggest of­fend­ers: most GNOME pro­grams), won’t.

The biggest prob­lem with this method is its in­con­sis­ten­cy - it’s im­pos­si­ble to know nec­es­sar­i­ly which pro­grams work, and which pro­grams sup­port proxy au­then­ti­ca­tion; and in ad­di­tion, one needs to re-en­ter one’s proxy set­tings in the en­vi­ron­ment vari­ables (bashrc, prob­a­bly), in the GNOME set­tings, and prob­a­bly in­di­vid­u­al­ly for some pro­grams too. On top of that, many GNOME pro­grams don’t (or didn’t) sup­port proxy au­then­ti­ca­tion… fi­nal­ly, it is im­pos­si­ble in the en­vi­ron­ment vari­able to spec­i­fy which con­nec­tions (like those to the lo­cal net­work) should be di­rect, and which should be through the proxy.

Con­sid­er a slight­ly bet­ter method, which is proxychains. In order to use prox­y­chains, one must type com­mands like so:

# In­stead of writ­ing
su­do apt-get in­stall ubun­tu-desk­top
# One must write
su­do prox­y­chains apt-get in­stall ubun­tu-desk­top

Okay, so that’s not too bad, if a lit­tle bit in­con­ve­nient. The good thing about this method is that prox­y­chains can “prox­i­fy” pro­grams that don’t sup­port proxy servers na­tive­ly. The proxy au­then­ti­ca­tion user­name and pass­word are al­so stored in one place on­ly: the prox­y­chains con­fig­u­ra­tion file. The on­ly two prob­lems with this method? 1. Typ­ing prox­y­chains be­fore ev­ery com­mand, and 2. The in­abil­i­ty of prox­y­chains (at least the most re­cent ver­sion) to make some con­nec­tions di­rect (i.e. those on the lo­cal net­work) and some to go through the proxy, just like the pre­vi­ous method.

Fine, so those meth­ods aren’t ide­al. What makes the Squid 3 method bet­ter? Well, on the sur­face it solves most, if not all, the prob­lems that the pre­vi­ous meth­ods had. It doesn’t re­quire au­then­ti­ca­tion (that is han­dled trans­par­ent­ly by the per­son­al proxy) and au­then­ti­ca­tion in­for­ma­tion is stored in one place on­ly (the squid.conf). This alone makes many pro­grams work much bet­ter. You can cache far more per­son­al­ized web data (the school’s proxy serv­ing hun­dreds of stu­dents prob­a­bly won’t cache data that you, per­son­al­ly fre­quent­ly use), and sav­ing proxy in­for­ma­tion in many dif­fer­ent places is okay, be­cause if your user­name and pass­word change, you don’t need to change it in all those dif­fer­ent places. Fi­nal­ly, one can still use prox­y­chains to prox­i­fy mis­be­hav­ing pro­grams, be­cause Squid can be con­fig­ured to con­nect to some ad­dress­es di­rect­ly in­stead of prox­y­ing through the par­ent proxy. Its biggest prob­lem is that run­ning Squid more or less re­quires Lin­ux.

Okay, enough ban­ter. Let’s learn how to do this thing. I’m us­ing Ubun­tu 8.04 server (on a vir­tu­al ma­chine), so the­se in­struc­tions may or may not be Ubun­tu-speci­fic. Here’s my con­fig file:

cache_ef­fec­tive_user proxy # Ubun­tu-speci­fic?
cache_ef­fec­tive_group proxy # Ubun­tu-speci­fic?

http_port 3128
http_ac­cess al­low all

cache_peer prx1 par­ent 3128 3190 no-query login=user­name:pass
cache_peer prx2 par­ent 3128 3190 no-query login=user­name:pass
cache_peer_ac­cess prx1 al­low all
cache_peer_ac­cess prx2 al­low all

hier­ar­chy_sto­plist cgi-bin ?

cache_mem 64 MB # How much mem­o­ry Squid us­es for cache.
                # Make low­er if you have less mem­o­ry
max­i­mum_ob­ject_size_in_mem­o­ry 64 KB # Make low­er if you have less mem­o­ry

cache_re­place­ment_pol­i­cy heap LFU­DA
cache_dir aufs /var/spool/squid3 6000 16 256
max­i­mum_ob­ject_size 16384 KB

ac­cess_log /var/log/squid3/ac­cess.log squid

shut­down_life­time 1 sec­ond

acl lo­cal-servers dst 127.0.0.1 192.168.1.0/24
nev­er_di­rect deny lo­cal-servers
nev­er_di­rect al­low all

There are two “par­ent” prox­ies in this file which this per­son­al proxy can ac­cess. Ob­vi­ous­ly, re­place user­name and pass­word with your own au­then­ti­ca­tion in­for­ma­tion. Al­so, don’t for­get to change cache_ef­fec­tive_user and cache_ef­fec­tive_group to your lik­ing or your dis­tro. Change the cache_re­place­ment_pol­i­cy to fit your caching needs, and fi­nal­ly, make sure to ed­it the acl lo­cal-servers to spec­i­fy which servers you do not want to proxy.

After do­ing this, you should be able to restart Squid and have ev­ery­thing work­ing! Next up: trans­par­ent prox­y­ing with ipt­a­bles.

And like ev­ery­thing else that Google puts out, it is a glo­ri­ous­ly in­cred­i­ble piece of work. It has the best char­ac­ter­is­tics a piece of soft­ware could have: it’s sim­ple, fast, re­spon­sive, and stable. I’ve been run­ning it for just a lit­tle while now, and while We­bKit’s ren­der­ing (espe­cial­ly of fonts) is slight­ly dif­fer­ent from Gecko’s, ev­ery­thing works just about the same as in Fire­fox… it’s all just way, way faster. Open­ing new tabs is faster. Us­ing Gmail is faster (thanks, no doubt, to the new V8 JavaScript engine). Writ­ing posts is faster. It’s re­al­ly quite in­cred­i­ble… in fact, pos­si­bly one of the most im­pres­sive open-source de­vel­op­ments to date, right up there with Fire­fox 1 and the Lin­ux ker­nel. Give it a spin your­self and see.