# cvmfs performance tuning at NP04
###### tags: `DAQ` `protoDUNE` `sysadmin`

## Configure squid proxy

1. Using `np04-web-proxy.cern.ch`, added the following lines to `/etc/squid/squid.conf`;
2. verify squid configuration `squid -k parse`
3. stop squid `systemctl stop squid`
4. increase file descriptor limit `ulimit -n 8192`
5. initialize squid cache disk `squid -z`
6. start squid `systemctl start squid`

coredump_dir /var/spool/squid
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern .               0       20%     4320

collapsed_forwarding on
minimum_expiry_time 0
maximum_object_size 1024 MB

cache_mem 128 MB
maximum_object_size_in_memory 128 KB
# 25 GB disk cache
cache_dir ufs /var/spool/squid 25000 16 256

## Configure cvmfs client

1. Under `/etc/cvmfs/config.d`, add two files named: `dunedaq.opensciencegrid.org.conf` and `dunedaq-development.opensciencegrid.org.conf` with their content as `CVMFS_HTTP_PROXY="http://np04-web-proxy.cern.ch:3128"`
2. verify cvmfs config by `cvmfs_config probe`.

## Test the new squid server with cvmfs

1. Wipe off current cache on the client side with `cvmfs_config wipecache`
2. Setting up the most recent nightly release with `dbt-create.sh -c ` option to clone the python virtual env, which is ~120 MB in size; the process took ~7 minutes;
3. Check the cache on the squid server side, `watch "du -sh /var/spool/squid"`, and watch the squid cache increases over time;
4. Applying the same configuration on another NP04 node, and repeat steps 1 and 2 above, and time the process... **AND it took ~6 seconds...**

==**The take-away is:** instead of using the shared squid server `http://ca-proxy.cern.ch:3128`, possibly used by many experiments, using our own dedicated squid server greatly improved the performance on the client side. This could be the CERN squid server being overloaded (cached files get refreshed more often or too many client requests), or client requests are load-balanced among multiple servers, which gives client requests a higher chance of hitting a server with cache misses.==