# Getting More GPUs for Colab Here's are there ways to acquire more GPUs if you've hit your GPU quota on free Colab. ## 1. Run your Colab on another Google account A hacky solution, but you can share your Colab with another Google account (e.g. a personal google account) and work off that for a new quota. ## 2. Spin up a Google Cloud Platform Virtual Machine We've also gotten GCP credits for everyone! Each student can get $50 of credits; refer to Ed for the redemption link. You can spin up a Colab-configured VM on GCP and have it power your Colab notebook. 1. Claim your Google cloud credits with your `brown.edu` account, but redeem them **on a personal Gmail account**. - If you redeem them on a Brown Google account, you will need to [submit an access request to OIT](https://ithelp.brown.edu/new-ticket) to permit you to create projects on Google Cloud Platform. 2. On Google Cloud Platform on the account you've redeemed the credits `console.cloud.google.com`, create a "New Project" by clicking the button on the top left. Once your project is set up, check that it is billing to your Education credits. ![](https://hackmd.io/_uploads/BJwdNcnW6.png) - If you have never used GCP before, you will need to request for additional GPU quota: - Type `Quotas` in the top search bar to get to the quotas page. - Search for compute.googleapis.com/gpus_all_regions in the filter. - Click the checkbox on "GPU (All Regions)", then click 'Edit Quotas' on the top right of the page. Submit a request for up to 2. - Once you receive an approval email, you can continue the steps in this guide. Approval should be pretty quick for a small request! 3. Go to [Colab on Google Marketplace](https://console.cloud.google.com/marketplace/product/colab-marketplace-image-public/colab) and click "Get Started"/"Launch". 4. Select` Machine Type > GPUs`. 1 x `NVIDIA T4` with a `n1-standard-4` will typically suffice for assignments. There's also a `Zone` dropdown at the top of this config screen. Because GPU allocations are crowded, you may need to try many different zones to get a zone that has a free GPU to allocate. Once done with configs, click "Deploy" and wait/hope for the instance to be successfully deployed (else you'll need to re-do the config with a new region). - Here are [areas with T4 GPUs](https://cloud.google.com/compute/docs/gpus/gpu-regions-zones), and here are [alternative configurations supported by T4 GPUs](https://cloud.google.com/compute/docs/gpus/) ![](https://hackmd.io/_uploads/Sytr6YBZ6.png) 5. Once it is successfully deployed, you'll get a screen with two important fields you'll need to keep: (1) "Instance" and (2) "Instance Zone". You'll also need to click on your project name in the top left to retrieve the third field you'll need: (3) Project ID. ![](https://hackmd.io/_uploads/S12m6FH-a.png) ![](https://hackmd.io/_uploads/BJ8NpYBWT.png) 6. Back on Colab, logged in on the account where you have the GCP project set up, click on the top right "Connect" and select "Connect to a custom GCE VM. ![](https://hackmd.io/_uploads/r1qB6tBWa.png) 7. Key in the fields from step 5 into these fields and Connect! You're done :) ![](https://hackmd.io/_uploads/S1CSptSba.png) 8. ❗️❗️**Remember to stop your VM *AND* its connected SSD storage when you're done using it**, else it will keep charging your credits.❗️❗️ - You can [check and stop your resources here](https://console.cloud.google.com/search;q=colab;searchScope=projects%2Fcs1460-397713;tab=resources?project=cs1460-397713). If you click on the resource name and navigate to the resource's screen, you can manually stop it. ![](https://hackmd.io/_uploads/rk3dpYrba.png) - Just *suspending* your VM is convenient becaues you can resume it easily without deploying it again, and it also stops GPU/CPU fees. But it **will** keep charging you for its attached SSD, which is about $1/day if you are using a 200gb SSD, **so make sure to completely STOP your VM instance eventually.** - If you see your credits depleting unexpectedly, you can go to Cloud Overview > Dashboard and in the "Billing" card on the top-right, click "View Detailed Charges" to see the charge breakdown. ![](https://hackmd.io/_uploads/B1LZoUHGT.png) Other resources: [Google's Guide](https://research.google.com/colaboratory/marketplace.html) <!-- ## 3. Tunnel into Oscar Brown's Center for Computation and Visualization also has a high-performance computing cluster: Ocean State Centre for Advanced Resources, or more lovingly, [Oscar](https://docs.ccv.brown.edu/oscar/). All Brown students can request a free account that allows you to use up to 2 GPUs. How do we make use this with a Colab notebook? At a high level: (i) we can set up a Jupyter server in Oscar, (ii) tunnel to that Jupyter server from your local machine, then (iii) plug our Colab into the local tunnel. Here's how you can use Oscar's GPUs on Colab: 1. Request a Oscar account by submitting [a ticket here](https://brown.co1.qualtrics.com/jfe/form/SV_0GtBE8kWJpmeG4B), under "Exploratory Account without PI" 2. Once approved, log into Oscar via your terminal by following CCV's guide [here](https://docs.ccv.brown.edu/oscar/connecting-to-oscar/ssh). 3. Once on Oscar, create a virtual environment for CS1460. You can do this in your home directory by entering `python3 -m venv ~/cs1460-env` 4. Activate the virtual environment and install some necessary packages for running a local Jupyter server on Colab by entering: ``` source ~/cs1460-env/bin/activate pip3 install notebook==6.5.2 jupyter_server==2.2.1 jupyter_client==8.0.2 jupyter_core==5.2.0 jupyter-events==0.6.3 jupyter_server_terminals==0.4.4 jupyter-http-over-ws==0.0.8 traitlets==5.9.0 urllib3==1.26.6 ``` - If this does not work, check that you are on Python 3.9 and above by entering `python3 --version` - If you are not on Python 3.9 and above: - Enter `deactivate` to deactivate the virtual environment. - Enter `rm -rf cs1460-env` to delete the environment folder. - Enter `module load python/3.9.0` - Check with `python3 --version` that you are now on Python 3.9 and above. - Finally, repeat Step 3 and 4. 5. Create a bash file by entering `vim colab.sh`, then enter insert mode by keying `i`, and paste the following code into the file (adapted from [CCV's guide](https://docs.ccv.brown.edu/oscar/jupyter-notebooks/jupyter-notebooks-on-oscar-1)). Key `Esc` and then enter `:wq` to save and exit the file (i.e., [Vim shortcuts](https://vim.rtorr.com/)). ``` #!/bin/bash #SBATCH -p gpu --gres=gpu:1 #SBATCH --mem=96G #SBATCH --nodes 1 #SBATCH --time 4:00:00 #SBATCH --job-name colab #SBATCH --output colab.txt ## print tunneling instructions to jupyter-log-{jobid}.txt XDG_RUNTIME_DIR="" XDG_RUNTIME_DIR="" ipnport=$(shuf -i8000-9999 -n1) ipnip=$(hostname -i) ## print tunneling instructions to jupyter-log-{jobid}.txt echo -e " Copy/Paste this in your local terminal to ssh tunnel with remote ----------------------------------------------------------------- ssh -N -L $ipnport:$ipnip:$ipnport $USER@ssh.ccv.brown.edu ----------------------------------------------------------------- Then open a browser on your local machine to the following address ------------------------------------------------------------------ http://localhost:$ipnport/?token=cs1460 ------------------------------------------------------------------ " module load python/3.9.0 module load openssl/1.1.1 module load cuda source ~/cs1460-env/bin/activate jupyter serverextension enable --py jupyter_http_over_ws jupyter notebook \ --no-browser \ --port=$ipnport \ --ip=0.0.0.0 \ --NotebookApp.allow_origin='https://colab.research.google.com' \ --NotebookApp.port_retries=0 \ --NotebookApp.token='cs1460' ``` 6. Enter `sbatch colab.sh` to queue the job. Wait until the job is active (which usually should be pretty quick): you can check by entering `myq`. 7. Once the job is active, enter `cat colab.txt`. This should print a ssh command with your login, as well as a localhost URL. - You can also check this file (`cat colab.txt`) for any error messages that print from your job. 9. Open another terminal window and copy/paste the ssh command, and complete Duo authentication if needed. 10. On your Colab notebook, on the top right "Connect" tab, click the dropdown arrow and select "Connect to local runtime". Copy and paste the localhost URL printed into your terminal. Click "Connect". 11. When you're done, you can retrieve your job's job ID by entering `myq`. You can enter `scancel <JOBID>` to cancel the job and free up GPU resources for other users. Congratulations! You're connected to Oscar via ssh tunnelling to use its GPUs :~) Notes: - You may need to remove Google-cloud-server specific imports, like `from google.colab import output`. You may also need to explicitly pip install some imports that would have otherwise come pre-installed in a Google-hosted runtime (e.g. `matplotlib`) - You can run terminal commands through Colab via entering `!<COMMAND>` in a cell. - You can name your environment and/or Notebook app token anything you want, of course. - You can change the time/GPUs requested in the header of your shell script. A free account can request up to 2 GPUs. - Your Colab code will be relative to your Oscar home directory. You can still drag/drop into the Files pane to upload files to your Oscar home directory. For bigger files (e.g, final project), you may want to use your `~/scratch` directory. --> -- Last updated 17 Oct 2023.