Homemade Scientific Clusters (in Python)

tags: archived open-science python juju charm solvcon pyconapac pyconapac2019

Resources

Demo Code

• Grid Engine Job and MPI application https://github.com/tai271828/demo-PyCon-APAC-2019
• The example cluster source code (the homemade cluster)
  • https://github.com/tai271828/sge-cluster-bundle
  • https://github.com/tai271828/charm-layer-sge-master
  • https://github.com/tai271828/charm-layer-sge-client
  • https://github.com/tai271828/charm-layer-sge-interface

Juju/Charm Ecosystem

• Charm-tools https://github.com/juju/charm-tools
• Charm store (Share your charm here!) https://jujucharms.com/store
• MaaS and it cli tool https://code.launchpad.net/maas
• Instruction to connect to GCE (Google Computing Engine) https://docs.jujucharms.com/2.5/en/clouds-gce
• Tutorial for beginners https://discourse.jujucharms.com/t/video-tutorial-deploying-openstack-on-to-6-bare-metal-servers-with-juju-and-maas/1606

Google Genomics team commandline tool

• https://googlegenomics.readthedocs.io/en/latest/use_cases/setup_gridengine_cluster_on_compute_engine/index.html
• (Deprecated) https://github.com/GoogleCloudPlatform/solutions-google-compute-engine-cluster-for-grid-engine

ElastiCluster

• https://elasticluster.readthedocs.io/en/latest/

Grid Engine

• http://wiki.gridengine.info/wiki/index.php/Main_Page

Beowulf cluster

• https://en.wikipedia.org/wiki/Beowulf_cluster

Singularity

• https://singularity.lbl.gov/

Assets

Some icons used in the slide

• https://www.flaticon.com/

Questions and Feedback

It's my honor to have your questions and feedback here…