---

tags: PythonGIS
title: Python in Puhti

---

# Lesson 7: Python in Puhti

* [Slides about Python on Puhti supercomputer](https://a3s.fi/gis-courses/pythongis_2022/PythonGIS_in_Puhti.pdf)
* [Exercise materials](https://github.com/csc-training/geocomputing/tree/master/python/puhti)

## Links
* [Puhti documentation](https://docs.csc.fi/computing/running/getting-started/)
* [Puhti web interface](https://puhti.csc.fi/)
* [cPouta documentation](https://docs.csc.fi/cloud/pouta/pouta-what-is/)
* [CSC Notebooks for courses](https://notebooks-beta.rahtiapp.fi)
* [CSC Computing Environment course](https://ssl.eventilla.com/enveff2022)
* [Geoscience related tools on Puhti](https://docs.csc.fi/apps/#geosciences)
* Most geopsatial python packages are available in [geoconda module](https://docs.csc.fi/apps/geoconda/)
* Questions? Missing software? Ask [e-mail]

## Editing files in Puhti
If you need to edit a file in Puhti there are four main options:

1. Open a graphical tool in Puhti. For example
* [Visual Studio code](https://docs.csc.fi/computing/webinterface/vscode/)
* [Jupyter](https://docs.csc.fi/computing/webinterface/jupyter/)
* [Spyder](https://docs.csc.fi/apps/geoconda) with for example geoconda module
3. Edit files in Puhti with commandline tools: nano, vim etc.
4. Edit Puhti files files from your local PC with a tool that supports that, see [Developing scripts remotely](https://docs.csc.fi/support/tutorials/remote-dev/)
5. Edit files on your local PC and copy them to Puhti.

## Questions

* Do we have LiDAR packages in python? for example 3d point clouds to convert elvation model or canopy height model?
* what tool would you use on your own computer for this?
* currently only arcMap and ArcGIS pro 3D analysis. Also "lidR" R package.
* lidR is available in Puhti
* From Python we have at the moment: laspy, laxpy, pdal, likely more in future. Which you would need?
* We had point cloud workshop some weeks ago, some new packages were suggested there: https://hackmd.io/vAsn2q-JSTiBszEdXbfTeg
* Some tools that can work with LiDAR data that we have on Puhti
* [PDAL](https://pdal.io/) /Python-pdal
* [laspy](https://pythonhosted.org/laspy/tut_part_1.html)
* [QGIS](https://docs.csc.fi/apps/qgis/)
* [PCL](https://docs.csc.fi/apps/pcl/)
* [Whiteboxtools](https://docs.csc.fi/apps/whiteboxtools/)
* [lastools](https://docs.csc.fi/apps/lastools/)
* [PCL/PCLpy](https://docs.csc.fi/apps/pcl/)
* we are also looking into getting CloudCompare to work on Puhti, would that be of interest to anyone?
* Something missing? -> [e-mail] :)
* Thank you

* Do we have any packages in python for digital aerial photogrammetry to make 3D point clouds. Soemthing similar that pix4D does?
* OpenDroneMap does at last partly similar processes as pix4D, but it is not Python.

* Is OpenDroneMap is open access?
* OpenDroneMap is open source, so no license needed. https://docs.csc.fi/apps/opendronemap/
* GeoPortti site has also instructions for installing OpenDroneMap web (has graphical interface) to cPouta: https://www.geoportti.fi/tools/instruments/ (at the end of page)
* UEF has plans in GeoPortti project to improve usability of OpenDroneMap in CSC computing environment, so if interested in this (for example as test-user) please send us e-mail.

* Is it better, timewise, to use data directly from Allas, or transfer it to Puhti?
* Accessing data from Allas directly is a bit slower than accessing data on Puhti, but in general processing time it should not matter much.

* So, what is the difference in running the code using the .sh file and the arrow? Is the arrow running it in the login node?
* Arrow in VScode: runs the script within the interactive session that we started from the web interface
* running the sh file from terminal with `sbatch filename.sh` submits a job to the computing node from the login node (-> login node shell)

> make sure to run the `sbatch ...` command from the login shell and not within VSCode terminal

> multiprocessing can be really powerful, highly suggest to check and try it out :)

----