Try   HackMD

Lesson 7: Python in Puhti

Editing files in Puhti

If you need to edit a file in Puhti there are four main options:

  1. Open a graphical tool in Puhti. For example
  2. Edit files in Puhti with commandline tools: nano, vim etc.
  3. Edit Puhti files files from your local PC with a tool that supports that, see Developing scripts remotely
  4. Edit files on your local PC and copy them to Puhti.

Questions

  • Do we have LiDAR packages in python? for example 3d point clouds to convert elvation model or canopy height model?

    • what tool would you use on your own computer for this?
      • currently only arcMap and ArcGIS pro 3D analysis. Also "lidR" R package.
      • lidR is available in Puhti
      • From Python we have at the moment: laspy, laxpy, pdal, likely more in future. Which you would need?
      • We had point cloud workshop some weeks ago, some new packages were suggested there: https://hackmd.io/vAsn2q-JSTiBszEdXbfTeg
    • Some tools that can work with LiDAR data that we have on Puhti
    • Something missing? -> servicedesk@csc.fi :)
    • Thank you

  • Do we have any packages in python for digital aerial photogrammetry to make 3D point clouds. Soemthing similar that pix4D does?

    • OpenDroneMap does at last partly similar processes as pix4D, but it is not Python.

  • Is OpenDroneMap is open access?

    • OpenDroneMap is open source, so no license needed. https://docs.csc.fi/apps/opendronemap/
    • GeoPortti site has also instructions for installing OpenDroneMap web (has graphical interface) to cPouta: https://www.geoportti.fi/tools/instruments/ (at the end of page)
    • UEF has plans in GeoPortti project to improve usability of OpenDroneMap in CSC computing environment, so if interested in this (for example as test-user) please send us e-mail.

  • Is it better, timewise, to use data directly from Allas, or transfer it to Puhti?

    • Accessing data from Allas directly is a bit slower than accessing data on Puhti, but in general processing time it should not matter much.

  • So, what is the difference in running the code using the .sh file and the arrow? Is the arrow running it in the login node?

    • Arrow in VScode: runs the script within the interactive session that we started from the web interface
    • running the sh file from terminal with sbatch filename.sh submits a job to the computing node from the login node (-> login node shell)

make sure to run the sbatch ... command from the login shell and not within VSCode terminal

multiprocessing can be really powerful, highly suggest to check and try it out :)

Last changed by 
 
GeospatialCSC
Geospatial team of CSC
0
590

Read more

Spatial data analysis with R

Link to this page: siili.rahtiapp.fi/Rspatial_23

May 9, 2023
EO tutorial comments

Comments and additions to EO tutorial draft EO tutorial draft What about Forestry TEP, Sen2agri etc? Earth Observation guide This guide aims to help researchers to work with Earth Observation (EO) data using CSC's computing resources. The purpose of this guide is to give an overview of available options, so it would be easier to decide if CSC has suitable services for your your EO reseach. It also helps you find the right data and tools for raster data based EO tasks. If you are interested in the fundamentals of EO, please check the resources and further reading section. What are the benefits of using EO data?

Jan 17, 2023
EO tutorial draft

Table of contents TODO: author names (?) STAC paragraph edit data storage paragraph edit Allas usage, more info Learning material links * Geoportti mention

Jan 10, 2023
GeoML

Practical machine learning with spatial data :::info This document: https://hackmd.io/@GeospatialCSC/GeoML Slides Exercise material Computing environment: www.puhti.csc.fi ::: Schedule :::spoiler Monday 7.11.

Nov 1, 2022
Read more from GeospatialCSC
Published on HackMD