Try   HackMD

PyQGIS: expanding QGIS's functionality with Python

Schedule

Tuesday 23.11

09:00 – 10:20 Introduction to Python in QGIS
10:20 - 10:40 Coffee break
10:40 – 12:00 Working with spatial data in PyQGIS

Wednesday 24.11

09:00 – 10:20 Automation with Model Builder and Processing tools
10:20 - 10:40 Coffee break
10:40 – 12:00 A look at PyQGIS plugin development

Course materials

Exercise environment

Exercises are done on your local laptop/PC. Please make sure you have these installed and available before the course starts:

  • QGIS 3.16 LTR, the exercises are tested to work with this version, newer versions should be ok also.
  • Zoom desktop if possible, but Zoom web-version is ok

Practicalities

  • 2 screens helps to keep Zoom, instructions and QGIS in workable sizes.
  • If you have only one screen, consider using a tablet/smartphone as a second screen.
  • Please ask the presenter to increase font size etc, if not readable for you
  • Headphones, for better audio.

During the course

  • If you have questions:
    • Preferably ask with audio
    • If less related to current topic / also later answer ok, ask here in HackMD
  • If you need help during exercises:
    • Ask with audio or chat.
    • Share your screen if asked for troubleshooting.
    • We might also use Zoom Breakout rooms or private chat for troubleshooting.
  • Please mute yourself when not talking but keep your video on during the whole course, if your Internet connection allows
  • During the breaks your are also welcome to have free chat in Zoom

QGIS with Anaconda

Note for the future (NOT needed for the course): If you want to keep QGIS Python separate from everything else on your computer:
QGIS is also available from Conda-forge channel: https://anaconda.org/conda-forge/qgis
-> You can create a separate conda environment with only QGIS:
conda create --name qgisenv
conda activate qgisenv
conda install -c conda-forge qgis
then you can start qgis by typing qgis
You can check python version within QGIS Python Console with

import sys
sys.version_info

And when you are done, you close the environment with conda deactivate
Or you can also use Docker (if you know about it ;) ): https://hub.docker.com/r/qgis/qgis/tags

QGIS on Puhti (CSC supercomputer)

Short QGIS on Puhti demo:
you need CSC user account
Using the new Puhti web interface: https://www.puhti.csc.fi
https://docs.csc.fi/apps/qgis/
There is more GUI applications available: https://docs.csc.fi/computing/webinterface/
Note: Not intended for processing, but only eg viewing files you have processed etc
All geocomputing information: https://research.csc.fi/geocomputing

Questions and Answers

  • Ask a question like this
    • Anyone can answer like this
      • Ask more about it like this

Day 2 Questions and some notes

  • what was the code for help?

    • processing.algorithmHelp("native:simplifygeometries")

  • Where does the output go when rerunning the algorithm?

    • currently, nowhere. We will come back to this.

  • how is ti possible that there's no simpler way to find the usable names to tools than running a for loop through all the tools?

    • you can eg hover over the alorithm in the processing toolbox
    • KE: ID can be found also from GUI -> find tool from processing toolbox (with search) -> Open tool page -> Help -> "Algorithm ID" is under Python code heading.

  • Slightly off-topic: there seems to be a way to use an external script editor, but the button for external editor doesn't do anything (in my QGIS at least). Do you need to change some settings for it to work?

  • where do you ACTUALLY define the field that we drop? i could not find it in the instructions

    • You define the fields when you run the code :)
    • but why not in the model?
      • The model is more general, both input file and Fields are defined at run time.
    • is it possible to define the fields in the model, and not leave it to the runner?
      • Try to leave out the Vector layer and Vector Field, and in define the input layer and fields directly in the algorithm inputs.

  • what was "Field to drop" setting in the Drop Field(s)?

    • leftmost button, change to 'model input'
    • parent layer needs to be defined

https://docs.qgis.org/3.22/en/docs/pyqgis_developer_cookbook/intro.html#using-pyqgis-in-standalone-scripts

  • General question: say you usually process GIS data with python by using e.g. Anaconda. What are the key benefits of adding QGIS in the mix? What can you do with QGIS+python that you can't do with Anaconda?
    • QGIS+python makes sense if you are anyway using QGIS already, also that you can see right away the results in the GUI might be a benefit
    • Python is great for sharing workflows for QGIS with scripts
    • Layout creation in QGIS with Python for reusability
    • Plugin development for QGIS
    • -> seems to be a matter of taste

Day 1 Questions and some notes

  • Combination possibility of PyQGIS and pandas etc?

    • Possible to use pyQGIS outside of QGIS
    • better to choose: either PyQGIS or other Python libraries

  • do we need anakonda for running pyqgis_practical_day1.ipynb? I mean will we use it or QGIS?

    • no, the notebooks are just for reading, not for interacting
    • we will copy code out from the notebook into the Python console in QGIS

  • How can we switch the software from Finnish to English language?

    • Options -> General -> User interface language + Restart
    • In Finnish: Asetukset >Valinnat>Yleiset

Moby Dick is not included for you, so you cannot read it, it is only an example of a very long text

  • can we save these lines as .py file, I mean not in console only. is there any way to run only one line of a code in python script in QGIS? without commenting out the other lines

    • you can start a script editor from top of python console (3rd button from left: show editor)
    • select lines in script editor> click right> run selected lines

  • Can also geopandas be used in pyhton console?

  • How to run a script with Path?

    • you can run any script written for this before from console exec(open('/ath/to/your/file/test.py'.encode('utf-8')).read())

  • What is feat, geom in geospatial analysis in python? I have seen them (words) used quite frequently.

    • I think it is just used as abbreviations for features and geometries in variable names, programmers are lazy
    • sometimes geom can also be the identifier of the geometry column in the attribute table, but this can have many different names
    • TL: feat = feature -> in this case, one object with geometry and attributes (like one polygon with population information)
    • TL: geom = geometry, points or list of points forming a line / polygon. The location of the points is in some coordinate reference system to tie them to real World locations. In this case, the CRS is EPSG:3067

  • paavo_layer = QgsVectorLayer(path=paavo_path, baseName="Paavo", providerLib="ogr") how do i know the providerLib is ogr in this case

    • TL: In this case, we know it's ogr because we define a path to a geopackage. Ogr is the general library for dealing with vector data (similar to GDAL is for raster data)
    • TL: Rather I should say, define a path to a vector layer in a geopackage (since geopackages can also include raster layers)

  • ogr also for shapefiles?

    • yes

Note: When copying multiple lines from the notebook, it will run every line, except for the last line (which can be run by pressing enter)
if the copied lines are a for loop you might have to press enter again (you can see that the console is still expecting input, when it is showing … (three dots instead of the normal >>>) )

Area is transformed to km2

  • I cannot type \ to Python console or script editor in QGIS but it works normally elsewhere. Is there a limitation to this or is it just my QGIS acting weird?

    • interetsing

  • Section 2.3 Creating Vector Layers, there is the big code block: I copy pasted it into the Script file and run selection but I get an error: “simplified_feat_list.append(feat) ^ IndentationError: unexpected indent”

  • Section 2.3: a list with this feature's name and population trimmed_attributes = [feat['name'], feat['pop']] -> What is the meaning of population property?

    • KE: the datasets are described in the beginning of the tutorial. pop is "Population in 2020". For paavo there is a link to Statistics Finland site for all explanations.
    • TL: This refers to the population attribute on the input layer. "Pop" is the name of the column that stores the population information. So, for example for feature that has the "name" Helsinki-Uusimaa, has the population of 1702678
    • In brief: it's a column name

  • If I want to change the tolerance of the simplified layer, do I need to rerun all the scripts and add a new layer?

    • Yes, it's probably the simplest to rerun the script. You can copy it to the editor to make rerunning simple. Also, you can manually remove the simplified layers as you go as to not clutter the screen.

  • I am doing this part: # pass the expression and the field definitions paavo.addExpressionField(student_exp, QgsField("perc_students", QVariant.Double, prec=1)) put I can't see the added layer in the qgis layers visually.

    • This particular expression doesn't define a new layer. Instead, it adds a new field to the paavo-layer. If you check the attribute table and scroll to the right end, you should see the new field.

Challenge X: use perc_students from before! Do not color a map like this with absolute values (they should always be related to area / population in certain area)

  • Hmm, in my QGIS 3.8 it says: NameError: name 'QgsClassificationQuantile' is not defined —> it is under some other name in this version of QGIS?
    • QgsClassificationQuantile came with QGIS 3.10
    • solution: update QGIS :)

Do not write below this line!

Last changed by 
 
GeospatialCSC
Geospatial team of CSC
0
813

Read more

Spatial data analysis with R

Link to this page: siili.rahtiapp.fi/Rspatial_23

May 9, 2023
EO tutorial comments

Comments and additions to EO tutorial draft EO tutorial draft What about Forestry TEP, Sen2agri etc? Earth Observation guide This guide aims to help researchers to work with Earth Observation (EO) data using CSC's computing resources. The purpose of this guide is to give an overview of available options, so it would be easier to decide if CSC has suitable services for your your EO reseach. It also helps you find the right data and tools for raster data based EO tasks. If you are interested in the fundamentals of EO, please check the resources and further reading section. What are the benefits of using EO data?

Jan 17, 2023
EO tutorial draft

Table of contents TODO: author names (?) STAC paragraph edit data storage paragraph edit Allas usage, more info Learning material links * Geoportti mention

Jan 10, 2023
GeoML

Practical machine learning with spatial data :::info This document: https://hackmd.io/@GeospatialCSC/GeoML Slides Exercise material Computing environment: www.puhti.csc.fi ::: Schedule :::spoiler Monday 7.11.

Nov 1, 2022
Read more from GeospatialCSC
Published on HackMD