# processing pipeline - next up

## issues:
* [ ] XML reading memhog
details in branch xml_relator
-> Stubi?
- https://stackoverflow.com/questions/7171140/using-python-iterparse-for-large-xml-files/7171543#7171543
- https://stackoverflow.com/questions/22380990/how-do-i-free-up-the-memory-used-by-an-lxml-etree
- https://pretagteam.com/question/why-is-lxmletreeiterparse-eating-up-all-my-memory

* [ ] design question
* [ ] "Population.strategy"
integration of strategy methods into classes
* [ ] _classmethod:_ handover still function
- need to id ismethod() on register, naming appropriately
+ can write clean, readable code as long as object itself
is returned or whatever is defined
* [ ] _staticmethod:_ handover is function
- id method is crufty, unelegant
* "strategy"
* _normal function:_
still possible as user-provided thing

## goals
### Gerald
* Levitate scenario preps in pipeline
* population exemption for homes in tolling zone
- select population
- manipulate
* network
- select links by frc/osm:highway
- manipulate

### Stubi
* [ ] the XML reading thing for XMLRelator
* [ ] poetry @Windows -> 1.1.12 self update
* [ ] feedback runde pipeline
- my drafts
- IF design

### Jo
* contribute code to load polygons
read_polygon/s
`getModePerArea.py`
class `ModeArea`
* [ ] done how ?
* polygon input: shapefile
* shapely
* fiona: read formats (shapefile)
alternative to geopandas (tough to install [@Johannes])
* [ ] prep for strategy declaration
* [ ] in a new file `formats/polygon.py`

## status

### XML handling
* network
* got old NetworkXML, *working* rather fine
* [ ] shift to XMLRelator ?
* events
the file size problems

## tasks
* network
* population

### Gerald
* [x] merge to main branch
* [x] finish 1st version of XMLRelator
* [x] data only
* [ ] full XML content
* [ ] example
* [ ] population inhabitants within polygon
for now: defined by first homes (assumed to be first activities, which is MATSim requirement), of the selected plan of each person
* [ ] polygon reader
-> Johannes code

## examples:
### for Stubi
1. exp. plans:
* starting before 8:00
proto_code: (XMLRelator steps, **NOT** the final Plans object/class)

  pick_legs = table['leg']['dep_time'] <= '08:00:00'
  parent_ids = get_parents(pick_legs.index)
  picked_plans = table['plan'].loc[parent_ids]

2. link volumes:
* all links with daily volumes
proto_code: (XMLRelator steps, **NOT** the final object/class)

  pick_legs = table['leg']['mode'].isin('car drt1 drt2 ...'.split())
  child_route_ids = get_children(pick_legs.index, type='route')
  all_crossed_linkIds = table['route'].loc[child_route_ids].__text__.str.split()
    all_crossed_linkId_counts =.value_count()