# Urban Analytics standup - 17/01/2024
* [name=Andy] managed to extract data from the Belgium census catalogue (have impolite things to say about semantic web as a result). Also had a bit of a deep dive into the code and issues for clim-recal.
* [name=Barbara] Got access to baskerville :tada:; worked on extended abstract for submission to GISRUK2024. Got some first results on whether image embeddings improve current air pollution and house price prediction models (Newcastle for now, more to come).
* [name=Sam] Developing the guidelines template and guide (currently under [review](https://github.com/alan-turing-institute/uatk-admin/pull/86), thanks Andy aynd Stuart for comments), fixing some repro issues in R pipeline very close to review now, assisting Anna with GPU issues, other project admin. Next week: REG tech talk prep, merge R repro work, SPC toolkit, look again to summarise QUANT and QUANT_RAMP code scoping
* [name=Bowen] Trying to apply the fitting algorithm in the demo of link SPC-data with clime-recal data to the whole UK, currently waiting for more resources of the virtual machine. Processing the contract and visa issues with HR and Stuart/Kelin.
* [name=Martin] All the known bugs and issues in the demoland pipeline are now fixed. Some testing is ahead and then we can properly document everything and call it done.
* [name= Hadrien] Improved SPC validation paper. Started looking into a multi-modal transport model with Hussein.
* [name= Hussein] Writing mini-proposal for work on creating a demand model + writing 1st phd paper
* [name=AnnaZ] Finally remote machine (in Azure) fully working!! Eternal gratitude to @sgreenbury. Some knitting projects while waiting for progress. Spoken with HOT people and ready to prepare consistent dataset, fun mapping ahead! :world_map:
* [name=Stuart] Presentation to LOTI data practitioners with Jen (some potential good follow ups for demoland and LTN tool). Exploring how to use LangChain to develop LLM agents that can interact with data. Discussions about the platform presentation to REG with Sam and Andy. Feedback on the new scoping issue for Sam.
* [name=Jen] The LOTI folks had some interesting questions on SPC and Demoland (the ONS Local folks in particular here). LOTI is also working on this “local sandbox” that could be interesting for us to feed into https://loti.london/blog/local-authority-sandbox/
* [name=Jen] DyME/clim-recal final 3 month [project roadmap](https://hackmd.io/OSp11AVwQ3-yuDIr34e0MA?view#Final-outputs---Jan-to-March-2024) in progress! alongside, the CSIRO workshop in Melbourne is slowly coming together with some more Turing folks getting involved... and also we have £18k (?!) approved for another workshop with UK public health agencies before March. This one is a collaboration with E&S and Health GCs
* [name=Chris] ATE side - finalising preparation of results for sensor site analysis. Short straights - proposed idea for project to DfT who showed interest, will share with wider group at workshop today. Idea summary: to dynamically generate OD matrices from observed flow on roads, and simulate demand using DTA models.
* [name= Dustin] Automating the use of a DfT Excel tool that calculates benefits from a proposed scheme. (Learning some general techniques for working with Excel files, and exploring some approaches to automatically "porting" to another language.) Continuing prototypes of the LTN web app rewrite and a new crossing severance tool.
* [name=Dani] GC contract (finally) signed (#yay), working on proposal for project with ESA, Charles University (Martin), and OECD.

# Urban Analytics standup - 10/01/2024
* [name=Jen] Lots of stakeholder presentations/workshops coming up! Next week a presentation on UATP to the LOTI data practitioners network (data folks from London local authorities) and at the end of Jan a presentation to the TfL Innovation board. In Feb, AI Climate & Health (DyME) workshop with CSIRO and SE Asian public health stakeholders. In March, Climate & Health workshop in the works with Public Health Wales/Scotland and of course the UA AI UK workshop. Already looking forward to the end of March...
* [name=Hadrien] Back to work!
* [name=AnnaZ] Update on the Azure linux machine functioning status: Finally have managed to install Nvidia drivers (thanks @Stuart!) and also have Tensoflow being able to see the GPU (thanks @sgreenbury!) which was not trivial at all. Now at the transfer of the data to the remote machine, and that should be it, I will finally be fully operative.
* [name=Barbara] Copied some data and code so I can (temporarily) use the GPU from my old university affiliation, so that I can scale up my code to create image embeddings from satellite images. Two different kinds, one with h3 masks and one version without. Still hoping to get access to Baskerville though.
* [name=Stuart] Lots of thining about ways that demoland can use LLM. Summarized in a document [here](https://hackmd.io/DEMUlu9mTCOG7xd9hx9Ivw?view) if anyone is interested. Some admin and planning meetings with Jen and other folks. Working with Sam and Andy to find ways to streamline the project boards, have a scoping template for new projects. Next week: make progress on PopGetter apps and put together a revised timeline for the deliverables. Start experimenting with lang chain and other tools for getting LLM agents to interact with external resources.
* [name=Andy] Working with Stuart+Sam on project board processes. A couple of days as leave/sick. Not enough time spent on popgetter :-(
* [name=Chris] ATE - scoping analysis to classify sites and surrounding area of cyclability for report on sensor sites collecting cycling data. Short Straights - Identified several methods for "dynamic flow to OD" method on road networks, which will form input to a DTA model. Planning for meetings with DfT to present ideas. In background developing front-end UI for AT-Access analysis. No blockers.
* [Martin] Fixing up some compatibility issues between different parts of the Demoland pipeline to make sure we can close this task as done.
* [Sam] Working with Stuart and Andy on refining issue/project tracking processes, returning to complete the R repro work on SPC pipeline. Next week: rerun SPC pipeline with parquet outputs and make available, look at running the QUANT_RAMP code, finalise talk schedule and updtae issue template
* [name=Wenhua] Working on modifying single site model in Netlogo, including updating drivers' behavior, adjusting charging rate and add new outputs. Also working on outcome distribution anlysis using pynetlogo for the new model. Next step will start with wider network modelling.
* [name=Dustin] Consolidating scheme data for ATE from multiple funding rounds and dealing with many data cleaning problems. Starting to read about AMAT in preparation for implementing it. Did a one-day exploration of using OSM data from 2011 and 2020 (<https://github.com/acteng/osm_historic>)
* [name=Dani] Not a lot new to report, reviewing Turing Fellow applications, which should come out later in the month. And lots of cool stuff on DemoLand, but mostly listening to what Stuart,Barbara and Martin report.
* [name=Hussein] working on first phd paper document. Will talk with Hadrian to scope out the creation of an activity-based model from SPC

# Urban Analytics standup - 03/01/2024

* [name=Dustin] For ATE, combining GeoJSON data collected about different funding rounds and consolidating scripts for cleaning/combining. Over the holiday, started a few experiments with a web rewrite of the LTN tool, some "edge bundling" and topological junction simplification, and a new tool to analyze crossings and severances/detours.
* [name=Andy] Various bits on admin catchup. More work on Belgium in popgetter - generalising to be able to pull in (almost) any table from Belgium Open Data Stats.
* [name=Hussein] working on skeleton document for my first phd paper. Also getting familiar with the open version of the national travel survey. can it be used to build an activity-based model, or do I need the secure version? I've added some useful notes to [this github issue](https://github.com/Hussein-Mahfouz/drt-potential/issues/19) and I'll pick up the conversation with Hadrian when he's back (also Dustin and Sam depending on availability / interest)
* [name=Barbara] not many updates since last meeting; worked on cleaning up the embeddings code and still waiting to get access to baskerville (by account reset through the baskerville team).

# Urban Analytics standup - 20/12/2023

* [name=Andy] Working on implenmenting popgetter metadata for Belgium. Various handover stuff with Greg, Sam and Stuart.
* [name=Martin] Managed to get a fully working pipeline that given a minimal input information generates all the files that are needed for the deployment of the demoland app. The final bit is to wrap that in a Docker container which is mostly done and will be finished by either today or tomorrow.
* [name=Dani] Trying to get the DemoLand extension over the line, almost there! Hopefully first thing in the new year it'll be signed.
* [name=Stuart] Finally managed to fix pyodide pickling bug for HistGradientBoost models upstream. Lots of meetings getting to know folks on the E&S side of things. Thinking a lot about llms and how they interact with geospatial data for Demoland. Worked with Andy on implementing the new metadata system in popgetter for US with the aim of getting the client apps to use it. Worked with Andy and Sam to think about streamlining the Project board going forward.
* [name=AnnaZ] Have managed to have the environment working on the Linux VM!! Still **TO_DO:** Install Nvidia drivers (no GPU otherwise), and obtain data on the machine.
Christmas party!! :christmas_tree::santa:
* [name=Chris] Building front-end UI for AT-Access Tool in Flask enjoying learning new tool. Considering how to construct OD Demand Matrix from road sensor flow data - challenging task particulary for rare event like extreme congestion, doing reading on topic.
* [name=Barbara] Wrote code for creating two different sets of embeddings for satellite images (one satlas and one IBM/NASA foundation model); now need to scale up to the whole dataset, however, **still** issues with the baskerville login, but hope to get it fixed (getting help from the research computing team). Otherwise, written updates on data preprocessing and milestones added to research project document.
* [name=Wenhua] Worked on updating the netlogo model by incorporating different scenarios of driver's behavior (e.g. stop to charging, stop for a break and for both), next step will work on adding dwell time of drivers and modify the charging rate
* [name=Hussein] Attended a [matsim workshop](https://github.com/casa-arup-city-modelling-lab/matsim-day) and looking at documentation of [pam](https://github.com/arup-group/pam/blob/main/examples/07_travel_survey_to_matsim.ipynb) to see how to build an activity-based model from the NTS. Will be useful to talk to Dustin / Hadrian / Sam on this for SPC
* [name=griff-rees] Change config of server post workshop to minimise cost. Positive feedback from the workshop.

# Urban Analytics standup - 13/12/2023
* [name=Stuart] Found a workaround for bug in scikit-learn that was impacting the use of some models in pyodide for the demoland project. Opening a PR to fix upstream on demoland. Working with Greg, Andy and Sam to workout handover stuff. Implemented a padantic version of the metadata spec for popgetter with Andy. Started thinking about LLMs for demoland and how we might integrate them with the platform.
* [name=AnnaZ] Have managed to have the Azure Linux VM up and running (thanks Stuart!). Prepared the talk slides for today' meeting. **Blocker:** still not managing to install python (conda) environment on the machine.
* [name=Barbara] Wrote code to create dataset of tiles (with masks and uniform masks) from the satellite images. And created embeddings of the tiles (using satlas pretrain), so the next step is to add them to the air pollution and house price models. Also, looking into how to compare embeddings between the datasets.
* [name=Wenhua] Working with sensitivity analysis in Pynetlogo running for one week to see any changes in the outcome, next step start with creating reginal network environment (e.g. locate service stations)
* [name=Chris] Developed analysis function which are efficent for AT access tool, now starting on front end UI - need to learn some new skills! On short straights implemented basic DTA model and now considering how to determine demand from flow data.
* [name=griff-rees] Refactored `clim-recal` to share `RStudio` deploy for workshop on Monday, deployed for workshop and particpated.
* [name=Greg] Handing over things - various meetings and documentation of tasks, responsibilities, etc.
* [name=Sam] Recovering from being unwell, working with Greg, Stuart, Andy on handover
* [name=Andy] Working with Stuart on metadata schema for popgetter. Working with Greg, Stuart and Sam on handover stuff.
* [name=Bowen] Writing the code & doc for link SPC data with clim-recal data. Meeting to plan my work for the new year with Stuart.
* [name=Hussein] Meta-issue for drt analysis pipeline + tweaks to the method

# Urban Analytics standup - 6/12/2023
* [name=Greg] Worked and will be working on handover until my last day at the Turing, especially on REG team next steps. Also preparing for the workshop on the 11th, working with Griff. No obvious blockers at the moment.
* [name=Chris] Further developed functions to measure active travel access and output some example areas for integration with proposed front-end interface. On short straights continuing with background reading on dynamic traffic assignment models and investigating open-source implementations. Arranging access to Baskerville for model development.
* [name=Barbara] Worked on preprocessing satellite image tiles with hexagon masks and without. Setting up code for extracting embeddings. Still issues with access to baskerville.
* [name=griff-rees] Worked through core issues on `docker` `RStudio` server deploy/accounts for `clim-recal` workshop.
* [name=Jon] Custom scenario UI for DemoLand --- a functional freehand draw tool for selecting multiple areas at once now exists! It also looks prettier now. Yay! (Thanks to Dustin for giving me lots of suggestions!)
* [name=Andy] General tidy up of popgetter repo - bringing it in line with scientific python standard, linting etc.
* [name=Martin] Finishing the pipeline to generate all the files needed to deploy a demoland app anywhere in England. It will eventually be wrapped into a Docker image that automatizes most of the steps. We have also spent some quality time on the demoland paper.
* [name=AnnaZ] Have managed to get access to the VM in Azure, not setting it up.
**Blocker:** struggling in creating python environment in the Linux machine.
For the time being, am working on the code, it will anyways be needed to be done once the VM is running.
GREG DO NOT LEAVE US!! :crying_cat_face: :heart:
* [name=Wenhua]Tried different bounds for different input parameters, such as number of chargers and arrival rates to run for one week to see the impact on the outputs (average charging time, average waiting time etc.) and started to create the reginal network modeling. Next step is continue with the nework modeling.
* [name=Mike S] Working on new Neuroaesthetic features for Colouring Cities, in collaboration with researchers in the US. Also finalising the Disaster Management features and other updates to consistency and usability of the interface.
# Urban Analytics standup - 29/11/2023
* [name=Stuart] Moved flat! Worked through the Quant code with Sam, trying to figure out what it does and how we can use it. Notes in this hackpad (https://hackmd.io/m0ZdFAvqS_iiSH_5MzpsAg). Working on next steps for popgetter. Flashing out a proposal for app templates for next weeks meeting.
* [name=AnnaZ] Have met with HOT people and @sgreenbury, got a few very good inputs/feedback from them and going ahead with the metric creation for fAIr. Keeping track of the progress [in this hackMD](https://hackmd.io/3XKMN6zwQiCa9JP0-gcdDw?view). Requested and obtained access to Azure through Turing (trial subscription). Tried to set-up the virtual machine (with the help of REG people) but having issues with it.
* [name= Barbara] Pre-processed satellite images and thought about hexagons and tiles. Created a more concrete research plan and list of various experiments connected to it. Problems with connecting to baskerville GPU, but hope these will be solved soon.
* [name=Chris] Packaged code to generate LTS scores from OSM tags, plan to continue to adapt this e.g., for walkability. Developped AT access functions to improve efficiency and correct spatial unit for intergration with matico. Short straights: started to implement open source traffic simulation model (DTA-Lite fyi).
* [name= Jen] Planning lots of end of year stakeholder related events. If you're interested in getting involved/joining any, lmk!
* 11 December climate projections workshop - Met Office & other climate data users
* 14 December - Greater London Authority (GLA) Data Day
* London Data Week 2024 core team recruiting
* (upcoming) 16 Jan 2024 - potential presentation to LOTI Data Practitioner's Network on UATP
* [name=Wenhua] Worked on sensitivity analysis experiment by varying peak and offpeak mean arrival rate and number of different types of chargers. Next step is to start with creating regional network modelling.
* [name=Hadrien] Validation paper coming together. Linked with other mobility projects.

# Urban Analytics standup - 22/11/2023
* [name=Jon] looking into lasso tools for DemoLand UI. Moved Azure API to Functions, rather than Web App (this costs less as machines are only started when receiving a request). Got Pyiodide custom scenario creation to build on the web :)
* [name=Bowen] Preparing the clim-recal workshop documentation. Discussed with Ruth how to link SPC data with Climate data, then attempted some ideas based on our discussion for link the 'tasmax' data in Glasgow.
* [name=Barbara] First experiments with creating image representations from satellite images with foundation models and pre-processing satellite images (created mask with hexagon of output index); thinking about how a potential paper on this would look like; hopefully soon access to some GPU power for further experiments.
* [name=griff-rees]
* [name=hadrien] Writing up validation article and meeting people (ARUP, Lewis Dijkstra, UCL).
* [name=Wenhua] Last week, Worked on modifying the sigle site netlogo model, adjusting peak and offpeak arrival rate and creating several new metrics (e.g. total energy used, average charging time). Next step to work on behaviorspace environemnt to see the performance in various scenarios, such as vary arrival rate and number of chargers).
* [name=Chris] Last week working on models for walkability and porosity to link in with transport initiatives CSNA York audit. Beginning work on Strava data to develops model to determine edge level cyclability costs. This week starting work with Stuart to develop front-end UI for AT access function in Matico. On short straights continuing background reading on ABM/Simulation.
* [name=Stuart] Continuing to explore new portable serialisation methods for models (for pyodide demoland). Popgetter metadata proposal. Working on fleshing out the Matico proposal for how it could be useful to UA. Met with Chris about potential Matico project.
* [name=Martin] We're discussing the conceptual structure of predictive models for demoland, whether we should or should not try to account for spatial heterogeneity. Technically, both solutions are now ready. Moving onto accessibility.
* [name=AnnaZ] Taken part to Professional development all-day training on Policy engagement (ask me about it). Started process for Azure subscription. Tried fAIr on a few different test sites.
* [name=Sam] Arup meeting, working with Anna for GPU access and to discuss model training, SPC toolkit. This week: begin looking at QUANT code, meeting with fAIr with Anna, SPC PRs, open SPENSER/general synthetic population issue with thoughts/context from recent meetings
* [name=Andy] Pulling together various strands in popgetter, into a (hopefully) conherent set of tasks.

# Urban Analytics standup - 08/11/2023

* [name=Jon] Not actually sure what I've done :upside_down_face: (after git log-ing demoland) More refactoring of DemoLand web app so that indicators are now no longer hard-coded. You can now remove / add indicators in the config (without breaking the website). Had a play with Stuart's pyodide work!
* [name=Stuart] 2 PRs for demoland adjusting the engine and the frontend for pyodide. Planning with Martin and Barbra to undestand how we want to structure the H3 whole country model for popgetter. Working with Jen on the presentation for GLA incubator. Still trying out some optomization of parquet files for popgetter. Next up for me, producing some slides for discussing dagster and popgetter at the team meeting. Fixing a few demoland bugs with the new model.
* [name=Martin] Starting work on nation-wide models for demoland prediction models that involve quite massive interpolation of data from enclosed tessellation used in the Urban Grammar project to H3 grid.
* [name=griff-rees] Merged branches for testing via `pytest`, linting via `pre-commit` and documentation via `quarto`. Aim to combine those for continuous-integration.
* [name=Dustin] Refining scheme sketcher for imminent use by a contractor to map lots of LCWIP schemes. Many changes to od2net for State of the Map -- tutorials, a new generalized route cost function, built-in examples, etc. See <https://od2net.org>
* [name=Barbara] Pre-processing of satellite data; downloading and tiling satellite images into different formats. Model preparation; Work on proposal drafting including validation methods and comparison of different model set ups.
* [name=Chris] ATE: Processing data from CSNA for use in ML experiments to predict LTS on road networks. Short straights: scoping exercise to understand how agent-based modeling and traffic simulation models can be used to characterise congestion events and assist intervention planning.
* [name=Sam] Scoping on QUANT, SPC reproducibility in R,and implemented builder for toolkit
* [name=Wenhua] Working on refining existing model (the single site model), added mage-watt chargers and built a queuing system at the charging station. Next step will work on adjusting the arrival rate of charging stations and also peprape simulation environment for network modelling
* [name=Andy] Got stuck on one of the popgetter tasks, but got it fixed with help from @sgreenbury and @wenhuajiang. Have a couple of things to follow up after the [traffic] Modelling World conf last week.
* [name=Hussein] PhD: Working on matching travel demand to existing bus routes - this [PR](https://github.com/Hussein-Mahfouz/drt-potential/pull/14)
* [name=AnnaZ] Working on builfing a metric to assess [fAIr](https://fair-dev.hotosm.org/) performance. Tracking ideas [here](https://hackmd.io/@annazan/H1PkFnRz6).
* [name=Jen] Great meeting in Australia with the CSIRO partners and now exploring how to connect up the February workshop with the SE Asian partners with some existing work at Turing including DyME, Demoland, etc. Also presentation to the GLA HSDS incubator with Stuart on Thurs!
* [name=griff-rees] `docker` branch of `clim-recal` merged, so continuous integration now covers testing, linting, and documentation building via `quarto` (Dustin: I'm ripping off your config from `uatk-spc` for that, thanks). We're planned for a workshop December 11. Probably targeting `RStudioServer` notebooks for that...

---
# Urban Analytics standup - 01/11/2023
* [name=Wenhua] Modify existing model in netlogo (e.g., add the locations of vehicles are queueing and mega-watt chargers, adjust model assumptions for driver's break). Next step: keep working on updating current model, extract road network and locations of charging sites from OSM for wider network modelling.
* [name=Andy] On leave today. Tomorrow will follow up on consolidating Dagster in popgetter. At (traffic) Modelling World, in Birmingham, with Dustin, tomorrow and Friday.
* [name=griff-rees] Worked through *hopefully* final changes for `clim-recal` branches for [testing](https://github.com/alan-turing-institute/clim-recal/pull/64) and [documentation](https://github.com/alan-turing-institute/clim-recal/pull/72). Also work on the manuscript for an input-output city model paper for Bowan.
* [name=Jon] Setting up UATP website. Fixing DemoLand bugs. No doubt I've introduced some more, thereby ensuring that I have work to do in the future
* [name=Hadrien] Pres at Hocus Focus - AI Fringe drinks reception. Small improvements to the validation method. Scoping for next steps (post-validation), likely will be new transport demand model. Next week: writing up paper about validation.
* [name=Greg] Worked on some clim-recal improvements and planning, created a spreadsheet with information on projects within SoCaR. Reviewed a couple of PRs for clim-recal.
* [name=Barbara] Worked on research proposal for including satellite imagery into the current Demoland prediction for air pollution and house pricing. Research on IBM/NASA foundation model and pretrained networks for image feature creation.
* [name=Dustin] Preparing for a contractor to use the Scheme Sketcher to map lots of LCWIP cycling plans by figuring out requirements and adjusting the tool. Got od2net to run entirely in-browser with WASM for small areas and simplified OD patterns, to let us experiment with route cost parameters interactively.
* [name=Dani] **IMPORTANT** - Lewis Dijkstra visit on Wednesday Nov. 8, free lunch! Ideally, everyone available **at the Office from 11am to 2pm**.
* [name=Stuart] Building out the platform website. Getting indexed filtering for MVTiles (mostly) working in rust. Wokring with Martin to get the new version of demoland_engine working in the browser (+some performance and organization improvements).
* [name=Martin] Refactored `demoland_engine` Python package to avoid the bottleneck caused by unnecessary sampling when no data is changed in a scenario and to move the data away from the wheel. That also allows us to easily plug-in other datasets once there is another deployment of the demoland app on a different local authority or using different set of geometries. The data is now hosted on Github to avoid Azure cost and access issues.
* [name=AnnaZ] Working on improving fAIr: understanding how to frame the next steps so that they are "measurable", good inputs from @sgreenbury and @yongrenjie. On its way: hackMD/other platform for keeping track and sharing ideas/progress with other users.
* [name=Sam] Releases for SPC, SPC toolkit + discussion with Hussein, working with Anna and Jon on fAIr utils, hocus focus with Hadrien. Next week: finalise reproducibility for R preproces scripts, finish initial impl of SPC toolkit, develop options for SPENSER next steps, add scoping query for QUANT
* [name=Bowen] For last week, I was working on the creating and test the docker container for the clim-recal, and I also commited the geodata folder to github. In addition, I also discussed the manuscript about the estios to upload.
* [name=Chris] ATE: formalising data-driven method to determine different user types for route-level cycle/walking access costs, and considering data sources (e.g., Strava) to validate methods. Short Straights: Finalising GNN on road network, and starting to think about agent-based models to support forecasting work.
* [name=Mike] Colouring Cities: Work on interface revamp for the core platform is now complete. Working on adding some extra data into Colouring London, including 1940s aerial images as an extra layer on the maps. Looking at new partners to integrate new data into the platform.
# Urban Analytics standup - 25/10/2023

#### Chair: Greg

#### Roundtable updates (please add your name and a quick update on what you have been working on the last week, and any help you need)

* [name=Wenhua] Summarize the findings of the literature review and extract applicable information for project 3; Specify model assumptions and implement the prototype model with various scenarios at a single site; prepare slides for DfT workshop on Thursday. Next step: start to consider modelling of a charging network and prepare the simulation environment for the network modelling
* [name=Chris] Building prototype functions for route-level active travel cost functions. Next week develop test environment with ground-truth to test function and start writing up more formally. On short straights, building graph neural network for national highways data and prep for workshop on Thursday.
* [name=Martin] Worked on the comparison of the effect of the spatial unit selection on the outcome of the modelling of air quality and house price within the demoland project. We get some indication on the better and worse choices but still some work to be done.
* [name=Stuart] Working with Jen on a presentation for HSDS Incubator. Exploring ways to make reading large parquet files from s3 fast and efficently (for popgetter). Continued to explore methods for running demoland in the browser with pyodide. Trying to figure out how to make PMTiles from rust with Dustin.
* [name=Dustin] Redesigned the ATIP sketch page to simplify user flow. Regenerating route snapper files for a bunch of areas in the UK, since some LAD boundaries changed. Working through some new UI improvements to drawing/editing routes. Preparing for the official start of ATE's contractor to use the sketch tool to map LCWIP plans everywhere. Figuring out what's realistic with od2net before State of the Map.
* [name=Hadrien] Added area queries to the validation method.
* [name=Andy] Mostly distracted by REG matters, but looking to pull together the different treads of work that are happening with popgetter.
* [name=Greg] Reviewed PRs for clim-recal, organised future sessions, created a document that lists all projects under SoCaR, St andrews conference. No blockers at the moment but might contact people in PMs for any help needed.
* [name=Barbara] Days off and applied for GPU baskerville cloud computing. Preperation of research proposal for adding satellite images to existing demoland predictions. Research about IBM/NASA foundation model.
* [name=AnnaZ] Taken part to the conference in St Andrews (that was cool! + enjoyed time in EDB with Dustin + train with @sgreenbury ). After proposal submission for the DSO call, got finally back to the work on the fAIr project and had some REG dedicated time with @sgreenbury and @yongrenjie NB: @anothersmith reach out if you want to be included in these semi-regular meetings.
* [name=Sam] Researching required updates for SPENSER/SPC to use 2021 census data, working with Anna and Jon on fAIr project
* [name=Griff] Apologies all, I'm between train stations with rough signal. Nearing a merge of testing/continuous integration config for `clim-recal` [#64](https://github.com/alan-turing-institute/clim-recal/pull/64) and config of rendering documentation via [`quarto`](https://github.com/alan-turing-institute/clim-recal/tree/documentation-quarto)
* [name=Jon] Since the last meeting I've pretty much just been preparing for / hanging out in Scotland, but recently also been working with Anna on setting up the computer vision project.

# Urban Analytics standup - 10/10/2023

#### Chair: Greg

#### Roundtable updates (please add your name and a quick update on what you have been working on the last week, and any help you need)

* [name=Dustin] Added importing of arbitrary GeoJSON into the ATIP sketch tool, with interactive validations for people to make their data fit our schema. Prototyping simpler UX designs for a "focus" mode in the tool. Moved route naming functionality to the route-snapper NPM package and cleaned up a redundant WASM library (with perf benefits). Renamed the routing engine project to od2net, fleshing out docs, adding support for calculating edge costs + LTS from external scripts.
* [name=AnnaZ] Worked mainly on the grant application, more details at the unconference time, as there is opportunities for the UA analytics programme to be involved. <br/>In parallel had a call with @sgreenbury and @yongrenjie about the structure of the ML modelling for building footprints generation.
* [name=Andy] Agreed on Dagster as the workflow management framework for popgetter and the technology platform. Have started with Stuart to port some of the existing code to Dagster, and explore some of its functionality. Will plan to present basic usage/tutorial for the rest of the team in then comming weeks.
* [name=Bowen] Checking the current code of the Dyme-cal project on the github, start to test the data-loading function with creating a new branch. I am still waiting for arrangement the access to Azure.
* [name=Barbara] Lots of reading and going over urban grammar code. Also had a meeting with Dani and discussions about how to move forward. Including foundation model for satellite images; expanding current research to multiple time points or including SAR images. We'll decide on next steps today.
* [name=Anna FS] Working on my grant application an preparing for a week of teaching in mid-October. Finishing some other papers in meantime.
* [name=Martin] Defining the limits for variables in demoland custom scenario definition; exploring spatial units and their effects on modelling outputs to figure out the optimal core unit to use in the v2 of demoland, that should be unit agnostic but we need to ensure that the models aren't rubbish.
* [name=Jon] DemoLand custom scenarios are done! (Well, it needs to be made prettier; but the functionality is complete, _including_ selecting multiple OAs at once)
* [name=Hadrien] Added outlier areas flagging and formatting inn a readable way for the user.
* [name=Chris] Developing pipeline from OSM to LTS models - prototype complete and integrating with routing engine. Working on accessiblity functions for different user profiles e.g., "confident", "eager by cautious" etc. On short straights working closer with project 1 on ML forecasting on road networks.
* [name= Jen] Meetings with LGA, LOTI, Met Office for various upcoming engagements. For DyME, facilitating a data transfer to our Exeter stakeholders and preparing for the trip to Australia with Gavin/Allaine.
* [name=Stuart] Worked on a proof of concept of running demoland-engine in the browser with pyodide. Set up dagster for popgetter. Contributed US Census DAG for pop getter.

# Urban Analytics standup - 04/10/2023

#### Chair: Greg

#### Roundtable updates (please add your name and a quick update on what you have been working on the last week, and any help you need)
* [name=Jen] Preparing to present the [stakeholder strategy](https://thealanturininstitute-my.sharepoint.com/:p:/r/personal/jding_turing_ac_uk/_layouts/15/doc2.aspx?sourcedoc=%7B20C2281B-9BCB-473B-A50B-DB771EC4D7F3%7D&file=UA%20Stakeholder%20Engagement%20Strategy.pptx&action=edit&mobileredirect=true&DefaultItemOpen=1&ct=1692346923091&wdOrigin=OFFICECOM-WEB.START.EDGEWORTH&cid=8257e96b-6a76-4252-8b93-5f29fec74098&wdPreviousSessionSrc=HarmonyWeb&wdPreviousSession=b16763df-635d-413f-b6bb-b9111a7ab59a) at UA monthly - primary goal is to make sure that everyone's work is reflected in the "state of" map and roadmap for the next 6 months. I'd like to check in with everyone next week to make sure I have captured all the stakeholders they are working with / want to work with.
* [name=Greg] Merged a PR in clim-recal and generated first batch of data. Onboarding for Bowen. Preparation for UATK workshop in St Andrews.
* [name=Chris] Initial prototype of LTS pipeline from OSM to edge level access cost for varying user types. Next week build out accessibility measure and functions, and to include measure of walkability to the pipeline. Planned session with Dustin to intergrate pipeline into routing engine. Continuing work to support short straights where possible.
* [name=Martin] Designed the workflow to generate generic demoland data anywhere in England
* [name=Bowen] Have been working on setting the existing code of clim-recal project run locally, and I will continue to do this and test the code for this week. I am still waiting for the acess of the azure to the VM, once I got it,I will check the data and the process code.
* [name=Dustin] Ported ATIP to svelte-maplibre, first making some upstream fixes with Stuart's help. Pivoting back to scheme sketching improvements for possible short-term use by an ATE contractor to map LCWIP plans. Cleaning up the routing-engines Python examples, trying to get results hosted somewhere nicely, and getting docs to some kind of MVP for use by other people.
* [name=Andy] Tested some patterns for caching in popgetter. Planning for discussion about pipelline management frameworks for this afternoon.
* [name=Sam] Added Scotland for popgetter, final refinements for refactoring [R pipeline](https://github.com/alan-turing-institute/uatk-spc/pull/66), on A/L 2-3/10
* [name=Barbara] accessing data and getting code to run for demoland and urban grammar to understand current workflow; access to hpc/gpu
* [name=Stuart] Have been working on bringing the demoland-engine to the browser using pyodide. Writing up projects for the landing site. Continuing to push improvements to svelte-maplibre upstream. **Help potentially wanted with anyone who has experince sliming down python code for WASM/pyodide**
* [name=Jon] Playing with SvelteKit (because why not) - Don't think it'll be needed for DemoLand yet but can definitely see it being useful if/when it gets bigger. Also general UI improvements to DemoLand custom scenario creation.
* [name=Hadrien] -- Enhancing validation method for the presentation in St Andrews (with new visuals + "unified pipeline")
* [name=Anna] Have decided to give up with trying (one) existing CNN models for computer vision for generating buildings footprints, as replicability is harder than I thought. Next: Will get back to the [working fAIr platform](https://github.com/hotosm/fAIr-utilities) and start a (yet another) model from scratch \[**help wanted**\]. In parallel, have been working on the application for the DSO-Turing call proposal