OSMaPaaralan

OSM + Map + Paaralan

Motivation

In the Philippines, school facilities are more than just places for learning. They are also used as:

  • shelters or evacuation sites during emergencies;
  • voting precincts during elections
  • de facto social facilities for local communities

Parsing the dataset from DepEd's EBEIS can help us map and identify more places using OpenStreetMap, including sitios (hamlets).

Certain business enterprises are restricted by law from establishing their operations within a certain distance from a school. OSM maps can help Local governments and communities verify whether businesses are following these restrictions or now.

In the context of humanitarian development activities, disaster risk reduction and planning, mapping schools is one way of helping communities and risk managers prepare themselves for emergencies.

Existing hazard maps and school location data can help determine whether such-and-such site or school is appropriate as an evacuation center for a particular hazard.

Workplan

Review and conflate the data set with updated information 2 from the "Masterlist of Public Schools" 5 released by DepEd to the public domain via the open data portal.

PHL Schools in OpenStreetMap

Last update: 2022-01-20 11:00
† - partial, as of last update
TL;DR 99% reviewed, 99% fixed


Cumulative schools in PH since 2012

Year Period Cumulative Diff, period Schools added, CY
2021 47,851 12,824
2020 35,027 14,245
2019 20,782 4,337
2018 16,445 2,241
2017 14,204 1,897
2016 12,307 2,448
2015 9,859 922
2014 8,937 1,503
2013 7,434 1,868
2012 5,566 -
Quarterly summary since 2012
Year Period Cumulative Diff, period Schools added, CY
2021 Q4 47,851 +1,213 12,824
Q3 46,638 +4,875
Q2 41,763 +3,034
Q1 38,729 +3,702
2020 Q4 35,027 +2,842 14,245
Q3 32,185 +4,246
Q2 27,939 +4,466
Q1 23,473 +2,691
2019 Q4 20,782 +2,139 4,337
Q3 18,643 +413
Q2 18,230 +705
Q1 17,525 +1,080
2018 Q4 16,445 +1,570 2,241
Q3 14,875 +523
Q2 14,352 +323
Q1 14,029 -175
2017 Q4 14,204 +286 1,897
Q3 13,918 +369
Q2 13,549 +564
Q1 12,985 +678
2016 12,307 2,448
2015 9,859 922
2014 8,937 1,503
2013 7,434 1,868
2012 5,566 -

Project history

2021 - As of 31st June, 87% of the project tasks has been reviewed, with 5,171 pending tasks.

2020 - In August, the project has reviewed fifty percent of the ~39k active task.

2019 - In September 2019, project is revived to complement the Tabang-AI initiative. A pull-request for the cleaned-up data was made in the repo.

2017 - By January 2017, the data has been taken down, but a copy has been uploaded in a back-up repo.

2016 - DepEd published a dataset with limited attributes in May 2016 4 in the Philippine government's Open Data portal.

2015- Over a year later, I re-discovered the effort, and expressed an interest in moving the project forward. Efforts were focused on getting the (1) formal permission from DepEd to conflate the data into OpenStreetMap; and (2) getting an up-to-date copy of the data.

2013 - During the 2013 State of the Map Philippines (SOTM:PH)0 Miguel Karlo Macariola of the Philippine Department of Education (DepEd) presented their MAPaaralan Project 1. The initial dataset provided by Miguel is available here 2, with the explicit permission letter 3 for it to be "incorporated into the OpenStreetMap project geodatabase and released under a free and open license."

In the Philippine presentation for the 2017 State of the Map Asia conference, it was reported that only 24% of Philppine villages are on the map. That's only 10,406 of the 42,036 official villages, and excluding numerous other populated settlements known as sitios and puroks.


Update logs

2020-08-13 Meeting with Bea Apolinario of DepEd-DRRMS, with Maning, Erwin

  • DRRMS is interested in better quality data
    • As of 2020, there are 48k DepEd schools (vs 39k in OSMaPaaralan tasks)
  • Pitched that current PH school data in OSM is now better (albeit incomplete) compared to what's in the internal DepEd db.
    • short-term: DRRM to coordinate with ICTS, about how DepEd can organize internal efforts to supplement OSMaPaaralan thru mapathons
    • next phase: release additional DepEd data added after 2012
      • OSM-PH cannot make any commitments on behalf of the OSM volunteer community (e.g. deadlines, "responsibilities");
      • any new data releases should follow the same manner as 2013, under Asec. Laguda, without any commitments from all parties, and compatible with ODbL.
      • for better DRRM context, it would be ideal if DRRMS can release addditional attributes, for example:
        • how many rooms? toilets?
        • or pax capacity for evacuees
        • water
          • water source
          • drinkable?
        • electricity
        • officially designated as shelter?

2020-03-03 - Maning introduces initiative to HOT-PH, who agrees to participate and validate the schools in PAM. Feye Andal is HOT-PH PoC.

2020-01-31 - Phone conversation with Macariola. DepEd re-scheduled Division-level IT Officers' meeting from quarterly to bi-annual. Next meeting will be on April, and will get back to co-ordinate a workshop/training during the meeting

2019-12-06 - Meeting with DepEd ICTS director, Abram Abanil, to conduct workshop/mapathon, to supplement mapping efforts and introduce the project to their staff and Division-level IT coordinators.

2019-09-25 - Data laundering completed, the micro-tasking challenge page is now online. Issue tracker is here

Earlier than 2019

2016-06-08 - The dataset from DepEd has been published in http://data.gov.ph/catalogue/dataset/201-enrolment-data-elementary-and-secondary, and as expected, was missing some of the attributes previously made available. Has reached out to Rey Laguda (outgoing department undersecretary) and Miguel Macariola to seek the authorization for the previous data set, or release a new version that included the missing attributes.

2016-05-20 - Latest from Miguel Macariola, they will be publishing their datasets via http://data.gov.ph , but the data fields may not be what we're expecting. I tried to convince him if they could release the same minimum attributes as the spreadsheet they released to Maning in 2013. Crossing fingers.

2016-05-04 - Miguel is back to Information and Communications Technology Service. They've validated the geographic coordinates of 93% of estimated 46,000 schools they have on record. Promised to follow-up the release letter/waiver/data-sharing agreement with her supervisor, Ms. Aida C. Yuvienco Director IV of the DepEd Information and Communications Technology Service (ICTS)

2016-02-01 - talked to Miguel Macariola this afternoon. (1) no updates because their database got fuxored :( ; (2) will talk to concerned official regarding release letter to OSM for present dataset. Also suggested dataset release through data.gov.ph

2015-09-29 - Talked to Miguel Macariola again, he promised to send updated data files. No news about the release letter. Mentioned other means of releasing datasets (i.e. to the public domain, versus Creative Commons, or ODbL)

2015-09-04 - Letter of release hasn't been signed, but not because of any objections from the agency or by the signatory. Miguel promised to follow-up with his boss, again.

Data set has been updated (but not released) by DSWD, and now includes geographic coordinates for 93% of public schools, and 44% of private schools. Miguel promised to release copies of these, too.

2015-08-10 - Called office, Miguel will be out, on field work, and won't be back until 24th Aug

2015-07-30 - No updates from DepEd. Follow-up next week, 3rd August

2015-07-14 - No authorization / data Authorization/Donation letter from DepEd to OSMF

2015-07-08 - 11,000~ public school records (with updates/field verification by DSWD); 5,000~ (out of 16,000) private school records

​​​​ Called with Mr. Miguel Macariola and he promised to send some of the data today;  still waiting to get complete dataset from DSWD.

2015-07-03 - Follow-up on Wed, 8th Jul. Talked to Mr. Miguel Macariola of DepEd, and he agreed to release updates of DepEd datasets. However, he's still waiting for updates to 6,000 public schools coming from DSWD who volunteered to update/verify their geocodes which he promised to send next week.

Of the 16,000 private schools in the country, they have 5,000 records geocoded. This data set will be donated to OSM as well.

These 11,000 records will make it to the second batch (and not part of the 30th Aug deadline. Miguel confirmed that 57,000 schools is incorrect, and mentions 46,000 as an approximate number.

Send letter to deped, and copy of letter of authorization/contribution to OSM to cover these donations.

2015-07-02 - Maning Sambale offered to host instance of MR using the OSMph server. This would be a fallback if the the main MR site declines to host this challenge, for any reason.

Coordinating for updates from DepEd PoC, Mr. Miguel Karlo Macariola but Pasig city is on holiday. Will get in touch again tomorrow morning. Agenda:

Discuss updated dataset from deped.gov.ph (2014Q4 54,518 records vs 46,486 in 2013) and whether the department is interested in continuing their contributions of select attributes (PIIs will be dropped) of the datasets to the geospatial database of OSM.

  • 7,914 duplicates in their 2014q4 data set. :(
  • Highlight the advantages and long-term impact of data release to the commons. A continuation of their interest back in 2013.
  • Prepare draft authorization/waiver.
  • Mention collaboration with USAID/ASFPH

2015-06-30 - Meeting with ASFPH Team : 9th July, final workplan + clean datasets ready for import to task manager

Clarify workplan, RFCs from OSM-PH community re select attributes to be imported. Highlight nfo that geographic coordinates are not to be imported, but only to be used as a reference and temporary place-holders for attribute information to be merged with existing (or new) features . Secondary task is to trace school grounds and buildings (where possible).

​​​​smaller datataset for testing asfph team ASAP

​​​​AidData Fellows will help with marketing the new task

​​​​we will spread the word here and in the US and the Philippines, during mapping workshops and events

​​​​Let's have a 1-5 minute elevator pitch written for this project, including a segway into the chosen platform for the geocoding-for use in any presentation

​​​​A 1 page quick start guide is another deliverable that will make the project more scalable 

​​​​Also, blog post published in the first tranche to raise state side awareness and possibly get geocoders to map schools for an hour or so in Williamsburg. 150 strong!

​​​​Sounds good! I know I promised a smaller dataset for you to practice on, but in the mean time, try out the existing challenges in the MapRoulette.org website to have a good feel for the platform. It's going to be very simiilar, anyway.


​​​​Things I'd like highlight: 

​​​​ading the geospatial attrributes into OSM improves nominatim search abilities. for the country;  

​​​​public schools are multi-functional facilities in PH - they serve as voting precincts during elections;

​​​​evacuation centers during crisis, a community centre for the community;

​​​​improve transparency and empower communities;

​​​​Folding this contribution into OSM will not only help the education department visualize their presence on a map, it empower communities to verify things for themselves. For example, by law, certain business establishments are not allowed a certain distance from schools. Stakeholders can monitor and visualize such things when school data is made available to them. 

​​​​completely the whole task by Aug. 30th!

Link to agenda page, for reference.

2015-06-21 - Five AidData summer fellows in the Philippines expressed interest in collaborating because we have similar project goals of mapping schools in PH. See asfph2015 . Will review my working datasets and report updates to the mailing list soon.

​​​​dear friends, the summer fellows cannot figure out the magical mailing list. #feelingdumb could anyone provide some insight on how to join by chance. With love from a silly fellow, -lu

​​​​Please send 100 USD to my bank account to access the mailing list. :)))


​​​​Great! I thought you would ask for 1000. Thanks for cutting us a sweet deal

​​​​https://lists.openstreetmap.org/listinfo/talk-ph

​​​​money coming soon, please provide you ssn with responding email ;) 

Concerns

​​​​Updated data set is available (check out the original CSV (46,486 records) from maPaaralan  versus this CSV  (46,603 records) from October 2014 , uploaded by Miguel Karlo Macariola)  which also contains more usable attributes. See Data Matrix below.

​​​​Integrated Schools - share the same name but  has two school_ids, one for primary, another for secondary. 

​​​​ same issue for schools offering night classes. May confuse other mappers.

​​​​Integrated schools in the same location may be tagged as one instead of two:

​​​​amenity=school

​​​​name=Somename Integrated School

​​​​isced:level=0-3

​​​​ref=123456;54312

​​​​comment=*



​​​​To Do

​​​​Ask Maning for updates. Sychronize his changes to my working version.

​​​​Contact Macariolia and inquire whether updated data, with additional attributes from EBEIS can be provided. Send Macariola copy of the tidy data after "data laundering" tasks are complete

​​​​data laundering : Convert charset to UTF-8. Spell out abbreviations (PS, ES, MHS, Sto., Pob., etc.), where appropriate (exceptions: Sr., Jr.), standardized name cases.


​​​​ Flag records that fail validation of:

​​​​Geographic coordinates 

​​​​Attribute validation - address (city, municipality) don't match geographic coordinates 

​​​​Missing coordinates

​​​​How useful is it for micro-tasking to use barangay centroid coordinates for records without GCs? It appears to me that centroid GCs will help mappers determine an approximate area and use the imagery to search for features that look like schools


​​​​Data Matrix

Column 1 contains proposed tags and corresponding columns from various data sources

​​​​Unwieldy table support in Hackpad . Check this out instead: https://docs.google.com/spreadsheets/d/1M41VkJLxuJdd1BXYrMe7L7eQvYPWDwHYu_rlHIL5uDk/edit?usp=sharing

Discuss following tags:

​​​​Capacity

​​​​How many students can the school accommodate?  May help determine the potential number of evacuees it can accommodate.


​​​​Rooms

​​​​Propose to adopt this although it is currently used just for hotels. Number of classrooms may help determine the potential number of evacuees a school can accommodate.


​​​​ISCED levels

​​​​isced:level is a proposed tag with over 16,000 occurrences in OSM. 

isced:level PH Educational System OSM tag (current)
0 Pre-school K amenity=kindergarten
1 Elementary 1-6 amenity=school
2 Junior High School 7-10 amenity=school
3 Senior High School 11-12 amenity=school
4 Technical-Vocational / Short Course Programs 2-year courses, vocational-technical institutes (non-university) leading to an associate's degree
5 First stage of tertiary education leading to a bachelor's Degree amenity=college
amenity=university
6 Second state of tertiary education leading to a master's or doctorate's degree amenity=university

​​​​See http://wiki.openstreetmap.org/wiki/Proposed_features/ISCED#Level


​​​​The alternative is to follow US convention, i.e. :

​​​​school=secondary

​​​​which isn't as widely used as the above tag (just 2,500 times).


​​​​Branch

​​​​The data set often show entity name like:

​​​​Subangdaku Technical-Vocational High School – Mandaue City Central National High School Extension – (Subangdaku National High School)


​​​​To minimize unnecessarily  complex names, proposed to tag entity as follows:

​​​​name=Subangdaku Technical-Vocational High School

​​​​branch=Mandaue City Central National High School

​​​​old_name=Subangdaku National High School 

​​​​ This tag could also address the issue above regarding Night schools


​​​​See http://wiki.openstreetmap.org/wiki/Key:branch for reference.


​​​​References

​​​​Wiki page - http://wiki.osm.org/wiki/WikiProject_Philippines/Data_import/DepEd_Schools

​​​​Discussion - DepED MAPAaralan to OSM Import

​​​​Topic: Is it acceptable to use coordinates from Google Maps and/or Earth on Wikipedia?

​​​​Link for reference, not that we're planning to use the coordinates we got as they are.

​​​​OSMph Mailing list: https://lists.openstreetmap.org/listinfo/talk-ph
Select a repo